best way to modify all repos in a workspace?

[dustingooding]

What's the best approach to, say, switch all repos in a workspace to a different branch if that branch exists? For example, 10 repos in my workspace are on 'master'. But I'm working on a new feature that affects multiple repos. I'd like to switch back and forth between my master branch and feature branch(es) easily without having to go to each repo root and "git checkout".

vcs-custom allows for this (in a way) by using 'vcs custom --args checkout myBranch".

(Also, if there's a more appropriate way to ask questions or suggest new features, I'll be glad to do that instead.)

[wjwwood]

There is no automatic tool for this that I know of. You could maintain multiple .rosinstall files in your source space and symbolic link one of them to .rosinstall depending on which one you want then you can do wstool up -t src or similar to get it to update them. Here is an example:

% tree -L 2 -a
.
└── src
    ├── .rosinstall -> indigo.rosinstall
    ├── catkin
    ├── groovy.rosinstall
    └── indigo.rosinstall

2 directories, 3 files

% vcs st
=== ./src/catkin (git) ===
On branch indigo-devel
Your branch is up-to-date with 'origin/indigo-devel'.

nothing to commit, working directory clean

% cd src

% rm .rosinstall

% ln -s groovy.rosinstall .rosinstall

% cd ..

% wstool up -t src
[catkin] Updating /private/tmp/wstool_ws/src/catkin
[catkin] Done.

% vcs st
=== ./src/catkin (git) ===
On branch groovy-devel
Your branch is up-to-date with 'origin/groovy-devel'.

nothing to commit, working directory clean

[dustingooding]

Maintaining a bunch of rosinstall files for all the various tickets/features I'm working on sounds like a nightmare. I'll probably work with vcs-custom for now. Consider this a feature request. As wstool is intended to manage a multi-repo workspace, having a way to change multiple repos at once seems a natural thing. Thanks for the reply.

[tkruse]

Hi,

wstool(based on vcstools) provides an abstraction over multiple SCMs (git, mercurial, bazaar, subversion). An operation to switch to a branch would require the notion of "branch" to be standardized accross SCM providers, something which cannot easily be done.

I think what you want could be achieved if you wrote custom scripts that make good use of the extended wstool info command:

#! /usr/bin/env sh
IFS=','
wstool info --only=scmtype,uri | while read scmval urival; do 
       cd $urival
       $scmval $@
done

name this ws-each, make it executable and put it in your workspace root, then ./ws-each checkout develop should run what you seem to want.

Providing a wstool command to run custom vcs commands like vcs-custom is not that difficult, but would require some volunteer effort to do the coding including regression tests. I cannot tell how much the community would benefit from that feature.

[wkentaro]

The ws-each command is easy to implement even if without wstool's subcommand (by using wstool info --only=localname,scmtype,path, grep, awk), but I think adding the command is a good idea to enhance wstool as a workspace managing tool.

I'm thinking the command will be something like git submodule foreach 'git pull origin master'.
And enabling to assign repository in workspace is good idea like wstool ws-each repo0 repo2 'git pull origin master'. (but in this case, ws-each may be inappropriate, I prefer foreach)

Do you have any idea? about

• implementing this kind of command
• name of the command

(Maybe I will have time to help to implement this feature after my project in next month.)

[wkentaro]

I moved from: #38 (comment)

[tkruse]

For the name and structure, I like wstool foreach [--bzr] [--git] [--hg] [--svn] [repository]+
On the wstool side, this could be largely copy&paste of the status command (see multiproject_cmd:cmd_status()).

Since we have shell command execution in vcstools, I can imagine reusing that, so the worked in cmd_foreach() could directly invoke vcstools.common:run_shell_command(...)

All of the arguments to run_shell_command might also be arguments to foreach, so maybe foreach would become:

wstool foreach [--bzr] [--git] [--hg] [--svn] [repository]+ [--shell] [--us] [--hide-stdout] [--timeout=double] [..no-filter]

I believe verbose must be set to true, and no_warn seems to be an obsolete argument. Some refactoring might be possible in vcstools.py, because the verbose argument and the _discard_line() functions are specific to vcs commands (and also evil).

[wkentaro]

So the todo list becomes like below?

• 1. refactor vcstools in particular: run_shell_command(), _discard_line()
• 2. Add cmd_foreach() to wstool

regarding run_shell_command() arguments, I'm thinking like below:

• no_warn arg is not used in run_shell_command (maybe removed in refactoring)
• always shell=False for security (according to: https://docs.python.org/2/library/subprocess.html#using-the-subprocess-module)
• always verbose=True so does not care about show_stdout because verbose overwrites (so refactoring _discard_line() is probably not required)
• --quiet option for no_filter=True otherwise no_filter=False
• --timeout=double for timeout=double otherwise timeout=None
• always us_env=False to get users' expected outputs

so the foreach's options will be
wstool foreach [--bzr] [--git] [--hg] [--svn] [repository]+ [--quiet] [--timeout=double] 'command'

What do you think about above?

[tkruse]

refactoring vcstools is optional, it is a bit dirty, but can be used as is with verbose=True, I think. Also this should not be refactored too much, as this is the most important part of vcstools which must not break (else ROS installation is broken).

I removed no_warn in the other branch by now.
I think most of your assumptions are reasonable, but if run_shell_command() has the options it has, it should be for good reason (we ran into problems without the options).

So it is quite possible users will have similar problems for their commands as we had for our commands, and would be happy to have those options (in particular shell and us_env).

[wkentaro]

Now I think the foreach command is easily created with function in shell like below.
And in most cases, this works, so maybe no need to create new subcommand at this point.

# bash

wstool_foreach () {
  local -a locations
  local e
  locations=$(wstool info --only=path)
  for e in $locations; do
    (cd $e && eval $@)
  done
}

wstool () {
  case "$1" in
    (foreach)
      shift; wstool_foreach $@
      ;;
    (*)
      command wstool $@
      ;;
  esac
}

now I can run

$ wstool foreach "[ -d .git ] && git pull origin master"

[tkruse]

going through bash is an clean and easy solution, the only small problem is that we cannot nicely put foreach into the tool "--help" that way. However we could put it into --help as extra text, and deliver it with the autocomplete functions. I am undecided whether to do so.

[wkentaro]

yeah, I mean this can be configured at shell of users who like this subcommand, and wstool officially does not support this.