use extend when sourcing devel or install setup.bash with rosws

[e1000i]

This is a suggestion based on my experience. Do you think will be useful to use rosws to setup easily workspace hierarchies and overlays?

I have a common workspace where all default packages from ros-distro that I need to modify for some reason rest. In order to avoid overwrite the default install i would like to overlay them over opt/ros ones, and on top of that put my current workspace.

Say I have: ws1, ws2 ws3 in such a way that ws2 and ws3 depends on packages from w1.

Manually I can source ws1/devel/setup.bash and after the current one I want to work, using for example, ws2/devel/setup.bash --extend

I found out that catkin_make automatically generates a .rosinstall file for the devel or install space. So I can use rosws to merge these spaces. But the only one remaining in ROS_PACKAGE_PATH is the last i have added to my ws .rosinstall. So there is no access to ws1 from ws2 or ws3

I have a kick fix that patches all my setup.sh to make them include any devel/sebup.bash using --extend option. That does the trick, but it is permanent.

If the maintainers / authors / users of this set of tools think this use case fit the design requirements of rosinstall and rosws I will gladly contribute a patch to allow using --extend for rosws generated setup.bash in oder to enable extension instead of overwrite when required.

An other consideration could be that wherever a install or devel space is merged, it seems to me that we do so because we want to extend our ws with the packages contained there. If this is true, the default behavior of rosws setup.sh should be using --extend right?

My idea is to share the full project development environment. This will allow students using it to easily setup everything and focus only on the area they are developing/working/studing, while being able to test it against the whole platform and merge quickly improvments in the other modules with rosws update powers.

Any other thought or consideration?

[wjwwood]

tl; dr

You shouldn't use rosws with catkin, but instead you should use wstool and the setup files generated by catkin packages/workspaces of packages.

If this information clears up the issue for you, then maybe consider closing this issue and possibly opening a different issue with a different proposal.

end tl; dr

I think there might be some overlapping confusion here, because it was not intended that anyone use rosws with catkin workspaces. rosws is designed to be used with rosbuild (rosmake) workspaces and therefore the setup.*sh files generated by rosws are not designed to work with catkin workspaces.

The tool that we intend people to use for catkin workspaces is wstool. It provides most of the features of rosws, but does not generate any setup.*sh files as that is the responsibility of the packages in the catkin workspace. The catkin workspaces drop a .rosinstall file in the root of their prefix (which is why there is one in /opt/ros/indigo for example) in order to allow building rosbuild workspaces on top of it (potentially using rosws). So there are a few things in catkin, like dropping this .rosinstall file, which make it possible to have rosbuild packages depend on catkin packages, but there is no compatibility layer to allow the use of rosbuild tools like rosws on workspaces of catkin packages.

If this is true, the default behavior of rosws setup.sh should be using --extend right?

So the setup.*sh files provided by rosws for rosbuild packages work slightly different from the setup.*sh files from catkin packages. The rosbuild setup files when sourced add only the packages local to it to the environment. For example, consider you have workspaces A, B, C, and D where B and C depend on packages in A and D depends on packages in B and C. You can build B and C separately and you can build D by sourcing A then B and C in either order (A, B, C or A, C, B). This is because the setup files for rosbuild are additive and cumulative.

In the case of catkin setup files, however, this would not work. When you build B you have only sourced A. So in the future sourcing B will set the environment to be A+B. But then if you source the result of C (which was built by sourcing A and building C) then your environment will be set to A+C without containing B. Note that in neither the B case nor the C case do you have to explicitly source A after building. If you want to build D then you need to source A, build B, source B, build C, source C, build D. then once you source D you'll get A+B+C+D. This is because the catkin setup files set the environment to what it had at build time plus what was built. So it is not additive but instead accumulates at build time.

This seems more inconvenient, but on the other hand the rosbuild way allows for inconsistent state because you can build a set of packages in one environment and run them in another. This can lead to subtle bugs when used in conjunction with overlaying packages at different versions. However, the catkin way is not perfect either because it cannot perfectly reset the environment, such that sourcing X then Y (where Y does not build on X) does not necessarily completely undo the sourcing of X. This is because arbitrary things can be done in the sourcing of setup files, not just setting of the main environment variables we normally modify.

We foresaw people wanting to emulate the rosbuild style setup files, so we added the --extend option on the catkin setup files which makes them behave cumulatively like the rosbuild ones.

Another note is that rosws would setup implicit chaining of setup files which makes them behave sort of like the catkin setup files. In this case you would build B against A, then you could source B which would implicitly source A.

We've bounced back and forth on this and its a problem that we actively look to solve in a more consistent way in our prototypes for the next iteration of the build system.

[tkruse]

+1 to all @wjwwood said.

IMO, The only nice chains has two elements, the first being /opt/ros/... (or a similar location which is not really a workspace, but a stable install location), and the second an actual workspace. All other chains are very likely to cause more damage than good.

So IMO, for setting up a dev env for students, the one and only thing you should optimize for is the time that is necessary to support the students when they run into problems. And this time grows a lot with chains. If you let them work in just a single workspace (no chain), they will be able to fix most of their problems by themselves doing a clean rebuild.

Possible confusion in setups like you describe happen when:

• packages in ws2 are built, packages in ws1 are changed and built, and then someone tries to run (without rebuilding) things in ws2
• packages in /opt/ros change, but the packages in ws1 are not rebuild, and someone tries to rebuild/run packages in ws2
• For shared workspaces instances (rather than each student having his own ws1): A student working on ws2 only goes home one day, another person changes ws1 in the meantime without telling the others, and the student comes back, makes changes, tries to run his code, but fails errors. As the student does not know ws1 has changed, he looks for the reason of failure in his changes instead of changes in ws1
• This list does not even include possible bugs in the ROS toolchain for chains, or students making tiny mistakes when setting up their workspaces

I would recommend to only share a ws1.rosinstall file listing all the packages that you would have put into ws1. Then duplicate all those packages into the workspaces of your students. The wstool merge command (same for rosws) can work with such files.

You could ask e.g. in ros-buildsystem SIG for advice on how to set up environments for teams of students, as several people in the ROS community have done this for their labs. This is also where you should reply to if you want to discuss this further.

[dirk-thomas]

Closing since the repository is about to be archived: see #126.