Forking URDFDom?

[davetcoleman]

There are many desired changes to the URDF spec that are needed for projects like MoveIt and Tesseract that have not been merged into this repo. This has been an issue for many years - the spec is basically unchanged while the SDF format goes through many many iterations. This is hurting the ROS project. A good discussion to get a sense of this issue is here

Based on today's MoveIt Maintainer meeting, we are considering forking this URDF project so that new features can be added. This is not an ideal outcome because we don't want to fracture the ROS community and have code duplication. As I understand it, ROS-Industrial already has forked it.

In order to prevent a fork, we believe URDFDom needs to be released within ROS, not Ubuntu Universe, so that changes can be released more frequently. We also believe we need a more aggressive policy in breaking things, especially between ROS versions, so forward progress can be made.

I may have missed some key points here as I don't follow every detail of the URDF project, but I've been following this project loosely for years and am trying to sum up my observations and the feedback I've heard from others.

How can we get the long list of PRs/improvements in this repo merged without the fear of breaking something?

@scpeters @jmirabel @rhaschke @tfoote @Levi-Armstrong

[jmirabel]

There are two different issues to be discussed:

1. the fact that the source code is frozen,
2. the fact that the binaries are released at a low frequency.

I cannot help for point 2. I can give my two cents on point 1 and summarize what has been said in #24 .

We also believe we need a more aggressive policy in breaking things, especially between ROS versions, so forward progress can be made.

I fully align with @scpeters points: An old parser should fail verbosely on a file with a too recent format version, without requiring to update the old parser.

This was the blocking point for #24. Some months (or years) after opening the PR, I proposed a solution which consists in:

• changing the main tag (currently, robot) into, for instance, urdf,
• and adding a version number to tag urdf.

I insist on this proposition because I prefer a consensus more than a fork.

[scpeters]

Let me first apologize for not responding to these pull requests that add features to urdfdom. Let me try to communicate my thoughts on some of these challenges:

I've spoken before about the issues with old parsers not recognizing new fields (#24 (comment)). There is a version attribute in the urdf.xsd schema at urdfdom, but I haven't seen it used. Ideally, each parser should know which versions it supports and give a warning or error if it tries to load a version that it doesn't recognize. For me, the lack of versioning is the biggest blocker to adding features to urdfdom.

Another challenge to adding features to urdfdom_headers with the current architecture is the potential for breaking API or ABI when modifying data structures. An alternative is using a PIMPL pattern with get/set functions instead of direct member access.

For sdformat, we are discussing the use of xml namespaces for custom elements and prefixes:

• https://bitbucket.org/osrf/sdf_tutorials/pull-requests/32/proposal-for-custom-elements-and/diff

[rhaschke]

I think we fully understand the constraints of maintaining such a key package. However, we should enter discussions how to address and overcome these concerns to allow some progress and avoid alternative forks. Hopefully, we can start this process at ROSCon.

[traversaro]

I personally fully agree with @davetcoleman suggestion that the URDF format should be allowed to evolve, and in particular with @jmirabel proposal.

On the top of that, I think some points that should be addressed in the discussion are the following:

• I always intended urdfdom_headers and urdfdom (together with urdf_parser_py) as reference implementations of the URDF "specification", that I always understood it is maintained in the ROS wiki at http://wiki.ros.org/urdf/XML . Ideally a proposed change to the URDF format (such as sensor refactoring #22 or Add Capsule primitives #24) should come in the form of a modification of the specifications, together with the modification of the reference implementations. However, what is the current process for proposing modifications to the URDF specification? REPs? I think it should be clarified the process for proposing changes to the URDF specification, for example by moving it to a GitHub repo and use the usual Pull Request workflow to propose. discuss and accept modifications.
• Once a version system is in place, should a given urdfdom version and related ROS distro provide support to load only URDF of the latest version, or also provide some kind of backword compatibility?
• As suggested by @scpeters, a clearly documented way of specifying URDF custom extensions (as done for example for glTF or COLLADA would be a great way to permit to subset of URDF users on agreeing on some new features, without being blocked by the process of getting a change in the official spec or getting the reference implementation released.

There are been a few related discussions in the past that I think could be interesting for people reading this thread:

• https://discourse.ros.org/t/urdf-ng-ros2-urdf2-discussion/511/20
• https://gist.github.com/mikedh/ce09d1bf83767fefded25be519a61564

[davetcoleman]

Thanks for all these very quick thoughts and feedback! Lots of good actionables here, that I agree should be discussed at ROSCon

the fact that the binaries are released at a low frequency.

This seems easily fixable by providing a Melodic & future Noetic release in the ROS 1 build farm, right? Just a matter of someone making the release...

[tfoote]

Making the releases is relatively easy, knowing what exactly to release is the hard part. Even just jumping it into a ROS package will run into issues of colliding symbols and breaking existing implementations that rely on the system packages.

There's definitely space to accelerate the rate of development. But we do need to develop the migration plan with versioning etc that can be implemented into the specification to enable the migration path so that maintenance updates can be pushed.

There's significant value in being upstream in Ubuntu, but it does slow down the release cadence possiblities and require significant planning ahead for major changes. I'm quite hesitant to support custom extensions for various elements. That is a way that you can get to the point that they all talk URDF, but nothing is fully compatible because they all use different custom extensions.

Getting the major stakeholder together at ROSCon sounds like a good idea.

[jmirabel]

Was this discussed at ROSCon ? I couldn't attend.

[clalancette]

Was this discussed at ROSCon ? I couldn't attend.

Yes, we had a meeting about it at ROSCon. We're going to send out some additional details this week, but the short of it is that we are going to attempt not to fork URDF, and see if we can make a bit more progress with it. There is also some talk of switching to a different file format, but we still have to finish discussing that and have a follow-up meeting.

[gavanderhoorn]

@clalancette: what's the current status on adding the version nr to the format?

[clalancette]

@clalancette: what's the current status on adding the version nr to the format?

It's been done in ros/urdf_parser_py#52 and ros/urdfdom#133. I've also released a new version of urdfdom_py into Melodic.

I think what is left to do is:

1. Merge ROS 2: Add in support for the version tag. (#52) urdf_parser_py#54 for ROS 2
2. Port Add in support for a version tag. urdfdom#133 to https://github.com/ros2/urdfdom
3. Release above changes for ROS 2
4. Release urdfdom as an Ubuntu package update
5. Announce on ROS Discourse

[gavanderhoorn]

Nice 👍

Seems I need to Watch even more repositories to not miss this sort of thing ;)

Should there be a 4. Announce on ROS Discourse?

[traversaro]

@clalancette There is any plan of propagating this change in the fork of urdfdom used in ROS2, i.e. https://github.com/ros2/urdfdom ? The fact that ROS2 install its own urdfdom that shadows the one used by ROS1/Gazebo is already quite error prone (see robotology-legacy/gym-ignition#118 (comment)), and if the two versions are not in sync that will get even worse.

[clalancette]

@clalancette There is any plan of propagating this change in the fork of urdfdom used in ROS2, i.e. https://github.com/ros2/urdfdom ? The fact that ROS2 install its own urdfdom that shadows the one used by ROS1/Gazebo is already quite error prone (see robotology/gym-ignition#118 (comment)), and if the two versions are not in sync that will get even worse.

Ug, I didn't even realize that. OK, I'll update the list above, we definitely need to get the change into the ROS2 repository as well.