Efficient pack development at scale

[jeking3]

SUMMARY

Currently stackstorm-exchange packs have a development workflow that requires the version to get bumped with every pull request. As part of a discussion in a pull request for the vSphere pack, some limitations of the current methodologies came up which will hinder scaling development. As the number of developers in the community grows, packs will need a better way to accumulate many changes between releases.

ISSUE TYPE

• Best Practices

OS / ENVIRONMENT / INSTALL METHOD

StackStorm-Exchange

DISCUSSION

In the vSphere pack (and likely other packs) today, every pull request requires a version bump. If two changes are committed to master and the version is not bumped, one cannot determine which version that they might have, since the master branch always has the version of the latest release in it, and there is no staging branch.

This will not scale, and a solution is needed to enable for expected growth in the community. The boost community for example solves this by maintaining a master branch which is always deemed to be the latest release, and a "develop" branch to which all pull requests are targeted. This allows each boost library to accumulate multiple changes in a holding area ("develop") until the maintainer deems them sound, good, etc. The maintainer then merges "develop" into "master" (this should be a simple fast-forward), and then the maintainer updates the version and performs release activities (a final CI build, tagging the release, and bumping any version info to whatever will be next.)

To translate this mechanism to a pack repository, one would:

1. Create a "develop" branch off "master".
2. Change github to direct all PRs to "develop".
3. Now multiple developers can submit as many changes as they need.
4. Maintainer decides it is time for a release, and fast-forwards master to develop, then commits a change to pack.yaml with the version bump, then tags the release and performs any other release related activities.

This has many benefits:

1. Allows a pack to accumulate multiple changes between releases.
2. Avoids collisions on pack.yaml if every PR would normally need to update it with a version bump.
3. Puts the decision of what is a patch/minor/major bump on the maintainer, not the submitter.

[bigmstone]

This approach could work, and does for a lot of other projects. On the ST2 main repo we've opted for a development master and release branches for specific releases so we can maintain multiple versions of ST2 at the same time if needed. (For instance pushing fixes for our paying customers for previous versions). We would likely take that approach on an exchange pack once the active development got to a level that required it. In the mean time if you'd like to coordinate releases with others I see no reason why a development branch couldn't be contributed to by various users to batch changes in a release. So if you'd like to take that approach with coordinated individuals that would like to contribute to the pack I would support that. To date we just haven't seen enough active development on the pack to warrant the modified release process.

[jeking3]

If the standard behavior for pack installation always used a tag called "latest" or "stable" then leaving master open as the collection of all new, unreleased changes seems reasonable. One could install a pack in a variety of stable ways to get what you want:

Today

# st2 pack install vsphere
    installs master
# st2 pack install vsphere=v0.7.7
    installs a specific release

Proposed

# st2 pack install vsphere
    defaults to installing the tag "stable" or "latest"
    fails if the repository has no stable tag,
      encourages opening an issue,
      and provides instructions to work around it
# st2 pack install vsphere=master
    installs the current development tip (you have to request possible instability)
# st2 pack install vsphere=v0.7.6
    installs a specific version

Pack maintainers would then be able to allow for pull requests on master without version bumps in pack.yaml, and control the release cycle including when a release is promoted to being stable. This would allow a pack to have a beta or testing tag that may be a fresh release, for example. This avoids the dual branch scenario. If release branches become necessary they would fit right into this strategy. Part of the release cycle might be to bump pack.yaml in master to something like v0.8.0-dev after making a release , while at release time to change it to v0.7.8, then tag it that way. The -dev designation would let people know they are not using a release.

[arm4b]

# st2 pack install vsphere
   installs master

I just checked, and you're right. This now sounds unexpected to me, too.

I think the improvement is much easier.
Instead of installing master on st2 pack install vsphere we should install from latest stable tagged release like vX.Y.Z, which is v0.7.7 in case of vsphere pack https://github.com/StackStorm-Exchange/stackstorm-vsphere/releases.

This will leave enough space for maneuver in master branch for someone maintaining the pack. Additionally, it doesn't enforce overcomplication like -dev and develop: pack contributors can ask to version every PR or ship releases with a bigger amount of changes when version is not bumped in a PR.
That would be also consistent with our model where we consider master as something "unstable" and git tag/release as stable between the majority of StackStorm repositories.

@jeking3 Do you think that would be a decent solution?

I'm surprised this doesn't happen and I was sure that we install from latest stable git tag/release in st2 pack ... by default, not from master. It's a good catch @jeking3!

[jeking3]

Yes, that sounds reasonable. This will solve the issues described. Your suggestion further reduces tagging requirements on the maintainer by having st2 pick the most recent semver tag when one is not specified.

[LindsayHill]

Agreed: st2 pack install should default to installing most recent tagged version, rather than installing from master.

New PRs can continue to be submitted & merged into master branch, but this will not automatically create new tag. This should only be done as part of a 'release' step that will incorporate bumping version in pack.yaml, and creating a new tag. The release process can then combine multiple PRs into one versioned release.

Needs:

• Changes to pack install code
• CI/CD process updates for Exchange to change how/when tags are created

Will also need some thought around:
1/ Installing from a file:/// location - this is a git repo. What is expected behavior here? Install from tag, or master?
2/ Users with their own internal Exchange index - what if they are not tagging their releases? How do we handle things if there is only master, no version tags? May need to have some methods of enforcing old-style "always install from master"

[arm4b]

New PRs can continue to be submitted & merged into master branch, but this will not automatically create new tag. This should only be done as part of a 'release' step that will incorporate bumping version in pack.yaml, and creating a new tag. The release process can then combine multiple PRs into one versioned release.

I think current release automation for StackStorm Exchange is flexible enough as is.

At the moment, only pushing to master with version change in pack.yaml will enforce new git tag/release creation via automation.
This way pack maintainers (who really need it) have a chance to ask contributors (or commit to a forked PR directly) and decide whether to change or not to change pack version which will trigger a new release. It means someone who needs it is already covered and we don't change the behavior for everyone else.

And so I think the improvement is scoped mostly to st2 core.

[LindsayHill]

At the moment, only pushing to master with version change in pack.yaml will enforce new git tag/release creation via automation.

I'm not sure about that - I believe it always tags code based on a merge to master, it doesn't check that the PR includes pack.yaml

[arm4b]

Quick example: StackStorm-Exchange/stackstorm-ansible#27
Exchange automation is smart enough to detect version change in pack.yaml and use that number for git tag/release and so for example it's possible to bump a major version 0.5.0 -> 1.0.0 via pack.yaml.

[jeking3]

It would be nice if the master on each pack's repository uses a SemVer compliant version with a development tag of "-dev", for example:

version: v0.8.0-dev

The reasoning for this is that if someone specifically asks for =master then you want the pack version to specify vx.x.x-dev to signify that it is not a release. Without this workflow it will look like a specific version is installed, when it is that version plus all the changes accumulated in master.

The release of a pack would then require two steps:

1. The maintainer commits a pack.yaml without a development tag. CI automation detects this and treats it as a release. CI would fail if a tag already exists with this version, so as to avoid human error. This makes a release out of master, where the CI automation creates a tag matching what's in pack.yaml. I'm not sure how github releases are inserted, but this probably happens here as well.
2. The maintainer commits another pack.yaml with a development tag. CI automation ignores this. Pulls from master once again look like development builds.

[arm4b]

You can use that format if you want manually via pack.yaml, semver-compliant version would be accepted and created as dedicated git tag. No reason to ignore it and we're not committed to change the automation for this specific use case.

---

Relying on latest git tag instead of master in st2 pack install makes sense, - thanks for raising it! 👍
This change may be incorporated in a future st2 release.