[RFC] Unify all kata components under a single repository

[ganeshmaharaj]

Description of problem

I will start off stating that I am writing this all down as someone who is new to the project and with my limited understanding. Please help me educate myself on things that i might be missing/mis-stating in this RFC.

The main components of kata are split across a few repositories. runtime, shim, agent, proxy. There are a couple of additional repositories that help test all the components together and a few more to aid with what is the final product. from talking to a few folks, the reasons for creating seperate repositories for each of those components were two-fold.

1. Allow folks to write an alternate shim or proxy which can be plug & play with all the other existing components.
2. Make sure that we are able to keep the components truly decoupled and use standard interfaces to communicate with each other without being heavily tied to each other.

I have come to understand that kata is now fairly stable in how the components are going to be working with each other and interfaces are also fairly well defined. With that as premise, I was wondering how the community would feel about merging the code bases of the key kata components into a single repository?

Pros:

• Less Paperwork (in the sense, one does not have to touch multiple repositories, open multiple issues for their fixes and modify yet another repository to add tests to their code and tell Jenkins to make sure it picks up all the changes while testing).
• Better developer workflow where users can have a single repository, make changes, be able to run both functional and unit tests with some ease (It is not really that hard today and has been very well automated but this will reduce the additional checks one might do to make sure how well they test their code).
• Release will be simplified on how many repositories need tagging, pinning, packaging, etc.
• Folks can still write an alternate agent or proxy and have it replace the existing code.

Cons:

• We are opening the door where changes might break the strict decoupling of components without realizing.
• Anything else you can think of?

I believe if we add add checks/processes that can verify we maintain the design ethos we have today for Kata even after moving all the sources into a single repository.

If we choose to move to a single repository, some things to consider.

1. What components/repositories will be merged into the new repository and what will still remain as a seperate repository?
   • I am thinking proxy, shim, agent, runtime, tests, documentation will all be merged into the new repository and all the others will remain as they are today.
2. how are we going to maintain each components history?
   • We can archive the current repositories and tell folks to refer to them in case they wish to view previous commits for each of those components.
3. Any changes to the CI systems to handle this?

[sboeuf]

Thanks for raising this point @ganeshmaharaj.
As you very well summarized the previous reasons why we chose to go with different repositories, it might make sense now to move to a single one. We would need to figure out the details about which repos we would like to see merged, and I think that as a starter, we could focus on having shim, proxy and agent hosted under the runtime repo (this might need to be renamed, idk).

IMO the main point in having this repo merging happening now, as you mentioned, is based on the fact that Kata reached a stable point regarding its architecture. The generic design and interfaces that we wanted before was to enforce proper design for each component, which is not a concern anymore now that it's very unlikely to see those components being modified.

I think the proposal makes a lot of sense as it would simplify everything related to "paperwork" and "releases", and fits well with the state of the project. By that, I mean now that we consider Kata as a stable project/architecture, we should cleanup/simplify the leftover complexity from the old days (I'm sure @egernst will appreciate 🙂)

[gnawux]

Hey guys,

In the pre-launch stage of the Kata Containers project,

we (@bergwolf @laijs and I) suggested we should not separate code into several repos.

While you (including @sboeuf and @sameo) suggested we should put each binary in a separated repo. And we accepted your suggestion. The discussion was done in Oct 2017 in Austin.

I don't think the pros and cons changed during the past one and a half year. I don't want to change the situation either.

[gnawux]

This is a topic has been well discussed before. And if no new interesting point was introduced, I will close this issue on Monday.

[ganeshmaharaj]

@gnawux I am sorry but I was not part of the project or had a chance to participate in that conversation back in 2017, so apologies if I am bringing back a done and dusted topic. If the pros and cons have not changed and still are valid and with time the project getting stable, would it not be a good time to revisit some of the old technical debt? This unification might help new users like me wrap their head around working with and contributing to kata. Will there be other issues that unification of repos will cause that I am not seeing?

[gnawux]

One more point, you guys mentioned

Less Paperwork (in the sense, one does not have to touch multiple repositories, open multiple issues for their fixes and modify yet another repository to add tests to their code and tell Jenkins to make sure it picks up all the changes while testing)

Yes, it's a bit too complicated to modify multiple for one thing, but it's not too troublesome because the code is maintained by a same group of persons actually and you may call them in slack and AC meeting.

If you want to make things easier, try to move intel/govmm code to kata-containers may be more appreciated. It's hard to send PR in intel/govmm, wait for approving, and update dep in kata-containers/runtime. It's only a small piece of code, right?

[ganeshmaharaj]

One such example of where unifying the sources will help is this. #1372 was part of a series of patches to switch the golang linter. The change in tests has already landed prior to this patch landing which means the new linter will start flagging the changes in runtime as issues and might fail PRs. (That is my understanding on how all this works). Not bringing this up to cause anyone discomfort, but just saying even though it is the same people who are approving changes, there is still a possibility that one such multi-repository change might slip the cracks. This might not be an issue if we put all the sources in a single location.

[teawater]

One such example of where unifying the sources will help is this. #1372 was part of a series of patches to switch the golang linter. The change in tests has already landed prior to this patch landing which means the new linter will start flagging the changes in runtime as issues and might fail PRs. (That is my understanding on how all this works). Not bringing this up to cause anyone discomfort, but just saying even though it is the same people who are approving changes, there is still a possibility that one such multi-repository change might slip the cracks. This might not be an issue if we put all the sources in a single location.

Do you have any other example? I didn't see any station like this will flood all the repo of kata except test issues that I will open a special issue to talk about.

[devimc]

@gnawux I agree with you

[sboeuf]

@gnawux

While you (including @sboeuf and @sameo) suggested we should put each binary in a separated repo. And we accepted your suggestion. The discussion was done in Oct 2017 in Austin.

Yes that's right, but @ganeshmaharaj is pointing the fact that the situation has changed since we understand clearly what's needed for Kata now.
I don't have strong opinion on going one way or the other, but the proposal from @ganeshmaharaj is valid and makes sense IMO.
A decision from one and a half year ago doesn't have to be unchanged if we realize this is not the best option for the project now.

[ganeshmaharaj]

Do you have any other example? I didn't see any station like this will flood all the repo of kata except test issues that I will open a special issue to talk about.

My history on this project is very limited, but i will speculate this is not the first time this has happened. Majority of them will be around tests, but i am sure there are other scenarios where at the very least two dependent patches in two different repositories did not land at the same time and there was a shuffle to get them all merged.

[ganeshmaharaj]

Just out of curiosity, is there a big negative to this effort that i am not seeing? I understand this was already looked at a year and half ago, but other than the one time effort to do this, are there other issues that will cause heart-ache in the long run? It seems like the pros still outweigh the cons and still seems like a good set of reasons to do this effort.

as far as govmm goes, i have little control over that project, but i will try and look at it and see if it is feasible.

[umarcor]

@ganeshmaharaj, IMHO the issues you describe can be addressed with github and optionally git:

In the specific case you mention, kata-containers/tests#1315 should have not been merged yet. #1372 is still open, so:

• Whoever decided to merge lint: Switch golang linter to golangci-lint tests#1315 already knew that and he/she didn't think it was relevant.
• Whoever proposed lint: Change go linter from gometalinter to golangci-lint #1372 did not properly identify the issue as conflictive.

Note that, should everything be in a single repo, it wouldn't be possible to merge kata-containers/tests#1315 until all the changes are done in all the subdirs. The ability to do so provides a power that has not been handled as expected. But it is desirable, indeed. Because this means that it can be done in other contexts, when it does make sense to merge a piece of a patchset while other parts are to be finished yet.

However, this is easy to fix: just document it. It should be explained somewhere how to mark a patchset and how to relate the issues of a patchset to each other, so that a maintainer knows when all the parts are ready.

Moreover this enhances distribution of responsabilities, as each part of the patchset can be reviewed by different people with access to some repos only. When all of them are approved, an 'admin' user can merge/push everything in a few minutes, so that it is not required to synchronize multiple repo maintainers. This could probably be automated through a bot, but that's probably me overthinking now.

Less Paperwork

Having multiple repositories provides multiple layers to organize access. In a single one it would be difficult to let some maintainers work on some of the components only. It would be done 'with pen and paper'. Nevertheless, as you have commented, this is not very useful ATM, because such features are not being used.

open multiple issues for their fixes and modify yet another repository.

Regarding opening multiple issues, I personally like each repo to have it's own tracker. But it is possible to enable a single one (say this one), and disable it in every other repo. GitHub allows to transfer issues between repos, so no history would be lost and all the references would be updated automatically (I think). Such a solution would be in the middle ground:

• Issues are discussed in a single centralize place.
• Patches are to be applied separatedly (see comments about the procedure above).

one does not have to touch multiple repositories

In order to avoid having to touch multiple repositories, it is possible to create a single dummy kata-containers/top and declare every other as a submodule of it. It is kind of redundant to go modules, but it allows to have submodules which are not golang packages. It is also easier to replace one of the components with a fork.

Release will be simplified on how many repositories need tagging, pinning, packaging, etc.

Right now all the packages are tagged at the same time with the same tag. As a result, it is possible (unlikely tho) that some of them are tagged multiple times without any commit in-between. If kata-containers/top was created, it could be not only dummy, but actually hold kata releases. That would be the only repo which users should check to download full pre-built packages. Since submodules would contain the info about which specific commit of each package is used for the build, packages could follow their own versioning according to semver. Top would be, in a sense, a composer with an aggregated changelog.

Folks can still write an alternate agent or proxy and have it replace the existing code.

Any user contributing a small piece to one of the packages is required to retrieve the full repo. This might not be relevant for active developers who need to touch multiple packages anyway, but it can be annoying for others. It might be worth checking what's the size of a git clone i) separated, ii) integrated through submodules and iii) merged in a single repo.

EDIT:

24.7M agent
3.8M  documentation
6.5M  proxy
31.8M runtime
80.7M shim

152.7M single [runtime (4x) + agent (6x) + documentation (40x) + proxy (23x) + shim (2x)]

How are we going to maintain each components history?

It is possible to merge the history of all the repositories in a single git repo, so no info is lost at all:

• Rewrite the history of each repo to move everything to a subdir.
• Create a new clean repo.
• Sequentially add packages as remotes and rebase remote master branches on top of the local master.

However, I don't know if it is possible to have the histories interleaved. See https://github.com/mbitsnbites/git-tools

We can archive the current repositories and tell folks to refer to them in case they wish to view previous commits for each of those components.

It is also possible to move all the issues, as commented above. I don't know if it is possible to move PR issues, tho (I don't think so).

[umarcor]

Do you have any other example? I didn't see any station like this will flood all the repo of kata except test issues that I will open a special issue to talk about.

To provide some insight, I merged runtime, agent, documentation, proxy and shim with https://github.com/mbitsnbites/git-tools. The result is https://github.com/umarcor/kata-single (note that I will remove it in the following days). It can be viewed with gource in a golang:alpine container. The X server for the container can be started easily with x11docker:

# x11docker -i --user=0 -- --runtime=runc -v /$(pwd)://src -- golang:alpine sh
...
/go # apk add gource git mesa-dri-swrast
/go # git clone https://github.com/umarcor/kata-single
/go # cd kata-single 
/go # gource -1920x1080 --key -a 0.05 -s 0.2 -i 0 --hide bloom,filenames --hash-seed 128 --date-format "%Y / %m / %d [%W]" --font-colour aaaaaa --caption-size 4

Since vendor directories are so large, you need to focus on the center. Each time someone touches (orange light) multiple edges, there is a potential scenario as the one described by @ganeshmaharaj.