Update composefs, unified storage by cgwalters · Pull Request #2044 · bootc-dev/bootc

cgwalters · 2026-03-04T21:18:36Z

The latest composefs-rs stores manifest and config objects and
the manifest becomes a GC root, so we can use that instead of .imginfo
sidecar files.

The flow now is:
bootloader entry -> deployment -> origin file
-> manifest digest -> manifest -> [config | objects]

For backward compatibility, fall back to the legacy .imginfo
file if the .origin does not contain a manifest_digest key.

Drop the really old hacky fallback that did network fetches.

Note the manifest becomes part of the GC root.

Closes: #1977

gemini-code-assist

Code Review

This pull request refactors how container image manifest and configuration are handled for composefs deployments. Instead of storing them in a separate .imginfo file, the manifest digest is now stored in the .origin file, and the manifest/config are read directly from the composefs repository. This is a good improvement that centralizes image metadata storage. The changes are consistent across the codebase, and backward compatibility for older deployments with .imginfo files is maintained. I've found one area for improvement regarding repository handling efficiency.

cgwalters · 2026-03-13T15:09:57Z

OK, this is passing tests now and I think is a notable cleanup.

I think this may be one of the last ~breaking changes for the composefs installation layout.

cgwalters · 2026-03-13T15:20:13Z

Actually thinking about this more...it feels like we should really be creating tags in the cfs repo instead of using this additional root stuff. That's effectively what ostree does with ostree/ refs. Will look at a PoC

Basically we create localhost/bootc-<manifestsha> tags that are "owned" by bootc, and our GC is:

walk over all bootloader entires, gather their origin images and load the manifest sha digest
load all localhost/bootc- prefixed tags in the cfs repo
Remove all unreferenced tags
Just run cfsctl gc effectively

In particular a key thing is that this works if the composefs repo holds other data such as app images.

Johan-Liebert1 · 2026-03-16T04:19:01Z

 needless_borrow = "allow"
 needless_borrows_for_generic_args = "allow"

+[patch."https://github.com/composefs/composefs-rs"]


I don't think we need this

We do it's for composefs/composefs-rs#263

Johan-Liebert1 · 2026-03-16T04:29:04Z


-    tracing::debug!("img_bootloader_diff: {img_bootloader_diff:#?}");
+    // Bootloader entries without a state dir are from interrupted cleanups.
+    let orphaned_boot_entries: Vec<_> = bootloader_entries


this will always be an empty vec iiuc? Since we always delete the bootloader entries first

The open_config() return type changed from a tuple to the OpenConfig struct. Point the bootc reverse-dep CI at bootc-dev/bootc#2044 which has the matching API update, until that PR is merged to main. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

The OCI digest cleanup breaks the bootc API boundary. Re-enable once bootc-dev/bootc#2044 lands on bootc main. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

cgwalters · 2026-04-24T20:36:40Z

Awesome! This one is finally passing all the tests.

We need to merge composefs/composefs-rs#285 and then rebase this, and it could use a bit more cleanup, but then should be good to go.

`just bcvk up` with `BOOTC_variant=composefs` silently ignored the variant — it read the env var but never passed `--composefs-backend` or related flags to `bcvk libvirt run`, so the VM always installed with the ostree backend. Extract a shared `BcvkInstallOpts` helper (new `bcvk.rs` module) that both `sysext.rs` and `tmt.rs` use to build bcvk CLI arguments from `BOOTC_*` environment variables. This fixes the sysext path and eliminates the duplicated firmware/install arg construction in tmt. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

The xtask run-tmt command now reads BOOTC_bootloader, BOOTC_filesystem, BOOTC_seal_state, and BOOTC_boot_type directly via clap env support. When BOOTC_variant=composefs, --composefs-backend is implied automatically. This means `just test-tmt` works correctly with composefs env vars set, without the Justfile needing to forward flags manually. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

Add a podman_client module for pulling container images through podman's native libpod HTTP API with streaming per-blob download progress displayed via indicatif. Starts a transient `podman system service` against bootc's custom storage root (reusing bind_storage_roots and setup_auth helpers) and talks to it over a Unix socket. The response NDJSON stream is read line-by-line via AsyncBufReadExt. Also fix get_ensure_imgstore() to work on composefs-only systems by falling back to physical_root when ostree is not initialized. Factor setup_auth() out of new_podman_cmd_in() so both the CLI podman commands and the API service share the same auth setup. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

The composefs backend has a multi-dimensional configuration space (variant, bootloader, boot_type, seal_state, filesystem) with non-obvious constraints between them. Add a clear reference table and common workflow examples to CONTRIBUTING.md, and a quick-start cheat sheet to the Justfile header. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

By far the biggest change here is to how we do our GC logic. Now, composefs-rs itself holds refs to the EROFS images; we just need to hold onto the images themselves. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

Add unified storage support to the composefs path, leveraging the new support for containers-storage. See: composefs/composefs-rs#285 Assisted-by: OpenCode (Claude Opus 4) Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>

Instead of reading the in memory filesystem to get /usr/lib/os-release get it from the mounted EROFS. This is also prep for providing backwards compatibility due to our newly introduced prefix `bootc_composefs-` where we'll need to create new boot entries and we can get the `os_id` from the mounted root Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com> Signed-off-by: Colin Walters <walters@verbum.org>

While finishing up GC, we had come up with the idea of prepending our boot binaries (UKI PEs, BLS directories) with a certain prefix and we ended up hard requiring these prefixes. If someone has an older version of bootc which they used to install their system with, then upgrade to a new version, many if not all of the important operations would cease to work. This basically handles the backwards compatibility of new binaries on older systems by prepending our custom prefix to all existing boot binaries Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com> Signed-off-by: Colin Walters <walters@verbum.org>

prepend_custom_prefix() already checks has_prefix on each entry and skips those that already have it, so it is safe to call on every boot with no extra cost when migration is already done. Running it unconditionally ensures it fires correctly on systems upgrading from older bootc versions that lacked the prefix. Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com> Signed-off-by: Colin Walters <walters@verbum.org>

Add some basic infra to mock up enough of an installed root to use in unit tests - specifically targeted for the bootloader logic. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

If a previous upgrade attempt was aborted partway through, the state directory may already exist with a /var symlink in place. Rather than crashing with EEXIST, silently skip creation when the symlink is already there. This is purely defensive — the primary guard is the digest check added to do_upgrade — but makes the system robust against leftover state from partial runs. Assisted-by: OpenCode (Claude Sonnet 4.5) Signed-off-by: Colin Walters <walters@verbum.org>

If the pulled image results in the same composefs digest as the currently booted system, there is no new state to write. Attempting to re-write the state crashes with EEXIST on the /var symlink (now separately hardened). This safely updates the origin file to record the new image reference instead, which is the correct behaviour when switching transport (e.g. registry: to containers-storage:) for the exact same image content. Assisted-by: OpenCode (Claude Sonnet 4.5) Signed-off-by: Colin Walters <walters@verbum.org>

When a user has previously opted into unified storage, all subsequent switch and upgrade operations should continue using that path. Previously only the target image was checked; this also checks the booted image, so that switching to a new image while already on unified storage automatically stays on the unified path without requiring an explicit flag. Assisted-by: OpenCode (Claude Sonnet 4.5) Signed-off-by: Colin Walters <walters@verbum.org>

When iterating with the bcvk dev VM, developers often have a locally built composefs image (localhost/bootc). Automatically using that if it exists avoids having to explicitly set BOOTC_BASE_IMAGE on every invocation, and ensures the VM uses the correct bootloader (e.g. systemd-boot) rather than the upstream base's grub. Assisted-by: OpenCode (Claude Sonnet 4.5) Signed-off-by: Colin Walters <walters@verbum.org>

cgwalters · 2026-05-01T21:13:48Z

OK! Now that we have a release out, I think we can get this one in! Obviously a huge change...

Johan-Liebert1

Overall looks okay. A few comments

Johan-Liebert1 · 2026-05-04T07:33:54Z

        }
    };

+    // Remove "root=" from kernel cmdline as systemd-auto-gpt-generator should use DPS


This was removed here 103a8cd

This was a rebase artifact/mistake

Well spotted!

Johan-Liebert1 · 2026-05-04T07:41:04Z

+        .context("Getting var symlink path")?;
+    match symlink(&var_target, state_path.join("var")) {
+        Ok(()) => {}
+        Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => {}


The case where var symlnk would already exist would be if the new pulled repository and hence deployment has the same fs-verity as one of the deployments already present. I think this case should be handled much earlier than populating the state directory. imo, this should be an error if var symlink exists

After live discussion, we decided for now to make bootc switch just error out when the image verity is identical for now (and we'd probably want to implement bootc switch --in-place or so that would be explicitly documented to not create a new bootloader entry, and would instead require that the image digest is identical)

The other option is to implement what ostree does with a "deployment counter" that creates distinct /etc as well, but it's not clear that's worth it.

github-actions Bot added the area/install Issues related to `bootc install` label Mar 4, 2026

bootc-bot Bot requested a review from gursewak1997 March 4, 2026 21:18

gemini-code-assist Bot reviewed Mar 4, 2026

View reviewed changes

Comment thread crates/lib/src/bootc_composefs/repo.rs Outdated

cgwalters force-pushed the prep-composefs-manifest branch 3 times, most recently from ef5cb0f to d1f366d Compare March 13, 2026 12:22

cgwalters marked this pull request as ready for review March 13, 2026 14:33

cgwalters enabled auto-merge (rebase) March 13, 2026 15:08

cgwalters requested a review from Johan-Liebert1 March 13, 2026 15:10

cgwalters disabled auto-merge March 13, 2026 15:19

cgwalters force-pushed the prep-composefs-manifest branch 2 times, most recently from 215b824 to e3b7826 Compare March 15, 2026 17:46

Johan-Liebert1 reviewed Mar 16, 2026

View reviewed changes

Comment thread crates/lib/src/bootc_composefs/state.rs

Johan-Liebert1 reviewed Mar 16, 2026

View reviewed changes

cgwalters marked this pull request as draft March 16, 2026 21:05

cgwalters force-pushed the prep-composefs-manifest branch 3 times, most recently from 1af0080 to e19a663 Compare April 3, 2026 21:20

cgwalters mentioned this pull request Apr 6, 2026

Update composefs-rs #2113

Closed

cgwalters force-pushed the prep-composefs-manifest branch from e19a663 to d5ba655 Compare April 6, 2026 23:42

github-actions Bot added area/system-reinstall-bootc Issues related to system-reinstall-botoc area/ostree Issues related to ostree area/documentation Updates to the documentation labels Apr 6, 2026

cgwalters force-pushed the prep-composefs-manifest branch from d5ba655 to c936426 Compare April 7, 2026 00:00

Johan-Liebert1 mentioned this pull request Apr 7, 2026

composefs: Backwards compatibility #2128

Closed

cgwalters force-pushed the prep-composefs-manifest branch 4 times, most recently from 8693354 to 2aaa2ab Compare April 22, 2026 19:17

cgwalters force-pushed the prep-composefs-manifest branch from 2aaa2ab to c9794c7 Compare April 24, 2026 16:14

cgwalters force-pushed the prep-composefs-manifest branch 2 times, most recently from 51218fd to a9dca00 Compare April 29, 2026 00:27

cgwalters added the ci/merge Run full CI suite (all OSes) — equivalent to merge queue label Apr 29, 2026

cgwalters and others added 14 commits May 1, 2026 16:53

composefs: Add harness and unit tests for shared boot entry GC

ba5d1d3

Add some basic infra to mock up enough of an installed root to use in unit tests - specifically targeted for the bootloader logic. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>

cgwalters force-pushed the prep-composefs-manifest branch from a9dca00 to f795e6f Compare May 1, 2026 20:57

cgwalters marked this pull request as ready for review May 1, 2026 20:57

bootc-bot Bot requested a review from ckyrouac May 1, 2026 20:57

Johan-Liebert1 reviewed May 4, 2026

View reviewed changes

This was referenced May 4, 2026

composefs backend lacks garbage collection for /boot and sysroot #1808

Open

Unified storage #20

Open

Conversation

cgwalters commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

cgwalters commented Mar 13, 2026

Uh oh!

cgwalters commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cgwalters commented Apr 24, 2026

Uh oh!

cgwalters commented May 1, 2026

Uh oh!

Johan-Liebert1 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cgwalters commented Mar 4, 2026 •

edited

Loading

cgwalters commented Mar 13, 2026 •

edited

Loading