Running `install to-disk` for a loop mounted disk fails on Fedora 39

[mvo5]

When trying to explore the images that bootc generates to explore how/if bootc-image-builder could use it directly (or for a start ensure the generated images are similar enough) I ran into the following issue on a standard Debian/unstable system:

$ podman --version
podman version 4.8.3

$ truncate -s 10G test-disk.img
$ sudo losetup -f test-disk.img

$ sudo podman run --env RUST_LOG=error,bootc_lib::install=debug  --rm --privileged --pid=host --security-opt label=type:unconfined_t quay.io/centos-bootc/fedora-bootc:eln bootc install to-disk /dev/loop2
DEBUG Target image reference: ostree-unverified-registry:quay.io/centos-bootc/fedora-bootc:eln
DEBUG Fetched manifest with digest sha256:145383779c284eab7e8067219dd0249532b4d9d77861bd74c029a53e07b59ce9
DEBUG Retargeting /tmp to host
DEBUG Retargeting /var/tmp to host
Mounting selinuxfs
DEBUG Target has SELinux, host=true
DEBUG Target image reference: ostree-unverified-registry:quay.io/centos-bootc/fedora-bootc:eln
DEBUG Fetched manifest with digest sha256:145383779c284eab7e8067219dd0249532b4d9d77861bd74c029a53e07b59ce9
DEBUG Target has SELinux, host=true
DEBUG Loaded install configuration
Mounting devtmpfs
Initializing partitions
DEBUG Created partition table
ERROR Creating rootfs: Rereading partition table: couldn't reread partition table: device may not support partitions: EINVAL: Invalid argument

Maybe related to using a loop device for testing? I looked at tests/kolainst/install and there /dev/vda is used instead.

[mvo5]

Fwiw, I treid --via-lookback but that did not work, presumably because #260 is too new? Maybe I need the -dev container instead? Sorry!

[mvo5]

Actually yes, with

$ sudo podman run --env RUST_LOG=error,bootc_lib::install=debug  --rm --privileged --pid=host --security-opt label=type:unconfined_t quay.io/centos-bootc/fedora-bootc-dev:eln bootc install to-disk --via-loopback /dev/loop2

things are downloading now, but:

Writing manifest to image destination
ERROR Installing to disk: Not a regular file (to be used via loopback): /dev/loop2

[cgwalters]

You need to use losetup -P for external loopback devices.

[cgwalters]

If you provided the loopback device, then don't specify --via-loopback. If you want bootc to handle the loopback (which would fix your omission of -P to losetup), then just give it a raw file and do pass --via-loopback.

[mvo5]

Thank you! Sorry for my confusion, I promise to writeup something in the docs and propose a PR so that I feel less bad about wasting your time. I am at this point now:

# truncate -s 10G test-disk2.img
# losetup -Pf test-disk2.img
# losetup 
/dev/loop8                0       0         0  0 /home/mvogt/tmp/test-disk2.img
podman run --env RUST_LOG=error,bootc_lib::install=debug  --rm --privileged --pid=host --security-opt label=type:unconfined_t quay.io/centos-bootc/fedora-bootc:eln bootc install to-disk /dev/loop8
DEBUG Target image reference: ostree-unverified-registry:quay.io/centos-bootc/fedora-bootc:eln
DEBUG Fetched manifest with digest sha256:afbe9a457e188ac6ee1366c1a315ca68707a223fb8244cc2c81d5e54b59972f0
DEBUG Retargeting /tmp to host
DEBUG Retargeting /var/tmp to host
DEBUG Target has SELinux, host=true
Mounting selinuxfs
DEBUG Target image reference: ostree-unverified-registry:quay.io/centos-bootc/fedora-bootc:eln
DEBUG Fetched manifest with digest sha256:afbe9a457e188ac6ee1366c1a315ca68707a223fb8244cc2c81d5e54b59972f0
DEBUG Target has SELinux, host=true
DEBUG Loaded install configuration
Mounting devtmpfs
Initializing partitions
DEBUG Created partition table
Creating filesystem
mke2fs 1.47.0 (5-Feb-2023)
Creating filesystem
Mounting /run/bootc/mounts/rootfs
Mounting /run/bootc/mounts/rootfs/boot
Creating ESP filesystem
Mounting /run/bootc/mounts/rootfs/boot/efi
Initializing ostree layout
DEBUG boot uuid=b7ef2b00-5b99-46f6-a6b5-ccd860fe5e43
Initializing sysroot
ostree/deploy/default initialized as OSTree stateroot
Creating initial deployment
Installed: ostree-unverified-registry:quay.io/centos-bootc/fedora-bootc:eln
   Digest: sha256:b87f89290c31b04391681e4dc9844f306dbd6ef38172709758a18f0d84da1fe7
Running bootupctl to install bootloader
error: boot data installation failed: installing component EFI: Updating EFI firmware variables: Clearing current EFI boot entry: Failed to invoke efibootmgr
ERROR Installing bootloader: Task Running bootupctl to install bootloader failed: ExitStatus(unix_wait_status(256))

[cgwalters]

Ah yeah, I have seen that before but not debugged it yet; but actually you probably want to specify --generic-image anyways which will suppress this.

Hmm very arguably we should default to --generic-image when writing to a loopback device.

[mvo5]

Fwiw, trying the new --via-loopback option gives me:

$ truncate -s 10G test-disk3.img
$ sudo podman run -v $(pwd):/host  --env RUST_LOG=error,bootc_lib::install=debug  --rm --privileged --pid=host --security-opt label=type:unconfined_t quay.io/centos-bootc/fedora-bootc-dev:eln bootc install to-disk --via-loopback /host/test-disk3.img
losetup: /host/test-disk3.img: failed to set up loop device: No such file or directory
ERROR Installing to disk: Task losetup failed: ExitStatus(unix_wait_status(256))
$ sudo podman run -v $(pwd):/host  --env RUST_LOG=error,bootc_lib::install=debug  --rm --privileged --pid=host --security-opt label=type:unconfined_t quay.io/centos-bootc/fedora-bootc-dev:eln ls /host/test-disk3.img
/host/test-disk3.img

[cgwalters]

Updated #281

[cgwalters]

losetup: /host/test-disk3.img: failed to set up loop device: No such file or directory

I hit this too and the fix is in
#278

[cgwalters]

And yeah before #278 we were leaking a loop device per invocation, so you need to manually losetup -d those leaked devices.

[mvo5]

Thank you so much for your help! I opened #282 to get a discussion started if we can make the examples more prominent in the README. WIth that I have a booting image now \o/

But:
a) I do not see serial output when passing -serial stdio ? when adding console=ttyS0 to grub things look happier
b) I see quite a few failurs on startup of the machine, e.g.

[FAILED] Failed to start dbus-broker.service - D-Bus System Message Bus.
[FAILED] Failed to start sshd-keygen@ecdsa.…penSSH ecdsa Server Key Generation.
...

which is maybe quite normal for a fedora-bootc-dev:eln but I wanted to mention it still :)

[cgwalters]

a) I do not see serial output when passing -serial stdio ? when adding console=ttyS0 to grub things look happier

That's currently expected, see also https://github.com/osbuild/bootc-image-builder/blob/113792d1263fece0e4547676018d4263115aa8ac/bib/cmd/bootc-image-builder/image.go#L98
The console stuff is...hard, but again I think the defaults need to come from the container image and not hardcoded in bib. Another case of osbuild/bootc-image-builder#18

This also touches on coreos/fedora-coreos-config#2785 (comment)

b) I see quite a few failurs on startup of the machine, e.g.

I don't see those here...

I believe at this point we've addressed the actionable issues here, so closing (but without prejudice, feel free to reopen or file new issues).