Skip to content

perf(convert): reduce rootfs copies from 3 to 1#110

Open
imlk0 wants to merge 7 commits intomasterfrom
fde-convert-slow
Open

perf(convert): reduce rootfs copies from 3 to 1#110
imlk0 wants to merge 7 commits intomasterfrom
fde-convert-slow

Conversation

@imlk0
Copy link
Copy Markdown
Collaborator

@imlk0 imlk0 commented Apr 29, 2026

Summary

  • Replace the intermediate rootfs.img file and final qemu-img convert step with a qcow2 overlay/snapshot architecture, reducing full rootfs copies from 3 → 1
  • Source image is modified through a writable overlay (source-mod), then forked into two independent snapshots (source-read for verity hash, source-write for dracut initrd generation)
  • Eliminates two full rootfs copies (~17s dd + ~32s qemu-img convert ≈ ~49s savings)
  • Remove --device / operate_on_device support: deprecate in-place device conversion in favor of file-based (--in/--out) only
  • Fix RPM version detection: rpm -q outputs error to stdout on uninstalled packages, which was captured as the version string
  • Fix unbound boot_part variable when source has no separate boot partition
  • Remove --wipe-freed-space option (no longer needed with qcow2 overlay architecture)
  • Update docs: quick-start.md, quick-start_zh.md — replace --device examples with --in/--out

Test plan

  • Run conversion on aliyun_3_x64 image — all 11 steps complete successfully
  • Verify output qcow2 has correct partition layout (BIOS boot, EFI, boot ext4, rootfs LUKS2)
  • Verify verity hash is computed correctly
  • Verify dracut generates initrd successfully
  • Timing: ~5m24s total

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

imlk0 added 2 commits April 29, 2026 20:12
…napshots

Replace the intermediate rootfs.img file and final qemu-img convert step
with qcow2 overlay/snapshot architecture. The source image is modified
through a writable overlay (source-mod), then forked into two independent
snapshots (source-read for verity hash, source-write for dracut). This
eliminates two full rootfs copies (~17s dd + ~32s qemu-img convert).

Additional fixes:
- Fix RPM version detection: rpm -q outputs error to stdout on uninstalled
  packages, which was captured as the version string
- Fix unbound boot_part variable when source has no separate boot partition
Deprecate in-place device conversion in favor of file-based (--in/--out)
conversion only. All device-mode code paths and conditional branches are
removed, simplifying the codebase.

Changes:
- Remove operate_on_device variable and all conditional branches
- Remove device-mode validation, confirmation prompt, and cleanup logic
- Keep --device/-d flag but emit a deprecation warning
- Remove --wipe-freed-space option (emit deprecation warning if used)
- Update help text to reflect deprecation
- Update docs: quick-start.md, quick-start_zh.md — replace --device
  examples with --in/--out and add deprecation notes (since v0.7.0)
@imlk0 imlk0 force-pushed the fde-convert-slow branch from aa6ba77 to ec85e19 Compare April 29, 2026 12:12
@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@imlk0 imlk0 force-pushed the fde-convert-slow branch from ec85e19 to 07fc888 Compare April 29, 2026 12:17
@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

Remove "Example 5: Encrypt a Real System Disk" section which duplicated
Example 1 after the --device removal. Renumber remaining examples
(4: KBS, 5: KMS). Update prerequisites and troubleshooting to remove
real-disk mentions. Add deprecation note at the top of both
quick-start docs.
@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

Replace ~70 lines of duplicated mount/unmount code with a single call to
setup_chroot_mounts, adding optional boot_override_part and efi_override_part
parameters to handle the update_initrd's output-device mount needs.
@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

@imlk0 imlk0 force-pushed the fde-convert-slow branch from 9ef0a95 to 0c97e2d Compare April 29, 2026 13:09
@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

- Add function-level shellcheck disable=SC2154 on step:update_initrd and
  step::prepare_output_and_snapshots (variables set in main() and used in
  these step functions)
- Remove unused output_boot_part_num variable
- Remove duplicate log::step from step::prepare_output_and_snapshots and
  step::copy_partitions (main() already logs the step)
@imlk0 imlk0 force-pushed the fde-convert-slow branch from 0c97e2d to 8a094cc Compare April 29, 2026 13:13
@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

Use guestfish to write EFI content directly to qcow2 file, bypassing the
NBD kernel page cache invalidation issue that caused FAT32 filesystem data
to be lost on qemu-nbd disconnect.

Also fix EFI partition detection in external.rs where the lsblk FSTYPE
column addition broke the partition filter, and add partprobe to ensure
partition device nodes are created after NBD connect.
@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

- Remove early mkfs.vfat + mount + cp for EFI partition in UKI mode;
  EFI is now dd'd in copy_partitions and restored via guestfish after
  NBD disconnect to avoid writeback-cache data loss.
- Save partition UUIDs with blkid before parted rm/mkpart operations,
  restore them with sgdisk -u afterward to preserve GPT GUIDs.
@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@shankailun-aliyun
Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants