lib: Add internal qcow2 compression for Nutanix image by thunderboltsid · Pull Request #2848 · coreos/coreos-assembler

thunderboltsid · 2022-05-06T20:03:56Z

Add a -c convert option for Nutanix image which will help reduce
size of the image for Nutanix.

This PR also sets skip_compression value to True for Nutanix variant
in order to ensure cosa compress doesn't gzip the image.

Fixes coreos/fedora-coreos-tracker#1191

openshift-ci · 2022-05-06T20:04:05Z

Hi @thunderboltsid. Thanks for your PR.

I'm waiting for a coreos member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

thunderboltsid · 2022-05-06T20:04:27Z

/hold this is a provisional PR

bgilbert · 2022-05-07T01:12:41Z

You'll also need to add a way for a VARIANTS dict to set skip-compression in the image metadata, so cmd-compress doesn't try to recompress the image.

/ok-to-test

thunderboltsid · 2022-05-10T13:08:30Z

You'll also need to add a way for a VARIANTS dict to set skip-compression in the image metadata, so cmd-compress doesn't try to recompress the image.

@bgilbert can you take another look? Is this what you were expecting?

sohankunkerkar · 2022-05-10T13:32:38Z

/retest

bgilbert · 2022-05-10T17:47:49Z

/retest

thunderboltsid · 2022-05-10T18:04:30Z

@bgilbert the rhcos test seems to be failing due to openshift/os#795

bgilbert · 2022-05-10T18:14:55Z

This appears to be a different error.

thunderboltsid · 2022-05-10T18:29:58Z

/retest-required

thunderboltsid · 2022-05-10T19:13:00Z

/retest-required

thunderboltsid · 2022-05-10T19:50:52Z

/retest-required

dustymabe · 2022-05-10T20:55:40Z

Note: before something like this lands we still need to make an announcement about it to the community.

bgilbert

This LGTM, though I haven't tested it.

thunderboltsid · 2022-05-11T18:24:15Z

/retest-required

miabbott · 2022-05-11T20:27:47Z

Looks like another issue with the repo mirror

Updating rpm-md repo 'rhel-86-appstream': cannot update repo 'rhel-86-appstream': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried; Last error: Status code: 403 for http://base-4-11-rhel86.ocp.svc.cluster.local/rhel-8-appstream/repodata/repomd.xml (IP: 172.30.132.86)

I put up another PR to switch from the 8.6 Beta content, but I'm not convinced it will solve the error.

openshift/release#28509

I would see if the job on openshift/os#798 passes and then trigger the jobs here if so.

thunderboltsid · 2022-05-11T22:51:43Z

/retest-required

thunderboltsid · 2022-05-12T05:26:30Z

/hold cancel

miabbott

This doesn't work as proposed. I tried building RHCOS 4.11 and generating a Nutanix artifact.

$ cosa buildextend-nutanix                        
[INFO]: CLI is a symlink for cmd-buildextend-nutanix
[INFO]: Target 'NUTANIX' is a Qemu Variant image
[INFO]: Targeting architecture: x86_64      
[INFO]: Targeting build: 411.85.202205121303-0
[INFO]: Processed build for: OpenShift 4 (RHCOS-x86_64) 411.85.202205121303-0
[INFO]: Processing the build artifacts    
[INFO]: Staging temp image: /srv/tmp/tmp2x7ohrl6/rhcos-411.85.202205121303-0-nutanix.x86_64.qcow2
...
+ qemu-img convert -f qcow2 -O qcow2 /srv/tmp/tmp2x7ohrl6/rhcos-411.85.202205121303-0-qemu.x86_64.qcow2.working -c "" /srv/tmp/tmp2x7ohrl6/rhcos-411.85.202205121303-0-nutanix.x86_64.qcow2
qemu-img: Could not open '': The 'file' block driver requires a file name
Error running command qemu-img

It looks like -c option requires a filename

thunderboltsid · 2022-05-12T13:24:41Z

It looks like -c option requires a filename

I don’t think -c requires a filename for the flag. But, I think because the -c flag is the “key” and "" is the value, the underlying command ends up copying the empty string as "" in the generated command.

miabbott · 2022-05-12T13:27:34Z

I don’t think -c requires a filename for the flag. But, I think because the -c flag is the “key” and "" is the value, the underlying command ends up not copying the empty string as "" in the generated command.

You are correct; I should avoid doing debug before the first coffee 😛

Add a `-c` convert option for Nutanix image which will help reduce size of the image for Nutanix.

Add a `skip_compression` property on `QemuVariantImage` based on the Set `skip_compression` value of the variant in `VARIANTS` dict. When `skip_compression` is set to True (default is False), the `mutate_image` method adds `skip-compression:True` to the `meta_patch`. This commit also sets `skip_compression` value to True for Nutanix variant.

miabbott

cosa buildextend-nutanix and cosa compress behaved as expected

openshift-ci Bot added the needs-ok-to-test label May 6, 2022

openshift-ci Bot added the do-not-merge/hold label May 6, 2022

openshift-ci Bot added ok-to-test and removed needs-ok-to-test labels May 7, 2022

bgilbert previously approved these changes May 10, 2022

View reviewed changes

openshift-ci Bot removed the do-not-merge/hold label May 12, 2022

miabbott suggested changes May 12, 2022

View reviewed changes

thunderboltsid dismissed bgilbert’s stale review via 8e72e12 May 12, 2022 13:30

thunderboltsid force-pushed the ntnx-qcow2-compression branch from dc9a101 to 8e72e12 Compare May 12, 2022 13:30

sohankunkerkar reviewed May 12, 2022

View reviewed changes

Comment thread src/cosalib/qemuvariants.py

thunderboltsid added 2 commits May 12, 2022 16:23

Add internal qcow2 compression for Nutanix image

78b0a80

Add a `-c` convert option for Nutanix image which will help reduce size of the image for Nutanix.

thunderboltsid force-pushed the ntnx-qcow2-compression branch from 8e72e12 to cd53df6 Compare May 12, 2022 14:23

sohankunkerkar approved these changes May 12, 2022

View reviewed changes

sohankunkerkar requested a review from miabbott May 12, 2022 14:55

miabbott approved these changes May 12, 2022

View reviewed changes

miabbott merged commit 3f8113b into coreos:main May 12, 2022

sohankunkerkar mentioned this pull request May 17, 2022

unpacked RAW image size for Nutanix reduced to 10GiB coreos/fedora-coreos-tracker#1198

Closed

dustymabe mentioned this pull request May 24, 2022

Roll out stable 36.20220505.3.2, testing 36.20220522.2.1, next 36.20220522.1.0 coreos/fedora-coreos-streams#508

Merged

bgilbert mentioned this pull request Jun 7, 2022

Roll out stable 36.20220522.3.0, testing 36.20220605.2.0, next 36.20220605.1.0 coreos/fedora-coreos-streams#512

Merged

Conversation

thunderboltsid commented May 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci Bot commented May 6, 2022

Uh oh!

thunderboltsid commented May 6, 2022

Uh oh!

bgilbert commented May 7, 2022

Uh oh!

thunderboltsid commented May 10, 2022

Uh oh!

sohankunkerkar commented May 10, 2022

Uh oh!

bgilbert commented May 10, 2022

Uh oh!

thunderboltsid commented May 10, 2022

Uh oh!

bgilbert commented May 10, 2022

Uh oh!

thunderboltsid commented May 10, 2022

Uh oh!

thunderboltsid commented May 10, 2022

Uh oh!

thunderboltsid commented May 10, 2022

Uh oh!

dustymabe commented May 10, 2022

Uh oh!

bgilbert left a comment

Choose a reason for hiding this comment

Uh oh!

thunderboltsid commented May 11, 2022

Uh oh!

miabbott commented May 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thunderboltsid commented May 11, 2022

Uh oh!

thunderboltsid commented May 12, 2022

Uh oh!

miabbott left a comment

Choose a reason for hiding this comment

Uh oh!

thunderboltsid commented May 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

miabbott commented May 12, 2022

Uh oh!

Uh oh!

miabbott left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

thunderboltsid commented May 6, 2022 •

edited

Loading

miabbott commented May 11, 2022 •

edited

Loading

thunderboltsid commented May 12, 2022 •

edited

Loading