ci: use the RHEL 8.5 repos on the mirror#718

Merged

openshift-merge-robot merged 2 commits intoopenshift:masterfrom

miabbott:ci_rhel85

Feb 24, 2022

Member

miabbott commented Feb 15, 2022

We want to test RHCOS using RHEL 8.5 until we can switch to using RHEL
8.6, so use the newly created 8.5 repo.

See: openshift/release#26193


          ci: use the RHEL 8.5 repos on the mirror

dc42838

We want to test RHCOS using RHEL 8.5 until we can switch to using RHEL
8.6, so use the newly created 8.5 repo.

See: openshift/release#26193

miabbott mentioned this pull request

Use the generic redhat-release package #713

Merged

openshift-ci Bot requested review from jmarrero and peberanek

February 15, 2022 18:30

openshift-ci Bot added the approved label

cgwalters approved these changes

View reviewed changes

ci/prow-build-test-qemu.sh

-              curl -L http://base-"${ocpver_mut}"-rhel8.ocp.svc.cluster.local > src/config/ocp.repo
+              # we want to use RHEL 8.5 for testing until we can start using 8.6
+              # see https://github.com/openshift/release/pull/26193
+              curl -L http://base-"${ocpver_mut}"-rhel85.ocp.svc.cluster.local > src/config/ocp.repo

Member

cgwalters Feb 15, 2022

This is fine as is, but it's another place to bump the RHEL version. Let's figure out how to dedup all this after recent PRs merge though, don't block here!

Member Author

miabbott Feb 15, 2022

Ack. Will do!

Member Author

miabbott commented Feb 15, 2022

Updating rpm-md repo 'rhel-8-advanced-virt': cannot update repo 'rhel-8-advanced-virt': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried; Last error: Status code: 404 for http://base-4-11-rhel85.ocp.svc.cluster.local/rhel-8-advanced-virt/repodata/repomd.xml

I think the repo definition needs a tweak, will chase that down

Member Author

miabbott commented Feb 16, 2022

I think the repo definition needs a tweak, will chase that down

openshift/release#26305

miabbott mentioned this pull request

Bug 2099811: manifest: bump to openvswitch2.16 #715

Merged

Member Author

miabbott commented Feb 16, 2022

/retest

Member Author

miabbott commented Feb 16, 2022

Messed up the baseurl for RHEL AV 😞 - openshift/release#26334

Contributor

HuijingHei commented Feb 22, 2022

/retest

Member Author

miabbott commented Feb 22, 2022

CI failed on the iso-install branch:

+ kola testiso -S --scenarios pxe-install,pxe-offline-install,iso-install,iso-offline-install,iso-live-login,iso-as-disk --output-dir tmp/kola-metal
Testing scenarios: [pxe-install pxe-offline-install iso-install iso-offline-install iso-live-login iso-as-disk]
Ignoring verification of signature on metal image
Could not open option rom '/usr/share/qemu/pxe-rtl8139.rom': No such file or directory
Successfully tested scenario pxe-install for 411.84.202202221344-0 on bios (metal)
Could not open option rom '/usr/share/qemu/pxe-rtl8139.rom': No such file or directory
Successfully tested scenario pxe-offline-install for 411.84.202202221344-0 on bios (metal)
Error: scenario iso-install: entered emergency.target in initramfs
2022-02-22T15:47:28Z cli: scenario iso-install: entered emergency.target in initramfs

But the failed artifacts didn't get saved, so not sure what happened. Will try to reproduce locally.

Member Author

miabbott commented Feb 22, 2022

Didn't reproduce locally

/retest

Member

jmarrero commented Feb 22, 2022

/lgtm

jmarrero approved these changes

View reviewed changes

openshift-ci Bot assigned jmarrero

openshift-ci Bot added the lgtm label

miabbott mentioned this pull request

Clean up POWER wrapper in libguestfish coreos/coreos-assembler#2718

Merged

openshift-bot commented Feb 22, 2022

/retest-required

Please review the full test history for this PR and help us cut down flakes.

9 similar comments

openshift-bot commented Feb 22, 2022

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented Feb 22, 2022

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented Feb 23, 2022

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented Feb 23, 2022

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented Feb 23, 2022

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented Feb 23, 2022

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented Feb 23, 2022

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented Feb 23, 2022

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot commented Feb 23, 2022

/retest-required

Please review the full test history for this PR and help us cut down flakes.

Member Author

miabbott commented Feb 23, 2022

/hold

Need to get a cosa pod on build02 to investigate why the iso-install is failing

openshift-ci Bot added the do-not-merge/hold label

Member Author

miabbott commented Feb 23, 2022 •

edited

Loading

OK, got at least some more info from console.txt in the pod:

[   17.295334] ignition[872]: Ignition 2.13.0                                                                                                                  
[   17.309174] systemd[1]: Starting Ignition (kargs)...                       
[   17.322253] ignition[872]: Stage: kargs                        
[   17.334175] systemd[1]: Starting dracut initqueue hook...            
[   17.348195] ignition[872]: reading system config file "/usr/lib/ignition/base.d/00-core.ign"                                                                
[  OK  ] Mounted /run/media/iso.                                     
         Starting Persist osmet files (ISO)...                           
         Mounting /sysroot...                                
[   17.394190] systemd[1]: Started dracut initqueue hook.                                                                                                      
[   17.423469] loop1: detected capacity change from 0 to 1039517696
[   17.435403] systemd[1]: Starting dracut pre-mount hook...                
[   17.462086] loop_set_status: loop1 () has still dirty pages (nrpages=2)                                                                                     
[   17.468483] ignition[872]: no config dir at "/usr/lib/ignition/base.platform.d/metal"                                                                       
[FAILED] Failed to mount /sysroot.                                                                                                                             
See 'systemctl status sysroot.mount' for details.                                                                                                              
[DEPEND] Dependency failed for Initrd Root File System.                                                                                                        
[DEPEND] Dependency failed for Reload Configuration from the Real Root.
[DEPEND] Dependency failed for Mount OSTree /var.                                                                                                              
[   17.577382] systemd[1]: Reached target Remote File Systems (Pre).
[  OK  ] Stopped Ignition (fetch-offline).         
[  OK  ] Stopped target Initrd Root Device.                                                                                                                    
[  OK  ] Stopped target Timers.                                                                                                                                
[  OK  ] Stopped Ignition (disks).                           
[  OK  [   17.663205] ignition[872]: kargs: kargs passed                                                                                                       
] Stopped Ignition (kargs).                           
[  OK  ] Stopped CoreOS Ignition User Config Setup.                                                                                                            
[  OK  ] Stopped Forward Password Requests to Clevis Directory Watch.                                                                                                                                                                                                                                                         
         [   17.710472] mount[902]: mount: /sysroot: failed to setup loop device for /run/media/iso/images/pxeboot/rootfs.img.                                 
Unmounting /run/ephemeral...

Searching for the loop device error hits on #666, with this observation which feels relevant:

I suspect there was something in the 8.5 bits that was causing the error, but the pkgdiff for the CI jobs isn't working, so we don't have a good snapshot of what changed.

@jlebon @bgilbert Any ideas around this one?

Frustratingly, building/testing RHCOS with RHEL 8.5 content works just fine on my F35 host so I can't reproduce this locally.

Contributor

bgilbert commented Feb 24, 2022 •

edited

Loading

Yeah, it looks consistent with #666. I'll reopen that bug and add some observations there.

bgilbert mentioned this pull request

testiso failing in 4.11 using RHEL 8.5; race mounting rootfs #666

Closed


          ci: disable testiso scenarios

4e8cc9c

All of the ISO-based `testiso` scenarios are failing due to openshift#666
so we are going to disable them until we get `util-linux` patched.

See also: https://bugzilla.redhat.com/show_bug.cgi?id=2058176

openshift-ci Bot removed the lgtm label

ravanelli approved these changes

View reviewed changes

Member

ravanelli left a comment

LGTM

Contributor

openshift-ci Bot commented Feb 24, 2022

@miabbott: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

saqibali-2k approved these changes

View reviewed changes

Contributor

saqibali-2k left a comment

/lgtm

openshift-ci Bot assigned saqibali-2k

openshift-ci Bot added the lgtm label

Contributor

openshift-ci Bot commented Feb 24, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters, jmarrero, miabbott, ravanelli, saqibali-2k

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [cgwalters,jmarrero,miabbott,ravanelli,saqibali-2k]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Member Author

miabbott commented Feb 24, 2022

/hold cancel

openshift-ci Bot removed the do-not-merge/hold label

openshift-merge-robot merged commit f6db7a2 into openshift:master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

cgwalters cgwalters approved these changes

peberanek Awaiting requested review from peberanek

+3 more reviewers

jmarrero jmarrero approved these changes

ravanelli ravanelli approved these changes

saqibali-2k saqibali-2k approved these changes

Labels