Skip to content

MCO-593: Consume MCO API and CRDs from openshift/api#3747

Merged
openshift-ci[bot] merged 8 commits into
openshift:masterfrom
jkyros:migrate-mco-api
Oct 19, 2023
Merged

MCO-593: Consume MCO API and CRDs from openshift/api#3747
openshift-ci[bot] merged 8 commits into
openshift:masterfrom
jkyros:migrate-mco-api

Conversation

@jkyros
Copy link
Copy Markdown
Member

@jkyros jkyros commented Jun 14, 2023

This just brings:

back into the MCO.

This is almost entirely find/replace except:

  • API helpers moved into apihelpers package (since they couldn't go with the API)
  • existing API removed from the MCO
  • hack scripts updated to no longer generate deepcopy code
  • hack scripts updated to copy MCO CRDs out of vendor directory when "make update" is run

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 14, 2023
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

openshift-ci-robot commented Jun 14, 2023

@jkyros: This pull request references MCO-593 which is a valid jira issue.

Details

In response to this:

Ignore this for now, I just want to run some tests and make sure what's going up in openshift/api is valid

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 14, 2023
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Jun 14, 2023

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 14, 2023
@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Jun 14, 2023

/test e2e-gcp-op

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Jun 15, 2023

/test e2e-aws-ovn-upgrade

@jkyros jkyros force-pushed the migrate-mco-api branch 2 times, most recently from a97bce5 to 648de33 Compare June 15, 2023 18:54
@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Jun 15, 2023

/test all

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Jun 15, 2023

/test verify

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Jun 15, 2023

/test all

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Jun 15, 2023

/test unit
/test verify

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Jun 15, 2023

/test e2e-gcp-op

1 similar comment
@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Jun 16, 2023

/test e2e-gcp-op

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Jun 16, 2023

/retest-required

@cdoern
Copy link
Copy Markdown
Contributor

cdoern commented Jun 21, 2023

@jkyros I think you need to modify the codegen to also fetch the operator.openshift.io group so we can generate listers for the generic CRD. I have been working on this as well in my branch :) The zz_generated... files in pkg/api need to be modified by hand as well I think

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 23, 2023
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 14, 2023
@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Aug 14, 2023

/test all

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Aug 15, 2023

/test e2e-aws-ovn-upgrade

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Aug 15, 2023

Oh, hah, we're failing the e2e-aws-ovn-upgrade test because master still has those certificate API fields as strings. We changed them to metav1.Time, but I guess this test upgrades from "whatever is in master" (rather than 4.13) to "whatever is in this PR", so we'd need to fix that in master, and then re-run e2e-aws-ovn-upgrade :

2023-08-15T05:03:40.989575279Z W0815 05:03:40.989527       1 reflector.go:533] github.com/openshift/client-go/machineconfiguration/informers/externalversions/factory.go:101: failed to list *v1.MachineConfigPool: parsing time "2033-08-12 01:47:54 +0000 UTC" as "2006-01-02T15:04:05Z07:00": cannot parse " 01:47:54 +0000 UTC" as "T"
2023-08-15T05:03:40.989575279Z E0815 05:03:40.989555       1 reflector.go:148] github.com/openshift/client-go/machineconfiguration/informers/externalversions/factory.go:101: Failed to watch *v1.MachineConfigPool: failed to list *v1.MachineConfigPool: parsing time "2033-08-12 01:47:54 +0000 UTC" as "2006-01-02T15:04:05Z07:00": cannot parse " 01:47:54 +0000 UTC" as "T"
2023-08-15T05:04:05.304139210Z W0815 05:04:05.304088       1 reflector.go:533] github.com/openshift/client-go/machineconfiguration/informers/externalversions/factory.go:101: failed to list *v1.ControllerConfig: parsing time "2033-08-12 01:47:54 +0000 UTC" as "2006-01-02T15:04:05Z07:00": cannot parse " 01:47:54 +0000 UTC" as "T"
2023-08-15T05:04:05.304139210Z E0815 05:04:05.304121       1 reflector.go:148] github.com/openshift/client-go/machineconfiguration/informers/externalversions/factory.go:101: Failed to watch *v1.ControllerConfig: failed to list *v1.ControllerConfig: parsing time "2033-08-12 01:47:54 +0000 UTC" as "2006-01-02T15:04:05Z07:00": cannot parse " 01:47:54 +0000 UTC" as "T"

Everything else is green though, so I think we've got it aside from that.

@jkyros jkyros changed the title [WIP] MCO-593: Consume MCO API and CRDs from openshift/api MCO-593: Consume MCO API and CRDs from openshift/api Aug 15, 2023
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

openshift-ci-robot commented Aug 15, 2023

@jkyros: This pull request references MCO-593 which is a valid jira issue.

Details

In response to this:

This just brings:

back into the MCO.

This is almost entirely find/replace except:

  • API helpers moved into apihelpers package (since they couldn't go with the API)
  • existing API removed from the MCO
  • hack scripts updated to no longer generate deepcopy code
  • hack scripts updated to copy MCO CRDs out of vendor directory when "make update" is run

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jkyros jkyros marked this pull request as ready for review August 15, 2023 05:46
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 15, 2023
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Oct 13, 2023

@jkyros: trigger 1 job(s) for the /payload-(job|aggregate) command

  • periodic-ci-openshift-hypershift-release-4.15-periodics-e2e-aws-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/c8d64630-699a-11ee-9f2b-dc030e30b910-0

@sinnykumari
Copy link
Copy Markdown
Contributor

Overall LGTM, we will need to look at failing HyperShift conformance job

jkyros and others added 6 commits October 13, 2023 18:29
We've been keeping the kubelet.yaml template in sync with the
openshift/api featuregates manually for quite awhile. A bunch of new
featuregates graduated, so we're going to do one more round, but we
do need to come back and do this the right way by reading the default
featuregates off the featureGateAccessor and injecing them into the
kubelet.yaml templates. The plumbing is already there, we just need to
use it.

I also added some additional verbosity to the skew test because I got
tired of having to hunt through the list for the missing gate :)
This updates our imports to read the API from openshift/api,
removes the generated and API packages (because client-go supersedes
them) and moves the api helper functions to their own package
"apihelpers" since they did not go with the API.

This also removes the "legacy" MCO api from the repo, so anyone using it
will either have to pin to an older version or migreate to using it from
openshift/api
There were a few API helper functions that were part of the API when it
lived here, but since they were only used here in the MCO, we didn't
migrate them to openshift/api, we just broke them out into a separate
apihelpers package.

Initially we thought we might be able to stuff them into
controller/common but with the way the test suite is setup, that results
in a dependency cycle, so a separate package was the way to go.
We merged some additional code since I went through and changed all the
imports and references, this just makes sure the new code we've added is
properly referencing the MCO API and its helpers.
We took these certificate observability fields out because they were the
wrong type and put in a storageversionmigration for them in 4.14, but
now that we're in 4.15, and we have a newly generated client-go for the
MCO that uses applyconfigurations/server-side apply, it matters that
we're not supplying the "whole object" every time -- "partially" applying
them like we were as a stopgap will now generate errors and fail.

This reintroduces those time fields to the MCO code so
certificate observability work again, and the "required field not
supplied" errors will cease.
@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Oct 14, 2023

Testing in #3976 it looks like it's actually the "CRDs from vendor" commit that breaks it. I might have changed the operator timing by removing the CRD sync from the operator somehow, or Hypershift cares that controllerconfig is part of the payload now for some reason? ( both possibilities are weird, but this seems like a weird thing )

If not having this PR in is blocking stuff, I can peel that CRD commit of and then we can troubleshoot that separately?

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Oct 18, 2023

As soon as this run fails, I'll peel the commit off (I just want to rule out a problem being fixed in the background):
/payload-job periodic-ci-openshift-hypershift-release-4.15-periodics-e2e-aws-ovn-conformance

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Oct 18, 2023

@jkyros: trigger 1 job(s) for the /payload-(job|aggregate) command

  • periodic-ci-openshift-hypershift-release-4.15-periodics-e2e-aws-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/10060be0-6d5a-11ee-8230-a87660254d7b-0

@cdoern
Copy link
Copy Markdown
Contributor

cdoern commented Oct 18, 2023

@jkyros looks like it failed?

This temporarily introduces a crds-sync.sh script that syncs the CRDs
out of vendor into their usual locations. It is intended to run during
"make update" and is a stopgap until we can figure out why sourcing the
the manifests out of vendor and having the CVO apply controllerconfig
instead of the MCO breaks hypershift external cloud provider
conformance.
@yuqi-zhang
Copy link
Copy Markdown
Contributor

/payload-job periodic-ci-openshift-hypershift-release-4.15-periodics-e2e-aws-ovn-conformance

Just another data point to see if we missed anything

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Oct 18, 2023

@yuqi-zhang: trigger 1 job(s) for the /payload-(job|aggregate) command

  • periodic-ci-openshift-hypershift-release-4.15-periodics-e2e-aws-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/dbc2fd90-6ded-11ee-81fd-4ee50d4e6af2-0

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Oct 18, 2023

cluster failure, and a pod sandbox failure
/test e2e-aws-ovn-upgrade
/test okd-scos-e2e-aws-ovn

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Oct 19, 2023

/test e2e-openstack

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Oct 19, 2023

@jkyros: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/okd-scos-e2e-aws-ovn 16f26e7 link false /test okd-scos-e2e-aws-ovn

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Oct 19, 2023

Alright, ci passed, hypershift conformance passed. I'll let this run overnight:
/payload 4.15 nightly blocking

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Oct 19, 2023

@jkyros: trigger 8 job(s) of type blocking for the nightly release of OCP 4.15

  • periodic-ci-openshift-release-master-nightly-4.15-e2e-aws-sdn-upgrade
  • periodic-ci-openshift-release-master-ci-4.15-e2e-azure-ovn-upgrade
  • periodic-ci-openshift-release-master-ci-4.15-upgrade-from-stable-4.14-e2e-gcp-ovn-rt-upgrade
  • periodic-ci-openshift-hypershift-release-4.15-periodics-e2e-aws-ovn-conformance
  • periodic-ci-openshift-release-master-nightly-4.15-e2e-aws-ovn-serial
  • periodic-ci-openshift-release-master-ci-4.15-e2e-aws-ovn-upgrade
  • periodic-ci-openshift-release-master-nightly-4.15-e2e-metal-ipi-ovn-ipv6
  • periodic-ci-openshift-release-master-nightly-4.15-e2e-metal-ipi-sdn-bm

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/51d7a4b0-6e50-11ee-99e0-c5c595fbd5a4-0

@jkyros
Copy link
Copy Markdown
Member Author

jkyros commented Oct 19, 2023

Oh wow that payload test was super green. 😄

The Azure one was the only one that failed, and it passed twice, failed twice. Failures were:

{  etcdHighCommitDurations was at or above info for at least 1m58s on platformidentification.JobType{Release:"4.15", FromRelease:"4.15", Platform:"azure", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 0s, firing for 1m58s:

which I'm reasonably confident this didn't affect.

@cdoern
Copy link
Copy Markdown
Contributor

cdoern commented Oct 19, 2023

/lgtm
/hold cancel

@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 19, 2023
@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Oct 19, 2023
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Oct 19, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cdoern, jkyros, yuqi-zhang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [cdoern,jkyros,yuqi-zhang]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants