Skip to content

MCO-607: MCO-237: Keep track of certs in ControllerConfigStatus#3756

Merged
openshift-merge-robot merged 1 commit into
openshift:masterfrom
cdoern:certVisibility
Jul 24, 2023
Merged

MCO-607: MCO-237: Keep track of certs in ControllerConfigStatus#3756
openshift-merge-robot merged 1 commit into
openshift:masterfrom
cdoern:certVisibility

Conversation

@cdoern
Copy link
Copy Markdown
Contributor

@cdoern cdoern commented Jun 23, 2023

Add ControllerCertificates to ControllerConfigStatus to keep track of all rotated certs This data gets updated when the template controller syncs. So this happens whenever the controllerConfig is modified either by a cert data rotation or otherwise.

The node controller also has access to this data and puts it into the proper MCP in order for the console team to properly access this data.

- What I did

Modified the CRD for the controllerConfig, added two new API types, and attached logic to the template controller to go through this data.

- How to verify it

oc describe controllerconfig should now have cert info in the status

oc describe mcp/... should now have cert expiry info in the status

- Description for the changelog

Added certificate information to the controllerconfig so customers can access the data easily.

@cdoern cdoern changed the title Keep track of certs in ControllerConfigStatus MCO-607: Keep track of certs in ControllerConfigStatus Jun 23, 2023
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

openshift-ci-robot commented Jun 23, 2023

@cdoern: This pull request references MCO-607 which is a valid jira issue.

Details

In response to this:

Add ControllerCertificates to ControllerConfigStatus to keep track of all rotated certs This data gets updated when the template controller syncs. So this happens whenever the controllerConfig is modified either by a cert data rotation or otherwise.

- What I did

Modified the CRD for the controllerConfig, added two new API types, and attached logic to the template controller to go through this data.

- How to verify it

oc describe controllerconfig should now have cert info in the status

- Description for the changelog

Added certificate information to the controllerconfig so customers can access the data easily.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 23, 2023
@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jun 23, 2023

/jira-refresh

@openshift-ci-robot
Copy link
Copy Markdown
Contributor

openshift-ci-robot commented Jun 26, 2023

@cdoern: This pull request references MCO-607 which is a valid jira issue.

Details

In response to this:

Add ControllerCertificates to ControllerConfigStatus to keep track of all rotated certs This data gets updated when the template controller syncs. So this happens whenever the controllerConfig is modified either by a cert data rotation or otherwise.

The node controller also has access to this data and puts it into the proper MCP in order for the console team to properly access this data.

- What I did

Modified the CRD for the controllerConfig, added two new API types, and attached logic to the template controller to go through this data.

- How to verify it

oc describe controllerconfig should now have cert info in the status

oc describe mcp/... should now have cert expiry info in the status

- Description for the changelog

Added certificate information to the controllerconfig so customers can access the data easily.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@cdoern cdoern changed the title MCO-607: Keep track of certs in ControllerConfigStatus MCO-607: MCO-237: Keep track of certs in ControllerConfigStatus Jun 26, 2023
@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jun 26, 2023

/jira-refresh

@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jun 26, 2023

/retest-required

.... not sure what just happened.

@cdoern cdoern force-pushed the certVisibility branch 6 times, most recently from 7c92d22 to 7c7c739 Compare June 27, 2023 13:57
@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jun 27, 2023

/retest-required

2 similar comments
@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jun 28, 2023

/retest-required

@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jul 11, 2023

/retest-required

@jkyros
Copy link
Copy Markdown
Member

jkyros commented Jul 11, 2023

Hey, that's pretty cool!

I'm not sure the signer is coming out quite right sometimes though (here, maybe?) :

  "controllerCertificates": [
                    {
                        "bundleFile": "KubeAPIServerServingCAData",
                        "notAfter": "2024-07-10 16:15:50 +0000 UTC",
                        "notBefore": "2023-07-11 16:15:49 +0000 UTC",
                        "signer": "\u000f��bj./�W�L�/��~/4�k2�]�O�T\u0008B��{� �Xy$�׃��h��K�;\u0026\u000e'��\u001a ��Q�Dz�aR��~n3���\u000e��#�]��6���t�����\u0006\u000c\u0016�v|�h\u001b����3���o{b�k\u001d\u001dB�1e4�@�4#]buw�@�h��\u000c܊#\u0002[�љ�Ae�,�t��'��\u001f�Y�!�q�G�\u001d֯\u000eS#�T�x\u0012�F\tS��\\�L�S�\u0011��ۿ#�F\\E\u0016�Oc�\u0012�\u0014)4�\u000b\u0005̬U�r���\u0010K\r�Ӄ�f�ŗ�5�xW\u0026y'\u0014���x��~\u0019�͞��;��",
                        "subject": "CN=openshift-kube-apiserver-operator_node-system-admin-signer@1689092150"
                    },
                    {
                        "bundleFile": "RootCAData",
                        "notAfter": "2033-07-08 16:04:12 +0000 UTC",
                        "notBefore": "2023-07-11 16:04:12 +0000 UTC",
                        "signer": "�\u0015$�qZ�H��a�4���a\u0014��[��d���gǟ�\"�\t�q�|�P�\u00149�c��y���W̔\u0000��$�\u0012\u003c��\\��q�VB������֬�[ڻ3E\rfen[\u0010���L�\u0017\u0008k�͟���m[\u0018�A4�S\r9\u0010\u0017��B�[\r$���F?r#ҝ�ـ��Ì��V�?h����=\u0002\u001f�\u003e\u0019�-\u0016\u0005\u001b]Ϋ0e�\u0014A���4�\u000b\u0019L��¼zٜ��5�h_T\u0001d8�F\u001f50|e\u001aҼڵ��U�=)��\u0004��f�@?���7������I��Yz\u0012�Nz�\u0008\u0026I�Ќ5\u0008K��E��",
                        "subject": "CN=root-ca,OU=openshift"
                    }
                ],

(I had to dump JSON because yaml was crying about control characters)

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 11, 2023
@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jul 12, 2023

@sergiordlr does this need QE approval to test the sanity of the output in the controllerconfig?

@sergiordlr
Copy link
Copy Markdown
Contributor

Hello. We have used IPI on AWS for the verification.

This is the info displayed in the controllerconfig

$ oc get controllerconfig machine-config-controller -o jsonpath='{.status.controllerCertificates}' | jq
[
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2024-07-12 07:52:44 +0000 UTC",
    "notBefore": "2023-07-13 07:52:43 +0000 UTC",
    "signer": "CN=openshift-kube-apiserver-operator_node-system-admin-signer@1689234764",
    "subject": "CN=openshift-kube-apiserver-operator_node-system-admin-signer@1689234764"
  },
  {
    "bundleFile": "RootCAData",
    "notAfter": "2033-07-10 07:38:41 +0000 UTC",
    "notBefore": "2023-07-13 07:38:41 +0000 UTC",
    "signer": "CN=root-ca,OU=openshift",
    "subject": "CN=root-ca,OU=openshift"
  }
]
  • Regarding RootCAData

The information is ok. It has the right issuer, subject and dates.

$ oc get controllerconfig machine-config-controller -o jsonpath='{.spec.rootCAData'}  |base64 -d |  while openssl x509 -noout -issuer -subject -dates ; do : ; done
issuer=OU = openshift, CN = root-ca
subject=OU = openshift, CN = root-ca
notBefore=Jul 13 07:38:41 2023 GMT
notAfter=Jul 10 07:38:41 2033 GMT
Could not read certificate from <stdin>
Unable to load certificate

  • Regarding KubeAPIServerServingCAData
$ oc get controllerconfig machine-config-controller -o jsonpath='{.spec.kubeAPIServerServingCAData'}  |base64 -d |  while openssl x509 -noout -issuer -subject -dates ; do : ; done
issuer=OU = openshift, CN = admin-kubeconfig-signer
subject=OU = openshift, CN = admin-kubeconfig-signer
notBefore=Jul 13 07:38:42 2023 GMT
notAfter=Jul 10 07:38:42 2033 GMT
issuer=OU = openshift, CN = kubelet-signer
subject=CN = kube-csr-signer_@1689234764
notBefore=Jul 13 07:52:43 2023 GMT
notAfter=Jul 14 07:38:47 2023 GMT
issuer=OU = openshift, CN = kubelet-signer
subject=OU = openshift, CN = kubelet-signer
notBefore=Jul 13 07:38:47 2023 GMT
notAfter=Jul 14 07:38:47 2023 GMT
issuer=OU = openshift, CN = kube-apiserver-to-kubelet-signer
subject=OU = openshift, CN = kube-apiserver-to-kubelet-signer
notBefore=Jul 13 07:38:48 2023 GMT
notAfter=Jul 12 07:38:48 2024 GMT
issuer=OU = openshift, CN = kube-control-plane-signer
subject=OU = openshift, CN = kube-control-plane-signer
notBefore=Jul 13 07:38:47 2023 GMT
notAfter=Jul 12 07:38:47 2024 GMT
issuer=OU = openshift, CN = kubelet-bootstrap-kubeconfig-signer
subject=OU = openshift, CN = kubelet-bootstrap-kubeconfig-signer
notBefore=Jul 13 07:38:44 2023 GMT
notAfter=Jul 10 07:38:44 2033 GMT
issuer=CN = openshift-kube-apiserver-operator_node-system-admin-signer@1689234764
subject=CN = openshift-kube-apiserver-operator_node-system-admin-signer@1689234764
notBefore=Jul 13 07:52:43 2023 GMT
notAfter=Jul 12 07:52:44 2024 GMT
Could not read certificate from <stdin>
Unable to load certificate

There are several certificates in kubeAPIServerServingCAData, but only the last one ( openshift-kube-apiserver-operator_node-system-admin-signer@1689234764) is displayed in the controllerconfig information, the rest of the certificates are ignored.

@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jul 13, 2023

thanks @sergiordlr I will push a fix for only the last certificate showing

@cdoern cdoern force-pushed the certVisibility branch 2 times, most recently from 24963f1 to 7978318 Compare July 13, 2023 13:59
@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jul 13, 2023

/retest-required

@sergiordlr
Copy link
Copy Markdown
Contributor

sergiordlr commented Jul 14, 2023

Verified using IPI on AWS.

This is the information reported by the ControllerConfig:

$ oc --kubeconfig=/tmp/veri1 get ControllerCOnfig machine-config-controller -o jsonpath={.status.controllerCertificates} | jq
[
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2033-07-11 13:52:25 +0000 UTC",
    "notBefore": "2023-07-14 13:52:25 +0000 UTC",
    "signer": "CN=admin-kubeconfig-signer,OU=openshift",
    "subject": "CN=admin-kubeconfig-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2023-07-15 13:52:29 +0000 UTC",
    "notBefore": "2023-07-14 14:06:49 +0000 UTC",
    "signer": "CN=kubelet-signer,OU=openshift",
    "subject": "CN=kube-csr-signer_@1689343610"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2023-07-15 13:52:29 +0000 UTC",
    "notBefore": "2023-07-14 13:52:29 +0000 UTC",
    "signer": "CN=kubelet-signer,OU=openshift",
    "subject": "CN=kubelet-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2024-07-13 13:52:30 +0000 UTC",
    "notBefore": "2023-07-14 13:52:30 +0000 UTC",
    "signer": "CN=kube-apiserver-to-kubelet-signer,OU=openshift",
    "subject": "CN=kube-apiserver-to-kubelet-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2024-07-13 13:52:29 +0000 UTC",
    "notBefore": "2023-07-14 13:52:29 +0000 UTC",
    "signer": "CN=kube-control-plane-signer,OU=openshift",
    "subject": "CN=kube-control-plane-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2033-07-11 13:52:26 +0000 UTC",
    "notBefore": "2023-07-14 13:52:26 +0000 UTC",
    "signer": "CN=kubelet-bootstrap-kubeconfig-signer,OU=openshift",
    "subject": "CN=kubelet-bootstrap-kubeconfig-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2024-07-13 14:06:51 +0000 UTC",
    "notBefore": "2023-07-14 14:06:50 +0000 UTC",
    "signer": "CN=openshift-kube-apiserver-operator_node-system-admin-signer@1689343610",
    "subject": "CN=openshift-kube-apiserver-operator_node-system-admin-signer@1689343610"
  },
  {
    "bundleFile": "RootCAData",
    "notAfter": "2033-07-11 13:52:24 +0000 UTC",
    "notBefore": "2023-07-14 13:52:24 +0000 UTC",
    "signer": "CN=root-ca,OU=openshift",
    "subject": "CN=root-ca,OU=openshift"
  }
]
  • Regarding RooCAData

The information is ok. I has the right issure, subject and dates:

$ oc get controllerconfig machine-config-controller -o jsonpath='{.spec.rootCAData'}  |base64 -d |  while openssl x509 -noout -issuer -subject -dates  -nameopt RFC2253 ; do : ; done
issuer=CN=root-ca,OU=openshift
subject=CN=root-ca,OU=openshift
notBefore=Jul 14 13:52:24 2023 GMT
notAfter=Jul 11 13:52:24 2033 GMT
  • Regarding KubeAPIServerServingCAData

The information is OK too. It has the right subject, issuer and dates for every certificate:

$  oc get controllerconfig machine-config-controller -o jsonpath='{.spec.kubeAPIServerServingCAData'}  |base64 -d |  while openssl x509 -noout -issuer -subject -dates -nameopt RFC2253; do echo "-----" ; done
issuer=CN=admin-kubeconfig-signer,OU=openshift
subject=CN=admin-kubeconfig-signer,OU=openshift
notBefore=Jul 14 13:52:25 2023 GMT
notAfter=Jul 11 13:52:25 2033 GMT
-----
issuer=CN=kubelet-signer,OU=openshift
subject=CN=kube-csr-signer_@1689343610
notBefore=Jul 14 14:06:49 2023 GMT
notAfter=Jul 15 13:52:29 2023 GMT
-----
issuer=CN=kubelet-signer,OU=openshift
subject=CN=kubelet-signer,OU=openshift
notBefore=Jul 14 13:52:29 2023 GMT
notAfter=Jul 15 13:52:29 2023 GMT
-----
issuer=CN=kube-apiserver-to-kubelet-signer,OU=openshift
subject=CN=kube-apiserver-to-kubelet-signer,OU=openshift
notBefore=Jul 14 13:52:30 2023 GMT
notAfter=Jul 13 13:52:30 2024 GMT
-----
issuer=CN=kube-control-plane-signer,OU=openshift
subject=CN=kube-control-plane-signer,OU=openshift
notBefore=Jul 14 13:52:29 2023 GMT
notAfter=Jul 13 13:52:29 2024 GMT
-----
issuer=CN=kubelet-bootstrap-kubeconfig-signer,OU=openshift
subject=CN=kubelet-bootstrap-kubeconfig-signer,OU=openshift
notBefore=Jul 14 13:52:26 2023 GMT
notAfter=Jul 11 13:52:26 2033 GMT
-----
issuer=CN=openshift-kube-apiserver-operator_node-system-admin-signer@1689343610
subject=CN=openshift-kube-apiserver-operator_node-system-admin-signer@1689343610
notBefore=Jul 14 14:06:50 2023 GMT
notAfter=Jul 13 14:06:51 2024 GMT
-----

  • Regarding MCP

Master pool is reporting the expiration time for the certificates like this:

$ oc get mcp master -o yaml  |grep certExpirys -B1 -A15
status:
  certExpirys:
  - bundle: KubeAPIServerServingCAData
    expiry: 2033-07-11 13:52:25 +0000 UTC
  - bundle: KubeAPIServerServingCAData
    expiry: 2023-07-15 13:52:29 +0000 UTC
  - bundle: KubeAPIServerServingCAData
    expiry: 2023-07-15 13:52:29 +0000 UTC
  - bundle: KubeAPIServerServingCAData
    expiry: 2024-07-13 13:52:30 +0000 UTC
  - bundle: KubeAPIServerServingCAData
    expiry: 2024-07-13 13:52:29 +0000 UTC
  - bundle: KubeAPIServerServingCAData
    expiry: 2033-07-11 13:52:26 +0000 UTC
  - bundle: KubeAPIServerServingCAData
    expiry: 2024-07-13 14:06:51 +0000 UTC
  conditions:

The same information is displayed for the worker pool

$ oc get mcp worker -o yaml  |grep certExpirys -B1 -A16
status:
  certExpirys:
  - bundle: KubeAPIServerServingCAData
    expiry: 2033-07-11 13:52:25 +0000 UTC
  - bundle: KubeAPIServerServingCAData
    expiry: 2023-07-15 13:52:29 +0000 UTC
  - bundle: KubeAPIServerServingCAData
  • We have confirmed that the certifiactes information is available in the ControllerConfig and the MCPs descriptions too.
$ oc describe controllerconfig  |grep "Controller Certificates"  -A 5
 Controller Certificates:
   Bundle File:        KubeAPIServerServingCAData
   Not After:          2033-07-11 13:52:25 +0000 UTC
   Not Before:         2023-07-14 13:52:25 +0000 UTC
   Signer:             CN=admin-kubeconfig-signer,OU=openshift
   Subject:            CN=admin-kubeconfig-signer,OU=openshift
...

$ oc describe mcp worker  | grep "Cert Exp" -A 5
 Cert Expirys:
   Bundle:  KubeAPIServerServingCAData
   Expiry:  2033-07-11 13:52:25 +0000 UTC
   Bundle:  KubeAPIServerServingCAData
   Expiry:  2023-07-15 13:52:29 +0000 UTC
   Bundle:  KubeAPIServerServingCAData
...

  • A rotation for the kub-apiserver certificates was executed.

The information in the ControllerConfig regarding the certificates was correct after the certificate rotation.

We have not found any problem. The only thing that we like to comment is that the information regarding the certificates' expiration date in the MCPs is a bit confusing from the user's point of view, the information is right, though.

@sergiordlr
Copy link
Copy Markdown
Contributor

Please, no problem has been found, but before adding the qe-approved label, could you please, guys, review the PR verification and confirm that the information that we are displaying now is the information that we want to display actually? So that I have the opportunity to test the PR in case you decide to add/remove some information.

Thank you very much!

@sergiordlr
Copy link
Copy Markdown
Contributor

sergiordlr commented Jul 14, 2023

I forgot to add that everytime we rotate the certificates, the number of certificates tracked by ControllerConfig increases. I'm not sure about how it can impact the readability of MCPs and ControllerConfig if we rotate the certificates very often.

For example, this is the number of certificates without any rotation

$ cat orig/controller-config-all-certs.json
[
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2033-07-11 08:04:36 +0000 UTC",
    "notBefore": "2023-07-14 08:04:36 +0000 UTC",
    "signer": "CN=admin-kubeconfig-signer,OU=openshift",
    "subject": "CN=admin-kubeconfig-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2023-07-15 08:04:40 +0000 UTC",
    "notBefore": "2023-07-14 08:19:42 +0000 UTC",
    "signer": "CN=kubelet-signer,OU=openshift",
    "subject": "CN=kube-csr-signer_@1689322783"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2023-07-15 08:04:40 +0000 UTC",
    "notBefore": "2023-07-14 08:04:40 +0000 UTC",
    "signer": "CN=kubelet-signer,OU=openshift",
    "subject": "CN=kubelet-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2024-07-13 08:04:41 +0000 UTC",
    "notBefore": "2023-07-14 08:04:41 +0000 UTC",
    "signer": "CN=kube-apiserver-to-kubelet-signer,OU=openshift",
    "subject": "CN=kube-apiserver-to-kubelet-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2024-07-13 08:04:41 +0000 UTC",
    "notBefore": "2023-07-14 08:04:41 +0000 UTC",
    "signer": "CN=kube-control-plane-signer,OU=openshift",
    "subject": "CN=kube-control-plane-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2033-07-11 08:04:37 +0000 UTC",
    "notBefore": "2023-07-14 08:04:37 +0000 UTC",
    "signer": "CN=kubelet-bootstrap-kubeconfig-signer,OU=openshift",
    "subject": "CN=kubelet-bootstrap-kubeconfig-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2024-07-13 08:19:42 +0000 UTC",
    "notBefore": "2023-07-14 08:19:41 +0000 UTC",
    "signer": "CN=openshift-kube-apiserver-operator_node-system-admin-signer@1689322781",
    "subject": "CN=openshift-kube-apiserver-operator_node-system-admin-signer@1689322781"
  },
  {
    "bundleFile": "RootCAData",
    "notAfter": "2033-07-11 08:04:34 +0000 UTC",
    "notBefore": "2023-07-14 08:04:34 +0000 UTC",
    "signer": "CN=root-ca,OU=openshift",
    "subject": "CN=root-ca,OU=openshift"
  }
]

And this is the same ControllerConfig after a certificate rotation

[
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2033-07-11 08:04:36 +0000 UTC",
    "notBefore": "2023-07-14 08:04:36 +0000 UTC",
    "signer": "CN=admin-kubeconfig-signer,OU=openshift",
    "subject": "CN=admin-kubeconfig-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2033-07-11 12:21:06 +0000 UTC",
    "notBefore": "2023-07-14 12:21:05 +0000 UTC",
    "signer": "CN=openshift-config_admin.kubeconfig-signer@1689337266",
    "subject": "CN=openshift-config_admin.kubeconfig-signer@1689337266"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2023-07-15 08:04:40 +0000 UTC",
    "notBefore": "2023-07-14 08:19:42 +0000 UTC",
    "signer": "CN=kubelet-signer,OU=openshift",
    "subject": "CN=kube-csr-signer_@1689322783"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2023-07-15 08:04:40 +0000 UTC",
    "notBefore": "2023-07-14 08:04:40 +0000 UTC",
    "signer": "CN=kubelet-signer,OU=openshift",
    "subject": "CN=kubelet-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2024-07-13 11:33:19 +0000 UTC",
    "notBefore": "2023-07-14 11:33:18 +0000 UTC",
    "signer": "CN=openshift-kube-apiserver-operator_kube-apiserver-to-kubelet-signer@1689334399",
    "subject": "CN=openshift-kube-apiserver-operator_kube-apiserver-to-kubelet-signer@1689334399"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2023-09-12 11:33:20 +0000 UTC",
    "notBefore": "2023-07-14 11:33:19 +0000 UTC",
    "signer": "CN=openshift-kube-apiserver-operator_kube-control-plane-signer@1689334399",
    "subject": "CN=openshift-kube-apiserver-operator_kube-control-plane-signer@1689334399"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2023-09-12 11:33:19 +0000 UTC",
    "notBefore": "2023-07-14 11:33:18 +0000 UTC",
    "signer": "CN=openshift-kube-apiserver-operator_kube-control-plane-signer@1689334399",
    "subject": "CN=openshift-kube-apiserver-operator_kube-control-plane-signer@1689334399"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2023-09-12 11:33:19 +0000 UTC",
    "notBefore": "2023-07-14 11:33:18 +0000 UTC",
    "signer": "CN=openshift-kube-apiserver-operator_kube-control-plane-signer@1689334399",
    "subject": "CN=openshift-kube-apiserver-operator_kube-control-plane-signer@1689334399"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2023-09-12 11:33:19 +0000 UTC",
    "notBefore": "2023-07-14 11:33:18 +0000 UTC",
    "signer": "CN=openshift-kube-apiserver-operator_kube-control-plane-signer@1689334399",
    "subject": "CN=openshift-kube-apiserver-operator_kube-control-plane-signer@1689334399"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2023-09-12 11:33:19 +0000 UTC",
    "notBefore": "2023-07-14 11:33:18 +0000 UTC",
    "signer": "CN=openshift-kube-apiserver-operator_kube-control-plane-signer@1689334399",
    "subject": "CN=openshift-kube-apiserver-operator_kube-control-plane-signer@1689334399"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2033-07-11 08:04:37 +0000 UTC",
    "notBefore": "2023-07-14 08:04:37 +0000 UTC",
    "signer": "CN=kubelet-bootstrap-kubeconfig-signer,OU=openshift",
    "subject": "CN=kubelet-bootstrap-kubeconfig-signer,OU=openshift"
  },
  {
    "bundleFile": "KubeAPIServerServingCAData",
    "notAfter": "2024-07-13 08:19:42 +0000 UTC",
    "notBefore": "2023-07-14 08:19:41 +0000 UTC",
    "signer": "CN=openshift-kube-apiserver-operator_node-system-admin-signer@1689322781",
    "subject": "CN=openshift-kube-apiserver-operator_node-system-admin-signer@1689322781"
  },
  {
    "bundleFile": "RootCAData",
    "notAfter": "2033-07-11 08:04:34 +0000 UTC",
    "notBefore": "2023-07-14 08:04:34 +0000 UTC",
    "signer": "CN=root-ca,OU=openshift",
    "subject": "CN=root-ca,OU=openshift"
  }
]

Before the rotation we had 8 certificates, after the rotation we have 13 certificates. I haven't rotated again, but I assume that likely after a second rotation we will have aprox 18 certificates and so on....

It is not a problem of this PR, this PR shows the information correctly. The certificates are added to the kube-apiserver-client-ca bundle.

@sergiordlr
Copy link
Copy Markdown
Contributor

sergiordlr commented Jul 17, 2023

With the latest change, the certificates information in MCPs is this one (subject has been added):

$ oc get mcp master -o yaml  |grep certExpirys -B1 -A15
status:
  certExpirys:
  - bundle: KubeAPIServerServingCAData
    expiry: 2033-07-14 09:04:22 +0000 UTC
    subject: CN=admin-kubeconfig-signer,OU=openshift
  - bundle: KubeAPIServerServingCAData
    expiry: 2023-07-18 09:04:26 +0000 UTC
    subject: CN=kube-csr-signer_@1689585558
  - bundle: KubeAPIServerServingCAData
    expiry: 2023-07-18 09:04:26 +0000 UTC
    subject: CN=kubelet-signer,OU=openshift
  - bundle: KubeAPIServerServingCAData
    expiry: 2024-07-16 09:04:27 +0000 UTC
    subject: CN=kube-apiserver-to-kubelet-signer,OU=openshift
  - bundle: KubeAPIServerServingCAData
    expiry: 2024-07-16 09:04:27 +0000 UTC
    subject: CN=kube-control-plane-signer,OU=openshift

No issues found.

@sergiordlr
Copy link
Copy Markdown
Contributor

Not a bug, but maybe we are showing too many logs regarding the certificates visibility in the MCO controller pod

I0717 10:07:38.389413       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389459       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389490       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389521       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389551       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389581       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389611       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389639       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:07:38.389678       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:28:51.057345       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057380       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057395       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057411       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057428       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057446       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057464       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057481       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:28:51.057490       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:28:51.546799       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546838       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546862       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546885       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546917       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546945       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546975       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.547000       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:28:51.547015       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:28:56.368807       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 10:28:56.368807       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 10:43:15.918835       1 template_controller.go:132] Re-syncing ControllerConfig due to secret pull-secret change
I0717 10:43:15.918929       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.918958       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.918977       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.918996       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.919013       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.919033       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.919054       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.919072       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:43:15.919081       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:57:11.925256       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925296       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925313       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925332       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925350       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925372       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925393       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925412       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:57:11.925419       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:57:12.431527       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431568       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431588       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431607       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431627       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431677       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431715       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431743       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:57:12.431755       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:57:17.235806       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 10:57:17.235806       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 11:18:53.447704       1 template_controller.go:132] Re-syncing ControllerConfig due to secret pull-secret change
I0717 11:18:53.447810       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447842       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447861       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447879       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447897       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447919       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447939       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447957       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 11:18:53.447967       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 11:25:32.793865       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.793910       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.793935       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.793960       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.793986       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.794014       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.794042       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.794067       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 11:25:32.794081       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 11:25:33.298325       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298370       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298393       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298420       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298447       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298490       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298532       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298557       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 11:25:33.298572       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 11:25:38.104786       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 11:25:38.104804       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 11:53:53.665820       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665864       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665891       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665918       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665947       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665975       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665997       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.666024       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 11:53:53.666038       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 11:53:54.160114       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160155       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160172       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160187       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160210       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160230       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160249       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160268       1 template_controller.go:452] Updating Controller Cerficiate RootCAData

@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jul 17, 2023

Not a bug, but maybe we are showing too many logs regarding the certificates visibility in the MCO controller pod

I0717 10:07:38.389413       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389459       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389490       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389521       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389551       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389581       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389611       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:07:38.389639       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:07:38.389678       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:28:51.057345       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057380       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057395       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057411       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057428       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057446       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057464       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.057481       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:28:51.057490       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:28:51.546799       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546838       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546862       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546885       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546917       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546945       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.546975       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:28:51.547000       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:28:51.547015       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:28:56.368807       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 10:28:56.368807       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 10:43:15.918835       1 template_controller.go:132] Re-syncing ControllerConfig due to secret pull-secret change
I0717 10:43:15.918929       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.918958       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.918977       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.918996       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.919013       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.919033       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.919054       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:43:15.919072       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:43:15.919081       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:57:11.925256       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925296       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925313       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925332       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925350       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925372       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925393       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:11.925412       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:57:11.925419       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:57:12.431527       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431568       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431588       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431607       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431627       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431677       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431715       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 10:57:12.431743       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 10:57:12.431755       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 10:57:17.235806       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 10:57:17.235806       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 11:18:53.447704       1 template_controller.go:132] Re-syncing ControllerConfig due to secret pull-secret change
I0717 11:18:53.447810       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447842       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447861       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447879       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447897       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447919       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447939       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:18:53.447957       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 11:18:53.447967       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 11:25:32.793865       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.793910       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.793935       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.793960       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.793986       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.794014       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.794042       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:32.794067       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 11:25:32.794081       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 11:25:33.298325       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298370       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298393       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298420       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298447       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298490       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298532       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:25:33.298557       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 11:25:33.298572       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 11:25:38.104786       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 11:25:38.104804       1 render_controller.go:565] BaseOSContainerImage=registry.build05.ci.openshift.org/ci-ln-tc6j2x2/stable@sha256:eb347b465d1c72324f0ab29e5131c47614c884d29be9bc2bf6657bc9c950865b
I0717 11:53:53.665820       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665864       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665891       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665918       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665947       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665975       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.665997       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:53.666024       1 template_controller.go:452] Updating Controller Cerficiate RootCAData
I0717 11:53:53.666038       1 template_controller.go:502] Detecting cert modification, syncing these changes to the true controllerConfig.
I0717 11:53:54.160114       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160155       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160172       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160187       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160210       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160230       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160249       1 template_controller.go:452] Updating Controller Cerficiate KubeAPIServerServingCAData
I0717 11:53:54.160268       1 template_controller.go:452] Updating Controller Cerficiate RootCAData

thanks for that.... leftover debug I will definitely remove that.

add ControllerCertificates to ControllerConfigStatus to keep track of all rotated certs
This data gets updated when the template controller syncs

also, add a way for the MCPs to keep track of cert expiration date for the console team

Signed-off-by: Charlie Doern <cdoern@redhat.com>
@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jul 17, 2023

/retest-required

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Jul 17, 2023

@cdoern: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/okd-scos-e2e-aws-ovn 80e7b4d link false /test okd-scos-e2e-aws-ovn

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jul 18, 2023

/retest-required

@sinnykumari
Copy link
Copy Markdown
Contributor

/test okd-images

@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jul 18, 2023

/test bootstrap-unit

@sergiordlr
Copy link
Copy Markdown
Contributor

Since no problem has been found and the information that we are displaying seems to be OK, we add the qe-approved label.

/label qe-approved

@openshift-ci openshift-ci Bot added the qe-approved Signifies that QE has signed off on this PR label Jul 20, 2023
@cdoern
Copy link
Copy Markdown
Contributor Author

cdoern commented Jul 24, 2023

Made a cleanup item for the above issues with the controllerConfig in the future: https://issues.redhat.com/browse/MCO-688

@cheesesashimi
Copy link
Copy Markdown
Member

/lgtm
/approved

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 24, 2023
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Jul 24, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cdoern, cheesesashimi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [cdoern,cheesesashimi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 3fb306d into openshift:master Jul 24, 2023
wking added a commit to wking/machine-config-operator that referenced this pull request Jan 2, 2024
These had snuck in back in 80e7b4d (keep track of certs in
ControllerConfigStatus, 2023-06-20, openshift#3756), 81136ed (implement the
ability for the MCO to handle image registry certificates, 2023-06-27,
names doesn't have any functional impact, but it's less confusing to
read if the package is refered to with a consistent prefix.  This
commit addresses all the duplicates turned up with:

  $ git grep -c '"github.com/openshift/api/machineconfiguration/v1"'  | grep '[.]go:' | grep -v 'vendor/\|:1$'
wking added a commit to wking/machine-config-operator that referenced this pull request Jan 2, 2024
These had snuck in back in 80e7b4d (keep track of certs in
ControllerConfigStatus, 2023-06-20, openshift#3756), 81136ed (implement the
ability for the MCO to handle image registry certificates, 2023-06-27, openshift#3770),
and similar.  Having the same package imported under multiple names
doesn't have any functional impact, but it's less confusing to read if
the package is refered to with a consistent prefix.  This commit
addresses all the duplicates turned up with:

  $ git grep -c '"github.com/openshift/api/machineconfiguration/v1"'  | grep '[.]go:' | grep -v 'vendor/\|:1$'
wking added a commit to wking/machine-config-operator that referenced this pull request Jan 2, 2024
These had snuck in back in 80e7b4d (keep track of certs in
ControllerConfigStatus, 2023-06-20, openshift#3756), 81136ed (implement the
ability for the MCO to handle image registry certificates, 2023-06-27, openshift#3770),
and similar.  Having the same package imported under multiple names
doesn't have any functional impact, but it's less confusing to read if
the package is refered to with a consistent prefix.  This commit
addresses all the duplicates turned up with:

  $ git grep -c '"github.com/openshift/api/machineconfiguration/v1"'  | grep '[.]go:' | grep -v 'vendor/\|:1$'
wking added a commit to wking/machine-config-operator that referenced this pull request Jan 2, 2024
These had snuck in back in 80e7b4d (keep track of certs in
ControllerConfigStatus, 2023-06-20, openshift#3756), 81136ed (implement the
ability for the MCO to handle image registry certificates, 2023-06-27, openshift#3770),
and similar.  Having the same package imported under multiple names
doesn't have any functional impact, but it's less confusing to read if
the package is refered to with a consistent prefix.  This commit
addresses all the duplicates turned up with:

  $ git grep -c '"github.com/openshift/api/machineconfiguration/v1"'  | grep '[.]go:' | grep -v 'vendor/\|:1$'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. qe-approved Signifies that QE has signed off on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants