DNM: Pin to RHCOS 47.190 #801

wking · 2018-12-06T07:33:06Z

I suspect 47.191 of causing:

Failing tests:

[Feature:Builds][Conformance] oc new-app  should succeed with a --name of 58 characters [Suite:openshift/conformance/parallel/minimal] [Suite:openshift/smoke-4]
[Feature:Builds][Smoke] result image should have proper labels set  Docker build from a template should create a image from "test-docker-build.json" template with proper Docker labels [Suite:openshift/conformance/parallel] [Suite:openshift/smoke-4]
[Feature:Builds][Smoke] result image should have proper labels set  S2I build from a template should create a image from "test-s2i-build.json" template with proper Docker labels [Suite:openshift/conformance/parallel] [Suite:openshift/smoke-4]

openshift-ci-robot · 2018-12-06T07:33:15Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [wking]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

I suspect 47.191 of causing [1]: Failing tests: [Feature:Builds][Conformance] oc new-app should succeed with a --name of 58 characters [Suite:openshift/conformance/parallel/minimal] [Suite:openshift/smoke-4] [Feature:Builds][Smoke] result image should have proper labels set Docker build from a template should create a image from "test-docker-build.json" template with proper Docker labels [Suite:openshift/conformance/parallel] [Suite:openshift/smoke-4] [Feature:Builds][Smoke] result image should have proper labels set S2I build from a template should create a image from "test-s2i-build.json" template with proper Docker labels [Suite:openshift/conformance/parallel] [Suite:openshift/smoke-4] And 47.190 causes those and one more [2]: Failing tests: [Feature:Builds][Conformance] oc new-app should succeed with a --name of 58 characters [Suite:openshift/conformance/parallel/minimal] [Suite:openshift/smoke-4] [Feature:Builds][Smoke] result image should have proper labels set Docker build from a template should create a image from "test-docker-build.json" template with proper Docker labels [Suite:openshift/conformance/parallel] [Suite:openshift/smoke-4] [Feature:Builds][Smoke] result image should have proper labels set S2I build from a template should create a image from "test-s2i-build.json" template with proper Docker labels [Suite:openshift/conformance/parallel] [Suite:openshift/smoke-4] [Feature:Platform][Suite:openshift/smoke-4] Managed cluster should have no crashlooping pods in core namespaces over two minutes [Suite:openshift/conformance/parallel] [1]: https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_installer/798/pull-ci-openshift-installer-master-e2e-aws/1978/build-log.txt [2]: https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_installer/801/pull-ci-openshift-installer-master-e2e-aws/1980/build-log.txt

wking · 2018-12-06T08:28:01Z

47.190 has an additional error:

Failing tests:

[Feature:Builds][Conformance] oc new-app  should succeed with a --name of 58 characters [Suite:openshift/conformance/parallel/minimal] [Suite:openshift/smoke-4]
[Feature:Builds][Smoke] result image should have proper labels set  Docker build from a template should create a image from "test-docker-build.json" template with proper Docker labels [Suite:openshift/conformance/parallel] [Suite:openshift/smoke-4]
[Feature:Builds][Smoke] result image should have proper labels set  S2I build from a template should create a image from "test-s2i-build.json" template with proper Docker labels [Suite:openshift/conformance/parallel] [Suite:openshift/smoke-4]
[Feature:Platform][Suite:openshift/smoke-4] Managed cluster should have no crashlooping pods in core namespaces over two minutes [Suite:openshift/conformance/parallel]

Let's go back to 47.188...

wking · 2018-12-06T13:44:43Z

Same with 47.188:

Failing tests:

[Feature:Builds][Conformance] oc new-app should succeed with a --name of 58 characters [Suite:openshift/conformance/parallel/minimal] [Suite:openshift/smoke-4]
[Feature:Builds][Smoke] result image should have proper labels set Docker build from a template should create a image from "test-docker-build.json" template with proper Docker labels [Suite:openshift/conformance/parallel] [Suite:openshift/smoke-4]
[Feature:Builds][Smoke] result image should have proper labels set S2I build from a template should create a image from "test-s2i-build.json" template with proper Docker labels [Suite:openshift/conformance/parallel] [Suite:openshift/smoke-4]

I dunno, maybe it's not RHCOS?

cgwalters · 2018-12-06T15:09:30Z

The only real changes in RHCOS that could affect this are the usual churn in origin (kubelet, cli, etc.).

wking · 2018-12-07T00:07:04Z

Digging into the 58-char issue, here's what killed it:

2018-12-06T09:14:23.925127369Z error: build error: Error determining manifest MIME type for docker://image-registry.openshift-image-registry.svc:5000/openshift/nodejs@sha256:c29f4b136d8d3640326a70d17afe35cbb93450839a79cb0ad79ec2a0f968c03b: pinging docker registry returned: Get https://image-registry.openshift-image-registry.svc:5000/v2/: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca")

I'll launch a new test and check the TLS for that endpoint.

/retest

wking · 2018-12-07T00:40:39Z

[core@ip-10-0-6-202 ~]$ openssl s_client -verify_return_error -connect image-registry.openshift-image-registry.svc:5000 </dev/null 
CONNECTED(00000003)
depth=1 OU = bootkube, CN = service-serving
verify error:num=20:unable to get local issuer certificate
---
Certificate chain
 0 s:/CN=image-registry.openshift-image-registry.svc
   i:/OU=bootkube/CN=service-serving
 1 s:/OU=bootkube/CN=service-serving
   i:/OU=openshift/CN=root-ca
---
Server certificate
-----BEGIN CERTIFICATE-----
MIID6DCCAtCgAwIBAgIIHDg0OqjylgMwDQYJKoZIhvcNAQELBQAwLTERMA8GA1UE
CxMIYm9vdGt1YmUxGDAWBgNVBAMTD3NlcnZpY2Utc2VydmluZzAeFw0xODEyMDcw
MDI5MzRaFw0yMDEyMDYwMDI5MzVaMDYxNDAyBgNVBAMTK2ltYWdlLXJlZ2lzdHJ5
Lm9wZW5zaGlmdC1pbWFnZS1yZWdpc3RyeS5zdmMwggEiMA0GCSqGSIb3DQEBAQUA
A4IBDwAwggEKAoIBAQCes5zUNROkMhxOZuCz6jVSmlCtkqmxyFjHqU1C9dB70i/0
mSxm2EY6BajZgaoOxsVgGke7WCSkONrEFm2fMyAVVvVUl7SPSqhEl08V56t0WT8O
2jiau+QPI5lXWK4i6//hYP2vmQMLAFnW4RiG8+kS/gz2fefCuPBBBPAyzEMZORVu
h9Kp7PsCI6zOCvC9Yq331Fd6YAKxY2kuinZ9TP5wZnD+7sP/FyVLCwt3HilOXTOI
NGU/cNiypsaFkU/CUfWIMNzxGPVkzXXsV/mqQ2UanxA1J+FFIqMr1FUAECXO+mdJ
TasKfeuQow6g0N1dTpR9dj45GeKhi/q0s9H/TahDAgMBAAGjggEBMIH+MA4GA1Ud
DwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8G
A1UdIwQYMBaAFEdDwPUsVYvERV2zaizaEjaA6HifMHEGA1UdEQRqMGiCK2ltYWdl
LXJlZ2lzdHJ5Lm9wZW5zaGlmdC1pbWFnZS1yZWdpc3RyeS5zdmOCOWltYWdlLXJl
Z2lzdHJ5Lm9wZW5zaGlmdC1pbWFnZS1yZWdpc3RyeS5zdmMuY2x1c3Rlci5sb2Nh
bDA1BgsrBgEEAZIIEWQCAQQmEyQyYzMzYWQyMC1mOWI3LTExZTgtOTljOS0wZTc3
ZWRiOTc0YWEwDQYJKoZIhvcNAQELBQADggEBAF45LrvpqI+xBChuqZxGTzDZZskd
FBpHsEGNC6obpqQn2bqC0H3T+/ibXMIB1yYzAEkKAYp7ISiO56Lr1IP32W7nbvhE
kcDCrVuknAWelfPO9C6OsACdwX8wO+miduOTTKSq1Ilgntg48aWAnvLlpFq5i5hz
Rbqxgs6DuJ4iUBHHw27LGigy8fmJL69n9i//Ox2Lo5yFhX2rPpAC6C/9F4ttAdjE
FILqLKVnOSa3snkYGLSxsRUIrJK7Iqcv/v1H7yWoUWIVDCFGEssnjPaNTVxxQW+f
Z1b9QImX3N20Wa1uX5YeGF+hNkDu8ChkJf/qCsMXWk3zJ/IBDvE7SFU3bjg=
-----END CERTIFICATE-----
subject=/CN=image-registry.openshift-image-registry.svc
issuer=/OU=bootkube/CN=service-serving
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2441 bytes and written 415 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 02A238CBD3D7DE4B65756371C07185AF470A48935C3546213B3DE5CC3CEFE709
    Session-ID-ctx: 
    Master-Key: EED10759E815DEAF9A0333893C427779444E4BCB18D9197F55DF555F54784F24F812C2B912AF731A837DADD06B58E028
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket:
    0000 - d9 22 fe e8 1c 42 e4 55-31 99 f5 5a 7b 7f 84 bf   ."...B.U1..Z{...
    0010 - 05 fb a8 85 82 1d e9 9f-98 82 7d de cf 96 db 45   ..........}....E
    0020 - 45 d1 aa b3 1c e4 92 f1-ee f2 78 9d 63 99 02 91   E.........x.c...
    0030 - d8 45 4c d2 6c 11 c1 a0-7c 42 80 8f a1 af 3b 62   .EL.l...|B....;b
    0040 - 0b b4 7e b2 09 f9 6e 95-34 a4 e2 09 35 2a ca 49   ..~...n.4...5*.I
    0050 - 84 fc fc 61 59 10 b4 d5-c6 23 14 60 0a a7 65 1f   ...aY....#.`..e.
    0060 - 8e 3f ff 65 f8 32 d6 5d-cf 63 8d eb e5 5a 64 8d   .?.e.2.].c...Zd.
    0070 - ff 2d 30 f2 dd 11 75 96-                          .-0...u.

    Start Time: 1544143153
    Timeout   : 300 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
---
DONE

Maybe they need to be appending the intermediate CA?

wking · 2018-12-07T00:52:16Z

Looks like the CVO died?

[core@ip-10-0-6-202 ~]$ sudo crictl ps -a
CONTAINER ID        IMAGE                                                                                                                          CREATED             STATE               NAME                           ATTEMPT
9c4b83408991b       03de8f11d9e07ee2b23be6d48dc849b9a5e24e4ab4c3ab758bdcd583b3b8fbd9                                                               17 minutes ago      Running             controller-manager             0
0733aabea0c34       registry.svc.ci.openshift.org/ci-op-2s5yvys3/release@sha256:3861328a19852ff283050a2d912fc6ce9780bf30504c40215fd471e8342ddf8a   17 minutes ago      Running             cluster-version-operator       1
cf5760b98b326       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:f82a3c247a4c59538a3d40ad1a2257383420440e15c4675b2e11ad620601bf98    17 minutes ago      Running             openshift-kube-apiserver       0
0a0b25edfb118       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:fa706a9606bec8014fc02ef35385d7efb34234442e08865e0b8cf8821b62e036    18 minutes ago      Exited              installer                      0
cff3ae6365927       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:f82a3c247a4c59538a3d40ad1a2257383420440e15c4675b2e11ad620601bf98    18 minutes ago      Running             openshift-apiserver            0
d7ae75c648742       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:b78062a9b95fd3864ea17d1ec26fd565b1b2b8261b8136f07e407dcbdd1d840e    19 minutes ago      Running             console                        0
2d43328a6163f       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:6667ac4aecae183dfd4e6ae4277dd86ca977e0a3b9feefee653043105503c6d6    19 minutes ago      Running             tuned                          0
db96eaf3e990f       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:b3ced03b1d2e181199dd387b6caa15ff051e2019a4dd22abaf5365de25d874b3    19 minutes ago      Running             console-operator               0
6f213373b1f4c       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:04b6a2c614fb3840782db830c5739ff00b74ba596c768b0c4de457b5584bdecd    19 minutes ago      Running             cluster-node-tuning-operator   0
32ed3134d9a46       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:b52fdf3cf4cc6b52d699ec355cad73989c969d54551e003a2a831ffe5d9379b9    19 minutes ago      Running             registry-ca-hostmapper         0
e055f5d713cc0       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:6e44c3cedc07017b91516b4c0456c4053898df51dcb5ea2daad1a553207bb332    20 minutes ago      Running             package-server                 0
a0143072369a1       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:cab9bf2289702dd3d92143f2802e6b4fe0dbb61b3dc4eec4837e41482fbd0c42    20 minutes ago      Running             machine-config-daemon          0
c93359f98a227       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:950f72cade6a484a6a500968191066945d5071a463805882619733a778a4add3    22 minutes ago      Running             machine-config-server          0
1a02da3dc5ac4       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:dad865393a21a7cf770626a000b5afef47937bd62b589c796ec10a51562b68f6    22 minutes ago      Running             operator                       1
6cd0f89ab60bc       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:4d0106d7428828c87ed905728742fbc11bd8b30d0c87165359699d0a475e2315    22 minutes ago      Running             kube-controller-manager        0
02d06b2a3cb83       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:fa706a9606bec8014fc02ef35385d7efb34234442e08865e0b8cf8821b62e036    22 minutes ago      Exited              installer                      0
76f85fb007afe       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:fd9dd199bc0115af013e4a78233f5920bdfd33136b376da5dc87532efa6a88e4    22 minutes ago      Exited              installer                      0
c73080768c53b       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:a3d70c94b7125507536dc97480aad8ccff36a3d12f63793e63249107fce3f63d    22 minutes ago      Running             machine-config-controller      0
0b7ff7556660c       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:4d0106d7428828c87ed905728742fbc11bd8b30d0c87165359699d0a475e2315    22 minutes ago      Running             scheduler                      0
38231c36a1728       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:936b4f986cfe791de81639bbccfb7a0886b4a1aa953ad2005ed064272c6f1446    23 minutes ago      Exited              installer                      0
33466abf21177       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:166628c841e80c65128e421557a44885d74450c0df041cab57e045e75fcf00d9    23 minutes ago      Running             machine-api-operator           0
dc28f05be8435       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:ce86e514320b680f39735323288cfd19caee5a9480b086b4b275454aef94136e    23 minutes ago      Running             dns-node-resolver              0
babaea31b095b       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:e4936a702d7d466a64a6a9359f35c7ad528bba7c35fe5c582a90e46f9051d8b8    23 minutes ago      Running             dns                            0
b94ad1b47e3c6       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:dad865393a21a7cf770626a000b5afef47937bd62b589c796ec10a51562b68f6    23 minutes ago      Exited              operator                       0
65ea41c93603f       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:4493151dde374cbf1171fcfcab9bef102181ddd174d71ac5b281470d40daece4    24 minutes ago      Running             operator                       0
9a46534947fb0       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:fd9dd199bc0115af013e4a78233f5920bdfd33136b376da5dc87532efa6a88e4    24 minutes ago      Running             operator                       0
7a1e10f9192cb       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:581af93fda80257651d621dade879e438f846a5bf39040dd0259006fc3b73820    24 minutes ago      Running             machine-approver-controller    0
500d5bd97f919       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:9c441d4a8a9fdc04e0e0e3a200867bbbf2549b84d05263f648b83165452c84c7    24 minutes ago      Running             cluster-autoscaler-operator    0
c15f3eeedbce7       registry.svc.ci.openshift.org/ci-op-2s5yvys3/release@sha256:3861328a19852ff283050a2d912fc6ce9780bf30504c40215fd471e8342ddf8a   24 minutes ago      Exited              cluster-version-operator       0
9823d3777e626       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:0f51e8c6713cf23fac9b4b61d3e10e453936c139ee9a58171090b5ffe7cd37ae    24 minutes ago      Running             openvswitch                    0
0e793e822f048       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:0f51e8c6713cf23fac9b4b61d3e10e453936c139ee9a58171090b5ffe7cd37ae    24 minutes ago      Running             sdn                            0
74db667360541       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:f82a3c247a4c59538a3d40ad1a2257383420440e15c4675b2e11ad620601bf98    25 minutes ago      Running             sdn-controller                 0
e91c0edfe47b1       registry.svc.ci.openshift.org/ci-op-2s5yvys3/stable@sha256:a8aa3e53cbaeae806210878f0c7b499b636a963b2a52f4d1eea6db3dfa2fdc98    26 minutes ago      Running             cluster-network-operator       0
3e7dbeb86eaff       quay.io/coreos/etcd@sha256:688e6c102955fe927c34db97e6352d0e0962554735b2db5f2f66f3f94cfe8fd1                                    30 minutes ago      Running             etcd-member                    0
bf44c01a96713       quay.io/coreos/kube-client-agent@sha256:d68f85b5ca3adccdc2f4a4c5263f1792798ed44a9b1d63a96004b6e283dc338d                       30 minutes ago      Exited              certs                          0
45b8f6ba88eff       registry.svc.ci.openshift.org/openshift/origin-v4.0@sha256:6dcda803990164d1e26fc20911455e437afa901de46ba60b9e1c5a591e2f073b    30 minutes ago      Exited              discovery                      0
[core@ip-10-0-6-202 ~]$ sudo crictl logs c15f3eeedbce7
...
I1207 00:31:31.287266       1 reflector.go:240] Listing and watching *v1.ClusterOperator from github.com/openshift/cluster-version-operator/vendor/github.com/openshift/client-go/config/informers/externalversions/factory.go:101
E1207 00:31:41.283969       1 reflector.go:205] github.com/openshift/cluster-version-operator/vendor/github.com/openshift/client-go/config/informers/externalversions/factory.go:101: Failed to list *v1.ClusterVersion: Get https://127.0.0.1:6443/apis/config.openshift.io/v1/clusterversions?limit=500&resourceVersion=0: net/http: TLS handshake timeout
E1207 00:31:41.292257       1 reflector.go:205] github.com/openshift/cluster-version-operator/vendor/github.com/openshift/client-go/config/informers/externalversions/factory.go:101: Failed to list *v1.ClusterOperator: Get https://127.0.0.1:6443/apis/config.openshift.io/v1/clusteroperators?limit=500&resourceVersion=0: net/http: TLS handshake timeout
E1207 00:31:41.726583       1 leaderelection.go:234] error retrieving resource lock openshift-cluster-version/version: configmaps "version" is forbidden: User "system:serviceaccount:openshift-cluster-version:default" cannot get configmaps in the namespace "openshift-cluster-version": no RBAC policy matched
E1207 00:31:41.726736       1 event.go:259] Could not construct reference to: '&v1.ConfigMap{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Data:map[string]string(nil), BinaryData:map[string][]uint8(nil)}' due to: 'no kind is registered for the type v1.ConfigMap'. Will not report event: 'Normal' 'LeaderElection' 'ip-10-0-6-202_510fb14b-3e90-4980-945c-109d7d5bdfb0 stopped leading'
I1207 00:31:41.726852       1 leaderelection.go:213] failed to renew lease openshift-cluster-version/version: timed out waiting for the condition
F1207 00:31:41.728854       1 start.go:123] leaderelection lost

Looks like the Kubernetes API server died? I poked around a bit more, but nothing stuck out before CI reaped the cluster.

openshift-ci-robot · 2018-12-07T00:57:41Z

@wking: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/prow/e2e-aws	`b06805f`	link	`/test e2e-aws`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

adambkaplan · 2018-12-07T03:00:11Z

My PR openshift/builder#29 is the likely offender. The builder code originally assumed that the service signing CA was mounted in one of two places:

/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt (v3.11 location), which came from builder service account token
/var/run/configs/openshift.io/service-ca.crt (temporary 4.0 location), which was injected into a ConfigMap via the openshift/service-serviing-cert-signer annotation.
Both of these were copied into the TLS certificate root in the builder pod /etc/pki/tls/certs, allowing buildah to trust the internal registry

openshft/builder#29 got rid of the 2nd copy, and instead assumed that the service signing CA was mounted in /var/run/configs/openshift.io/certs.d/<internal-registry-host>. This got through review and CI for the following reasons:

We suspect (but cannot confirm) that the service signing CA was only being mounted via route 2 above - our code assumed that we still receiving service-ca.crt from the builder service account.
We were mounting the cluster CA from the builder service account and adding it to the system certs directory, which we thought would allow the internal registry to be trusted. (PR did not change this)
We did not have the e2e-aws CI job enabled in the origin/builder repo, which has a few build-related smoke tests enabled.

This has been reverted in openshift/builder#30

wking · 2018-12-07T03:54:52Z

/close

openshift-ci-robot · 2018-12-07T03:54:53Z

@wking: Closed this PR.

Details

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Dec 6, 2018

openshift-ci-robot requested review from abhinavdahiya and hardys December 6, 2018 07:33

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 6, 2018

wking force-pushed the pin-to-coreos-47.190 branch from aa2add7 to b06805f Compare December 6, 2018 08:28

openshift-ci-robot closed this Dec 7, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DNM: Pin to RHCOS 47.190 #801

DNM: Pin to RHCOS 47.190 #801

Uh oh!

wking commented Dec 6, 2018

Uh oh!

openshift-ci-robot commented Dec 6, 2018

Uh oh!

wking commented Dec 6, 2018

Uh oh!

wking commented Dec 6, 2018

Uh oh!

cgwalters commented Dec 6, 2018

Uh oh!

wking commented Dec 7, 2018 •

edited

Loading

Uh oh!

wking commented Dec 7, 2018

Uh oh!

wking commented Dec 7, 2018

Uh oh!

openshift-ci-robot commented Dec 7, 2018

Uh oh!

adambkaplan commented Dec 7, 2018

Uh oh!

wking commented Dec 7, 2018

Uh oh!

openshift-ci-robot commented Dec 7, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

DNM: Pin to RHCOS 47.190 #801

DNM: Pin to RHCOS 47.190 #801

Uh oh!

Conversation

wking commented Dec 6, 2018

Uh oh!

openshift-ci-robot commented Dec 6, 2018

Uh oh!

wking commented Dec 6, 2018

Uh oh!

wking commented Dec 6, 2018

Uh oh!

cgwalters commented Dec 6, 2018

Uh oh!

wking commented Dec 7, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wking commented Dec 7, 2018

Uh oh!

wking commented Dec 7, 2018

Uh oh!

openshift-ci-robot commented Dec 7, 2018

Uh oh!

adambkaplan commented Dec 7, 2018

Uh oh!

wking commented Dec 7, 2018

Uh oh!

openshift-ci-robot commented Dec 7, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

wking commented Dec 7, 2018 •

edited

Loading