etcd: Use gcr.io/etcd-development/etcd as source

[hakman]

gcr.io/k8s-staging-etcd is not actually needed when promoting etcd images, as the primary mirror is gcr.io/etcd-development/etcd. This should simplify the release process.
As proof of concept, also promoting v3.5.23:

% crane digest gcr.io/etcd-development/etcd:v3.5.23
sha256:e1864730b4b0979f669c1b3b5b0f378b8df2a394f7a22ea36577d4a8ae9e1204

% crane manifest gcr.io/etcd-development/etcd:v3.5.23
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 3650,
         "digest": "sha256:35d9aa2b8584643ff795b5670c4d821eb3a9ae1ebfe12410246dc75d9183b39e",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 3650,
         "digest": "sha256:88b5e48236125ae7b5159d81500dcbe12736c6b4cc2b13750fd5c02943cdbfbc",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 3650,
         "digest": "sha256:2965c60817c5ca1cb138198cc40fdffa6dae577eaa051e1a6b2d134670d11b7f",
         "platform": {
            "architecture": "ppc64le",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 3650,
         "digest": "sha256:4375c7dfe98bb1d01a1b1571b494ad4e88fdfe419e3de0a43632ab320956ebae",
         "platform": {
            "architecture": "s390x",
            "os": "linux"
         }
      }
   ]
}

/cc @ahrtr @joshjms @ameukam

[hakman]

/hold for comments

[joshjms]

Oh wow you can do that. I suppose this will simplify things a lot more (so we don't have to push to registry.k8s.io at all hmm).

[joshjms]

/approve

[joshjms]

Actually, should we make adding the registry and promoting the images a different commit (or PR)?

[hakman]

Oh wow you can do that. I suppose this will simplify things a lot more (so we don't have to push to registry.k8s.io at all hmm).

No, you just won't need to push to gcr.io/k8s-staging-etcd. The promo PRs will still be needed to push to registry.k8s.io/etcd.

Actually, should we make adding the registry and promoting the images a different commit (or PR)?

Don't actually have a preference on this, single PR was simpler and proves that it works.

[ameukam]

This could have been a single image with multi-arch.

[hakman]

It also that, I think the plan is to drop the *-arch images in the next minor release.
If registry.k8s.io is supposed to be mirror, has to have all tags available via the main mirror.

[ahrtr]

Question: some places are using registry.k8s.io/etcd, should we keep them consistent?

see kubernetes/kubernetes#134251

[hakman]

@ahrtr Not sure what you mean. This change would just meant that you don't have to push images to gcr.io/etcd-development/etcd before creating promo PRs. gcr.io/etcd-development/etcd would be used as staging bucket`.

[ahrtr]

you don't have to push images to gcr.io/etcd-development/etcd

You mean gcr.io/k8s-staging-etcd? thx for the clarification in etcd-io/etcd#20756 (comment). I didn't know the details/workflow.

It seems that it makes etcd as a special case as I see that all other images are using gcr.io/k8s-staging-*.

I understood your intention now, but I am not sure what's the impact.

[hakman]

The other projects use registry.k8s.io as the main and, in most cases, only mirror.
There is no value in using gcr.io/k8s-staging-* when you already have a gcr.io/etcd-development with all images published there.
Obviously, you can switch to using gcr.io/k8s-staging-etcd at any time in the future if you decide to have a different release workflow.

[ahrtr]

Based on the discussion in etcd-io/etcd#20756, we don't need to build etcd image using cluster/images/etcd anymore, instead we use etcd officially released image directly. So no image will be pushed to gcr.io/k8s-staging-etcd any more. So we have to use gcr.io/etcd-development/etcd as the source here?

[hakman]

Yes, or push manually or with a script to gcr.io/k8s-staging-etcd, though, I don't think there's much value in that.

[ahrtr]

OK, makes sense now. thx again for the clafication.

[ahrtr]

The registry is gcr.io/etcd-development/etcd, I assume the suffix /etcd will automatically added somewhere.

[hakman]

Should be coming from here:

k8s.io/registry.k8s.io/images/k8s-staging-etcd/images.yaml

Line 1 in 2c5d008

- name: etcd

[ahrtr]

/lgtm
/approve

/hold for others to review

Please feel free to cancel the hold.

Thanks

[ameukam]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahrtr, ameukam, hakman, joshjms

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ameukam]
• registry.k8s.io/images/k8s-staging-etcd/OWNERS [ahrtr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hakman]

Let's see if there's anything else needed to adjust.
/unhold

[hakman]

Looks like it worked just fine:

$ crane ls registry.k8s.io/etcd | grep 3.5.23
3.5.23-0
v3.5.23
v3.5.23-amd64
v3.5.23-arm64
v3.5.23-ppc64le
v3.5.23-s390x

[upodroid]

Hi All

Can we have this PR reverted, particularly the image registry change? We need to start publishing images via Prow into GCR. Images are built and pushed manually which we need to fix ASAP.

Also, gcr.io/etcd-development/* should be deprecated after the next minor release and removed after a year.

[ahrtr]

Also, gcr.io/etcd-development/* should be deprecated after the next minor release and removed after a year.

You are proposing to remove a registry which has been used by users for about 10 years!

[upodroid]

You are proposing to remove a registry which has been used by users for about 10 years!

Yes(but not right now), the Kubernetes project went through the same exercise a few years ago and built a cost-efficient image registry.

The good news is that gcr.io/etcd-development/* doesn't cost a lot to serve today; hopefully its cost should reduce as we direct users to registry.k8s.io.

[marosset]

This image doesn't have Windows flavors (which are used in some e2e tests)

Previously etcd images published to registry.k8s.io did include windows flavors.

regctl manifest get registry.k8s.io/etcd:3.6.4-0
Name:        registry.k8s.io/etcd:3.6.4-0
MediaType:   application/vnd.docker.distribution.manifest.list.v2+json
Digest:      sha256:e36c081683425b5b3bc1425bc508b37e7107bb65dfa9367bf5a80125d431fa19

Manifests:

  Name:      registry.k8s.io/etcd:3.6.4-0@sha256:71170330936954286be203a7737459f2838dd71cc79f8ffaac91548a9e079b8f
  Digest:    sha256:71170330936954286be203a7737459f2838dd71cc79f8ffaac91548a9e079b8f
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64

  Name:      registry.k8s.io/etcd:3.6.4-0@sha256:867ecac79776bf83ce7dee030a3b14eaa4a1cda2898df7e25ed3524a9f809fd8
  Digest:    sha256:867ecac79776bf83ce7dee030a3b14eaa4a1cda2898df7e25ed3524a9f809fd8
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v7

  Name:      registry.k8s.io/etcd:3.6.4-0@sha256:5db83f9e7ee85732a647f5cf5fbdf85652afa8561b66c99f20756080ebd82ea5
  Digest:    sha256:5db83f9e7ee85732a647f5cf5fbdf85652afa8561b66c99f20756080ebd82ea5
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm64

  Name:      registry.k8s.io/etcd:3.6.4-0@sha256:8fbb16da31eb870d31b541e591b89504125373cc4e5d682bf6214ad08eb376c6
  Digest:    sha256:8fbb16da31eb870d31b541e591b89504125373cc4e5d682bf6214ad08eb376c6
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/ppc64le

  Name:      registry.k8s.io/etcd:3.6.4-0@sha256:14a4b7ef3df0910c311b5a89f4c2e4fa6270717a2a6b9271b810e770a26b9ac1
  Digest:    sha256:14a4b7ef3df0910c311b5a89f4c2e4fa6270717a2a6b9271b810e770a26b9ac1
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/s390x

  Name:      registry.k8s.io/etcd:3.6.4-0@sha256:7682a4e72f88f7a0546a78befbf848810527a30b93b729936bdda59dc03ef8cc
  Digest:    sha256:7682a4e72f88f7a0546a78befbf848810527a30b93b729936bdda59dc03ef8cc
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  windows/amd64
  OSVersion: 10.0.17763.7558

  Name:      registry.k8s.io/etcd:3.6.4-0@sha256:314419e0383b72dfb740986b8cea10e8c4e44f5eab528ef1d5d26133b92d5320
  Digest:    sha256:314419e0383b72dfb740986b8cea10e8c4e44f5eab528ef1d5d26133b92d5320
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  windows/amd64
  OSVersion: 10.0.20348.3932

cc @zylxjtu

[hakman]

Can we have this PR reverted, particularly the image registry change? We need to start publishing images via Prow into GCR. Images are built and pushed manually which we need to fix ASAP.

@upodroid gcr.io/etcd-development/* is currently the official/main mirror. I understand that it's not ideal, but there's no value in reverting the change until the whole release process changes and registry.k8s.io/etcd becomes the primary mirror.