Skip to content

[Task]: Migrate from archived kaniko for building SDK container images #35269

@RyuSA

Description

@RyuSA

What needs to happen?

GoogleContainerTools/kaniko was recently publicly archived.
GoogleContainerTools/kaniko#3502

I remembered that Apache Beam uses kaniko for its SDK builds, so I checked the repository and confirmed it is still being used in one place for building the SDK container image:
https://github.com/apache/beam/blob/v2.65.0/sdks/python/apache_beam/runners/portability/sdk_container_builder.py#L250-L252

While I haven't heard anything about the Kaniko container repository being deleted, it might be a good idea to migrate to a different tool sooner rather than later IF the pipeline is still in use.

For reference, the kaniko community has suggested podman, buildah, or buildKit as potential migration paths. Both are tools for building OCI images and seem like viable alternatives to kaniko.
GoogleContainerTools/kaniko#3348

BuildKit provides a build environment within a container image (Note: it requires privileged access).
https://hub.docker.com/r/moby/buildkit

Next steps:

  • Confirm if this pipeline is still actively in use.
  • Outline the requirements for the replacement tool.

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions