From 6252fc6db0f5bc4aa9ecdeab55a19070d01d46d7 Mon Sep 17 00:00:00 2001 From: Fan Yang Date: Thu, 2 Oct 2025 13:48:35 +0200 Subject: [PATCH 1/3] Add troubleshoot volume mounts for ephemeral container --- .../debugging-distroless-images.md | 51 +++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/content/chainguard/chainguard-images/troubleshooting/debugging-distroless-images.md b/content/chainguard/chainguard-images/troubleshooting/debugging-distroless-images.md index c68f8bdea5..d0c317c443 100644 --- a/content/chainguard/chainguard-images/troubleshooting/debugging-distroless-images.md +++ b/content/chainguard/chainguard-images/troubleshooting/debugging-distroless-images.md @@ -143,6 +143,57 @@ tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN To facilitate accessing processes running in other containers without having to specify a target, you may enable [process namespace sharing](https://kubernetes.io/docs/tasks/configure-pod-container/share-process-namespace/) within your Pod setup. It's also worth noting that the filesystem may not be accessible due to default user permissions. +### Troubleshooting Volume Mounts + +Kubernetes starts the ephemeral container without mounting the same volumes in the target container. As a result, directories like `/var/log`, which are backed by a volume in the original container, will appear empty or missing in the ephemeral container. + +Kubernetes does not currently offer a built-in way to automatically replicate volume mounts from the main container to the ephemeral debug container. There is ongoing discussion in the Kubernetes community regarding this limitation, including proposals for features like `--with-volume-mounts[=]` which would allow mounting the same volumes in ephemeral containers. This feature is not yet available. + +To work around this issue, you can manually create a copy of the pod definition with a new debug container that replicates the necessary volume mounts. + +Proxy the Kubernetes API: +``` +kubectl proxy +``` + +Patch the pod to add an ephemeral container: +``` +curl http://localhost:8001/api/v1/namespaces/default/pods//ephemeralcontainers \ +-X PATCH \ +-H 'Content-Type: application/strategic-merge-patch+json' \ +-d ' +{ + "spec": { + "ephemeralContainers": [ + { + "name": "debugger", + "command": ["sh"], + "image": "cgr.dev/chainguard/wolfi-base", + "targetContainerName": "", + "stdin": true, + "tty": true, + "volumeMounts": [ + { + "mountPath": "/var/log", + "name": "varlog", + "readOnly": true + } + ] + } + ] + } +}' +``` +- Replace `` and `` with your actual values. +- Be sure to match the `volumeMounts` spec to those in your target container. + +Attach to the debug container: +``` +kubectl attach -c debugger -ti +``` + +In the ephemeral container shell, you should now see the expected contents within `/var/log` or other paths you’ve mounted. + For more strategies on how to debug production distroless containers, check the [Kubernetes documentation on debugging running Pods](https://kubernetes.io/docs/tasks/debug/debug-application/debug-running-pod/). ## Resources to Learn More From 2c6738334c74a2c947f0741776540e94d669cc07 Mon Sep 17 00:00:00 2001 From: Fan Yang Date: Thu, 5 Mar 2026 17:36:56 +0100 Subject: [PATCH 2/3] update CA FAQ based on a custmer ticket --- .../chainguard/chainguard-images/features/ca-docs/faq.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/content/chainguard/chainguard-images/features/ca-docs/faq.md b/content/chainguard/chainguard-images/features/ca-docs/faq.md index e7855eb740..7a86daca99 100644 --- a/content/chainguard/chainguard-images/features/ca-docs/faq.md +++ b/content/chainguard/chainguard-images/features/ca-docs/faq.md @@ -98,6 +98,15 @@ Custom Assembly provides the tools and defaults for secure image creation — bu {{< blurb/why_ca >}} +## How do tags work for a custom assembly image (base image vs. added packages)? + +For a custom assembly image based on a Chainguard image: + +* The semantic tag (for example, vX.Y.Z) always reflects the base image version. +* Any added packages do not get their own version tags. +* When an added package is updated and available, your custom assembly is rebuilt automatically, and the latest moves to the new digest that includes the updated package. +* The semantic tag remains tied to the same base image version until the base image itself is updated and a new semantic tag is created. + ## Custom Assembly Troubleshooting Build failures can occur for a number of reasons, including the following: From deb733078a96498e0a2126bdaeb256df4a980f57 Mon Sep 17 00:00:00 2001 From: Mark Drake <33191761+SharpRake@users.noreply.github.com> Date: Thu, 5 Mar 2026 08:52:26 -0800 Subject: [PATCH 3/3] Update content/chainguard/chainguard-images/features/ca-docs/faq.md Signed-off-by: Mark Drake <33191761+SharpRake@users.noreply.github.com> --- .../chainguard/chainguard-images/features/ca-docs/faq.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/content/chainguard/chainguard-images/features/ca-docs/faq.md b/content/chainguard/chainguard-images/features/ca-docs/faq.md index 7a86daca99..326bb5afe0 100644 --- a/content/chainguard/chainguard-images/features/ca-docs/faq.md +++ b/content/chainguard/chainguard-images/features/ca-docs/faq.md @@ -100,12 +100,9 @@ Custom Assembly provides the tools and defaults for secure image creation — bu ## How do tags work for a custom assembly image (base image vs. added packages)? -For a custom assembly image based on a Chainguard image: +For a Custom Assembly image, the semantic tag (for example, `vX.Y.Z`) always reflects the version of the Chainguard container image on which it is based. Any added packages do not get their own version tags. -* The semantic tag (for example, vX.Y.Z) always reflects the base image version. -* Any added packages do not get their own version tags. -* When an added package is updated and available, your custom assembly is rebuilt automatically, and the latest moves to the new digest that includes the updated package. -* The semantic tag remains tied to the same base image version until the base image itself is updated and a new semantic tag is created. +When an added package is updated and available, your Custom Assembly image is rebuilt automatically and the new build that includes the updated package becomes the `latest` image. The semantic tag remains tied to the same base image digest until the base image itself is updated and a new semantic tag is created. ## Custom Assembly Troubleshooting