Inconsistent behavior for symlink file COPY compared with docker build

[ZedYeung]

Actual behavior
A clear and concise description of what the bug is.
For docker build, copy symlink would copy the actual file.

Expected behavior
A clear and concise description of what you expected to happen.
For kaniko build, copy symlink would copy the symlink file.

To Reproduce
Steps to reproduce the behavior:

1. ...FROM any image with /bin/sh(e.g. golang)
2. ...FROM scratch
3. ...COPY --from=0 /bin/sh /bin/sh

Additional Information

• Dockerfile

FROM golang:1.13
FROM gcr.io/kaniko-project/executor
COPY --from=0 /bin/sh /bin/sh

Please provide either the Dockerfile you're trying to build or one that can reproduce this error.

• Build Context
  Please provide or clearly describe any files needed to build the Dockerfile (ADD/COPY commands)
• Kaniko Image (fully qualified with digest)
• Personal thought
  Actually, /bin/sh is a symlink point to /bin/dash, but kaniko would only copy the symlink, which makes /bin/sh fail to work. In comparison, docker would copy the /bin/dash into /bin/sh.
  In kaniko, I found that the Dir symlink would be resolved here
  So far, the dir symlink would be resolved but the file symlink would be ignored. Any concern about that? Why they are treated in a different way?

Triage Notes for the Maintainers

Description	Yes/No
Please check if this a new feature you are proposing
Please check if the build works in docker but not in kaniko
Please check if this error is seen when you use --cache flag
Please check if your dockerfile is a multistage dockerfile

[wilmardo]

Was about to open the same issue but instead I will add it here for some more elaborate explanation:

libstdc++.so.6 is a symlink like so in stage 0:
/usr/lib/libstdc++.so.6 -> libstdc++.so.6.0.27

COPY --from=0 /usr/lib/libstdc++.so.6 /usr/lib/

This should result in the libstdc++.so.6.0.27 file placed as /usr/lib/libstdc++.so.6 in the current stage.

To Reproduce
Steps to reproduce the behavior:

Create the following Dockerfile

FROM alpine:3.11
RUN apk add gcc && \
    ls -l /usr/lib/libstdc++.so.6

FROM scratch
COPY --from=0 /usr/lib/libstdc++.so.6 /usr/lib/

Run kaniko:

# /kaniko/executor --no-push --context=$(pwd)
INFO[0000] Resolved base name alpine:3.11 to alpine:3.11
INFO[0000] Resolved base name scratch to scratch
INFO[0000] Resolved base name alpine:3.11 to alpine:3.11
INFO[0000] Resolved base name scratch to scratch
INFO[0000] Retrieving image manifest alpine:3.11
INFO[0001] Retrieving image manifest alpine:3.11
INFO[0002] Built cross stage deps: map[0:[/usr/lib/libstdc++.so.6]]
INFO[0002] Retrieving image manifest alpine:3.11
INFO[0003] Retrieving image manifest alpine:3.11
INFO[0009] Unpacking rootfs as cmd RUN apk add gcc &&     ls -l /usr/lib/libstdc++.so.6 requires it.
INFO[0009] Taking snapshot of full filesystem...
INFO[0009] RUN apk add gcc &&     ls -l /usr/lib/libstdc++.so.6
INFO[0009] cmd: /bin/sh
INFO[0009] args: [-c apk add gcc &&     ls -l /usr/lib/libstdc++.so.6]
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/10) Installing libgcc (9.2.0-r3)
(2/10) Installing libstdc++ (9.2.0-r3)
(3/10) Installing binutils (2.33.1-r0)
(4/10) Installing gmp (6.1.2-r1)
(5/10) Installing isl (0.18-r0)
(6/10) Installing libgomp (9.2.0-r3)
(7/10) Installing libatomic (9.2.0-r3)
(8/10) Installing mpfr4 (4.0.2-r1)
(9/10) Installing mpc1 (1.1.0-r1)
(10/10) Installing gcc (9.2.0-r3)
Executing busybox-1.31.1-r8.trigger
OK: 101 MiB in 24 packages
lrwxrwxrwx    1 root     root            19 Jan 13 19:11 /usr/lib/libstdc++.so.6 -> libstdc++.so.6.0.27
INFO[0015] Taking snapshot of full filesystem...
INFO[0017] Saving file /usr/lib/libstdc++.so.6 for later use.
INFO[0017] Deleting filesystem...
INFO[0018] No base image, nothing to extract
INFO[0018] Unpacking rootfs as cmd COPY --from=0 /usr/lib/libstdc++.so.6 /usr/lib/ requires it.
INFO[0018] Taking snapshot of full filesystem...
INFO[0018] COPY --from=0 /usr/lib/libstdc++.so.6 /usr/lib/
error building image: error building stage: failed to execute command: symlink libstdc++.so.6.0.27 /usr/lib/libstdc++.so.6: no such file or directory

For reference the docker build output:

# docker build . -t issue
Sending build context to Docker daemon  944.1kB
Step 1/4 : FROM alpine:3.11
 ---> c85b8f829d1f
Step 2/4 : RUN apk add gcc &&     ls -l /usr/lib/libstdc++.so.6
 ---> Running in 7b66ae1b432e
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/10) Installing libgcc (9.2.0-r3)
(2/10) Installing libstdc++ (9.2.0-r3)
(3/10) Installing binutils (2.33.1-r0)
(4/10) Installing gmp (6.1.2-r1)
(5/10) Installing isl (0.18-r0)
(6/10) Installing libgomp (9.2.0-r3)
(7/10) Installing libatomic (9.2.0-r3)
(8/10) Installing mpfr4 (4.0.2-r1)
(9/10) Installing mpc1 (1.1.0-r1)
(10/10) Installing gcc (9.2.0-r3)
Executing busybox-1.31.1-r8.trigger
OK: 101 MiB in 24 packages
lrwxrwxrwx    1 root     root            19 Jan 13 19:19 /usr/lib/libstdc++.so.6 -> libstdc++.so.6.0.27
Removing intermediate container 7b66ae1b432e
 ---> 2be48046ec6f
Step 3/4 : FROM scratch
 --->
Step 4/4 : COPY --from=0 /usr/lib/libstdc++.so.6 /usr/lib/
 ---> 5284a8f4ed9d
Successfully built 5284a8f4ed9d
Successfully tagged issue:latest

Thanks @ZedYeung for opening and if I could help in some way to move this forwarde @tejal29 please let me know!

[tejal29]

Thanks @ZedYeung and @wilmardo for all the information.
I was able to track this down and have a PR out.

The major changes are.

1. For cross stage dependency symlinks, we did not copy the target source. The temporary workspace, only had a dead symlink.
2. When copying a symlink from one stage to another, we know also copy the target.

Thanks
Tejal

[gokuatkai]

Hello, I am using version 0.23.0 and I am facing this issue too, Docker build copy the targetfile, but kaniko build just copy the symlink file. Was it released yet?