Skip to content

Fix dockerfile for druid image #15264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
abhishekagarwal87 merged 3 commits into from
Nov 1, 2023
Merged

Fix dockerfile for druid image #15264

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e16da90
Fix dockerfile
Oct 26, 2023
16ddc1c
Move to java17
Oct 30, 2023
22e28b3
tag alpine
Oct 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 16 additions & 13 deletions distribution/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
# under the License.
#

ARG JDK_VERSION=11
ARG JDK_VERSION=17
Copy link
Copy Markdown
Contributor

@gianm gianm Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am good with this, and in fact I prefer it 🙂


# The platform is explicitly specified as x64 to build the Druid distribution.
Copy link
Copy Markdown
Contributor

@petermarshallio petermarshallio Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @georgew5656! I was having real issues getting this to work on my M2 ;)

Is --platform still required here?

Copy link
Copy Markdown
Contributor Author

@georgew5656 georgew5656 Oct 31, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't think this druid image actually supports multiarch images right now, when I go to the druid dockerhub i only see a amd64 image which would explain why it doesn't work on a arm based mac

Copy link
Copy Markdown
Contributor Author

@georgew5656 georgew5656 Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abhishekagarwal87 do you know how this image gets built? is it a pipeline somewhere?

Copy link
Copy Markdown
Contributor

@abhishekagarwal87 abhishekagarwal87 Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a bit of black box for me as well. Apache infra jobs builds and pushes the image as far as I know.

Copy link
Copy Markdown
Contributor

@petermarshallio petermarshallio Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I shoulda said (!) was I managed to build an arm Docker image using your fix - but yeah the official image would be nicer :)

Copy link
Copy Markdown
Contributor

@jon-wei jon-wei Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the dockerhub and the official image is being built right now from a dockerhub automated build on the tags in github:

Screen Shot 2023-10-31 at 12 55 15 PM

Copy link
Copy Markdown
Contributor

@jon-wei jon-wei Oct 31, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a way to pass in a target platform for the automated build, but it seems like it'd be possible to have another Dockerfile configured for arm64, and then we could add a new automated build rule that reads from that instead

Screen Shot 2023-10-31 at 1 07 27 PM

Copy link
Copy Markdown
Contributor

@jon-wei jon-wei Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. what this repo is showing https://github.com/cgiraldo/docker-hello-multiarch

Copy link
Copy Markdown
Contributor

@jon-wei jon-wei Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option might be to move the docker image building to a github action on tags only, have some action that calls the appropriate buildx command, and pushes a multi-arch image to dockerhub

Copy link
Copy Markdown
Contributor

@gianm gianm Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The automated dockerhub builds seem nice because then we don't have to worry about dockerhub credentials in our GHA stuff.

# This is because it's not able to build the distribution on arm64 due to dependency problem of web-console. See: https://github.com/apache/druid/issues/13012
Expand Down Expand Up @@ -49,17 +49,8 @@ RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plu
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid

FROM busybox:1.34.1-glibc as busybox

FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <dev@druid.apache.org>"

COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]

# Predefined builtin arg, see: https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope
FROM alpine:3 as bash-static
ARG TARGETARCH

#
# Download bash-static binary to execute scripts that require bash.
# Although bash-static supports multiple platforms, but there's no need for us to support all those platform, amd64 and arm64 are enough.
Expand All @@ -73,12 +64,24 @@ RUN if [ "$TARGETARCH" = "arm64" ]; then \
echo "Unsupported architecture ($TARGETARCH)" && exit 1; \
fi; \
echo "Downloading bash-static from ${BASH_URL}" \
&& wget ${BASH_URL} -O /bin/bash \
&& chmod 755 /bin/bash
&& wget ${BASH_URL} -O /bin/bash
Comment thread
abhishekagarwal87 marked this conversation as resolved.

FROM busybox:1.35.0-glibc as busybox

FROM gcr.io/distroless/java$JDK_VERSION-debian12
LABEL maintainer="Apache Druid Developers <dev@druid.apache.org>"

COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]


RUN addgroup -S -g 1000 druid \
&& adduser -S -u 1000 -D -H -h /opt/druid -s /bin/sh -g '' -G druid druid


COPY --from=bash-static /bin/bash /bin/bash
RUN chmod 755 /bin/bash

COPY --chown=druid:druid --from=builder /opt /opt
COPY distribution/docker/druid.sh /druid.sh
COPY distribution/docker/peon.sh /peon.sh
Expand Down