Skip to content

Fix permission problems in docker #11299

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
clintropolis merged 7 commits into from
Jun 2, 2021
Merged

Fix permission problems in docker #11299

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
eafe977
Create /opt/data to fix permission problem
FrankChen021 May 25, 2021
4c1556a
eliminate symlink to avoid compatibility problem on AWS Fargate
FrankChen021 May 25, 2021
60038ad
Add a workaround section
FrankChen021 May 25, 2021
be1336a
Update instruction for named volume
FrankChen021 May 26, 2021
28e4b80
Use named volume in docker-compose
FrankChen021 May 28, 2021
51ba67d
Revert some doc change
FrankChen021 May 28, 2021
c8a1229
Resolve review comments
FrankChen021 May 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions distribution/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluat
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& ln -s /opt/apache-druid-${VERSION} /opt/druid
&& mv /opt/apache-druid-${VERSION} /opt/druid

FROM amd64/busybox:1.30.0-glibc as busybox

Expand All @@ -56,9 +56,13 @@ RUN addgroup -S -g 1000 druid \

COPY --chown=druid:druid --from=builder /opt /opt
COPY distribution/docker/druid.sh /druid.sh
RUN mkdir /opt/druid/var \
&& chown druid:druid /opt/druid/var \
&& chmod 775 /opt/druid/var

# create necessary directories which could be mounted as volume
# /opt/druid/var is used to keep individual files(e.g. log) of each Druid service
# /opt/shared is used to keep segments and task logs shared among Druid services
RUN mkdir /opt/druid/var /opt/shared \
&& chown druid:druid /opt/druid/var /opt/shared \
&& chmod 775 /opt/druid/var /opt/shared

USER druid
VOLUME /opt/druid/var
Expand Down
7 changes: 4 additions & 3 deletions distribution/docker/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ volumes:
broker_var: {}
coordinator_var: {}
router_var: {}
druid_shared: {}


services:
Expand All @@ -51,7 +52,7 @@ services:
image: apache/druid:0.22.0
container_name: coordinator
volumes:
- ./storage:/opt/data
- druid_shared:/opt/shared
- coordinator_var:/opt/druid/var
depends_on:
- zookeeper
Expand Down Expand Up @@ -83,7 +84,7 @@ services:
image: apache/druid:0.22.0
container_name: historical
volumes:
- ./storage:/opt/data
- druid_shared:/opt/shared
- historical_var:/opt/druid/var
depends_on:
- zookeeper
Expand All @@ -100,7 +101,7 @@ services:
image: apache/druid:0.22.0
container_name: middlemanager
volumes:
- ./storage:/opt/data
- druid_shared:/opt/shared
- middle_var:/opt/druid/var
depends_on:
- zookeeper
Expand Down
6 changes: 3 additions & 3 deletions distribution/docker/environment
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,12 @@ druid_metadata_storage_connector_password=FoolishPassword
druid_coordinator_balancer_strategy=cachingCost

druid_indexer_runner_javaOptsArray=["-server", "-Xmx1g", "-Xms1g", "-XX:MaxDirectMemorySize=3g", "-Duser.timezone=UTC", "-Dfile.encoding=UTF-8", "-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager"]
druid_indexer_fork_property_druid_processing_buffer_sizeBytes=268435456
druid_indexer_fork_property_druid_processing_buffer_sizeBytes=256MiB

druid_storage_type=local
druid_storage_storageDirectory=/opt/data/segments
druid_storage_storageDirectory=/opt/shared/segments
druid_indexer_logs_type=file
druid_indexer_logs_directory=/opt/data/indexing-logs
druid_indexer_logs_directory=/opt/shared/indexing-logs

druid_processing_numThreads=2
druid_processing_numMergeBuffers=2
Expand Down
6 changes: 5 additions & 1 deletion docs/tutorials/docker.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,11 @@ The Druid source code contains [an example `docker-compose.yml`](https://github.

### Compose file

The example `docker-compose.yml` will create a container for each Druid service, as well as Zookeeper and a PostgreSQL container as the metadata store. Deep storage will be a local directory, by default configured as `./storage` relative to your `docker-compose.yml` file, and will be mounted as `/opt/data` and shared between Druid containers which require access to deep storage. The Druid containers are configured via an [environment file](https://github.com/apache/druid/blob/{{DRUIDVERSION}}/distribution/docker/environment).
The example `docker-compose.yml` will create a container for each Druid service, as well as ZooKeeper and a PostgreSQL container as the metadata store.

It will also create a named volumes `druid_shared`, which is mounted as `opt/shared` in container, as deep storage to keep and share segments and task logs among Druid services.

The Druid containers are configured via an [environment file](https://github.com/apache/druid/blob/{{DRUIDVERSION}}/distribution/docker/environment).

### Configuration

Expand Down