Ingestion fails in Docker Compose

[avalanchy]

Ingestion fails in Docker Compose.

Affected Version

0.17.0

Description

• Cluster size: Default from Docker Compose
• Configurations in use: Default from Docker Compose
• Steps to reproduce the problem
  1. Start the docker with docker-compose -f distribution/docker/docker-compose.yml up
  2. Load data with "Example data" option wizard.
• The ingestion task will fail with IO error:

  java.io.IOException: Unable to create directory /opt/data/segments/intermediate_pushes/e6798a9c-296e-4504-a256-2c1900c41cfb
      at org.apache.commons.io.FileUtils.forceMkdir(FileUtils.java:2491) ~[commons-io-2.6.jar:2.6]
      at org.apache.druid.segment.loading.LocalDataSegmentPusher.push(LocalDataSegmentPusher.java:86) ~[druid-server-0.17.0.jar:0.17.0]
      at org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$mergeAndPush$4(AppenderatorImpl.java:791) ~[druid-server-0.17.0.jar:0.17.0]
      at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:87) ~[druid-core-0.17.0.jar:0.17.0]
      at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:115) ~[druid-core-0.17.0.jar:0.17.0]
      at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:105) ~[druid-core-0.17.0.jar:0.17.0]
      at org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:787) ~[druid-server-0.17.0.jar:0.17.0]
      at org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$push$1(AppenderatorImpl.java:657) ~[druid-server-0.17.0.jar:0.17.0]
      at com.google.common.util.concurrent.Futures$1.apply(Futures.java:713) [guava-16.0.1.jar:?]
      at com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:861) [guava-16.0.1.jar:?]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_232]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_232]
      at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
  

• A quick patch requires changing the permissions for middlemanager:

  docker-compose exec -u root middlemanager chown -R druid:druid /opt/data
  

  Probably coordinator and historical are also affected.

[jacekchalupka]

Since docker-compose.yaml mounts ./storage to /opt/data the other workaround is to precreate the folder and set it owner to 1000:1000 and 755 permissions:

mkdir storage
chmod 755 storage
chown 1000:1000 storage

I think proper solution would be to modify Dockerfile, so it would precreate a file in the same manner as it does with /opt/druid/var:

RUN addgroup -S -g 1000 druid \
 && adduser -S -u 1000 -D -H -h /opt/druid -s /bin/sh -g '' -G druid druid \
 && mkdir -p /opt/druid/var \
 && mkdir -p /opt/data \
 && chown -R druid:druid /opt \
 && chmod 775 /opt/druid/var \
 && chmod 775 /opt/data

[avalanchy]

I haven't tested but it looks like the problem is fixed by #10785.

[FrankChen021]

Resolved by #11299