HDDS-14452. Created a single container ozone quickstart image #49
Conversation
|
Hi @adoroszlai, I have created this PR against the "latest" branch for now. |
ptlrs
force-pushed
the
ozone-single-container
branch
from
00547c9 to
60061fe
Compare
adoroszlai
left a comment
adoroszlai
left a comment
There was a problem hiding this comment.
Thanks @ptlrs for working on this.
Do you think we should create a "quickstart" branch for this image so that it can live separately?
It definitely needs a separate branch, but I'd prefer all-in-one as Docker tag, hence ozone-all-in-one as branch name.
| --build-arg OZONE_VERSION \ | ||
| -t apache/ozone:dev \ | ||
| $@ - < Dockerfile | ||
| -t apache/ozone-quickstart:latest \ |
There was a problem hiding this comment.
apache/ozone-quickstart repository does not exist, but this build.sh is only for developers' local build, so it can use any tag.
| ARG OZONE_VERSION=2.1.0 | ||
| ARG OZONE_URL="https://www.apache.org/dyn/closer.lua?action=download&filename=ozone/${OZONE_VERSION}/ozone-${OZONE_VERSION}.tar.gz" | ||
| # Environment variables for Ozone configuration | ||
| ENV CORE-SITE.XML_fs.defaultFS=ofs://localhost \ |
There was a problem hiding this comment.
To simplify things, we can add these as actual config files (ozone-site.xml, core-site.xml, etc.) instead of environment variables.
| mkdir -p /data/metadata /data/hdds /var/log/hadoop | ||
| chown -R hadoop:hadoop /data/metadata /data/hdds /var/log/hadoop | ||
| chmod -R 755 /data |
There was a problem hiding this comment.
Can this be done in Dockerfile instead?
| -p 9876:9876 \ | ||
| -p 9874:9874 \ | ||
| -p 19864:19864 \ | ||
| -p 9878:9878 \ | ||
| -p 9888:9888 \ | ||
| -p 14000:14000 \ |
There was a problem hiding this comment.
I think only S3 Gateway and Recon ports should be included in this quickstart.
| --name ozone \ | ||
| -p 9876:9876 -p 9874:9874 -p 19864:19864 \ | ||
| -p 9878:9878 -p 9888:9888 -p 14000:14000 \ | ||
| -e OZONE-SITE.XML_ozone.server.default.replication=3 \ |
There was a problem hiding this comment.
This all-in-one config has a single datanode. Default replication 3 is setting up for failure.
| -v ozone-metadata:/data/metadata \ | ||
| -v ozone-hdds:/data/hdds \ | ||
| -v ozone-logs:/var/log/hadoop \ | ||
| apache/ozone-quickstart:latest |
There was a problem hiding this comment.
apache/ozone-quickstart does not exist. I'd prefer to tag this image as all-in-one, so this would be apache/ozone:all-in-one.
| ARG OZONE_RUNNER_IMAGE=apache/ozone-runner | ||
| ARG OZONE_RUNNER_VERSION=20260106-1-jdk21 | ||
| ARG OZONE_VERSION=2.1.0 | ||
| ARG OZONE_RUNNER_IMAGE=apache/ozone |
There was a problem hiding this comment.
OZONE_RUNNER_IMAGE is overridden by GitHub Actions workflow with ghcr.io/apache/ozone-runner (to use GitHub-local base image in CI), so this won't work.
Please rename the variables to OZONE_IMAGE and OZONE_IMAGE_VERSION here, in build.sh and in .github/workflows/build.yaml.
| # Expose all service ports | ||
| # SCM ports | ||
| EXPOSE 9876 9860 | ||
| # OM ports | ||
| EXPOSE 9874 9862 | ||
| # DataNode ports | ||
| EXPOSE 19864 9882 | ||
| # S3 Gateway ports | ||
| EXPOSE 9878 19878 | ||
| # Recon ports | ||
| EXPOSE 9888 | ||
| # HttpFS ports | ||
| EXPOSE 14000 |
There was a problem hiding this comment.
EXPOSE is not functional, just documentation. Since this is for quickstart, I don't think we should expose internal ports like Datanode, OM, SCM.
|
|
||
| ## What is the Quickstart Image? | ||
|
|
||
| The quickstart image is a single-container version of Apache Ozone that runs all services (SCM, OM, DataNode, S3 Gateway, Recon, HttpFS) in one container. It's designed for: |
There was a problem hiding this comment.
| The quickstart image is a single-container version of Apache Ozone that runs all services (SCM, OM, DataNode, S3 Gateway, Recon, HttpFS) in one container. It's designed for: | |
| The quickstart image is a single-container version of Apache Ozone that runs all services (SCM, OM, DataNode, S3 Gateway, Recon, HttpFS). It's designed for: |
There was a problem hiding this comment.
I think this doc should replace README.md (and consider removing or updating CONTRIBUTING.md on this branch).
2 participants
Created a docker image based off of the slim Ozone image.
This image will provide a single container with all ozone components up and running.
This will be useful for quick POCs with Ozone.
https://issues.apache.org/jira/browse/HDDS-14452
Thanks @ssulav and @SaketaChalamchala for co-authoring this.