HDDS-11631. Add Run with Docker Compose guide #101

ptlrs · 2024-10-31T05:46:54Z

What changes were proposed in this pull request?

Added Run with Docker Compose guide to the Developer Guide section

Please describe your PR in detail:

Added a Run with Docker compose guide
Added new keywords to fix spelling linter errors
Added a placeholder page to document docker/docker-compose related issues
Added mermaid package to render images via pnpm add which updated package.json and pnpm-lock.yaml
Updated the pnpm version to fix incompatible lock file version error when running docker-compose

What is the link to the Apache Jira?

https://issues.apache.org/jira/browse/HDDS-11631

How was this patch tested?

Ran the website locally via docker-compose
CI:
~~https://github.com/ptlrs/ozone-site/actions/runs/11606604449~~
~~https://github.com/ptlrs/ozone-site/actions/runs/11845847017~~
~~https://github.com/ptlrs/ozone-site/actions/runs/12627156091~~
~~https://github.com/ptlrs/ozone-site/actions/runs/12627665227~~
https://github.com/ptlrs/ozone-site/actions/runs/12705583595

Ran the website locally:

errose28

Thanks for adding this @ptlrs. There's some more information we can add to this page too:

Provide a quick summary of the various compose clusters available for HA and security.
Explain how to make configuration changes with the docker-config file.

cspell.yaml

docs/06-troubleshooting/15-docker.md

docs/08-developer-guide/02-run/02-docker-compose.md

errose28 · 2024-11-06T00:17:41Z

package.json

-    "@docusaurus/preset-classic": "3.3.2",
    "@docusaurus/plugin-pwa": "3.3.2",
+    "@docusaurus/preset-classic": "3.3.2",
+    "@docusaurus/theme-mermaid": "3.3.2",


I checked out this change and ran docker compse up --build to build an image with the new dependencies but got this error. Not sure exactly what the cause is:

ozone-site-site-1 | ozone-site-site-1 | > doc-site@0.1.0 start /ozone-site ozone-site-site-1 | > docusaurus start --port 3001 "--host" "0.0.0.0" ozone-site-site-1 | ozone-site-site-1 | [INFO] Starting the development server... ozone-site-site-1 | ozone-site-site-1 | [ERROR] Error: Docusaurus was unable to resolve the "@docusaurus/theme-mermaid" theme. Make sure one of the following packages are installed: ozone-site-site-1 | - @docusaurus/theme-mermaid ozone-site-site-1 | - @docusaurus/docusaurus-theme-theme-mermaid ozone-site-site-1 | at resolveModuleName (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/moduleShorthand.js:41:15) ozone-site-site-1 | at normalizeShorthand (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/configs.js:76:60) ozone-site-site-1 | at /ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/configs.js:90:63 ozone-site-site-1 | at Array.map (<anonymous>) ozone-site-site-1 | at loadPluginConfigs (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/configs.js:90:48) ozone-site-site-1 | at async initPlugins (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/init.js:42:27) ozone-site-site-1 | at async /ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/plugins.js:136:36 ozone-site-site-1 | at async loadSite (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/site.js:127:45) ozone-site-site-1 | at async createReloadableSite (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/commands/start/utils.js:46:16) ozone-site-site-1 | at async Command.start (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/commands/start/start.js:22:28) ozone-site-site-1 | [INFO] Docusaurus version: 3.3.2 ozone-site-site-1 | Node version: v20.18.0 ozone-site-site-1 | ELIFECYCLE Command failed with exit code 1. ozone-site-site-1 exited with code 1

I saw that I faced the same problem after uninstalling docusaurus/theme-mermaid.
I was able to resolve it by reinstalling the package again on my mac.
Next, I ran docker compse up --build and then ran it with no issues.

errose28 · 2024-11-06T00:18:07Z

package.json

@@ -1,5 +1,5 @@
 {
-  "packageManager": "pnpm@8.7.1",
+  "packageManager": "pnpm@9.12.3",


Can you clarify why we need the pnpm version bump? Is mermaid incompatible with older versions?

With pnpm@8.7.1, running docker-compose fails with the following error:

0.280 ! Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-8.7.1.tgz 1.109 ERR_PNPM_LOCKFILE_BREAKING_CHANGE Lockfile /ozone-site/pnpm-lock.yaml not compatible with current pnpm 1.109 1.109 Run with the --force parameter to recreate the lockfile. ------ failed to solve: process "/bin/sh -c pnpm install --prod --frozen-lockfile" did not complete successfully: exit code: 1 `docker-compose` process finished with exit code 17

I added the docusaurus/theme-mermaid package via pnpm add @docusaurus/theme-mermaid@3.3.2 on my mac. This updated the pnpm-lock.yaml and the package.json files.

@ptlrs can you run pnpm --version?
One reason for this error would be because the lockfile was generated on my machine running v8 and yours might have installed v9.
The reason for using v8 was because Ozone dist also uses pnpm v8 and hence local testing becomes easier when working on both the repos

errose28 · 2024-11-06T00:30:30Z

docs/08-developer-guide/02-run/02-docker-compose.md

+
+This will pull the official Apache Ozone images from Docker Hub.
+
+## Architecture Diagram


To avoid duplicate content I think we should leave this information for the page dedicated to Ozone architecture. Default ports will be covered in docs/administrator-guide/configuration/basic/network/. This may mean we don't need to add mermaid in this PR.

As a new user, it is not immediately apparent how the Ozone services will be divvied across containers. It could be the case that all Ozone services run within one container. There are several possible configurations.

I added the mermaid diagram to visualize how many containers will be available and how each Ozone service maps to those containers.

It just so happens that each Ozone service is a separate container and the container diagram looks similar to what could easily have been an Ozone architecture diagram. Perhaps the title and wording add to the confusion.

If it is still a cause of confusion, I can remove the diagram. Not a problem.

docs/08-developer-guide/02-run/02-docker-compose.md

adoroszlai

I suggest writing the page on Docker for users first. We should refer developers to that guide for basics.

errose28 · 2024-11-14T22:52:17Z

Good suggestion Attila. Rishabh and I talked on a call and he had the idea of creating a new compose file with all the configs baked so that new users can pull just that one file instead of a whole release tarball where the configs live outside the docker compose definition. For now when writing the docker quick start we can refer to the current process of pulling a full release though.

ptlrs · 2025-01-06T05:30:03Z

I suggest writing the page on Docker for users first. We should refer developers to that guide for basics.

The PR #122 adds the Try Ozone With Docker page

ptlrs · 2025-01-24T08:31:16Z

Hi @adoroszlai, @errose28 could you please review this PR now that #122 is merged. Thanks.

ptlrs · 2025-04-30T17:12:25Z

Hi @kerneltime, @errose28, @adoroszlai. Could you please review this PR?

jojochuang

lgtm

ptlrs · 2025-12-16T19:38:21Z

Thank you @jojochuang for updating this PR and merging it in.
Thank you @errose28 @adoroszlai @devabhishekpal for the reviews.

github-actions bot added the website-v2 Tasks for the new ozone website targeting the HDDS-9225-website-v2 branch label Oct 31, 2024

errose28 self-requested a review November 1, 2024 17:31

errose28 added the docs Changes updating documentation on the website label Nov 1, 2024

errose28 reviewed Nov 6, 2024

View reviewed changes

ptlrs requested a review from errose28 November 14, 2024 21:32

adoroszlai reviewed Nov 14, 2024

View reviewed changes

ptlrs requested a review from adoroszlai January 6, 2025 05:35

adoroszlai mentioned this pull request Feb 6, 2025

HDDS-10794. Update docker compose user doc apache/ozone#7822

Merged

ptlrs and others added 9 commits December 16, 2025 10:24

HDDS-11631.Added Running Ozone From Docker Compose guide

d652f2f

HDDS-11631. Fix linter errors

70f5bbc

HDDS-11631. Fix review comments

c0d5ccd

HDDS-11631. Fix review comments

2fb97c6

HDDS-11631. Link to the Try Ozone with Docker page

9832c14

HDDS-11631. Fix review comments

3bb4ad0

HDDS-11631. Fix review comments

8f64f57

HDDS-11631. Fix review comments

8fb963c

Fix wrong reference

894ebf0

jojochuang force-pushed the HDDS-11631-run-with-docker-compose-guide branch from 021e0b1 to 894ebf0 Compare December 16, 2025 18:37

jojochuang marked this pull request as ready for review December 16, 2025 18:43

jojochuang reviewed Dec 16, 2025

View reviewed changes

jojochuang merged commit 329c80a into apache:HDDS-9225-website-v2 Dec 16, 2025
11 checks passed

ptlrs deleted the HDDS-11631-run-with-docker-compose-guide branch December 16, 2025 19:36


		This will pull the official Apache Ozone images from Docker Hub.

		## Architecture Diagram

HDDS-11631. Add Run with Docker Compose guide #101

HDDS-11631. Add Run with Docker Compose guide #101

Uh oh!

Conversation

ptlrs commented Oct 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

What is the link to the Apache Jira?

How was this patch tested?

Uh oh!

errose28 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

errose28 Nov 6, 2024

Choose a reason for hiding this comment

Uh oh!

ptlrs Nov 14, 2024

Choose a reason for hiding this comment

Uh oh!

errose28 Nov 6, 2024

Choose a reason for hiding this comment

Uh oh!

ptlrs Nov 13, 2024

Choose a reason for hiding this comment

Uh oh!

devabhishekpal Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

errose28 Nov 6, 2024

Choose a reason for hiding this comment

Uh oh!

ptlrs Nov 13, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adoroszlai left a comment

Choose a reason for hiding this comment

Uh oh!

errose28 commented Nov 14, 2024

Uh oh!

ptlrs commented Jan 6, 2025

Uh oh!

ptlrs commented Jan 24, 2025

Uh oh!

ptlrs commented Apr 30, 2025

Uh oh!

jojochuang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ptlrs commented Dec 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ptlrs commented Oct 31, 2024 •

edited

Loading