From 3c67b93cd7759e770c87187aea5d6877f46fff28 Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Tue, 20 Jun 2023 17:54:41 -0700 Subject: [PATCH 01/15] Update for M1 --- docs/tutorials/tutorial-jupyter-docker.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index b5aa939db89b..23e17c3910c1 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -33,12 +33,15 @@ You can run the following combination of applications: * [Jupyter only](#start-only-the-jupyter-container) * [Jupyter and Druid](#start-jupyter-and-druid) * [Jupyter, Druid, and Kafka](#start-jupyter-druid-and-kafka) +* [Kafka and Jupyter](#start-kafka-and-jupyter) ## Prerequisites Jupyter in Docker requires that you have **Docker** and **Docker Compose**. We recommend installing these through [Docker Desktop](https://docs.docker.com/desktop/). +For ARM-based devices, consult the section [Tutorial setup for ARM-based devices]() for tutorial configuration instructions. + ## Launch the Docker containers You run Docker Compose to launch Jupyter and optionally Druid or Kafka. @@ -85,6 +88,17 @@ In the same directory as `docker-compose.yaml` and `environment`, start the appl DRUID_VERSION={{DRUIDVERSION}} docker compose --profile all-services up -d ``` +### Start Kafka and Jupyter + +Running Druid in Docker requires the `environment` file as well as the `DRUID_VERSION` environment variable. + +In the same directory as `docker-compose.yaml` and `environment`, start the application: + +```bash +DRUID_VERSION={{DRUIDVERSION}} docker compose --profile kafka-jupyter up -d +``` + + ### Update image from Docker Hub If you already have a local cache of the Jupyter image, you can update the image before running the application using the following command: @@ -193,6 +207,13 @@ as well as the [Python client for Druid](tutorial-jupyter-index.md#python-api-fo You should now be able to access and complete the tutorials. +## Tutorial setup for ARM-based devices +1. Start Druid. You can use the [Quickstart (local)](./index.md) instance. The tutorials + assume that you are using the quickstart, so no authentication or authorization + is expected unless explicitly mentioned. +2. Start Kafka and Jupyter using the command from [Start Kafka and Jupyter](#start-kafka-and-jupyter). +3. Before starting each tutorial, open the notebook and edit the `host` variable to `"host.docker.internal"`. + ## Learn more See the following topics for more information: From c5691f9602bea91ea534fda3b94e24724530f20b Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Wed, 21 Jun 2023 10:06:29 -0700 Subject: [PATCH 02/15] Update some text --- docs/tutorials/tutorial-jupyter-docker.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index 23e17c3910c1..3157335714fb 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -40,7 +40,7 @@ You can run the following combination of applications: Jupyter in Docker requires that you have **Docker** and **Docker Compose**. We recommend installing these through [Docker Desktop](https://docs.docker.com/desktop/). -For ARM-based devices, consult the section [Tutorial setup for ARM-based devices]() for tutorial configuration instructions. +For ARM-based devices, consult the section [Tutorial setup for ARM-based devices](#tutorial-setup-for-arm-based-devices) for instructions. ## Launch the Docker containers @@ -212,7 +212,7 @@ You should now be able to access and complete the tutorials. assume that you are using the quickstart, so no authentication or authorization is expected unless explicitly mentioned. 2. Start Kafka and Jupyter using the command from [Start Kafka and Jupyter](#start-kafka-and-jupyter). -3. Before starting each tutorial, open the notebook and edit the `host` variable to `"host.docker.internal"`. +3. Before starting a tutorial, open the notebook and edit the `host` variable to `"host.docker.internal"`. ## Learn more From ce38b53e00a94aac8281a7c7e2f84446c3d06d5c Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Wed, 21 Jun 2023 12:31:39 -0700 Subject: [PATCH 03/15] update some text --- docs/tutorials/tutorial-jupyter-docker.md | 29 ++++++++++++++++------- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index 3157335714fb..7f1b31acc3aa 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -40,7 +40,7 @@ You can run the following combination of applications: Jupyter in Docker requires that you have **Docker** and **Docker Compose**. We recommend installing these through [Docker Desktop](https://docs.docker.com/desktop/). -For ARM-based devices, consult the section [Tutorial setup for ARM-based devices](#tutorial-setup-for-arm-based-devices) for instructions. +For ARM-based devices, see [Tutorial setup for ARM-based devices](#tutorial-setup-for-arm-based-devices) for instructions. ## Launch the Docker containers @@ -66,6 +66,8 @@ docker compose --profile jupyter up -d The Docker Compose file assigns `8889` for the Jupyter port. You can override the port number by setting the `JUPYTER_PORT` environment variable before starting the Docker application. +If Druid is running local to the same machine as Jupyter, before starting a tutorial, set the `host` variable to `host.docker.internal` within the notebook. + ### Start Jupyter and Druid Running Druid in Docker requires the `environment` file as well as an environment variable named `DRUID_VERSION`, @@ -89,8 +91,7 @@ DRUID_VERSION={{DRUIDVERSION}} docker compose --profile all-services up -d ``` ### Start Kafka and Jupyter - -Running Druid in Docker requires the `environment` file as well as the `DRUID_VERSION` environment variable. +If you already have Druid running locally, you can run only Kafka and Jupyter in the Docker Compose environment to complete the tutorials. In the same directory as `docker-compose.yaml` and `environment`, start the application: @@ -98,6 +99,7 @@ In the same directory as `docker-compose.yaml` and `environment`, start the appl DRUID_VERSION={{DRUIDVERSION}} docker compose --profile kafka-jupyter up -d ``` +If Druid is running local to the same machine as Jupyter, open the tutorial and set the `host` variable to `host.docker.internal` before starting. ### Update image from Docker Hub @@ -208,11 +210,22 @@ as well as the [Python client for Druid](tutorial-jupyter-index.md#python-api-fo You should now be able to access and complete the tutorials. ## Tutorial setup for ARM-based devices -1. Start Druid. You can use the [Quickstart (local)](./index.md) instance. The tutorials - assume that you are using the quickstart, so no authentication or authorization - is expected unless explicitly mentioned. -2. Start Kafka and Jupyter using the command from [Start Kafka and Jupyter](#start-kafka-and-jupyter). -3. Before starting a tutorial, open the notebook and edit the `host` variable to `"host.docker.internal"`. + +For ARM-based devices, follow this setup to start Druid externally, while keeping Kafka and Jupyter within the Docker Compose environment: +1. Start Druid using the `start-druid` script. You can follow [Quickstart (local)](./index.md) instructions. The tutorials + assume that you are using the quickstart, so no authentication or authorization is expected unless explicitly mentioned. +2. Start either Kafka and Jupyter or only Jupyter using the following commands in the same directory as `docker-compose.yaml` and `environment`: + + ```bash + # Start Kafka and Jupyter + DRUID_VERSION={{DRUIDVERSION}} docker compose --profile kafka-jupyter up -d + + # Start only Jupyter + ```bash + docker compose --profile jupyter up -d + ``` + +3. If Druid is running local to the same machine as Jupyter, open the tutorial and set the `host` variable to `host.docker.internal` before starting. ## Learn more From f23e14ab46bcb1a3ac2318fcb20066c7f10ca3c1 Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Wed, 21 Jun 2023 13:22:42 -0700 Subject: [PATCH 04/15] format' --- docs/tutorials/tutorial-jupyter-docker.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index 7f1b31acc3aa..03be60f5dda3 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -212,6 +212,7 @@ You should now be able to access and complete the tutorials. ## Tutorial setup for ARM-based devices For ARM-based devices, follow this setup to start Druid externally, while keeping Kafka and Jupyter within the Docker Compose environment: + 1. Start Druid using the `start-druid` script. You can follow [Quickstart (local)](./index.md) instructions. The tutorials assume that you are using the quickstart, so no authentication or authorization is expected unless explicitly mentioned. 2. Start either Kafka and Jupyter or only Jupyter using the following commands in the same directory as `docker-compose.yaml` and `environment`: From d125290a9386c42d701710ca5dcc14b879106fa2 Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Thu, 22 Jun 2023 10:04:45 -0700 Subject: [PATCH 05/15] link?? --- docs/tutorials/tutorial-jupyter-docker.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index 03be60f5dda3..401612099e51 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -231,6 +231,6 @@ For ARM-based devices, follow this setup to start Druid externally, while keepin ## Learn more See the following topics for more information: -* [Jupyter Notebook tutorials](tutorial-jupyter-index.md) for the available Jupyter Notebook-based tutorials for Druid -* [Tutorial: Run with Docker](docker.md) for running Druid from a Docker container +* [Jupyter Notebook tutorials](./tutorial-jupyter-index.md) for the available Jupyter Notebook-based tutorials for Druid +* [Tutorial: Run with Docker](./docker.md) for running Druid from a Docker container From 03feb45dd74293c9465d2076aa92733cba876194 Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Thu, 22 Jun 2023 11:12:53 -0700 Subject: [PATCH 06/15] D: --- docs/tutorials/tutorial-jupyter-docker.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index 401612099e51..9bfa837c96d3 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -91,7 +91,7 @@ DRUID_VERSION={{DRUIDVERSION}} docker compose --profile all-services up -d ``` ### Start Kafka and Jupyter -If you already have Druid running locally, you can run only Kafka and Jupyter in the Docker Compose environment to complete the tutorials. +If you already have Druid running externally, you can run only Kafka and Jupyter in the Docker Compose environment to complete the tutorials. In the same directory as `docker-compose.yaml` and `environment`, start the application: @@ -231,6 +231,5 @@ For ARM-based devices, follow this setup to start Druid externally, while keepin ## Learn more See the following topics for more information: -* [Jupyter Notebook tutorials](./tutorial-jupyter-index.md) for the available Jupyter Notebook-based tutorials for Druid -* [Tutorial: Run with Docker](./docker.md) for running Druid from a Docker container - +* [Jupyter Notebook tutorials](tutorial-jupyter-index.md) for the available Jupyter Notebook-based tutorials for Druid +* [Tutorial: Run with Docker](docker.md) for running Druid from a Docker container \ No newline at end of file From 07f46eb41fa5b7efc1584a7b86b53cf41064e4ec Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Thu, 22 Jun 2023 11:34:06 -0700 Subject: [PATCH 07/15] Note for Kafka and Jupyter use case --- docs/tutorials/tutorial-jupyter-docker.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index 9bfa837c96d3..5dba41192468 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -91,7 +91,7 @@ DRUID_VERSION={{DRUIDVERSION}} docker compose --profile all-services up -d ``` ### Start Kafka and Jupyter -If you already have Druid running externally, you can run only Kafka and Jupyter in the Docker Compose environment to complete the tutorials. +If you already have Druid running externally, such as an existing cluster or a dedicated infrastructure for Druid, you can run only Kafka and Jupyter in the Docker Compose environment to complete the tutorials. In the same directory as `docker-compose.yaml` and `environment`, start the application: @@ -99,7 +99,9 @@ In the same directory as `docker-compose.yaml` and `environment`, start the appl DRUID_VERSION={{DRUIDVERSION}} docker compose --profile kafka-jupyter up -d ``` -If Druid is running local to the same machine as Jupyter, open the tutorial and set the `host` variable to `host.docker.internal` before starting. +Please note that if you have an external Druid instance running on a different machine than the one hosting the Docker Compose environment, you will need to change the `host` variable in the notebook tutorial to the hostname or address of the machine where Druid is running. This will establish a connection with the external Druid cluster. + +If Druid is running local to the same machine as Jupyter, you can set the `host` variable to `host.docker.internal` before starting. ### Update image from Docker Hub From a42129e7beba8af1caa261e0709ca3800b4d98e3 Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Thu, 22 Jun 2023 12:12:34 -0700 Subject: [PATCH 08/15] fix link issue --- docs/tutorials/tutorial-jupyter-docker.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index 5dba41192468..ce908e6e1d37 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -224,7 +224,6 @@ For ARM-based devices, follow this setup to start Druid externally, while keepin DRUID_VERSION={{DRUIDVERSION}} docker compose --profile kafka-jupyter up -d # Start only Jupyter - ```bash docker compose --profile jupyter up -d ``` From 3eed7dd4bec80f8da821ce33baa30d3ea69b5034 Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Thu, 22 Jun 2023 12:15:45 -0700 Subject: [PATCH 09/15] whitespace --- docs/tutorials/tutorial-jupyter-docker.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index ce908e6e1d37..80561b3582ca 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -91,6 +91,7 @@ DRUID_VERSION={{DRUIDVERSION}} docker compose --profile all-services up -d ``` ### Start Kafka and Jupyter + If you already have Druid running externally, such as an existing cluster or a dedicated infrastructure for Druid, you can run only Kafka and Jupyter in the Docker Compose environment to complete the tutorials. In the same directory as `docker-compose.yaml` and `environment`, start the application: From 2624208c214d9c9360f68604b52fc968095fe10b Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Tue, 27 Jun 2023 11:01:24 -0700 Subject: [PATCH 10/15] update to victoria's feedback --- docs/tutorials/tutorial-jupyter-docker.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index 80561b3582ca..6c6bfaa9fe70 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -40,7 +40,7 @@ You can run the following combination of applications: Jupyter in Docker requires that you have **Docker** and **Docker Compose**. We recommend installing these through [Docker Desktop](https://docs.docker.com/desktop/). -For ARM-based devices, see [Tutorial setup for ARM-based devices](#tutorial-setup-for-arm-based-devices) for instructions. +For ARM-based devices, see [Tutorial setup for ARM-based devices](#tutorial-setup-for-arm-based-devices). ## Launch the Docker containers @@ -56,7 +56,7 @@ access the files in `druid/examples/quickstart/jupyter-notebooks/docker-jupyter` ### Start only the Jupyter container -If you already have Druid running locally, you can run only the Jupyter container to complete the tutorials. +If you already have Druid running locally, you can run the Docker containers for Jupyter only. In the same directory as `docker-compose.yaml`, start the application: ```bash @@ -66,7 +66,10 @@ docker compose --profile jupyter up -d The Docker Compose file assigns `8889` for the Jupyter port. You can override the port number by setting the `JUPYTER_PORT` environment variable before starting the Docker application. -If Druid is running local to the same machine as Jupyter, before starting a tutorial, set the `host` variable to `host.docker.internal` within the notebook. +If Druid is running local to the same machine as Jupyter, open the tutorial and set the `host` variable to `host.docker.internal` before starting. For example: +```python +host = "host.docker.internal" +``` ### Start Jupyter and Druid @@ -92,7 +95,7 @@ DRUID_VERSION={{DRUIDVERSION}} docker compose --profile all-services up -d ### Start Kafka and Jupyter -If you already have Druid running externally, such as an existing cluster or a dedicated infrastructure for Druid, you can run only Kafka and Jupyter in the Docker Compose environment to complete the tutorials. +If you already have Druid running externally, such as an existing cluster or a dedicated infrastructure for Druid, you can run the Docker containers for Kafka and Jupyter only. In the same directory as `docker-compose.yaml` and `environment`, start the application: @@ -102,7 +105,10 @@ DRUID_VERSION={{DRUIDVERSION}} docker compose --profile kafka-jupyter up -d Please note that if you have an external Druid instance running on a different machine than the one hosting the Docker Compose environment, you will need to change the `host` variable in the notebook tutorial to the hostname or address of the machine where Druid is running. This will establish a connection with the external Druid cluster. -If Druid is running local to the same machine as Jupyter, you can set the `host` variable to `host.docker.internal` before starting. +If Druid is running local to the same machine as Jupyter, open the tutorial and set the `host` variable to `host.docker.internal` before starting. For example: +```python +host = "host.docker.internal" +``` ### Update image from Docker Hub @@ -228,7 +234,10 @@ For ARM-based devices, follow this setup to start Druid externally, while keepin docker compose --profile jupyter up -d ``` -3. If Druid is running local to the same machine as Jupyter, open the tutorial and set the `host` variable to `host.docker.internal` before starting. +3. If Druid is running local to the same machine as Jupyter, open the tutorial and set the `host` variable to `host.docker.internal` before starting. For example: + ```python + host = "host.docker.internal" + ``` ## Learn more From a32cb226183ccd9d7bcaf31ef5bfe63c52f25cde Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Mon, 3 Jul 2023 12:17:43 -0700 Subject: [PATCH 11/15] update compose yaml for kafka fix --- .../docker-jupyter/docker-compose-local.yaml | 6 ++++-- .../jupyter-notebooks/docker-jupyter/docker-compose.yaml | 6 ++++-- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/examples/quickstart/jupyter-notebooks/docker-jupyter/docker-compose-local.yaml b/examples/quickstart/jupyter-notebooks/docker-jupyter/docker-compose-local.yaml index 3d7baef90525..197b8b3722e2 100644 --- a/examples/quickstart/jupyter-notebooks/docker-jupyter/docker-compose-local.yaml +++ b/examples/quickstart/jupyter-notebooks/docker-jupyter/docker-compose-local.yaml @@ -58,12 +58,14 @@ services: # To learn about configuring Kafka for access across networks see # https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/ - "9092:9092" + - '9094:9094' depends_on: - zookeeper environment: - KAFKA_BROKER_ID=1 - - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092 - - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092 + - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093,EXTERNAL://:9094 + - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092,EXTERNAL://localhost:9094 + - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT,PLAINTEXT:PLAINTEXT - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181 - ALLOW_PLAINTEXT_LISTENER=yes - KAFKA_ENABLE_KRAFT=false diff --git a/examples/quickstart/jupyter-notebooks/docker-jupyter/docker-compose.yaml b/examples/quickstart/jupyter-notebooks/docker-jupyter/docker-compose.yaml index e6f2cd95ae9d..932df6a22b43 100644 --- a/examples/quickstart/jupyter-notebooks/docker-jupyter/docker-compose.yaml +++ b/examples/quickstart/jupyter-notebooks/docker-jupyter/docker-compose.yaml @@ -58,12 +58,14 @@ services: # To learn about configuring Kafka for access across networks see # https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/ - "9092:9092" + - '9094:9094' depends_on: - zookeeper environment: - KAFKA_BROKER_ID=1 - - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092 - - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092 + - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093,EXTERNAL://:9094 + - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092,EXTERNAL://localhost:9094 + - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT,PLAINTEXT:PLAINTEXT - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181 - ALLOW_PLAINTEXT_LISTENER=yes - KAFKA_ENABLE_KRAFT=false From b90d602451d56264c1c4ea8d19ec3e8b3554ba9e Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Mon, 3 Jul 2023 17:00:45 -0700 Subject: [PATCH 12/15] Update instructions for boostrap.server, connecting Druid and Kafka --- docs/tutorials/tutorial-jupyter-docker.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index 6c6bfaa9fe70..7856accb8dad 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -110,6 +110,8 @@ If Druid is running local to the same machine as Jupyter, open the tutorial and host = "host.docker.internal" ``` +To enable Druid to ingest data from Kafka within the Docker Compose environment, update the `bootstrap.server` property in the Kafka ingestion spec to `localhost:9094` before ingesting. For reference, see [more on consumer properties](../development/extensions-core/kafka-supervisor-reference.md#more-on-consumerproperties). + ### Update image from Docker Hub If you already have a local cache of the Jupyter image, you can update the image before running the application using the following command: @@ -238,6 +240,7 @@ For ARM-based devices, follow this setup to start Druid externally, while keepin ```python host = "host.docker.internal" ``` +4. If using Kafka to handle the data stream that will be ingested into Druid and Druid is running local to the same machine, update the consumer property `bootstrap.server` to `localhost:9094`. ## Learn more From c7ef00422d5dd0115d333a31a950c651faffdbc6 Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Wed, 5 Jul 2023 09:43:30 -0700 Subject: [PATCH 13/15] fix bootstrap.server to bootstrap.servers --- docs/tutorials/tutorial-jupyter-docker.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index 7856accb8dad..fbec6b922de4 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -110,7 +110,7 @@ If Druid is running local to the same machine as Jupyter, open the tutorial and host = "host.docker.internal" ``` -To enable Druid to ingest data from Kafka within the Docker Compose environment, update the `bootstrap.server` property in the Kafka ingestion spec to `localhost:9094` before ingesting. For reference, see [more on consumer properties](../development/extensions-core/kafka-supervisor-reference.md#more-on-consumerproperties). +To enable Druid to ingest data from Kafka within the Docker Compose environment, update the `bootstrap.servers` property in the Kafka ingestion spec to `localhost:9094` before ingesting. For reference, see [more on consumer properties](../development/extensions-core/kafka-supervisor-reference.md#more-on-consumerproperties). ### Update image from Docker Hub @@ -240,7 +240,7 @@ For ARM-based devices, follow this setup to start Druid externally, while keepin ```python host = "host.docker.internal" ``` -4. If using Kafka to handle the data stream that will be ingested into Druid and Druid is running local to the same machine, update the consumer property `bootstrap.server` to `localhost:9094`. +4. If using Kafka to handle the data stream that will be ingested into Druid and Druid is running local to the same machine, update the consumer property `bootstrap.servers` to `localhost:9094`. ## Learn more From 436cb956531ac7003f6a500b5fdac69f69c93b3c Mon Sep 17 00:00:00 2001 From: demo-kratia <56242907+demo-kratia@users.noreply.github.com> Date: Mon, 10 Jul 2023 15:44:06 -0700 Subject: [PATCH 14/15] rewording and moving order of commands --- docs/tutorials/tutorial-jupyter-docker.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index fbec6b922de4..27f1b7a08494 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -103,9 +103,10 @@ In the same directory as `docker-compose.yaml` and `environment`, start the appl DRUID_VERSION={{DRUIDVERSION}} docker compose --profile kafka-jupyter up -d ``` -Please note that if you have an external Druid instance running on a different machine than the one hosting the Docker Compose environment, you will need to change the `host` variable in the notebook tutorial to the hostname or address of the machine where Druid is running. This will establish a connection with the external Druid cluster. +If you have an external Druid instance running on a different machine than the one hosting the Docker Compose environment, change the `host` variable in the notebook tutorial to the hostname or address of the machine where Druid is running. If Druid is running local to the same machine as Jupyter, open the tutorial and set the `host` variable to `host.docker.internal` before starting. For example: + ```python host = "host.docker.internal" ``` @@ -226,14 +227,14 @@ For ARM-based devices, follow this setup to start Druid externally, while keepin 1. Start Druid using the `start-druid` script. You can follow [Quickstart (local)](./index.md) instructions. The tutorials assume that you are using the quickstart, so no authentication or authorization is expected unless explicitly mentioned. -2. Start either Kafka and Jupyter or only Jupyter using the following commands in the same directory as `docker-compose.yaml` and `environment`: +2. Start either Jupyter only or Jupyter and Kafka using the following commands in the same directory as `docker-compose.yaml` and `environment`: ```bash - # Start Kafka and Jupyter - DRUID_VERSION={{DRUIDVERSION}} docker compose --profile kafka-jupyter up -d - # Start only Jupyter docker compose --profile jupyter up -d + + # Start Kafka and Jupyter + DRUID_VERSION={{DRUIDVERSION}} docker compose --profile kafka-jupyter up -d ``` 3. If Druid is running local to the same machine as Jupyter, open the tutorial and set the `host` variable to `host.docker.internal` before starting. For example: From b22004aeaeb784ea959d94692dc2019eb9a5012d Mon Sep 17 00:00:00 2001 From: Charles Smith Date: Tue, 11 Jul 2023 08:30:15 -0700 Subject: [PATCH 15/15] Update docs/tutorials/tutorial-jupyter-docker.md --- docs/tutorials/tutorial-jupyter-docker.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/tutorial-jupyter-docker.md b/docs/tutorials/tutorial-jupyter-docker.md index 27f1b7a08494..41d3848c1c10 100644 --- a/docs/tutorials/tutorial-jupyter-docker.md +++ b/docs/tutorials/tutorial-jupyter-docker.md @@ -56,7 +56,7 @@ access the files in `druid/examples/quickstart/jupyter-notebooks/docker-jupyter` ### Start only the Jupyter container -If you already have Druid running locally, you can run the Docker containers for Jupyter only. +If you already have Druid running locally or on another machine, you can run the Docker containers for Jupyter only. In the same directory as `docker-compose.yaml`, start the application: ```bash