From 5199f38121fc9a9db9a59cb4369ed3870031f1aa Mon Sep 17 00:00:00 2001 From: Bryce Mecum Date: Wed, 25 Jan 2023 15:56:47 -0900 Subject: [PATCH 1/6] First draft of opentelemetry user guide --- docs/source/cpp/opentelemetry.rst | 87 +++++++++++++++++++++++++++++++ docs/source/cpp/user_guide.rst | 1 + 2 files changed, 88 insertions(+) create mode 100644 docs/source/cpp/opentelemetry.rst diff --git a/docs/source/cpp/opentelemetry.rst b/docs/source/cpp/opentelemetry.rst new file mode 100644 index 00000000000..bdb3cd07423 --- /dev/null +++ b/docs/source/cpp/opentelemetry.rst @@ -0,0 +1,87 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +.. _cpp_opentelemetry: + +============= +OpenTelemetry +============= + +Portions of Arrow C++ are instrumented with the `OpenTelemetry +`_ C++ SDK which makes it possible to generate +detailed tracing information which can be analyzed in other tools. + +Creating a Build with OpenTelemetry Tracing Enabled +--------------------------------------------------- + +OpenTelemetry tracing is not turned on by default so you must first create a +custom build of Arrow C++ with tracing turned on. See :ref:`Building Arrow C++ +` for general instructions on creating a custom build. + +To enable tracing, specify ``ARROW_WITH_OPENTELEMETRY=ON`` when generating your +build. You may also want to specify ``CMAKE_BUILD_TYPE=RelWithDebInfo`` in order +to get representative timings while retaining debug information. + +Exporting Tracing Information +----------------------------- + +By default, no tracing information is exported until a tracing backend has been +specified. The choice of tracing backend is controlled with the +:envvar:`_ARROW_TRACING_BACKEND` environment variable. Possible values are: + + - ``ostream``: emit textual log messages to stdout + - ``otlp_http``: emit JSON traces to a HTTP server (by default, the endpoint + URL is "http://localhost:4318/v1/traces") + - ``arrow_otlp_stdout``: emit JSON traces to stdout + - ``arrow_otlp_stderr``: emit JSON traces to stderr + +For example, to enable exporting JSON traces to stdout, set:: + + export ARROW_TRACING_BACKEND=arrow_otlp_stdout + +At this point, running the program you've linked to your custom build of +Arrow C++ should produce JSON traces on stdout. + +Visualizing Traces with Jaeger UI +--------------------------------- + +Analyzing trace information exported to stdout/stderr may involve writing custom +processing scripts. As an alternative -- or even a complement -- to this +process, the "all-in-one" `Jaeger `_ `Docker +`_ image is a relatively straightforward way of +visualizing trace data and is suitable for local development and testing. + +Note: This assumes you have `Docker `_ installed. + +First, change your tracing backend to ``otlp_http``:: + + export ARROW_TRACING_BACKEND=otlp_http + +Then start the Jaeger all-in-one container:: + + docker run \ + -e COLLECTOR_OTLP_ENABLED=true \ + -p 16686:16686 \ + -p 4317:4317 \ + -p 4318:4318 \ + jaegertracing/all-in-one:1.35 + +Now you should be able to run your program and view any traces in a web browser +at http://localhost:16686. Note that unlike with other methods of exporting +traces, no output will be made to stdout/stderr. However, if you tail your +Docker container logs, you should see output when traces are received by the +all-in-one container. diff --git a/docs/source/cpp/user_guide.rst b/docs/source/cpp/user_guide.rst index 22f01ac6b4d..6a426b6c107 100644 --- a/docs/source/cpp/user_guide.rst +++ b/docs/source/cpp/user_guide.rst @@ -41,4 +41,5 @@ User Guide flight gdb threading + opentelemetry env_vars From b6aa54f71124ecb6063e9e84fa31570c59fb7c72 Mon Sep 17 00:00:00 2001 From: Bryce Mecum Date: Thu, 26 Jan 2023 12:12:49 -0900 Subject: [PATCH 2/6] Update docs/source/cpp/opentelemetry.rst Co-authored-by: Matthijs Brobbel --- docs/source/cpp/opentelemetry.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/cpp/opentelemetry.rst b/docs/source/cpp/opentelemetry.rst index bdb3cd07423..aaecd05b410 100644 --- a/docs/source/cpp/opentelemetry.rst +++ b/docs/source/cpp/opentelemetry.rst @@ -41,7 +41,7 @@ Exporting Tracing Information By default, no tracing information is exported until a tracing backend has been specified. The choice of tracing backend is controlled with the -:envvar:`_ARROW_TRACING_BACKEND` environment variable. Possible values are: +:envvar:`ARROW_TRACING_BACKEND` environment variable. Possible values are: - ``ostream``: emit textual log messages to stdout - ``otlp_http``: emit JSON traces to a HTTP server (by default, the endpoint From 44db600b4147d9229b0f275849c436366e49ac5a Mon Sep 17 00:00:00 2001 From: Bryce Mecum Date: Thu, 26 Jan 2023 12:14:06 -0900 Subject: [PATCH 3/6] Update docs/source/cpp/opentelemetry.rst Co-authored-by: Matthijs Brobbel --- docs/source/cpp/opentelemetry.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/cpp/opentelemetry.rst b/docs/source/cpp/opentelemetry.rst index aaecd05b410..00390e90c64 100644 --- a/docs/source/cpp/opentelemetry.rst +++ b/docs/source/cpp/opentelemetry.rst @@ -44,7 +44,7 @@ specified. The choice of tracing backend is controlled with the :envvar:`ARROW_TRACING_BACKEND` environment variable. Possible values are: - ``ostream``: emit textual log messages to stdout - - ``otlp_http``: emit JSON traces to a HTTP server (by default, the endpoint + - ``otlp_http``: emit OTLP JSON encoded traces to a HTTP server (by default, the endpoint URL is "http://localhost:4318/v1/traces") - ``arrow_otlp_stdout``: emit JSON traces to stdout - ``arrow_otlp_stderr``: emit JSON traces to stderr From 4e437ce10dd3f20915fee8ca2c8079109e8b1213 Mon Sep 17 00:00:00 2001 From: Bryce Mecum Date: Thu, 26 Jan 2023 12:16:52 -0900 Subject: [PATCH 4/6] Hard wrap last commit --- docs/source/cpp/opentelemetry.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/cpp/opentelemetry.rst b/docs/source/cpp/opentelemetry.rst index 00390e90c64..db6c0ac4469 100644 --- a/docs/source/cpp/opentelemetry.rst +++ b/docs/source/cpp/opentelemetry.rst @@ -44,8 +44,8 @@ specified. The choice of tracing backend is controlled with the :envvar:`ARROW_TRACING_BACKEND` environment variable. Possible values are: - ``ostream``: emit textual log messages to stdout - - ``otlp_http``: emit OTLP JSON encoded traces to a HTTP server (by default, the endpoint - URL is "http://localhost:4318/v1/traces") + - ``otlp_http``: emit OTLP JSON encoded traces to a HTTP server (by default, + the endpoint URL is "http://localhost:4318/v1/traces") - ``arrow_otlp_stdout``: emit JSON traces to stdout - ``arrow_otlp_stderr``: emit JSON traces to stderr From 2f2e58387d16839b4220104b42b3a162f0289836 Mon Sep 17 00:00:00 2001 From: Bryce Mecum Date: Thu, 26 Jan 2023 12:17:23 -0900 Subject: [PATCH 5/6] Duplicate changes from opentelemetry.rst to env_vars.rst Just keeping these two lists in sync with one another --- docs/source/cpp/env_vars.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/cpp/env_vars.rst b/docs/source/cpp/env_vars.rst index e4f8f9046a1..f9627c6a30d 100644 --- a/docs/source/cpp/env_vars.rst +++ b/docs/source/cpp/env_vars.rst @@ -67,11 +67,11 @@ that changing their value later will have an effect. The backend where to export `OpenTelemetry `_-based execution traces. Possible values are: - - ``ostream``: emit textual log messages to stdout; - - ``otlp_http``: emit JSON traces to a HTTP server (by default, the endpoint - URL is "http://localhost:4318/v1/traces"); - - ``arrow_otlp_stdout``: emit JSON traces to stdout; - - ``arrow_otlp_stderr``: emit JSON traces to stderr. + - ``ostream``: emit textual log messages to stdout + - ``otlp_http``: emit OTLP JSON encoded traces to a HTTP server (by default, + the endpoint URL is "http://localhost:4318/v1/traces") + - ``arrow_otlp_stdout``: emit JSON traces to stdout + - ``arrow_otlp_stderr``: emit JSON traces to stderr If this variable is not set, no traces are exported. From f225dbcaafa5aaf1b5009a9fcca16d98d44a4cee Mon Sep 17 00:00:00 2001 From: Bryce Mecum Date: Thu, 26 Jan 2023 12:19:26 -0900 Subject: [PATCH 6/6] Match list formatting from elsewhere in doc --- docs/source/cpp/env_vars.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/cpp/env_vars.rst b/docs/source/cpp/env_vars.rst index f9627c6a30d..911d45af9c2 100644 --- a/docs/source/cpp/env_vars.rst +++ b/docs/source/cpp/env_vars.rst @@ -67,11 +67,11 @@ that changing their value later will have an effect. The backend where to export `OpenTelemetry `_-based execution traces. Possible values are: - - ``ostream``: emit textual log messages to stdout + - ``ostream``: emit textual log messages to stdout; - ``otlp_http``: emit OTLP JSON encoded traces to a HTTP server (by default, - the endpoint URL is "http://localhost:4318/v1/traces") - - ``arrow_otlp_stdout``: emit JSON traces to stdout - - ``arrow_otlp_stderr``: emit JSON traces to stderr + the endpoint URL is "http://localhost:4318/v1/traces"); + - ``arrow_otlp_stdout``: emit JSON traces to stdout; + - ``arrow_otlp_stderr``: emit JSON traces to stderr. If this variable is not set, no traces are exported.