Logging prototype

[reyang]

The goal:

• Prototype the logging API that can be used by both the SDK consumer and SDK author.
  • If we could find an existing C++ logging API that is widely adopted + cross platform, we would want to leverage it instead of reinventing wheels.
• Implement the Processor and Exporter logic similar like SpanProcessor and trace exporter.
• Implement ElasticSearch Log Exporter which sends batches of logs from the SDK directly to ElasticSearch via HTTP REST API.

The stretch goal:

• Provide a console log exporter for local debugging purpose.
• Provide a way to associate trace id / span id and other context/baggage to log entries.
• Implement OTLP Exporter https://github.com/open-telemetry/opentelemetry-proto/tree/master/opentelemetry/proto/logs/v1.

Non-goal:

• Provide log sampler interface and implementation.
• Implement Fluentd/Fluentbit exporter.
• Implement local file exporter.

Initial list of work items:

• API to fetch Logger (e.g. LoggerProvider)
• Logging API
• Log Processor interface
• Log Exporter interface
• SimpleExportProcessor similar like this https://github.com/open-telemetry/opentelemetry-cpp/blob/2973baaf476a92e9d11d55e82f1dad78fefe6de1/sdk/include/opentelemetry/sdk/trace/simple_processor.h.
• BatchExportProcessor similar like this https://github.com/open-telemetry/opentelemetry-cpp/blob/master/sdk/src/trace/batch_span_processor.cc.
• Console Exporter
• ElasticSearch Exporter

[reyang]

Seems I cannot assign the issue to @MarkSeufert and @xukaren at this moment. Please reply back once you are member of the OpenTelemetry community and I'll assign this to you. Thanks!

@alolita @MarkSeufert @xukaren

[reyang]

@maxgolov @lalitb @ThomsonTan FYI

[MarkSeufert]

Seems I cannot assign the issue to @MarkSeufert and @xukaren at this moment. Please reply back once you are member of the OpenTelemetry community and I'll assign this to you. Thanks!

@alolita @MarkSeufert @xukaren

@reyang I also just got my membership!
I'm excited to get started on this issue :)

[alolita]

Thanks @reyang

[maxgolov]

I would suggest we consider the following approach:

• for structured logging - capitalize on top of existing std::map<nostd::string_view, common::AttributeValue> to express any arbitrary strongly typed key-value property bag. Example here: https://github.com/maxgolov/opentelemetry-cpp/blob/4004c2d1ebc6bd9b1e55eaf9310726fc939f9de5/examples/TraceStreamer/main.cpp#L42

• for unstructured logging (plain UTF-8 string) - make sure we respect the verbosity levels, similar to other languages / OT spec for that.

• consider the industry "best practices", looking how API structure is done in mature / prominent logging libraries in

1. Java: https://logging.apache.org/log4j/2.x/manual/api.html
2. .NET: https://docs.microsoft.com/en-us/aspnet/core/fundamentals/logging/?view=aspnetcore-3.1

• ensure that we can provide an option to turn off the logging at build time with a macro, like it's done in log4cxx:
  https://logging.apache.org/log4cxx/latest_stable/apidocs/group___logging_macros.html

My initial thinking is we can have the following concepts:

• ILogger - akin to Tracer, but returned with something that is a GetLogger API call. Concrete Logger implementation could differ depending on environment, need, preferences..
• ILogger->LogEvent(...) with overrides similar to log4j, .NET ILogger, and/or log4cxx, consolidating the best practices of the worlds, to make onboarding easy.
• macros; and possibly compile-time template of ILogger->LogEvent(T), where customers may provide their own header-only implementation of a transform of a complex type T into OpenTelemetry primitives, where T - is a class, struct, or collection.

[tigrannajaryan]

Great to see this initiative.

Please consider structuring implementation in a way that also allows other C++ logging libraries to use the functionality that you expose to emit logs in OpenTelemetry-compliant manner. So for example if Boost.Log in the future decides to support OpenTelemetry they can use Otel C++ Logging SDK for that.

This is for example how Otel Java is structured: open-telemetry/opentelemetry-java#1524

[reyang]

Thanks @tigrannajaryan. FYI here goes a list of ways of logging in .NET https://github.com/open-telemetry/opentelemetry-dotnet/blob/a7bbefd0fc3e45e2291438e4f3d9702ffbfbe7f2/docs/logs/getting-started/Program.cs#L22.