-
Notifications
You must be signed in to change notification settings - Fork 850
Adding Zipkin exporter #320
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
28c6463
ec63dec
01dfed0
9d8d807
c3e415e
cb06b40
93331ff
e93e7b0
8946f4a
0e21ed8
4736004
b20310c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| # Changelog | ||
|
|
||
| ## Unreleased | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| OpenTelemetry Zipkin Exporter | ||
| ============================= | ||
|
|
||
| |pypi| | ||
|
|
||
| .. |pypi| image:: https://badge.fury.io/py/opentelemetry-ext-zipkin.svg | ||
| :target: https://pypi.org/project/opentelemetry-ext-zipkin/ | ||
|
|
||
| This library allows to export tracing data to `Zipkin <https://zipkin.io/>`_. | ||
|
|
||
| Installation | ||
| ------------ | ||
|
|
||
| :: | ||
|
|
||
| pip install opentelemetry-ext-zipkin | ||
|
|
||
|
|
||
| Usage | ||
| ----- | ||
|
|
||
| The **OpenTelemetry Zipkin Exporter** allows to export `OpenTelemetry`_ traces to `Zipkin`_. | ||
| This exporter always send traces to the configured Zipkin collector using HTTP. | ||
|
|
||
|
|
||
| .. _Zipkin: https://zipkin.io/ | ||
| .. _OpenTelemetry: https://github.com/open-telemetry/opentelemetry-python/ | ||
|
|
||
| .. code:: python | ||
|
|
||
| from opentelemetry import trace | ||
| from opentelemetry.ext import zipkin | ||
| from opentelemetry.sdk.trace import TracerSource | ||
| from opentelemetry.sdk.trace.export import BatchExportSpanProcessor | ||
|
|
||
| trace.set_preferred_tracer_source_implementation(lambda T: TracerSource()) | ||
| tracer = trace.tracer_source().get_tracer(__name__) | ||
|
|
||
| # create a ZipkinSpanExporter | ||
| zipkin_exporter = zipkin.ZipkinSpanExporter( | ||
| service_name="my-helloworld-service", | ||
| # optional: | ||
| # host_name="localhost", | ||
| # port=9411, | ||
| # endpoint="/api/v2/spans", | ||
| # protocol="http", | ||
| # ipv4="", | ||
| # ipv6="", | ||
| # retry=False, | ||
| ) | ||
|
|
||
| # Create a BatchExportSpanProcessor and add the exporter to it | ||
| span_processor = BatchExportSpanProcessor(zipkin_exporter) | ||
|
|
||
| # add to the tracer | ||
| trace.tracer_source().add_span_processor(span_processor) | ||
|
|
||
| with tracer.start_as_current_span("foo"): | ||
| print("Hello world!") | ||
|
|
||
| The `examples <./examples>`_ folder contains more elaborated examples. | ||
|
|
||
| References | ||
| ---------- | ||
|
|
||
| * `Zipkin <https://zipkin.io/>`_ | ||
| * `OpenTelemetry Project <https://opentelemetry.io/>`_ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| # Copyright 2019, OpenTelemetry Authors | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # | ||
| [metadata] | ||
| name = opentelemetry-ext-zipkin | ||
| description = Zipkin Span Exporter for OpenTelemetry | ||
| long_description = file: README.rst | ||
| long_description_content_type = text/x-rst | ||
| author = OpenTelemetry Authors | ||
| author_email = cncf-opentelemetry-contributors@lists.cncf.io | ||
| url = https://github.com/open-telemetry/opentelemetry-python/ext/opentelemetry-ext-zipkin | ||
| platforms = any | ||
| license = Apache-2.0 | ||
| classifiers = | ||
| Development Status :: 3 - Alpha | ||
| Intended Audience :: Developers | ||
| License :: OSI Approved :: Apache Software License | ||
| Programming Language :: Python | ||
| Programming Language :: Python :: 3 | ||
| Programming Language :: Python :: 3.4 | ||
| Programming Language :: Python :: 3.5 | ||
| Programming Language :: Python :: 3.6 | ||
| Programming Language :: Python :: 3.7 | ||
|
|
||
| [options] | ||
| python_requires = >=3.4 | ||
| package_dir= | ||
| =src | ||
| packages=find_namespace: | ||
| install_requires = | ||
| requests~=2.7 | ||
| opentelemetry-api | ||
| opentelemetry-sdk | ||
|
|
||
| [options.packages.find] | ||
| where = src | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # Copyright 2019, OpenTelemetry Authors | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| import os | ||
|
|
||
| import setuptools | ||
|
|
||
| BASE_DIR = os.path.dirname(__file__) | ||
| VERSION_FILENAME = os.path.join( | ||
| BASE_DIR, "src", "opentelemetry", "ext", "zipkin", "version.py" | ||
| ) | ||
| PACKAGE_INFO = {} | ||
| with open(VERSION_FILENAME) as f: | ||
| exec(f.read(), PACKAGE_INFO) | ||
|
|
||
| setuptools.setup(version=PACKAGE_INFO["__version__"]) |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,184 @@ | ||||||
| # Copyright 2019, OpenTelemetry Authors | ||||||
| # | ||||||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||||||
| # you may not use this file except in compliance with the License. | ||||||
| # You may obtain a copy of the License at | ||||||
| # | ||||||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||||||
| # | ||||||
| # Unless required by applicable law or agreed to in writing, software | ||||||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||||||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||||||
| # See the License for the specific language governing permissions and | ||||||
| # limitations under the License. | ||||||
|
|
||||||
| """Zipkin Span Exporter for OpenTelemetry.""" | ||||||
|
|
||||||
| import json | ||||||
| import logging | ||||||
| from typing import Optional, Sequence | ||||||
|
|
||||||
| import requests | ||||||
|
|
||||||
| from opentelemetry.sdk.trace.export import SpanExporter, SpanExportResult | ||||||
| from opentelemetry.trace import Span, SpanContext, SpanKind | ||||||
|
|
||||||
| DEFAULT_ENDPOINT = "/api/v2/spans" | ||||||
| DEFAULT_HOST_NAME = "localhost" | ||||||
| DEFAULT_PORT = 9411 | ||||||
| DEFAULT_PROTOCOL = "http" | ||||||
| DEFAULT_RETRY = False | ||||||
| ZIPKIN_HEADERS = {"Content-Type": "application/json"} | ||||||
|
|
||||||
| SPAN_KIND_MAP = { | ||||||
| SpanKind.INTERNAL: None, | ||||||
| SpanKind.SERVER: "SERVER", | ||||||
| SpanKind.CLIENT: "CLIENT", | ||||||
| SpanKind.PRODUCER: "PRODUCER", | ||||||
| SpanKind.CONSUMER: "CONSUMER", | ||||||
| } | ||||||
|
|
||||||
| SUCCESS_STATUS_CODES = (200, 202) | ||||||
|
|
||||||
| logger = logging.getLogger(__name__) | ||||||
|
|
||||||
|
|
||||||
| class ZipkinSpanExporter(SpanExporter): | ||||||
| """Zipkin span exporter for OpenTelemetry. | ||||||
|
|
||||||
| Args: | ||||||
| service_name: Service that logged an annotation in a trace.Classifier | ||||||
| when query for spans. | ||||||
| host_name: The host name of the Zipkin server | ||||||
| port: The port of the Zipkin server | ||||||
| endpoint: The endpoint of the Zipkin server | ||||||
| protocol: The protocol used for the request. | ||||||
| ipv4: Primary IPv4 address associated with this connection. | ||||||
| ipv6: Primary IPv6 address associated with this connection. | ||||||
| retry: Set to True to configure the exporter to retry on failure. | ||||||
| """ | ||||||
|
|
||||||
| def __init__( | ||||||
| self, | ||||||
| service_name: str, | ||||||
| host_name: str = DEFAULT_HOST_NAME, | ||||||
| port: int = DEFAULT_PORT, | ||||||
| endpoint: str = DEFAULT_ENDPOINT, | ||||||
| protocol: str = DEFAULT_PROTOCOL, | ||||||
| ipv4: Optional[str] = None, | ||||||
| ipv6: Optional[str] = None, | ||||||
c24t marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| retry: Optional[str] = DEFAULT_RETRY, | ||||||
| ): | ||||||
| self.service_name = service_name | ||||||
| self.host_name = host_name | ||||||
| self.port = port | ||||||
| self.endpoint = endpoint | ||||||
| self.protocol = protocol | ||||||
| self.url = "{}://{}:{}{}".format( | ||||||
| self.protocol, self.host_name, self.port, self.endpoint | ||||||
| ) | ||||||
| self.ipv4 = ipv4 | ||||||
| self.ipv6 = ipv6 | ||||||
| self.retry = retry | ||||||
|
|
||||||
| def export(self, spans: Sequence[Span]) -> SpanExportResult: | ||||||
| zipkin_spans = self._translate_to_zipkin(spans) | ||||||
| result = requests.post( | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. general question: does the SpanExporterer have any handling for when an export takes too long? I was going to suggest putting a timeout on this request, but it seems like, to keep the export working, we would need to provide a more general timeout mechanism, potentially in the caller to export.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good point, i don't know that the exporter currently does anything here. From some quick testing, it looks like the exporter could hang indefinitely waiting for an export to complete. |
||||||
| url=self.url, data=json.dumps(zipkin_spans), headers=ZIPKIN_HEADERS | ||||||
| ) | ||||||
|
|
||||||
| if result.status_code not in SUCCESS_STATUS_CODES: | ||||||
| logger.error( | ||||||
| "Traces cannot be uploaded; status code: %s, message %s", | ||||||
| result.status_code, | ||||||
| result.text, | ||||||
| ) | ||||||
|
|
||||||
| if self.retry: | ||||||
| return SpanExportResult.FAILED_RETRYABLE | ||||||
| return SpanExportResult.FAILED_NOT_RETRYABLE | ||||||
| return SpanExportResult.SUCCESS | ||||||
|
|
||||||
| def _translate_to_zipkin(self, spans: Sequence[Span]): | ||||||
|
|
||||||
| local_endpoint = { | ||||||
| "serviceName": self.service_name, | ||||||
| "port": self.port, | ||||||
| } | ||||||
|
|
||||||
| if self.ipv4 is not None: | ||||||
| local_endpoint["ipv4"] = self.ipv4 | ||||||
|
|
||||||
| if self.ipv6 is not None: | ||||||
| local_endpoint["ipv6"] = self.ipv6 | ||||||
|
|
||||||
| zipkin_spans = [] | ||||||
| for span in spans: | ||||||
| context = span.get_context() | ||||||
| trace_id = context.trace_id | ||||||
| span_id = context.span_id | ||||||
|
|
||||||
| # Timestamp in zipkin spans is int of microseconds. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. is there a reference we can link here on that?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. added a link to the docs |
||||||
| # see: https://zipkin.io/pages/instrumenting.html | ||||||
| start_timestamp_mus = _nsec_to_usec_round(span.start_time) | ||||||
| duration_mus = _nsec_to_usec_round(span.end_time - span.start_time) | ||||||
|
|
||||||
| zipkin_span = { | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it looks like there's no handling of the "debug" field here, which could be extracted from the boolean flags in the TraceOptions.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. theoretically we could include debug as a bit in trace_options, or use the sampled bit and consider that debug: https://www.w3.org/TR/trace-context-1/#trace-flags
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. i didn't see anything referencing a debug field in the zipkin API https://zipkin.io/zipkin-api/#/default/post_spans. can you elaborate on this?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Documentation is in that page under Models - Span https://zipkin.io/zipkin-api/#/default/post_spans
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Easy enough to use the sampling bit. |
||||||
| "traceId": format(trace_id, "x"), | ||||||
| "id": format(span_id, "x"), | ||||||
| "name": span.name, | ||||||
| "timestamp": start_timestamp_mus, | ||||||
| "duration": duration_mus, | ||||||
| "localEndpoint": local_endpoint, | ||||||
| "kind": SPAN_KIND_MAP[span.kind], | ||||||
| "tags": _extract_tags_from_span(span.attributes), | ||||||
| "annotations": _extract_annotations_from_events(span.events), | ||||||
| } | ||||||
|
|
||||||
| if context.trace_options.sampled: | ||||||
| zipkin_span["debug"] = 1 | ||||||
|
|
||||||
| if isinstance(span.parent, Span): | ||||||
| zipkin_span["parentId"] = format( | ||||||
| span.parent.get_context().span_id, "x" | ||||||
| ) | ||||||
| elif isinstance(span.parent, SpanContext): | ||||||
| zipkin_span["parentId"] = format(span.parent.span_id, "x") | ||||||
|
|
||||||
| zipkin_spans.append(zipkin_span) | ||||||
| return zipkin_spans | ||||||
|
|
||||||
| def shutdown(self) -> None: | ||||||
| pass | ||||||
|
|
||||||
|
|
||||||
| def _extract_tags_from_span(attr): | ||||||
| if not attr: | ||||||
| return None | ||||||
| tags = {} | ||||||
| for attribute_key, attribute_value in attr.items(): | ||||||
| if isinstance(attribute_value, (int, bool, float)): | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. is this case allowed? it seems like we should do some enforcement on the creation of attributes spans, rather than try to catch it on the flush side here.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Attributes are assigned one of the following values:
Looks like int is not one of them though, i'll remove it.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Although that isn't enforced at this point... I can create a separate ticket for this
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm in issues anyway: #347. Thanks! |
||||||
| value = str(attribute_value) | ||||||
| elif isinstance(attribute_value, str): | ||||||
| value = attribute_value[:128] | ||||||
| else: | ||||||
| logger.warning("Could not serialize tag %s", attribute_key) | ||||||
| continue | ||||||
| tags[attribute_key] = value | ||||||
| return tags | ||||||
|
|
||||||
|
|
||||||
| def _extract_annotations_from_events(events): | ||||||
| return ( | ||||||
| [ | ||||||
| {"timestamp": _nsec_to_usec_round(e.timestamp), "value": e.name} | ||||||
| for e in events | ||||||
| ] | ||||||
| if events | ||||||
| else None | ||||||
| ) | ||||||
|
|
||||||
|
|
||||||
| def _nsec_to_usec_round(nsec): | ||||||
| """Round nanoseconds to microseconds""" | ||||||
| return (nsec + 500) // 10 ** 3 | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # Copyright 2019, OpenTelemetry Authors | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| __version__ = "0.3.dev0" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # Copyright 2019, OpenTelemetry Authors | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we enforce this on the package? I have a feeling it will make it hard for people to adopt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what would you recommend here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nevermind, I think I forgot the usage of the compatible release qualifier. This looks good.