Add a basic http OpAMP client

[xrmx]

Description

This introduces a basic OpAMP http client for handling remote configuration. The client implements a bunch of capabilities (ReportsStatus, ReportsHeartbeat, AcceptsRemoteConfig, ReportsRemoteConfig) that are enough to get a remote config from an opamp server, parse it, apply it and ack it. Since OTel / OpAMP do not standardize APIs, config options or environment variables the distros are required to provide code doing so.
OTel Python distros would need to provide their own message handler callback that implements the actual change of whatever configuration their backends sends.

In practice distro would need to do something like the following:

from opentelemetry._opamp import messages
from opentelemetry._opamp.agent import OpAMPAgent
from opentelemetry._opamp.client import OpAMPClient
from opentelemetry._opamp.proto import opamp_pb2 as opamp_pb2


def opamp_handler(agent: OpAMPAgent, client: OpAMPClient, message: opamp_pb2.ServerToAgent):
    for config_filename, config in messages._decode_remote_config(message.remote_config):
        print("do something")


class MyOpenTelemetryConfigurator(_OTelSDKConfigurator):
    def _configure(self, **kwargs):
        super()._configure(**kwargs)

        enable_opamp = False
        endpoint = os.environ.get(OTEL_OPAMP_ENDPOINT)
        if endpoint:
            # this is not great but we don't have the calculated resource attributes around
            # see https://github.com/open-telemetry/opentelemetry-python/pull/4646 for creating
            # an entry point distros can implement
            resource = OTELResourceDetector().detect()
            agent_identifying_attributes = {
                "service.name": resource.attributes.get("service.name"),
            }
            opamp_client = OpAMPClient(
                endpoint=endpoint,
                agent_identifying_attributes=agent_identifying_attributes,
            )
            opamp_agent = OpAMPAgent(
                interval=30,
                message_handler=opamp_handler,
                client=opamp_client,
            )
            opamp_agent.start()

The module is called _opamp because it's a bit early to standardize on an api. The code is divided roughly in:

• agent: handles threads and queues for sending messages to the server, supports sending heartbeat messages at a fixed interval
• client: expose the api to build and send the OpAMP messages
• messages: wrappers to simplify protobuf serialization
• transports: http backends, currently there is only one using requests (to match the exporters)

OpAMP reference: https://opentelemetry.io/docs/specs/opamp/.

This is tested against https://github.com/elastic/opentelemetry-collector-components/ that is using the opamp-go implementation.

TODO:

• write some docs with usage examples

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• tox

Does This PR Require a Core Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[danielgblanco]

@open-telemetry/opamp-spec-approvers could you help review this? thanks!

[Kludex]

I can't unresolve threads, but I've replied above.

[tigrannajaryan]

Thank you for working on this.

This is tested against https://github.com/elastic/opentelemetry-collector-components/ that is using the opamp-go implementation.

One thing that would be great to add is interoperability tests between Go and Python implementations. Ideally we would have both pairs (Client in Python, Server in Go and the vice versa) connecting and performing OpAMP exchanges to make sure the pair works correctly together.

OpAMP Go implementation presumably is the most complete implementation at the moment, so all other languages (including Python) could use it as a reference implementation to test against. I think we can spend a bit time adding any necessary tooling to opamp-go that makes this possible (mock servers, clients that probe capabilities, etc). Unfortunately I don't have time myself but if anyone wants to work on the design and implementation I can dedicate some time to review the design.

[xrmx]

Thank you for working on this.

This is tested against https://github.com/elastic/opentelemetry-collector-components/ that is using the opamp-go implementation.

One thing that would be great to add is interoperability tests between Go and Python implementations. Ideally we would have both pairs (Client in Python, Server in Go and the vice versa) connecting and performing OpAMP exchanges to make sure the pair works correctly together.

This PR has some recorded e2e tests doing this, we don't test many scenarios but at least we test against a real response.

OpAMP Go implementation presumably is the most complete implementation at the moment, so all other languages (including Python) could use it as a reference implementation to test against. I think we can spend a bit time adding any necessary tooling to opamp-go that makes this possible (mock servers, clients that probe capabilities, etc). Unfortunately I don't have time myself but if anyone wants to work on the design and implementation I can dedicate some time to review the design.

That would be helpful indeed

[aabmass]

Giving this an approving "python review" but I don't know much about OpAmp. Looks good enough to merge though as an initial prototype with proper disclaimers.

🚢

[kelseyma]

Hi @xrmx, thanks so much for creating this! I’ve been using the client for a PoC and had a few questions as I was going through the changes.

• I noticed that opamp-go and the Java opamp-client define some additional callbacks. I was wondering if there are any plans to add similar callbacks to the Python client, particularly around error response handling?
• Are there plans to support additional content types for configs, or potentially generalize the config methods to work for any content type?
• Are there any plans to add more message options, or possibly introduce a helper to construct messages automatically based on updated fields (similar to opamp-go’s NextMessage)?

Totally understand if these are out of scope for this PR, wanted to ask in case there's been any discussion around them. I’m also happy to move this discussion to a new issue and take on the implementation work if that would be helpful. Thanks again!

[xrmx]

Hi @xrmx, thanks so much for creating this! I’ve been using the client for a PoC and had a few questions as I was going through the changes.

* I noticed that `opamp-go` and the Java `opamp-client` define some additional callbacks. I was wondering if there are any plans to add similar callbacks to the Python client, particularly around error response handling?

I think can change the interface to match theirs, now you have just one callback and you have to do that on your own

* Are there plans to support additional content types for configs, or potentially generalize the config methods to work for any content type?

This stuff is not specified so I've implemented only what we are using :) Of course we can generalize that. Maybe some specs will come from open-telemetry/opentelemetry-specification#4738

* Are there any plans to add more message options, or possibly introduce a helper to construct messages automatically based on updated fields (similar to opamp-go’s `NextMessage`)?

Wasn't aware of that, I'll need to check that

Totally understand if these are out of scope for this PR, wanted to ask in case there's been any discussion around them. I’m also happy to move this discussion to a new issue and take on the implementation work if that would be helpful. Thanks again!

May be out of scope for this PR but perfectly fine to discuss. Thanks!

In the last push I've made this package released on its own so we can iterate on this without interfering with the releases.

[pmcollins]

Ran changes locally and LGTM. Thanks for doing this.