Skip to content

Add support for supplemental configuration#1371

Merged
jefchien merged 5 commits intomainfrom
ot-build
Oct 4, 2024
Merged

Add support for supplemental configuration#1371
jefchien merged 5 commits intomainfrom
ot-build

Conversation

@jefchien
Copy link
Contributor

@jefchien jefchien commented Oct 4, 2024

Description of changes

  • Adds support for new components
Receivers Processors Extensions
awsecscontainermetricsreceiver attributesprocessor ballastextension
filelogreceiver deltatorateprocessor ecsobserver
jaegerreceiver groupbytraceprocessor filestorage
kafkareceiver k8sattributesprocessor healthcheckextension
prometheusreceiver memorylimiterprocessor pprofextension
statsdreceiver metricsgenerationprocessor sigv4authextension
zipkinreceiver probabilisticsamplerprocessor zpagesextension
spanprocessor
tailsamplingprocessor
  • Updates config-translator to ignore YAML files in the config-downloader directory (or /etc/cwaconfig when running in a container)
    • start-amazon-cloudwatch-agent will detect YAML files in the same directory and will add them as supplemental configurations in lexical order
  • Supports conflict detection when merging in the supplemental configuration file
    • Prevents components and pipelines from being overwritten

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

Added unit tests for functional changes. Ran integration tests (https://github.com/aws/amazon-cloudwatch-agent/actions/runs/11170956432) to verify that there are no regressions if not enabled.

Built and tested the agent manually

With merge conflicts:

2024-09-28T19:40:53Z E! Error running agent: failed to merge OTEL configs: merge conflict in receivers: [otlp], service::pipelines: [metrics]

Without merge conflicts:

2024-09-28T20:19:27Z D! Merging multiple OTEL configurations: [/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_test.yaml /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.yaml]
2024-09-28T20:19:27Z I! Merged OTEL configuration: 
exporters:
    awscloudwatchlogs/emf_logs:
        certificate_file_path: ""
        emf_only: true
        endpoint: ""
        imds_retries: 1
        local_mode: false
        log_group_name: emf/logs/default
        log_retention: 0
        log_stream_name: i-051caf5fb177e145e
        max_retries: 2
        middleware: agenthealth/logs
        no_verify_ssl: false
        num_workers: 8
        profile: ""
        proxy_address: ""
        raw_log: true
        region: us-east-1
        request_timeout_seconds: 30
        resource_arn: ""
        retry_on_failure:
            enabled: true
            initial_interval: 5s
            max_elapsed_time: 5m0s
            max_interval: 30s
            multiplier: 1.5
            randomization_factor: 0.5
        role_arn: ""
        sending_queue:
            enabled: true
            num_consumers: 1
            queue_size: 1000
    awsemf:
        certificate_file_path: ""
        detailed_metrics: false
        dimension_rollup_option: ZeroAndSingleDimensionRollup
        disable_metric_extraction: false
        eks_fargate_container_insights_enabled: false
        endpoint: ""
        enhanced_container_insights: false
        imds_retries: 0
        local_mode: false
        log_group_name: ""
        log_retention: 0
        log_stream_name: ""
        max_retries: 2
        namespace: ""
        no_verify_ssl: false
        num_workers: 8
        output_destination: cloudwatch
        profile: ""
        proxy_address: ""
        region: ""
        request_timeout_seconds: 30
        resource_arn: ""
        resource_to_telemetry_conversion:
            enabled: false
        retain_initial_value_of_delta_metric: false
        role_arn: ""
        version: "1"
    awsxray:
        certificate_file_path: ""
        endpoint: ""
        imds_retries: 0
        index_all_attributes: false
        local_mode: false
        max_retries: 2
        no_verify_ssl: false
        num_workers: 8
        profile: ""
        proxy_address: ""
        region: ""
        request_timeout_seconds: 30
        resource_arn: ""
        role_arn: ""
extensions:
    agenthealth/logs:
        is_usage_data_enabled: true
        stats:
            operations:
                - PutLogEvents
            usage_flags:
                mode: EC2
                region_type: EC2M
    health_check:
        check_collector_pipeline:
            enabled: false
            exporter_failure_threshold: 5
            interval: 5m
        endpoint: 0.0.0.0:13133
        include_metadata: false
        max_request_body_size: 0
        path: /
processors:
    batch/emf_logs:
        metadata_cardinality_limit: 1000
        send_batch_max_size: 0
        send_batch_size: 8192
        timeout: 5s
    batch/metrics:
        metadata_cardinality_limit: 1000
        send_batch_max_size: 0
        send_batch_size: 8192
        timeout: 1m0s
    batch/traces:
        metadata_cardinality_limit: 1000
        send_batch_max_size: 0
        send_batch_size: 50
        timeout: 1s
receivers:
    awsxray:
        dialer:
            timeout: 0s
        endpoint: 0.0.0.0:2000
        proxy_server:
            aws_endpoint: ""
            certificate_file_path: ""
            dialer:
                timeout: 0s
            endpoint: 0.0.0.0:2000
            imds_retries: 0
            local_mode: false
            profile: ""
            proxy_address: ""
            region: ""
            role_arn: ""
            service_name: xray
        transport: udp
    otlp:
        protocols:
            grpc:
                dialer:
                    timeout: 0s
                endpoint: 0.0.0.0:4317
                include_metadata: false
                max_concurrent_streams: 0
                max_recv_msg_size_mib: 0
                read_buffer_size: 524288
                transport: tcp
                write_buffer_size: 0
            http:
                endpoint: 0.0.0.0:4318
                include_metadata: false
                logs_url_path: /v1/logs
                max_request_body_size: 0
                metrics_url_path: /v1/metrics
                traces_url_path: /v1/traces
    tcplog/emf_logs:
        encoding: utf-8
        id: tcp_input
        listen_address: 0.0.0.0:25888
        operators: []
        retry_on_failure:
            enabled: false
            initial_interval: 0s
            max_elapsed_time: 0s
            max_interval: 0s
        type: tcp_input
    udplog/emf_logs:
        encoding: utf-8
        id: udp_input
        listen_address: 0.0.0.0:25888
        multiline:
            line_end_pattern: .^
            line_start_pattern: ""
            omit_pattern: false
        operators: []
        retry_on_failure:
            enabled: false
            initial_interval: 0s
            max_elapsed_time: 0s
            max_interval: 0s
        type: udp_input
service:
    extensions:
        - agenthealth/logs
        - health_check
    pipelines:
        logs/emf_logs:
            exporters:
                - awscloudwatchlogs/emf_logs
            processors:
                - batch/emf_logs
            receivers:
                - tcplog/emf_logs
                - udplog/emf_logs
        metrics:
            exporters:
                - awsemf
            processors:
                - batch/metrics
            receivers:
                - otlp
        traces:
            exporters:
                - awsxray
            processors:
                - batch/traces
            receivers:
                - otlp
                - awsxray
    telemetry:
        logs:
            development: false
            disable_caller: false
            disable_stacktrace: false
            encoding: console
            error_output_paths:
                - stderr
            level: debug
            output_paths:
                - /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log
            sampling:
                enabled: true
                initial: 2
                thereafter: 500
                tick: 10s
        metrics:
            address: ""
            level: None
        traces: {}

2024-09-28T20:19:27Z I! {"caller":"service@v0.103.0/service.go:115","msg":"Setting up own telemetry..."}
2024-09-28T20:19:27Z I! {"caller":"service@v0.103.0/service.go:156","msg":"Skipped telemetry setup.","address":"","level":"None"}
2024-09-28T20:19:27Z W! {"caller":"awsemfexporter@v0.103.0/emf_exporter.go:99","msg":"the default value for DimensionRollupOption will be changing to NoDimensionRollupin a future release. See https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23997 for moreinformation","kind":"exporter","data_type":"metrics","name":"awsemf"}
2024-09-28T20:19:27Z I! {"caller":"awsxrayreceiver@v0.103.0/receiver.go:45","msg":"Going to listen on endpoint for X-Ray segments","kind":"receiver","name":"awsxray","data_type":"traces","udp":"0.0.0.0:2000"}
2024-09-28T20:19:27Z I! {"caller":"udppoller/poller.go:95","msg":"Listening on endpoint for X-Ray segments","kind":"receiver","name":"awsxray","data_type":"traces","udp":"0.0.0.0:2000"}
2024-09-28T20:19:27Z I! {"caller":"awsxrayreceiver@v0.103.0/receiver.go:56","msg":"Listening on endpoint for X-Ray segments","kind":"receiver","name":"awsxray","data_type":"traces","udp":"0.0.0.0:2000"}
2024-09-28T20:19:27Z I! {"caller":"service@v0.103.0/service.go:182","msg":"Starting CWAgent...","Version":"1.300042.1-43-g248c1e06-dirty","NumCPU":2}
2024-09-28T20:19:27Z I! {"caller":"extensions/extensions.go:34","msg":"Starting extensions..."}
2024-09-28T20:19:27Z I! {"caller":"extensions/extensions.go:37","msg":"Extension is starting...","kind":"extension","name":"health_check"}
2024-09-28T20:19:27Z I! {"caller":"healthcheckextension@v0.103.0/healthcheckextension.go:32","msg":"Starting health_check extension","kind":"extension","name":"health_check","config":{"Endpoint":"0.0.0.0:13133","TLSSetting":null,"CORS":null,"Auth":null,"MaxRequestBodySize":0,"IncludeMetadata":false,"ResponseHeaders":null,"CompressionAlgorithms":null,"Path":"/","ResponseBody":null,"CheckCollectorPipeline":{"Enabled":false,"Interval":"5m","ExporterFailureThreshold":5}}}
2024-09-28T20:19:27Z W! {"caller":"internal@v0.103.0/warning.go:42","msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks. Enable the feature gate to change the default and remove this warning.","kind":"extension","name":"health_check","documentation":"https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks","feature gate ID":"component.UseLocalHostAsDefaultHost"}
2024-09-28T20:19:27Z I! {"caller":"extensions/extensions.go:52","msg":"Extension started.","kind":"extension","name":"health_check"}
2024-09-28T20:19:27Z I! {"caller":"extensions/extensions.go:37","msg":"Extension is starting...","kind":"extension","name":"agenthealth/logs"}
2024-09-28T20:19:27Z I! {"caller":"extensions/extensions.go:52","msg":"Extension started.","kind":"extension","name":"agenthealth/logs"}
2024-09-28T20:19:27Z W! {"caller":"internal@v0.103.0/warning.go:42","msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks. Enable the feature gate to change the default and remove this warning.","kind":"receiver","name":"otlp","data_type":"traces","documentation":"https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks","feature gate ID":"component.UseLocalHostAsDefaultHost"}
2024-09-28T20:19:27Z I! {"caller":"zapgrpc/zapgrpc.go:176","msg":"[core] [Server #1]Server created","grpc_log":true}
2024-09-28T20:19:27Z I! {"caller":"otlpreceiver@v0.103.0/otlp.go:102","msg":"Starting GRPC server","kind":"receiver","name":"otlp","data_type":"traces","endpoint":"0.0.0.0:4317"}
2024-09-28T20:19:27Z I! {"caller":"otlpreceiver@v0.103.0/otlp.go:152","msg":"Starting HTTP server","kind":"receiver","name":"otlp","data_type":"traces","endpoint":"0.0.0.0:4318"}
2024-09-28T20:19:27Z I! {"caller":"awsxrayreceiver@v0.103.0/receiver.go:90","msg":"X-Ray TCP proxy server started","kind":"receiver","name":"awsxray","data_type":"traces"}
2024-09-28T20:19:27Z I! {"caller":"zapgrpc/zapgrpc.go:176","msg":"[core] [Server #1 ListenSocket #2]ListenSocket created","grpc_log":true}
2024-09-28T20:19:27Z I! {"caller":"adapter/receiver.go:46","msg":"Starting stanza receiver","kind":"receiver","name":"tcplog/emf_logs","data_type":"logs"}
2024-09-28T20:19:27Z I! {"caller":"adapter/receiver.go:46","msg":"Starting stanza receiver","kind":"receiver","name":"udplog/emf_logs","data_type":"logs"}
2024-09-28T20:19:27Z I! {"caller":"healthcheck/handler.go:132","msg":"Health Check state change","kind":"extension","name":"health_check","status":"ready"}
2024-09-28T20:19:27Z I! {"caller":"service@v0.103.0/service.go:208","msg":"Everything is ready. Begin running and processing data."}
2024-09-28T20:19:27Z W! {"caller":"localhostgate/featuregate.go:63","msg":"The default endpoints for all servers in components will change to use localhost instead of 0.0.0.0 in a future version. Use the feature gate to preview the new default.","feature gate ID":"component.UseLocalHostAsDefaultHost"}

Requirements

Before commit the code, please do the following steps.

  1. Run make fmt and make fmt-sh
  2. Run make lint

@jefchien jefchien requested a review from a team as a code owner October 4, 2024 14:58
movence
movence reviewed Oct 4, 2024
View reviewed changes
mitali-salvi
mitali-salvi previously approved these changes Oct 4, 2024
View reviewed changes
okankoAMZ
okankoAMZ approved these changes Oct 4, 2024
View reviewed changes
Copy link
Contributor

@okankoAMZ okankoAMZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@jefchien jefchien merged commit f5edb3f into main Oct 4, 2024
@jefchien jefchien deleted the ot-build branch October 4, 2024 21:12
@jefchien jefchien mentioned this pull request Feb 16, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@movence movence movence left review comments

@mitali-salvi mitali-salvi mitali-salvi approved these changes

@okankoAMZ okankoAMZ okankoAMZ approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@jefchien @movence @mitali-salvi @okankoAMZ @musa-asad