Skip to content

feat: enable log forwarding through otel collector#186

Merged
florentianayuwono merged 26 commits intomainfrom
feat/log-forwarding-isd-5146
Apr 29, 2026
Merged

feat: enable log forwarding through otel collector#186
florentianayuwono merged 26 commits intomainfrom
feat/log-forwarding-isd-5146

Conversation

@florentianayuwono
Copy link
Copy Markdown
Collaborator

@florentianayuwono florentianayuwono commented Apr 22, 2026

What this PR does

Add action to allow workflow authors to opt in to forwarding specific log files from self-hosted GitHub runners to Loki through the OpenTelemetry Collector snap.

Why we need it

So that workflow author can opt-in to forwarding specific log files from their job to Grafana, and view application logs alongside system metrics without changing their existing logging approach.

Checklist

  • Changes comply with the project's coding standards and guidelines (see CONTRIBUTING.md and STYLE.md)
  • CONTRIBUTING.md has been updated upon changes to the contribution/development process (e.g. changes to the way tests are run)
  • Technical author has been assigned to review the PR in case of documentation changes (usually *.md files)
  • I updated docs/changelog.md with user-relevant changes
  • I used AI to assist with preparing this PR
  • I added or updated tests as needed (unit and integration)
  • If integration test modules are used: I updated the workflow configuration
    (e.g., in .github/workflows/integration_tests.yaml, ensure the modules list is correct)
  • If this PR involves a Grafana dashboard: I added a screenshot of the dashboard
  • If this PR involves Terraform: terraform fmt passes and tflint reports no errors
  • If this PR involves Rockcraft: I updated the version

florentianayuwono and others added 9 commits April 22, 2026 22:53
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…v v8 (#182)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* fix(dashboard): remove dead override hiding job queue time series

The "Job queue time" panel had a leftover hideSeriesFrom override that
excluded every series except one specific named expression. Combined
with the panel's current query (which already aggregates with sum by
(le)), the override hid all data, leaving an empty chart.

The override is obsolete now that the query collapses every label
except le into a single combined histogram, so removing it restores
visualisation without any other change.

* ci: pin astral-sh/setup-uv to v8.1.0

The astral-sh/setup-uv repository does not publish a floating v8 major
tag (only v8.0.0 and v8.1.0 specific tags exist), so referencing @v8
fails to resolve and breaks the workflow on every PR.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Christopher Bartz <christopher.bartz@canonical.com>
* chore(docs): update contributing guidelines (charmkeeper)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(docs): build errors

* chore: clarify PR checklist and remove unnecessary item

* chore(docs): revert changes in CONTRIBUTING.md

* fix(docs): whoops we're ignoring the changelog

* fix: update pr checklist to incorporate previous items, remove duplicate items

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new composite GitHub Action that can opt workflows into forwarding selected log files from self-hosted runners to Loki via an OpenTelemetry Collector snap.

Changes:

  • Introduces actions/enable-log-forwarding composite action that writes an otelcol config fragment and restarts the collector.
  • Adds unit tests and a CI workflow (including a self-hosted smoke test) for the new action.
  • Documents usage in a new how-to guide and records the change in the docs changelog.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
docs/how-to/enable-log-forwarding.md New how-to page describing how to use the log-forwarding action and query hints.
docs/changelog.md Adds a 2026-04-22 entry documenting the new action.
actions/enable-log-forwarding/tests/test_enable_log_forwarding.py Unit tests for parsing inputs, endpoint resolution, exporter detection, and config generation.
actions/enable-log-forwarding/enable-log-forwarding.py Action implementation that generates an otelcol config fragment, installs it, and restarts the collector.
actions/enable-log-forwarding/action.yaml Composite action definition and inputs wiring to the Python script.
.github/workflows/enable_log_forwarding_action_tests.yaml CI workflow to run unit tests and a self-hosted smoke test for the action.

Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
Comment thread docs/how-to/enable-log-forwarding.md Outdated
Comment thread actions/enable-log-forwarding/action.yaml Outdated
Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
Comment thread docs/how-to/enable-log-forwarding.md Outdated
Comment thread docs/how-to/enable-log-forwarding.md
Comment thread actions/enable-log-forwarding/action.yaml Outdated
Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
Comment thread docs/how-to/enable-log-forwarding.md
Copy link
Copy Markdown
Contributor

@erinecon erinecon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a how-to guide with this PR! Some initial comments and questions

Comment thread docs/how-to/enable-log-forwarding.md Outdated
Comment thread docs/how-to/enable-log-forwarding.md Outdated
Comment thread docs/how-to/enable-log-forwarding.md Outdated
Comment thread docs/how-to/enable-log-forwarding.md Outdated
Comment thread docs/how-to/enable-log-forwarding.md
Comment thread docs/how-to/enable-log-forwarding.md Outdated
Comment thread docs/how-to/enable-log-forwarding.md Outdated
Comment thread docs/how-to/enable-log-forwarding.md
Copy link
Copy Markdown
Member

@yanksyoon yanksyoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial review done, thank you!

Comment thread .github/workflows/enable_log_forwarding_action_tests.yaml Outdated
Comment thread .github/workflows/enable_log_forwarding_action_tests.yaml Outdated
Comment thread .github/workflows/enable_log_forwarding_action_tests.yaml Outdated
Comment thread actions/enable-log-forwarding/action.yaml
Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
Comment thread actions/enable_log_forwarding/tests/test_enable_log_forwarding.py
florentianayuwono and others added 4 commits April 23, 2026 10:21
Co-authored-by: Erin Conley <erin.conley@canonical.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Erin Conley <erin.conley@canonical.com>
Comment thread docs/how-to/index.rst
Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
Comment thread actions/enable-log-forwarding/enable-log-forwarding.py Outdated
florentianayuwono and others added 3 commits April 23, 2026 20:02
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Comment thread docs/how-to/enable-log-forwarding.md Outdated
florentianayuwono and others added 4 commits April 24, 2026 01:04
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Comment thread docs/how-to/index.rst
Copy link
Copy Markdown
Member

@yanksyoon yanksyoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more nits, almost there! Thanks @florentianayuwono :)

Comment thread .github/workflows/enable_log_forwarding_action_tests.yaml
Comment thread .github/workflows/enable_log_forwarding_action_tests.yaml Outdated
Comment thread actions/enable_log_forwarding/tests/test_enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/tests/test_enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/tests/test_enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
Copy link
Copy Markdown
Collaborator Author

@florentianayuwono florentianayuwono left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review session from weii

Comment thread actions/enable_log_forwarding/tests/test_enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/collector_config.j2 Outdated
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
Comment thread docs/how-to/enable-log-forwarding.md Outdated
Comment thread docs/how-to/enable-log-forwarding.md
Copy link
Copy Markdown
Contributor

@erinecon erinecon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve the documentation changes 👍 since there's code-based changes in this PR, I didn't provide an official approval.

Spotted a couple more nits + responded to a Copilot comment

Comment thread docs/how-to/enable-log-forwarding.md
Comment thread docs/how-to/enable-log-forwarding.md Outdated
Comment thread docs/how-to/index.rst
Comment thread docs/how-to/index.rst
Co-authored-by: Erin Conley <erin.conley@canonical.com>
Comment thread actions/enable_log_forwarding/action.yaml Outdated
Comment thread actions/enable_log_forwarding/action.yaml Outdated
Comment thread actions/enable_log_forwarding/action.yaml Outdated
Comment thread actions/enable_log_forwarding/action.yaml Outdated
Comment thread tox.ini
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
Comment thread actions/enable_log_forwarding/enable_log_forwarding.py Outdated
florentianayuwono and others added 3 commits April 29, 2026 11:21
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Comment thread actions/enable-log-forwarding/tests/test_enable_log_forwarding.py Outdated
Comment thread actions/enable-log-forwarding/enable_log_forwarding.py
Co-authored-by: Copilot <copilot@github.com>
@florentianayuwono florentianayuwono merged commit a435a1b into main Apr 29, 2026
40 checks passed
@florentianayuwono florentianayuwono deleted the feat/log-forwarding-isd-5146 branch April 29, 2026 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants