Skip to content

Conversation

@pront
Copy link
Member

@pront pront commented Nov 10, 2025

Summary

Unifies reload metrics by removing duplicate config_reload_rejected and config_reloaded metrics in favor of existing component_errors_total and reloaded_total metrics. This significantly reduces complexity and simplifies monitoring.

  • Now component_errors_total with error_code="reload" include a reason tag with values:

    • global_options_changed - Non-reloadable global options changed
    • global_diff_failed - Failed to compute global config diff
    • topology_build_failed - New topology failed to build/healthcheck
    • restore_failed - Rollback failed (fatal)
  • Refactored reload_config_and_respawn

    • return type from Result<(bool, Option<ReloadRejectionReason>), ()> to a much cleaner Result<(), ReloadError>
    • this allows for handling all emit! logic in a single place

We can consider this a non-breaking change - the removed metrics were never actually emitted due to abug, and adding the reason tag is a non-breaking change. Happy we caught this early.

Vector configuration

#data_dir: /Users/pavlos.rontidis/CLionProjects/vector/pront/data/data_dir_0

api:
  enabled: true

sources:
  s0:
    type: internal_metrics
    scrape_interval_secs: 10

transforms:
  filter_config_metrics:
    type: filter
    inputs: ["s0"]
    condition:
      type: vrl
      source: |
        starts_with!(.name, "reload")

sinks:
  console:
    inputs: ["filter_config_metrics"]
    type: console
    encoding:
      codec: "json"
#      pretty: true

How did you test this PR?

Run:

cargo run -- -w --config /Users/pavlos.rontidis/CLionProjects/vector/pront/configs/reload_metrics.yaml

Edited the fields multiple times and observed the internal metrics.

Also, updated existing unit tests.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

relates: #24150

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

@github-actions github-actions bot added the domain: topology Anything related to Vector's topology code label Nov 10, 2025
@pront pront force-pushed the pront-unify-config-reload-error-metrics branch from fd9be93 to 9ed688f Compare November 10, 2025 20:49
@pront pront added this to the 0.51.1 milestone Nov 10, 2025
@pront pront force-pushed the pront-unify-config-reload-error-metrics branch from 9ed688f to 077d729 Compare November 10, 2025 20:57
@pront pront force-pushed the pront-unify-config-reload-error-metrics branch from 077d729 to 9bc5907 Compare November 10, 2025 21:30
@pront pront force-pushed the pront-unify-config-reload-error-metrics branch from 9bc5907 to c8e5c5d Compare November 10, 2025 21:35
@pront pront marked this pull request as ready for review November 10, 2025 21:38
@pront pront requested a review from a team as a code owner November 10, 2025 21:38
@datadog-vectordotdev
Copy link

datadog-vectordotdev bot commented Nov 10, 2025

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 0d54efe | Docs | Was this helpful? Give us feedback!

@pront pront enabled auto-merge November 11, 2025 14:45
@pront pront added this pull request to the merge queue Nov 11, 2025
Merged via the queue into master with commit f90e7b5 Nov 11, 2025
45 checks passed
@pront pront deleted the pront-unify-config-reload-error-metrics branch November 11, 2025 15:30
thomasqueirozb pushed a commit that referenced this pull request Nov 11, 2025
#24203)

* chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError

* update topology_doesnt_reload_new_data_dir test

* chore(dev): cargo fmt

* strengthen tests, since now we have a better interface
thomasqueirozb pushed a commit that referenced this pull request Nov 11, 2025
#24203)

* chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError

* update topology_doesnt_reload_new_data_dir test

* chore(dev): cargo fmt

* strengthen tests, since now we have a better interface
thomasqueirozb added a commit that referenced this pull request Nov 11, 2025
* chore(ci): reorganize integration test files (#24108)

* chore(ci): reorganize integration test files

* revert nats data move

* fix paths to data

* fix more paths

* amqp fix

* Fix logstash int tests shared files paths

* Rename scripts/integration -> tests/integration

* scripts/integration -> tests/integration in source files

* Fix scripts->tests

* Fix gcp paths

* Fix http-client pem path

* move nats data

---------

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* chore(vdev): move all utils in a new utils folder (#24143)

* chore(vdev): move all utils in a new utils folder

* move remaining files to utils dir

* move remaining files to utils dir

* fmt

* chore(vdev): make modules visible to rustfmt (#24162)

* Remove mod inside cli_subcommands macro

* cargo fmt

* chore(vdev): apply vdev rust check fixes

* Link to rustfmt issue

* fix(vdev): release prepare vrl version pinning (#24158)

* Add --dry-run to release prepare

* Add error handling and checks to pin_vrl_version

* Add wrapper to toml

* Remove wrapper, parse as Table instead

* Fix vrl pinning logic

* Enable preserve_order feature in toml crate

* Use dependency instead of whole toml

* Fix dry run docs

* Fix dry run wording

* refactor to use toml_edit

* Add update_vrl_to_version to add unit test

* Use indoc in prepare.rs

* Remove preserve_order feature

* chore(deps): update VRL to add missing stdlib fns from 0.28 (#24178)

* chore(ci): temporarily remove homebrew publish step from publish workflow (#24185)

This temporarily removes the publish-homebrew job from the publish
workflow to address issue #24139. This is step 1 of the plan to fix
the homebrew publishing process.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>

* fix(blackhole sink): disable rate limiting for periodic stats messages (#24190)

* fix(blackhole sink): disable rate limiting for periodic stats messages

The blackhole sink's informational "Collected events" messages were being
rate-limited since rate limiting was enabled by default in #24024. This is
undesirable because:

1. These are deliberately scheduled periodic messages (controlled by the
   `print_interval_secs` config option), not error conditions that could
   flood the logs
2. Users explicitly configure the frequency - rate limiting defeats that
   explicit configuration and breaks user expectations
3. The interval timer already prevents log flooding, making additional
   rate limiting redundant
4. The blackhole sink is used for debugging/testing, where predictable
   output is essential

This fix adds `internal_log_rate_limit = false` to both info! calls,
similar to how the console sink disables rate limiting for its critical
operational messages.

Fixes #24188

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: add changelog fragment for blackhole sink rate limiting fix

---------

Co-authored-by: Claude <noreply@anthropic.com>

* chore(internal logs): Disable rate limiting for critical internal error logs (#24192)

* Improve debug log in aws_ecs_metrics

* Add internal_log_rate_limit = false to config reload errors

* Disable rate limiting for lua build failure

* Disable rate limiting for amqp build failure

* Add internal_log_rate_limit = false to important failures during reload

* Disable log rate limit for repeated events in process.rs

* Update src/config/watcher.rs

* Fix message format

* fix(tracing): prevent panic for traces without standard fields (#24191)

* fix(tracing): prevent panic for traces without std fields

* forbid unwrap in this lib

* clippy fix

* improve events_with_custom_fields_no_message_dont_panic, advance to next window

* chore(tracing): do not rate limit utlization report (#24202)

* chore(internal metrics): move config_reload_* metrics to VectorReload* (#24203)

* chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError

* update topology_doesnt_reload_new_data_dir test

* chore(dev): cargo fmt

* strengthen tests, since now we have a better interface

* chore(releasing): Add 0.51.0 known issues (#24211)

* Format fixes

* Add 0.51.0 known issues

* Add dots

* Reword log panic known issue

* Reword VRL missing fns

* Backtick versions

* Update website/cue/reference/releases/0.51.0.cue

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Fix cue docs fmt

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Bump vector version to 0.51.1

* Bump VRL to 0.28.1

* Generate 0.51.1 cue

* Add vrl_changelog

* Remove whitespace

* chore(releasing): Updated distribution/install.sh vector version to 0.51.1

* chore(releasing): Add 0.51.1 to versions.cue

* chore(releasing): Created release md file

* Add description

* Reorder items and fix format

* Finally fix styling

* Bump date

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
thomasqueirozb added a commit that referenced this pull request Nov 13, 2025
* chore(ci): reorganize integration test files (#24108)

* chore(ci): reorganize integration test files

* revert nats data move

* fix paths to data

* fix more paths

* amqp fix

* Fix logstash int tests shared files paths

* Rename scripts/integration -> tests/integration

* scripts/integration -> tests/integration in source files

* Fix scripts->tests

* Fix gcp paths

* Fix http-client pem path

* move nats data

---------

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* chore(vdev): move all utils in a new utils folder (#24143)

* chore(vdev): move all utils in a new utils folder

* move remaining files to utils dir

* move remaining files to utils dir

* fmt

* chore(vdev): make modules visible to rustfmt (#24162)

* Remove mod inside cli_subcommands macro

* cargo fmt

* chore(vdev): apply vdev rust check fixes

* Link to rustfmt issue

* fix(vdev): release prepare vrl version pinning (#24158)

* Add --dry-run to release prepare

* Add error handling and checks to pin_vrl_version

* Add wrapper to toml

* Remove wrapper, parse as Table instead

* Fix vrl pinning logic

* Enable preserve_order feature in toml crate

* Use dependency instead of whole toml

* Fix dry run docs

* Fix dry run wording

* refactor to use toml_edit

* Add update_vrl_to_version to add unit test

* Use indoc in prepare.rs

* Remove preserve_order feature

* chore(deps): update VRL to add missing stdlib fns from 0.28 (#24178)

* chore(ci): temporarily remove homebrew publish step from publish workflow (#24185)

This temporarily removes the publish-homebrew job from the publish
workflow to address issue #24139. This is step 1 of the plan to fix
the homebrew publishing process.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>

* fix(blackhole sink): disable rate limiting for periodic stats messages (#24190)

* fix(blackhole sink): disable rate limiting for periodic stats messages

The blackhole sink's informational "Collected events" messages were being
rate-limited since rate limiting was enabled by default in #24024. This is
undesirable because:

1. These are deliberately scheduled periodic messages (controlled by the
   `print_interval_secs` config option), not error conditions that could
   flood the logs
2. Users explicitly configure the frequency - rate limiting defeats that
   explicit configuration and breaks user expectations
3. The interval timer already prevents log flooding, making additional
   rate limiting redundant
4. The blackhole sink is used for debugging/testing, where predictable
   output is essential

This fix adds `internal_log_rate_limit = false` to both info! calls,
similar to how the console sink disables rate limiting for its critical
operational messages.

Fixes #24188

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: add changelog fragment for blackhole sink rate limiting fix

---------

Co-authored-by: Claude <noreply@anthropic.com>

* chore(internal logs): Disable rate limiting for critical internal error logs (#24192)

* Improve debug log in aws_ecs_metrics

* Add internal_log_rate_limit = false to config reload errors

* Disable rate limiting for lua build failure

* Disable rate limiting for amqp build failure

* Add internal_log_rate_limit = false to important failures during reload

* Disable log rate limit for repeated events in process.rs

* Update src/config/watcher.rs

* Fix message format

* fix(tracing): prevent panic for traces without standard fields (#24191)

* fix(tracing): prevent panic for traces without std fields

* forbid unwrap in this lib

* clippy fix

* improve events_with_custom_fields_no_message_dont_panic, advance to next window

* chore(tracing): do not rate limit utlization report (#24202)

* chore(internal metrics): move config_reload_* metrics to VectorReload* (#24203)

* chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError

* update topology_doesnt_reload_new_data_dir test

* chore(dev): cargo fmt

* strengthen tests, since now we have a better interface

* chore(releasing): Add 0.51.0 known issues (#24211)

* Format fixes

* Add 0.51.0 known issues

* Add dots

* Reword log panic known issue

* Reword VRL missing fns

* Backtick versions

* Update website/cue/reference/releases/0.51.0.cue

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Fix cue docs fmt

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Bump vector version to 0.51.1

* Bump VRL to 0.28.1

* Generate 0.51.1 cue

* Add vrl_changelog

* Remove whitespace

* chore(releasing): Updated distribution/install.sh vector version to 0.51.1

* chore(releasing): Add 0.51.1 to versions.cue

* chore(releasing): Created release md file

* Add description

* Reorder items and fix format

* Finally fix styling

* Bump date

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

chore(releasing): pull in `internal_logs` fix into 0.51.1 (#24225)

* fix(internal_logs source): remove rate limit (#24218)

* fix(internal_logs source): remove rate limit

* add changelog

* Debug commit

* Fix validated - Revert "Debug commit"

This reverts commit c7b9ec9.

* add unit test

* fix check-events

* Pull in internal_logs source fix

* Add internal_logs known issue

* Bump date

* Update website/cue/reference/releases/0.51.0.cue

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
thomasqueirozb added a commit that referenced this pull request Nov 13, 2025
* chore(ci): reorganize integration test files (#24108)

* chore(ci): reorganize integration test files

* revert nats data move

* fix paths to data

* fix more paths

* amqp fix

* Fix logstash int tests shared files paths

* Rename scripts/integration -> tests/integration

* scripts/integration -> tests/integration in source files

* Fix scripts->tests

* Fix gcp paths

* Fix http-client pem path

* move nats data

---------

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* chore(vdev): move all utils in a new utils folder (#24143)

* chore(vdev): move all utils in a new utils folder

* move remaining files to utils dir

* move remaining files to utils dir

* fmt

* chore(vdev): make modules visible to rustfmt (#24162)

* Remove mod inside cli_subcommands macro

* cargo fmt

* chore(vdev): apply vdev rust check fixes

* Link to rustfmt issue

* fix(vdev): release prepare vrl version pinning (#24158)

* Add --dry-run to release prepare

* Add error handling and checks to pin_vrl_version

* Add wrapper to toml

* Remove wrapper, parse as Table instead

* Fix vrl pinning logic

* Enable preserve_order feature in toml crate

* Use dependency instead of whole toml

* Fix dry run docs

* Fix dry run wording

* refactor to use toml_edit

* Add update_vrl_to_version to add unit test

* Use indoc in prepare.rs

* Remove preserve_order feature

* chore(deps): update VRL to add missing stdlib fns from 0.28 (#24178)

* chore(ci): temporarily remove homebrew publish step from publish workflow (#24185)

This temporarily removes the publish-homebrew job from the publish
workflow to address issue #24139. This is step 1 of the plan to fix
the homebrew publishing process.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>

* fix(blackhole sink): disable rate limiting for periodic stats messages (#24190)

* fix(blackhole sink): disable rate limiting for periodic stats messages

The blackhole sink's informational "Collected events" messages were being
rate-limited since rate limiting was enabled by default in #24024. This is
undesirable because:

1. These are deliberately scheduled periodic messages (controlled by the
   `print_interval_secs` config option), not error conditions that could
   flood the logs
2. Users explicitly configure the frequency - rate limiting defeats that
   explicit configuration and breaks user expectations
3. The interval timer already prevents log flooding, making additional
   rate limiting redundant
4. The blackhole sink is used for debugging/testing, where predictable
   output is essential

This fix adds `internal_log_rate_limit = false` to both info! calls,
similar to how the console sink disables rate limiting for its critical
operational messages.

Fixes #24188

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: add changelog fragment for blackhole sink rate limiting fix

---------

Co-authored-by: Claude <noreply@anthropic.com>

* chore(internal logs): Disable rate limiting for critical internal error logs (#24192)

* Improve debug log in aws_ecs_metrics

* Add internal_log_rate_limit = false to config reload errors

* Disable rate limiting for lua build failure

* Disable rate limiting for amqp build failure

* Add internal_log_rate_limit = false to important failures during reload

* Disable log rate limit for repeated events in process.rs

* Update src/config/watcher.rs

* Fix message format

* fix(tracing): prevent panic for traces without standard fields (#24191)

* fix(tracing): prevent panic for traces without std fields

* forbid unwrap in this lib

* clippy fix

* improve events_with_custom_fields_no_message_dont_panic, advance to next window

* chore(tracing): do not rate limit utlization report (#24202)

* chore(internal metrics): move config_reload_* metrics to VectorReload* (#24203)

* chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError

* update topology_doesnt_reload_new_data_dir test

* chore(dev): cargo fmt

* strengthen tests, since now we have a better interface

* chore(releasing): Add 0.51.0 known issues (#24211)

* Format fixes

* Add 0.51.0 known issues

* Add dots

* Reword log panic known issue

* Reword VRL missing fns

* Backtick versions

* Update website/cue/reference/releases/0.51.0.cue

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Fix cue docs fmt

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Bump vector version to 0.51.1

* Bump VRL to 0.28.1

* Generate 0.51.1 cue

* Add vrl_changelog

* Remove whitespace

* chore(releasing): Updated distribution/install.sh vector version to 0.51.1

* chore(releasing): Add 0.51.1 to versions.cue

* chore(releasing): Created release md file

* Add description

* Reorder items and fix format

* Finally fix styling

* Bump date

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

chore(releasing): pull in `internal_logs` fix into 0.51.1 (#24225)

* fix(internal_logs source): remove rate limit (#24218)

* fix(internal_logs source): remove rate limit

* add changelog

* Debug commit

* Fix validated - Revert "Debug commit"

This reverts commit c7b9ec9.

* add unit test

* fix check-events

* Pull in internal_logs source fix

* Add internal_logs known issue

* Bump date

* Update website/cue/reference/releases/0.51.0.cue

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
github-merge-queue bot pushed a commit that referenced this pull request Nov 13, 2025
* chore(releasing): prepare v0.51.1 release (#24214)

* chore(ci): reorganize integration test files (#24108)

* chore(ci): reorganize integration test files

* revert nats data move

* fix paths to data

* fix more paths

* amqp fix

* Fix logstash int tests shared files paths

* Rename scripts/integration -> tests/integration

* scripts/integration -> tests/integration in source files

* Fix scripts->tests

* Fix gcp paths

* Fix http-client pem path

* move nats data

---------

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* chore(vdev): move all utils in a new utils folder (#24143)

* chore(vdev): move all utils in a new utils folder

* move remaining files to utils dir

* move remaining files to utils dir

* fmt

* chore(vdev): make modules visible to rustfmt (#24162)

* Remove mod inside cli_subcommands macro

* cargo fmt

* chore(vdev): apply vdev rust check fixes

* Link to rustfmt issue

* fix(vdev): release prepare vrl version pinning (#24158)

* Add --dry-run to release prepare

* Add error handling and checks to pin_vrl_version

* Add wrapper to toml

* Remove wrapper, parse as Table instead

* Fix vrl pinning logic

* Enable preserve_order feature in toml crate

* Use dependency instead of whole toml

* Fix dry run docs

* Fix dry run wording

* refactor to use toml_edit

* Add update_vrl_to_version to add unit test

* Use indoc in prepare.rs

* Remove preserve_order feature

* chore(deps): update VRL to add missing stdlib fns from 0.28 (#24178)

* chore(ci): temporarily remove homebrew publish step from publish workflow (#24185)

This temporarily removes the publish-homebrew job from the publish
workflow to address issue #24139. This is step 1 of the plan to fix
the homebrew publishing process.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>

* fix(blackhole sink): disable rate limiting for periodic stats messages (#24190)

* fix(blackhole sink): disable rate limiting for periodic stats messages

The blackhole sink's informational "Collected events" messages were being
rate-limited since rate limiting was enabled by default in #24024. This is
undesirable because:

1. These are deliberately scheduled periodic messages (controlled by the
   `print_interval_secs` config option), not error conditions that could
   flood the logs
2. Users explicitly configure the frequency - rate limiting defeats that
   explicit configuration and breaks user expectations
3. The interval timer already prevents log flooding, making additional
   rate limiting redundant
4. The blackhole sink is used for debugging/testing, where predictable
   output is essential

This fix adds `internal_log_rate_limit = false` to both info! calls,
similar to how the console sink disables rate limiting for its critical
operational messages.

Fixes #24188

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: add changelog fragment for blackhole sink rate limiting fix

---------

Co-authored-by: Claude <noreply@anthropic.com>

* chore(internal logs): Disable rate limiting for critical internal error logs (#24192)

* Improve debug log in aws_ecs_metrics

* Add internal_log_rate_limit = false to config reload errors

* Disable rate limiting for lua build failure

* Disable rate limiting for amqp build failure

* Add internal_log_rate_limit = false to important failures during reload

* Disable log rate limit for repeated events in process.rs

* Update src/config/watcher.rs

* Fix message format

* fix(tracing): prevent panic for traces without standard fields (#24191)

* fix(tracing): prevent panic for traces without std fields

* forbid unwrap in this lib

* clippy fix

* improve events_with_custom_fields_no_message_dont_panic, advance to next window

* chore(tracing): do not rate limit utlization report (#24202)

* chore(internal metrics): move config_reload_* metrics to VectorReload* (#24203)

* chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError

* update topology_doesnt_reload_new_data_dir test

* chore(dev): cargo fmt

* strengthen tests, since now we have a better interface

* chore(releasing): Add 0.51.0 known issues (#24211)

* Format fixes

* Add 0.51.0 known issues

* Add dots

* Reword log panic known issue

* Reword VRL missing fns

* Backtick versions

* Update website/cue/reference/releases/0.51.0.cue

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Fix cue docs fmt

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Bump vector version to 0.51.1

* Bump VRL to 0.28.1

* Generate 0.51.1 cue

* Add vrl_changelog

* Remove whitespace

* chore(releasing): Updated distribution/install.sh vector version to 0.51.1

* chore(releasing): Add 0.51.1 to versions.cue

* chore(releasing): Created release md file

* Add description

* Reorder items and fix format

* Finally fix styling

* Bump date

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

chore(releasing): pull in `internal_logs` fix into 0.51.1 (#24225)

* fix(internal_logs source): remove rate limit (#24218)

* fix(internal_logs source): remove rate limit

* add changelog

* Debug commit

* Fix validated - Revert "Debug commit"

This reverts commit c7b9ec9.

* add unit test

* fix check-events

* Pull in internal_logs source fix

* Add internal_logs known issue

* Bump date

* Update website/cue/reference/releases/0.51.0.cue

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Delete changelog entries

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
elkh510 pushed a commit to elkh510/vector that referenced this pull request Nov 19, 2025
vectordotdev#24203)

* chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError

* update topology_doesnt_reload_new_data_dir test

* chore(dev): cargo fmt

* strengthen tests, since now we have a better interface
elkh510 pushed a commit to elkh510/vector that referenced this pull request Nov 19, 2025
* chore(releasing): prepare v0.51.1 release (vectordotdev#24214)

* chore(ci): reorganize integration test files (vectordotdev#24108)

* chore(ci): reorganize integration test files

* revert nats data move

* fix paths to data

* fix more paths

* amqp fix

* Fix logstash int tests shared files paths

* Rename scripts/integration -> tests/integration

* scripts/integration -> tests/integration in source files

* Fix scripts->tests

* Fix gcp paths

* Fix http-client pem path

* move nats data

---------

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* chore(vdev): move all utils in a new utils folder (vectordotdev#24143)

* chore(vdev): move all utils in a new utils folder

* move remaining files to utils dir

* move remaining files to utils dir

* fmt

* chore(vdev): make modules visible to rustfmt (vectordotdev#24162)

* Remove mod inside cli_subcommands macro

* cargo fmt

* chore(vdev): apply vdev rust check fixes

* Link to rustfmt issue

* fix(vdev): release prepare vrl version pinning (vectordotdev#24158)

* Add --dry-run to release prepare

* Add error handling and checks to pin_vrl_version

* Add wrapper to toml

* Remove wrapper, parse as Table instead

* Fix vrl pinning logic

* Enable preserve_order feature in toml crate

* Use dependency instead of whole toml

* Fix dry run docs

* Fix dry run wording

* refactor to use toml_edit

* Add update_vrl_to_version to add unit test

* Use indoc in prepare.rs

* Remove preserve_order feature

* chore(deps): update VRL to add missing stdlib fns from 0.28 (vectordotdev#24178)

* chore(ci): temporarily remove homebrew publish step from publish workflow (vectordotdev#24185)

This temporarily removes the publish-homebrew job from the publish
workflow to address issue vectordotdev#24139. This is step 1 of the plan to fix
the homebrew publishing process.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>

* fix(blackhole sink): disable rate limiting for periodic stats messages (vectordotdev#24190)

* fix(blackhole sink): disable rate limiting for periodic stats messages

The blackhole sink's informational "Collected events" messages were being
rate-limited since rate limiting was enabled by default in vectordotdev#24024. This is
undesirable because:

1. These are deliberately scheduled periodic messages (controlled by the
   `print_interval_secs` config option), not error conditions that could
   flood the logs
2. Users explicitly configure the frequency - rate limiting defeats that
   explicit configuration and breaks user expectations
3. The interval timer already prevents log flooding, making additional
   rate limiting redundant
4. The blackhole sink is used for debugging/testing, where predictable
   output is essential

This fix adds `internal_log_rate_limit = false` to both info! calls,
similar to how the console sink disables rate limiting for its critical
operational messages.

Fixes vectordotdev#24188

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: add changelog fragment for blackhole sink rate limiting fix

---------

Co-authored-by: Claude <noreply@anthropic.com>

* chore(internal logs): Disable rate limiting for critical internal error logs (vectordotdev#24192)

* Improve debug log in aws_ecs_metrics

* Add internal_log_rate_limit = false to config reload errors

* Disable rate limiting for lua build failure

* Disable rate limiting for amqp build failure

* Add internal_log_rate_limit = false to important failures during reload

* Disable log rate limit for repeated events in process.rs

* Update src/config/watcher.rs

* Fix message format

* fix(tracing): prevent panic for traces without standard fields (vectordotdev#24191)

* fix(tracing): prevent panic for traces without std fields

* forbid unwrap in this lib

* clippy fix

* improve events_with_custom_fields_no_message_dont_panic, advance to next window

* chore(tracing): do not rate limit utlization report (vectordotdev#24202)

* chore(internal metrics): move config_reload_* metrics to VectorReload* (vectordotdev#24203)

* chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError

* update topology_doesnt_reload_new_data_dir test

* chore(dev): cargo fmt

* strengthen tests, since now we have a better interface

* chore(releasing): Add 0.51.0 known issues (vectordotdev#24211)

* Format fixes

* Add 0.51.0 known issues

* Add dots

* Reword log panic known issue

* Reword VRL missing fns

* Backtick versions

* Update website/cue/reference/releases/0.51.0.cue

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Fix cue docs fmt

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Bump vector version to 0.51.1

* Bump VRL to 0.28.1

* Generate 0.51.1 cue

* Add vrl_changelog

* Remove whitespace

* chore(releasing): Updated distribution/install.sh vector version to 0.51.1

* chore(releasing): Add 0.51.1 to versions.cue

* chore(releasing): Created release md file

* Add description

* Reorder items and fix format

* Finally fix styling

* Bump date

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

chore(releasing): pull in `internal_logs` fix into 0.51.1 (vectordotdev#24225)

* fix(internal_logs source): remove rate limit (vectordotdev#24218)

* fix(internal_logs source): remove rate limit

* add changelog

* Debug commit

* Fix validated - Revert "Debug commit"

This reverts commit c7b9ec9.

* add unit test

* fix check-events

* Pull in internal_logs source fix

* Add internal_logs known issue

* Bump date

* Update website/cue/reference/releases/0.51.0.cue

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Delete changelog entries

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
elkh510 pushed a commit to elkh510/vector that referenced this pull request Nov 19, 2025
vectordotdev#24203)

* chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError

* update topology_doesnt_reload_new_data_dir test

* chore(dev): cargo fmt

* strengthen tests, since now we have a better interface
elkh510 pushed a commit to elkh510/vector that referenced this pull request Nov 19, 2025
* chore(releasing): prepare v0.51.1 release (vectordotdev#24214)

* chore(ci): reorganize integration test files (vectordotdev#24108)

* chore(ci): reorganize integration test files

* revert nats data move

* fix paths to data

* fix more paths

* amqp fix

* Fix logstash int tests shared files paths

* Rename scripts/integration -> tests/integration

* scripts/integration -> tests/integration in source files

* Fix scripts->tests

* Fix gcp paths

* Fix http-client pem path

* move nats data

---------

Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* chore(vdev): move all utils in a new utils folder (vectordotdev#24143)

* chore(vdev): move all utils in a new utils folder

* move remaining files to utils dir

* move remaining files to utils dir

* fmt

* chore(vdev): make modules visible to rustfmt (vectordotdev#24162)

* Remove mod inside cli_subcommands macro

* cargo fmt

* chore(vdev): apply vdev rust check fixes

* Link to rustfmt issue

* fix(vdev): release prepare vrl version pinning (vectordotdev#24158)

* Add --dry-run to release prepare

* Add error handling and checks to pin_vrl_version

* Add wrapper to toml

* Remove wrapper, parse as Table instead

* Fix vrl pinning logic

* Enable preserve_order feature in toml crate

* Use dependency instead of whole toml

* Fix dry run docs

* Fix dry run wording

* refactor to use toml_edit

* Add update_vrl_to_version to add unit test

* Use indoc in prepare.rs

* Remove preserve_order feature

* chore(deps): update VRL to add missing stdlib fns from 0.28 (vectordotdev#24178)

* chore(ci): temporarily remove homebrew publish step from publish workflow (vectordotdev#24185)

This temporarily removes the publish-homebrew job from the publish
workflow to address issue vectordotdev#24139. This is step 1 of the plan to fix
the homebrew publishing process.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>

* fix(blackhole sink): disable rate limiting for periodic stats messages (vectordotdev#24190)

* fix(blackhole sink): disable rate limiting for periodic stats messages

The blackhole sink's informational "Collected events" messages were being
rate-limited since rate limiting was enabled by default in vectordotdev#24024. This is
undesirable because:

1. These are deliberately scheduled periodic messages (controlled by the
   `print_interval_secs` config option), not error conditions that could
   flood the logs
2. Users explicitly configure the frequency - rate limiting defeats that
   explicit configuration and breaks user expectations
3. The interval timer already prevents log flooding, making additional
   rate limiting redundant
4. The blackhole sink is used for debugging/testing, where predictable
   output is essential

This fix adds `internal_log_rate_limit = false` to both info! calls,
similar to how the console sink disables rate limiting for its critical
operational messages.

Fixes vectordotdev#24188

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: add changelog fragment for blackhole sink rate limiting fix

---------

Co-authored-by: Claude <noreply@anthropic.com>

* chore(internal logs): Disable rate limiting for critical internal error logs (vectordotdev#24192)

* Improve debug log in aws_ecs_metrics

* Add internal_log_rate_limit = false to config reload errors

* Disable rate limiting for lua build failure

* Disable rate limiting for amqp build failure

* Add internal_log_rate_limit = false to important failures during reload

* Disable log rate limit for repeated events in process.rs

* Update src/config/watcher.rs

* Fix message format

* fix(tracing): prevent panic for traces without standard fields (vectordotdev#24191)

* fix(tracing): prevent panic for traces without std fields

* forbid unwrap in this lib

* clippy fix

* improve events_with_custom_fields_no_message_dont_panic, advance to next window

* chore(tracing): do not rate limit utlization report (vectordotdev#24202)

* chore(internal metrics): move config_reload_* metrics to VectorReload* (vectordotdev#24203)

* chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError

* update topology_doesnt_reload_new_data_dir test

* chore(dev): cargo fmt

* strengthen tests, since now we have a better interface

* chore(releasing): Add 0.51.0 known issues (vectordotdev#24211)

* Format fixes

* Add 0.51.0 known issues

* Add dots

* Reword log panic known issue

* Reword VRL missing fns

* Backtick versions

* Update website/cue/reference/releases/0.51.0.cue

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Fix cue docs fmt

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Bump vector version to 0.51.1

* Bump VRL to 0.28.1

* Generate 0.51.1 cue

* Add vrl_changelog

* Remove whitespace

* chore(releasing): Updated distribution/install.sh vector version to 0.51.1

* chore(releasing): Add 0.51.1 to versions.cue

* chore(releasing): Created release md file

* Add description

* Reorder items and fix format

* Finally fix styling

* Bump date

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

chore(releasing): pull in `internal_logs` fix into 0.51.1 (vectordotdev#24225)

* fix(internal_logs source): remove rate limit (vectordotdev#24218)

* fix(internal_logs source): remove rate limit

* add changelog

* Debug commit

* Fix validated - Revert "Debug commit"

This reverts commit c7b9ec9.

* add unit test

* fix check-events

* Pull in internal_logs source fix

* Add internal_logs known issue

* Bump date

* Update website/cue/reference/releases/0.51.0.cue

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* Delete changelog entries

---------

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: topology Anything related to Vector's topology code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants