dynamic modules: add asyncDataSource for the module binary using local.filename by kanurag94 · Pull Request #43333 · envoyproxy/envoy

kanurag94 · 2026-02-05T03:15:29Z

Commit Message: dynamic modules: add asyncDataSource for the module binary using local.filename
Additional Description: The PR adds support for fetching dynamic module binary from one of the RemoteDataSource. This currently implements local.filename support. Remote fetching will be added in follow up PRs.
Risk Level: Low
Testing: Added unit tests + Manually tested different scenarios.
Docs Changes: API docs already a part of this PR
Release Notes: Added
Platform Specific Features: NA
[Optional Runtime guard:] Not needed
[Optional Fixes #Issue]: NA
[Optional Fixes commit #PR or SHA]: NA
[Optional Deprecated:]: NA
[Optional API Considerations:] NA

The API usage is like below:

http_filters:
  - name: envoy.extensions.filters.http.dynamic_modules
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter
      filter_name: "hello"
      filter_config:
        "@type": "type.googleapis.com/google.protobuf.StringValue"
        value: |
      dynamic_module_config:
        module:
          local:
            filename: "/path/to/libhello.so"
        do_not_close: true

AI Disclaimer: Some of the UTs have been generated using Claude Code

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

repokitteh-read-only · 2026-02-05T03:15:36Z

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #43333 was opened by kanurag94.

see: more, trace.

wbpcode · 2026-02-05T12:05:04Z

this should be helpful to distribute dynamic module without customize the image. cc @agrawroh @mathetake what do you think about this API?

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

mathetake · 2026-02-06T02:18:51Z

I don't remember exactly what happens there but are we going to have a same issue as Wasm remote fetch bug or are we good here?

wbpcode · 2026-02-06T02:51:48Z

I don't remember exactly what happens there but are we going to have a same issue as Wasm remote fetch bug or are we good here?

I think bug is bug, although we don't care wasm too much, but we can fix bug for dynamic module. The key is do we want this feature. I inclined to keep it because my previously referred reasons. But still open to listen you and @agrawroh 's suggestion.

mathetake · 2026-02-06T03:53:39Z

No I meant I remember there was a fundamental limitation in remote resource but maybe I am mistaken. If you see no problem then I think this is useful

agrawroh · 2026-02-09T03:52:23Z

+1 This would be a really useful feature to have!

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

kanurag94 · 2026-02-09T13:30:33Z

/retest

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

kanurag94 · 2026-02-10T04:15:42Z

/retest

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

wbpcode · 2026-03-04T03:40:19Z

Could we split the local.filename support and the remote fetcher and keep the simplest local.filename support in this PR because it would be easier to get agreement on that.
Then, we can make another PR to support actual remote source fetching?

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

wbpcode · 2026-03-04T10:05:20Z

you can move test/extensions/filters/http/dynamic_modules/config_test.cc test to test/extensions/dynamic_modules/http/config_test.cc because we put all tests of dynamic modules there.

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

kanurag94 · 2026-03-04T11:16:03Z

+  // via ``local.filename``.
+  //
+  // When both ``name`` and ``module`` are set, ``module`` takes precedence.
+  config.core.v3.AsyncDataSource module = 6;


I named it module instead of source as per the patterns we have in the existing code where a datasource signifies what it is fetching. Reading dynamic_module_config.module also seemed better than dynamic_module_config.source which might mean configSource and not moduleSource.

kanurag94 · 2026-03-04T11:19:23Z

Thank you again for the review @wbpcode. Apologies that I wasn't able to understand that we wanted to split out remote mode from this PR.
I have now kept the PR very minimal (thanks to your suggestions). Will create a new PR for remote support.

I think this PR is cleanup and all previous comments from Takeshi were addressed

…inary (#43818) **Commit Message**: dynamic_modules: add remote HTTP support for asyncDataSource module binary **Additional Description**: Extends the asyncDataSource support (added in #43333 for local.filename) to support fetching dynamic module binaries from remote HTTP sources via module.remote in DynamicModuleConfig. The module is downloaded asynchronously during listener initialization with SHA256 verification, written to a temporary file, and loaded via dlopen. If the remote fetch fails, the filter is not installed and requests pass through (fail-open). **Risk Level**: Low **Testing**: Added unit tests + Manually tested **Docs Changes**: API docs already a part of this PR **Release Notes**: Added --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

@type

…l.filename (envoyproxy#43333) **Commit Message**: dynamic modules: add asyncDataSource for the module binary using local.filename **Additional Description**: The PR adds support for fetching dynamic module binary from one of the RemoteDataSource. This currently implements local.filename support. Remote fetching will be added in follow up PRs. **Risk Level**: Low **Testing**: Added unit tests + Manually tested different scenarios. **Docs Changes**: API docs already a part of this PR **Release Notes**: Added Platform Specific Features: NA [Optional Runtime guard:] Not needed [Optional Fixes #Issue]: NA [Optional Fixes commit #PR or SHA]: NA [Optional Deprecated:]: NA [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] NA --- The API usage is like below: ``` http_filters: - name: envoy.extensions.filters.http.dynamic_modules typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter filter_name: "hello" filter_config: "@type": "type.googleapis.com/google.protobuf.StringValue" value: | dynamic_module_config: module: local: filename: "/path/to/libhello.so" do_not_close: true ``` *AI Disclaimer*: Some of the UTs have been generated using Claude Code --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com> Signed-off-by: bjmask <11672696+bjmask@users.noreply.github.com>

…inary (envoyproxy#43818) **Commit Message**: dynamic_modules: add remote HTTP support for asyncDataSource module binary **Additional Description**: Extends the asyncDataSource support (added in envoyproxy#43333 for local.filename) to support fetching dynamic module binaries from remote HTTP sources via module.remote in DynamicModuleConfig. The module is downloaded asynchronously during listener initialization with SHA256 verification, written to a temporary file, and loaded via dlopen. If the remote fetch fails, the filter is not installed and requests pass through (fail-open). **Risk Level**: Low **Testing**: Added unit tests + Manually tested **Docs Changes**: API docs already a part of this PR **Release Notes**: Added --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com> Signed-off-by: bjmask <11672696+bjmask@users.noreply.github.com>

@type

…l.filename (envoyproxy#43333) **Commit Message**: dynamic modules: add asyncDataSource for the module binary using local.filename **Additional Description**: The PR adds support for fetching dynamic module binary from one of the RemoteDataSource. This currently implements local.filename support. Remote fetching will be added in follow up PRs. **Risk Level**: Low **Testing**: Added unit tests + Manually tested different scenarios. **Docs Changes**: API docs already a part of this PR **Release Notes**: Added Platform Specific Features: NA [Optional Runtime guard:] Not needed [Optional Fixes #Issue]: NA [Optional Fixes commit #PR or SHA]: NA [Optional Deprecated:]: NA [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] NA --- The API usage is like below: ``` http_filters: - name: envoy.extensions.filters.http.dynamic_modules typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter filter_name: "hello" filter_config: "@type": "type.googleapis.com/google.protobuf.StringValue" value: | dynamic_module_config: module: local: filename: "/path/to/libhello.so" do_not_close: true ``` *AI Disclaimer*: Some of the UTs have been generated using Claude Code --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

**Commit Message**: dynamic_modules: remote source: add caching for fetched modules **Motivation** Follow up from #43818 and #43333 Next PR: `nack_on_cache_miss` support (NACK on cache miss + background fetch) **Additional Description**: `newDynamicModuleFromBytes` already writes fetched modules to a deterministic path based on SHA256. This PR adds a check before the async fetch: if the file already exists on disk, load it directly and skip the HTTP fetch entirely. It is a simple approach that forms base for the next PR. **Risk Level**: Low **Testing**: Added unit tests **Docs Changes**: NA **Release Notes**: Added Platform Specific Features: NA [Optional Runtime guard:] Not needed [Optional Fixes #Issue]: NA [Optional Fixes commit #PR or SHA]: NA [Optional Deprecated:]: NA [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] NA --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

@type

…l.filename (envoyproxy#43333) **Commit Message**: dynamic modules: add asyncDataSource for the module binary using local.filename **Additional Description**: The PR adds support for fetching dynamic module binary from one of the RemoteDataSource. This currently implements local.filename support. Remote fetching will be added in follow up PRs. **Risk Level**: Low **Testing**: Added unit tests + Manually tested different scenarios. **Docs Changes**: API docs already a part of this PR **Release Notes**: Added Platform Specific Features: NA [Optional Runtime guard:] Not needed [Optional Fixes #Issue]: NA [Optional Fixes commit #PR or SHA]: NA [Optional Deprecated:]: NA [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] NA --- The API usage is like below: ``` http_filters: - name: envoy.extensions.filters.http.dynamic_modules typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter filter_name: "hello" filter_config: "@type": "type.googleapis.com/google.protobuf.StringValue" value: | dynamic_module_config: module: local: filename: "/path/to/libhello.so" do_not_close: true ``` *AI Disclaimer*: Some of the UTs have been generated using Claude Code --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

…inary (envoyproxy#43818) **Commit Message**: dynamic_modules: add remote HTTP support for asyncDataSource module binary **Additional Description**: Extends the asyncDataSource support (added in envoyproxy#43333 for local.filename) to support fetching dynamic module binaries from remote HTTP sources via module.remote in DynamicModuleConfig. The module is downloaded asynchronously during listener initialization with SHA256 verification, written to a temporary file, and loaded via dlopen. If the remote fetch fails, the filter is not installed and requests pass through (fail-open). **Risk Level**: Low **Testing**: Added unit tests + Manually tested **Docs Changes**: API docs already a part of this PR **Release Notes**: Added --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

…yproxy#43974) **Commit Message**: dynamic_modules: remote source: add caching for fetched modules **Motivation** Follow up from envoyproxy#43818 and envoyproxy#43333 Next PR: `nack_on_cache_miss` support (NACK on cache miss + background fetch) **Additional Description**: `newDynamicModuleFromBytes` already writes fetched modules to a deterministic path based on SHA256. This PR adds a check before the async fetch: if the file already exists on disk, load it directly and skip the HTTP fetch entirely. It is a simple approach that forms base for the next PR. **Risk Level**: Low **Testing**: Added unit tests **Docs Changes**: NA **Release Notes**: Added Platform Specific Features: NA [Optional Runtime guard:] Not needed [Optional Fixes #Issue]: NA [Optional Fixes commit #PR or SHA]: NA [Optional Deprecated:]: NA [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] NA --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

…yproxy#43974) **Commit Message**: dynamic_modules: remote source: add caching for fetched modules **Motivation** Follow up from envoyproxy#43818 and envoyproxy#43333 Next PR: `nack_on_cache_miss` support (NACK on cache miss + background fetch) **Additional Description**: `newDynamicModuleFromBytes` already writes fetched modules to a deterministic path based on SHA256. This PR adds a check before the async fetch: if the file already exists on disk, load it directly and skip the HTTP fetch entirely. It is a simple approach that forms base for the next PR. **Risk Level**: Low **Testing**: Added unit tests **Docs Changes**: NA **Release Notes**: Added Platform Specific Features: NA [Optional Runtime guard:] Not needed [Optional Fixes #Issue]: NA [Optional Fixes commit #PR or SHA]: NA [Optional Deprecated:]: NA [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] NA --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com> Signed-off-by: Xuyang Tao <taoxuy@google.com>

…yproxy#43974) **Commit Message**: dynamic_modules: remote source: add caching for fetched modules **Motivation** Follow up from envoyproxy#43818 and envoyproxy#43333 Next PR: `nack_on_cache_miss` support (NACK on cache miss + background fetch) **Additional Description**: `newDynamicModuleFromBytes` already writes fetched modules to a deterministic path based on SHA256. This PR adds a check before the async fetch: if the file already exists on disk, load it directly and skip the HTTP fetch entirely. It is a simple approach that forms base for the next PR. **Risk Level**: Low **Testing**: Added unit tests **Docs Changes**: NA **Release Notes**: Added Platform Specific Features: NA [Optional Runtime guard:] Not needed [Optional Fixes #Issue]: NA [Optional Fixes commit #PR or SHA]: NA [Optional Deprecated:]: NA [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] NA --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com> Signed-off-by: Nick Shipilov <nick.shipilov.n@gmail.com>

…yproxy#43974) **Commit Message**: dynamic_modules: remote source: add caching for fetched modules **Motivation** Follow up from envoyproxy#43818 and envoyproxy#43333 Next PR: `nack_on_cache_miss` support (NACK on cache miss + background fetch) **Additional Description**: `newDynamicModuleFromBytes` already writes fetched modules to a deterministic path based on SHA256. This PR adds a check before the async fetch: if the file already exists on disk, load it directly and skip the HTTP fetch entirely. It is a simple approach that forms base for the next PR. **Risk Level**: Low **Testing**: Added unit tests **Docs Changes**: NA **Release Notes**: Added Platform Specific Features: NA [Optional Runtime guard:] Not needed [Optional Fixes #Issue]: NA [Optional Fixes commit #PR or SHA]: NA [Optional Deprecated:]: NA [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] NA --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

@type

…l.filename (envoyproxy#43333) **Commit Message**: dynamic modules: add asyncDataSource for the module binary using local.filename **Additional Description**: The PR adds support for fetching dynamic module binary from one of the RemoteDataSource. This currently implements local.filename support. Remote fetching will be added in follow up PRs. **Risk Level**: Low **Testing**: Added unit tests + Manually tested different scenarios. **Docs Changes**: API docs already a part of this PR **Release Notes**: Added Platform Specific Features: NA [Optional Runtime guard:] Not needed [Optional Fixes #Issue]: NA [Optional Fixes commit #PR or SHA]: NA [Optional Deprecated:]: NA [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] NA --- The API usage is like below: ``` http_filters: - name: envoy.extensions.filters.http.dynamic_modules typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter filter_name: "hello" filter_config: "@type": "type.googleapis.com/google.protobuf.StringValue" value: | dynamic_module_config: module: local: filename: "/path/to/libhello.so" do_not_close: true ``` *AI Disclaimer*: Some of the UTs have been generated using Claude Code --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

…inary (envoyproxy#43818) **Commit Message**: dynamic_modules: add remote HTTP support for asyncDataSource module binary **Additional Description**: Extends the asyncDataSource support (added in envoyproxy#43333 for local.filename) to support fetching dynamic module binaries from remote HTTP sources via module.remote in DynamicModuleConfig. The module is downloaded asynchronously during listener initialization with SHA256 verification, written to a temporary file, and loaded via dlopen. If the remote fetch fails, the filter is not installed and requests pass through (fail-open). **Risk Level**: Low **Testing**: Added unit tests + Manually tested **Docs Changes**: API docs already a part of this PR **Release Notes**: Added --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

…yproxy#43974) **Commit Message**: dynamic_modules: remote source: add caching for fetched modules **Motivation** Follow up from envoyproxy#43818 and envoyproxy#43333 Next PR: `nack_on_cache_miss` support (NACK on cache miss + background fetch) **Additional Description**: `newDynamicModuleFromBytes` already writes fetched modules to a deterministic path based on SHA256. This PR adds a check before the async fetch: if the file already exists on disk, load it directly and skip the HTTP fetch entirely. It is a simple approach that forms base for the next PR. **Risk Level**: Low **Testing**: Added unit tests **Docs Changes**: NA **Release Notes**: Added Platform Specific Features: NA [Optional Runtime guard:] Not needed [Optional Fixes #Issue]: NA [Optional Fixes commit #PR or SHA]: NA [Optional Deprecated:]: NA [Optional [API Considerations](https://github.com/envoyproxy/envoy/blob/main/api/review_checklist.md):] NA --------- Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

kanurag94 added 2 commits February 3, 2026 10:26

dynamic modules: add remote fetching for the module binary

74ae885

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

Merge remote-tracking branch 'upstream/main' into dym-loading

7f80388

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

wbpcode assigned wbpcode, agrawroh and mathetake Feb 5, 2026

kanurag94 added 4 commits February 5, 2026 17:44

add tests and cleanup

810558e

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

attempt fixing coverage

ec4159f

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

attempt fixing coverage

93caee8

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

lower coverage percentage for dynamic_modules filter

13d1089

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

kanurag94 added 4 commits February 9, 2026 13:23

fix issues related to fetch

b708a8f

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

Merge remote-tracking branch 'upstream/main' into dym-loading

ed84cf4

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

Merge remote-tracking branch 'origin/dym-loading' into dym-loading

d4a09af

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

fix lint

52c7e77

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

lower coverage due proto validation not invoking lines

ec9edb5

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

add changelog

5113c7e

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

kanurag94 marked this pull request as ready for review February 11, 2026 13:25

kanurag94 requested review from agrawroh, mathetake, mattklein123 and wbpcode as code owners February 11, 2026 13:25

repokitteh-read-only Bot added the api label Feb 11, 2026

repokitteh-read-only Bot assigned RyanTheOptimist Feb 11, 2026

kanurag94 requested review from mathetake and wbpcode March 3, 2026 14:56

try keeping local.filename support only

4568023

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

kanurag94 added 2 commits March 4, 2026 15:42

move the test files

798b887

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

use loadFromYamlAndValidate

b611d90

Signed-off-by: Anurag Aggarwal <kanurag94@gmail.com>

kanurag94 commented Mar 4, 2026

View reviewed changes

kanurag94 changed the title ~~dynamic modules: add remote fetching for the module binary~~ dynamic modules: add asyncDataSource for the module binary using local.filename Mar 4, 2026

wbpcode approved these changes Mar 4, 2026

View reviewed changes

repokitteh-read-only Bot removed the api label Mar 4, 2026

wbpcode merged commit 059d8c6 into envoyproxy:main Mar 4, 2026
29 of 30 checks passed

kanurag94 mentioned this pull request Mar 9, 2026

dynamic_modules: add remote HTTP support for asyncDataSource module binary #43818

Merged

kanurag94 mentioned this pull request Mar 17, 2026

dynamic_modules: remote source: add caching for fetched modules #43974

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dynamic modules: add asyncDataSource for the module binary using local.filename#43333

dynamic modules: add asyncDataSource for the module binary using local.filename#43333
wbpcode merged 26 commits into
envoyproxy:mainfrom
kanurag94:dym-loading

kanurag94 commented Feb 5, 2026 •

edited

Loading

Uh oh!

repokitteh-read-only Bot commented Feb 5, 2026

Uh oh!

wbpcode commented Feb 5, 2026

Uh oh!

mathetake commented Feb 6, 2026

Uh oh!

wbpcode commented Feb 6, 2026 •

edited

Loading

Uh oh!

mathetake commented Feb 6, 2026

Uh oh!

agrawroh commented Feb 9, 2026

Uh oh!

kanurag94 commented Feb 9, 2026

Uh oh!

kanurag94 commented Feb 10, 2026

Uh oh!

wbpcode commented Mar 4, 2026

Uh oh!

wbpcode commented Mar 4, 2026

Uh oh!

kanurag94 Mar 4, 2026

Uh oh!

kanurag94 commented Mar 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

kanurag94 commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

repokitteh-read-only Bot commented Feb 5, 2026

Uh oh!

wbpcode commented Feb 5, 2026

Uh oh!

mathetake commented Feb 6, 2026

Uh oh!

wbpcode commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mathetake commented Feb 6, 2026

Uh oh!

agrawroh commented Feb 9, 2026

Uh oh!

kanurag94 commented Feb 9, 2026

Uh oh!

kanurag94 commented Feb 10, 2026

Uh oh!

wbpcode commented Mar 4, 2026

Uh oh!

wbpcode commented Mar 4, 2026

Uh oh!

kanurag94 Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

kanurag94 commented Mar 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kanurag94 commented Feb 5, 2026 •

edited

Loading

wbpcode commented Feb 6, 2026 •

edited

Loading