feat(translator): support ext-auth dynamic metadata options (Envoy to auth server)

[nareddyt]

What this PR does / why we need it:

Added support for specifying dynamic metadata namespaces that External Authorization services can access in SecurityPolicy API

This follows the structure and CRD naming conventions established in #5023.

NOTE: This PR only introduces fields for "sending metadata to ext-authz server". It does NOT handle the reverse operation of "parsing ext-authz HTTP headers and turning them into dynamic metadata on Envoy" (which #4163 was initially opened for). However, this PR structures the CRD changes in a way that allows for the reverse operation to be easily added.

Which issue(s) this PR fixes:
Not aware of any pre-existing issue describing this specific use case. #4163 is related but for the reverse operation.

Release Notes: Yes

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.76%. Comparing base (64f3576) to head (e2bda53).
Report is 5 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6453      +/-   ##
==========================================
+ Coverage   70.75%   70.76%   +0.01%     
==========================================
  Files         220      220              
  Lines       37757    37767      +10     
==========================================
+ Hits        26715    26727      +12     
+ Misses       9482     9479       -3     
- Partials     1560     1561       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[TheRushingWookie]

Just a nit

[nareddyt]

@guydc PTAL, addressed all review comments. Would like your input on our idea to expose both typed and untyped namespaces, as Confluent has a use case for typed namespaces

[nareddyt]

/retest

[guydc]

LGTM.
Would be nice to extend the e2es to cover this feature, e.g. have grcp ext-auth consume metadata and add it to the request headers through the Check Response.

[nareddyt]

LGTM. Would be nice to extend the e2es to cover this feature, e.g. have grcp ext-auth consume metadata and add it to the request headers through the Check Response.

Good idea, done. I tested static untyped route metadata in the e2e test. IMO there is no additional value is testing the other 3 fields (typed route metadata, typed + untyped listener metadata).

[guydc]

LGTM. Thanks for adding the e2e test.

[nareddyt]

/retest

[arkodg]

hey @nareddyt can we limit this feature to route metadata ?
filters in the data plane may map to different resources which are tied to different actors, so prefer avoiding it until we've thought through all the implications

[nareddyt]

@arkodg we do have a few use cases at Confluent where we need to propagate filter metadata from other OSS filters. We had a similar discussion in #6453 (comment)

Are there any specific implications you are worried about? I can limit it to route metadata if really required, but then we can no longer use this feature and will still need an extension server modification.

[arkodg]

would A CEL validtion be tricky be add for this case ?

• allow filter metdata if target is Gateway, else dont

[nareddyt]

would A CEL validtion be tricky be add for this case ?

• allow filter metdata if target is Gateway, else dont

Just trying to understand what that would solve?

IIRC we have precedent for filter metadata in EnvoyExtensionPolicy via ExtProc. https://gateway.envoyproxy.io/docs/api/extension_types/#extprocmetadata . This supports untyped filter metadata. And ExtProc can target both a HTTPRoute or a Gateway.

I didn't see any sort of similar validations in ExtProc CRD, so just trying to understand if they are necessary.

[nareddyt]

Just want to clear up some confusion: Envoy allows attaching xDS metadata to multiple resources, such as routes, clusters, listeners. Envoy filters also can modify per-request dynamic metadata.

The ext_authz callout should allow sending all these types of metadata, regardless of whether the SecurityPolicy is targeting a Gateway vs HTTPRoute. That should be orthogonal

[arkodg]

Just want to clear up some confusion: Envoy allows attaching xDS metadata to multiple resources, such as routes, clusters, listeners. Envoy filters also can modify per-request dynamic metadata.

The ext_authz callout should allow sending all these types of metadata, regardless of whether the SecurityPolicy is targeting a Gateway vs HTTPRoute. That should be orthogonal

if a SecurityPolicy can target a Gateway its in the same namespace as the Gateway, and implies same persona is creating them.

Envoy Gateway does not gives direct access to xds filters, except in the EnvoyProxy resource to order xds filters which is applied at the Gateway level

There's ExtensionServer and EnvoyExtensionPolicy which is the back door enabled by the admin in the startup EnvoyGateway config

[nareddyt]

Thanks @arkodg , I understand the concern now. Unfortunately this will not work for Confluent's use case. Let me close this PR and rethink it. Perhaps I will go with a custom extension server mutation on our end for the time being.