Skip to content

Baggage discovery #6779

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
istio-testing merged 15 commits into from
Jan 23, 2026
Merged

Baggage discovery #6779

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
4c87a04
Add Baggage metadata propagation
keithmattix Jan 21, 2026
c429b9d
clang-tidy
keithmattix Jan 21, 2026
7d22875
basics for baggage discovery downstream
grnmeira Jan 22, 2026
72394b4
removing unnecessary tests
grnmeira Jan 22, 2026
fba6e72
reverting crazy claude changes in release-binary.sh
grnmeira Jan 22, 2026
5d84f5a
fixing tests, fixing baggage key tokens
grnmeira Jan 22, 2026
08d5751
removing comment
grnmeira Jan 22, 2026
52467c8
Merge branch 'experimental-ambient-multicluster-telemetry' of github.…
grnmeira Jan 22, 2026
0e0806f
make lint
grnmeira Jan 22, 2026
efd48f5
fixing unit tests for metadata_object
grnmeira Jan 22, 2026
7407537
make lint
grnmeira Jan 22, 2026
dbf83ef
suggestions from PR
grnmeira Jan 23, 2026
ac7c5ef
clarifying use of mappings for baggage and field access
grnmeira Jan 23, 2026
1a87c89
make lint
grnmeira Jan 23, 2026
70f2b9d
Merge branch 'experimental-ambient-multicluster-telemetry' of github.…
grnmeira Jan 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 37 additions & 14 deletions extensions/common/metadata_object.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,10 @@ namespace Istio {
namespace Common {

namespace {
static absl::flat_hash_map<absl::string_view, BaggageToken> ALL_BAGGAGE_TOKENS = {

// This maps field names into baggage tokens. We use it to decode field names
// when WorkloadMetadataObject content is accessed through the Envoy API.
static absl::flat_hash_map<absl::string_view, BaggageToken> ALL_METADATA_FIELDS = {
{NamespaceNameToken, BaggageToken::NamespaceName},
{ClusterNameToken, BaggageToken::ClusterName},
{ServiceNameToken, BaggageToken::ServiceName},
Expand All @@ -36,6 +39,22 @@ static absl::flat_hash_map<absl::string_view, BaggageToken> ALL_BAGGAGE_TOKENS =
{InstanceNameToken, BaggageToken::InstanceName},
};

// This maps baggage keys into baggage tokens. We use it to decode baggage keys
// coming over the wire when building WorkloadMetadataObject.
static absl::flat_hash_map<absl::string_view, BaggageToken> ALL_BAGGAGE_TOKENS = {
Copy link
Copy Markdown
Contributor

@krinkinmu krinkinmu Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this used?

Copy link
Copy Markdown
Contributor

@keithmattix keithmattix Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used in getField() which is in-turn used to generatae baggage

Copy link
Copy Markdown
Contributor Author

@grnmeira grnmeira Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ALL_BAGGAGE_TOKENS is used to convert baggage into metadata, and ALL_METADATA_FIELDS is used for creating baggage from metadata. At first I thought we had getField being used also in other parts of the code, but looks like we don't. I'll unify the mappings and change the tests accordingly.

Copy link
Copy Markdown
Contributor Author

@grnmeira grnmeira Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I'm afraid WorkloadMetadtaObject can be accessed from Envoy's API and we may need to maintain compatibility. So if there's any use out there to access data using the old telemetry labels, we probably don't want to break that.

Copy link
Copy Markdown
Contributor Author

@grnmeira grnmeira Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added some comments, I hope it makes things clearer.

{NamespaceNameBaggageToken, BaggageToken::NamespaceName},
{ClusterNameBaggageToken, BaggageToken::ClusterName},
{ServiceNameBaggageToken, BaggageToken::ServiceName},
{ServiceVersionBaggageToken, BaggageToken::ServiceVersion},
{AppNameBaggageToken, BaggageToken::AppName},
{AppVersionBaggageToken, BaggageToken::AppVersion},
{DeploymentNameBaggageToken, BaggageToken::WorkloadName},
{PodNameBaggageToken, BaggageToken::WorkloadName},
{CronjobNameBaggageToken, BaggageToken::WorkloadName},
{JobNameBaggageToken, BaggageToken::WorkloadName},
{InstanceNameBaggageToken, BaggageToken::InstanceName},
};

static absl::flat_hash_map<absl::string_view, WorkloadType> ALL_WORKLOAD_TOKENS = {
{PodSuffix, WorkloadType::Pod},
{DeploymentSuffix, WorkloadType::Deployment},
Expand Down Expand Up @@ -69,13 +88,13 @@ std::string WorkloadMetadataObject::baggage() const {
}
// Map the workload metadata fields to baggage tokens
const std::vector<std::pair<absl::string_view, absl::string_view>> field_to_baggage = {
{Istio::Common::NamespaceNameToken, "k8s.namespace.name"},
{Istio::Common::ClusterNameToken, "k8s.cluster.name"},
{Istio::Common::ServiceNameToken, "service.name"},
{Istio::Common::ServiceVersionToken, "service.version"},
{Istio::Common::AppNameToken, "app.name"},
{Istio::Common::AppVersionToken, "app.version"},
{Istio::Common::InstanceNameToken, "k8s.instance.name"},
{Istio::Common::NamespaceNameToken, Istio::Common::NamespaceNameBaggageToken},
{Istio::Common::ClusterNameToken, Istio::Common::ClusterNameBaggageToken},
{Istio::Common::ServiceNameToken, Istio::Common::ServiceNameBaggageToken},
{Istio::Common::ServiceVersionToken, Istio::Common::ServiceVersionBaggageToken},
{Istio::Common::AppNameToken, Istio::Common::AppNameBaggageToken},
{Istio::Common::AppVersionToken, Istio::Common::AppVersionBaggageToken},
{Istio::Common::InstanceNameToken, Istio::Common::InstanceNameBaggageToken},
};

for (const auto& [field_name, baggage_key] : field_to_baggage) {
Expand Down Expand Up @@ -320,8 +339,8 @@ std::string serializeToStringDeterministic(const google::protobuf::Struct& metad

WorkloadMetadataObject::FieldType
WorkloadMetadataObject::getField(absl::string_view field_name) const {
const auto it = ALL_BAGGAGE_TOKENS.find(field_name);
if (it != ALL_BAGGAGE_TOKENS.end()) {
const auto it = ALL_METADATA_FIELDS.find(field_name);
Copy link
Copy Markdown
Contributor

@keithmattix keithmattix Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a contract change; even if the naming is clearer, we CANNOT change the contract. Please revert

Copy link
Copy Markdown
Contributor Author

@grnmeira grnmeira Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ALL_METADATA_FIELDS contains the original contract change. Keeping the ALL_BAGGAGE_TOKENS name seemed confusing to me.

Copy link
Copy Markdown
Contributor

@keithmattix keithmattix Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see what you're doing. Make sure you double check all of the other filters in istio-proxy to ensure they're not depending on the old naming. If possible, I'd prefer to split the rename into its own change later just so we and future reviewers can focus on the substantive changes

Copy link
Copy Markdown
Contributor Author

@grnmeira grnmeira Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this change is reasonably safe. ALL_BAGGAGE_TOKENS (the old name) and even ALL_METADATA_FIELDS are not used anywhere else in the repo (checked with a find grep).

if (it != ALL_METADATA_FIELDS.end()) {
switch (it->second) {
case BaggageToken::NamespaceName:
return namespace_name_;
Expand Down Expand Up @@ -389,15 +408,19 @@ convertBaggageToWorkloadMetadata(absl::string_view data, absl::string_view ident
case BaggageToken::AppVersion:
app_version = parts.second;
break;
case BaggageToken::WorkloadName:
Copy link
Copy Markdown
Contributor

@keithmattix keithmattix Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the change if this worked before?

Copy link
Copy Markdown
Contributor Author

@grnmeira grnmeira Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is old cold, it worked with the previous baggage keys. Now that the workload type is embedded in baggage key we have to get the type and the name from the WorkloadName token. The workload name continues the same piece of code. But the type needs to be extract from a key now, different from before.

Copy link
Copy Markdown
Contributor Author

@grnmeira grnmeira Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note how it now comes from part.first, which is the key, instead of part.second, which is the value.

Copy link
Copy Markdown
Contributor

@keithmattix keithmattix Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh good catch; yeah the ztunnel format and old pilot format are apparently different

case BaggageToken::WorkloadName: {
workload = parts.second;
std::vector<absl::string_view> splitWorkloadKey = absl::StrSplit(parts.first, ".");
if (splitWorkloadKey.size() >= 2 && splitWorkloadKey[0] == "k8s") {
workload_type = fromSuffix(splitWorkloadKey[1]);
}
break;
case BaggageToken::WorkloadType:
workload_type = fromSuffix(parts.second);
break;
}
case BaggageToken::InstanceName:
instance = parts.second;
break;
default:
break;
}
}
}
Expand Down
15 changes: 15 additions & 0 deletions extensions/common/metadata_object.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ enum class BaggageToken {
InstanceName,
};

// Field names accessible from WorkloadMetadataObject.
constexpr absl::string_view NamespaceNameToken = "namespace";
constexpr absl::string_view ClusterNameToken = "cluster";
constexpr absl::string_view ServiceNameToken = "service";
Expand All @@ -79,6 +80,20 @@ constexpr absl::string_view InstanceNameToken = "name";
constexpr absl::string_view LabelsToken = "labels";
constexpr absl::string_view IdentityToken = "identity";

// Field names used to translate baggage content into
// WorkloadMetadataObject information.
constexpr absl::string_view NamespaceNameBaggageToken = "k8s.namespace.name";
Comment thread
keithmattix marked this conversation as resolved.
constexpr absl::string_view ClusterNameBaggageToken = "k8s.cluster.name";
constexpr absl::string_view ServiceNameBaggageToken = "service.name";
constexpr absl::string_view ServiceVersionBaggageToken = "service.version";
constexpr absl::string_view AppNameBaggageToken = "app.name";
constexpr absl::string_view AppVersionBaggageToken = "app.version";
constexpr absl::string_view DeploymentNameBaggageToken = "k8s.deployment.name";
constexpr absl::string_view PodNameBaggageToken = "k8s.pod.name";
constexpr absl::string_view CronjobNameBaggageToken = "k8s.cronjob.name";
constexpr absl::string_view JobNameBaggageToken = "k8s.job.name";
constexpr absl::string_view InstanceNameBaggageToken = "k8s.instance.name";

constexpr absl::string_view InstanceMetadataField = "NAME";
constexpr absl::string_view NamespaceMetadataField = "NAMESPACE";
constexpr absl::string_view ClusterMetadataField = "CLUSTER_ID";
Expand Down
38 changes: 20 additions & 18 deletions extensions/common/metadata_object_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,9 @@ TEST(WorkloadMetadataObjectTest, ConversionWithLabels) {
TEST(WorkloadMetadataObjectTest, Conversion) {
{
const auto r = convertBaggageToWorkloadMetadata(
"type=deployment,workload=foo,cluster=my-cluster,"
"namespace=default,service=foo-service,revision=v1alpha3,app=foo-app,version=latest");
"k8s.deployment.name=foo,k8s.cluster.name=my-cluster,"
"k8s.namespace.name=default,service.name=foo-service,service.version=v1alpha3,app.name=foo-"
"app,app.version=latest");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("service")), "foo-service");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("revision")), "v1alpha3");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("type")), DeploymentSuffix);
Expand All @@ -97,14 +98,14 @@ TEST(WorkloadMetadataObjectTest, Conversion) {
}

{
const auto r =
convertBaggageToWorkloadMetadata("type=pod,name=foo-pod-435,cluster=my-cluster,namespace="
"test,service=foo-service,revision=v1beta2");
const auto r = convertBaggageToWorkloadMetadata(
"k8s.pod.name=foo-pod-435,k8s.cluster.name=my-cluster,k8s.namespace.name="
"test,k8s.instance.name=foo-instance-435,service.name=foo-service,service.version=v1beta2");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("service")), "foo-service");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("revision")), "v1beta2");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("type")), PodSuffix);
EXPECT_EQ(absl::get<absl::string_view>(r->getField("workload")), "");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("name")), "foo-pod-435");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("workload")), "foo-pod-435");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("name")), "foo-instance-435");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("namespace")), "test");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("cluster")), "my-cluster");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("app")), "");
Expand All @@ -113,23 +114,23 @@ TEST(WorkloadMetadataObjectTest, Conversion) {
}

{
const auto r =
convertBaggageToWorkloadMetadata("type=job,name=foo-job-435,cluster=my-cluster,namespace="
"test,service=foo-service,revision=v1beta4");
const auto r = convertBaggageToWorkloadMetadata(
"k8s.job.name=foo-job-435,k8s.cluster.name=my-cluster,k8s.namespace.name="
"test,k8s.instance.name=foo-instance-435,service.name=foo-service,service.version=v1beta4");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("service")), "foo-service");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("revision")), "v1beta4");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("type")), JobSuffix);
EXPECT_EQ(absl::get<absl::string_view>(r->getField("workload")), "");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("name")), "foo-job-435");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("workload")), "foo-job-435");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("name")), "foo-instance-435");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("namespace")), "test");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("cluster")), "my-cluster");
checkStructConversion(*r);
}

{
const auto r =
convertBaggageToWorkloadMetadata("type=cronjob,workload=foo-cronjob,cluster=my-cluster,"
"namespace=test,service=foo-service,revision=v1beta4");
const auto r = convertBaggageToWorkloadMetadata(
"k8s.cronjob.name=foo-cronjob,k8s.cluster.name=my-cluster,"
"k8s.namespace.name=test,service.name=foo-service,service.version=v1beta4");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("service")), "foo-service");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("revision")), "v1beta4");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("type")), CronJobSuffix);
Expand All @@ -141,8 +142,9 @@ TEST(WorkloadMetadataObjectTest, Conversion) {
}

{
const auto r = convertBaggageToWorkloadMetadata(
"type=deployment,workload=foo,namespace=default,service=foo-service,revision=v1alpha3");
const auto r =
convertBaggageToWorkloadMetadata("k8s.deployment.name=foo,k8s.namespace.name=default,"
"service.name=foo-service,service.version=v1alpha3");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("service")), "foo-service");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("revision")), "v1alpha3");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("type")), DeploymentSuffix);
Expand All @@ -153,7 +155,7 @@ TEST(WorkloadMetadataObjectTest, Conversion) {
}

{
const auto r = convertBaggageToWorkloadMetadata("namespace=default");
const auto r = convertBaggageToWorkloadMetadata("k8s.namespace.name=default");
EXPECT_EQ(absl::get<absl::string_view>(r->getField("namespace")), "default");
checkStructConversion(*r);
}
Expand Down
24 changes: 24 additions & 0 deletions source/extensions/filters/http/peer_metadata/filter.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,23 @@ void BaggagePropagationMethod::inject(const StreamInfo::StreamInfo&, Http::Heade
headers.setReference(Headers::get().Baggage, value_);
}

BaggageDiscoveryMethod::BaggageDiscoveryMethod() {}

absl::optional<PeerInfo> BaggageDiscoveryMethod::derivePeerInfo(const StreamInfo::StreamInfo&,
Http::HeaderMap& headers,
Context&) const {
const auto baggage_header = headers.get(Headers::get().Baggage);
if (baggage_header.empty()) {
return {};
}
const auto baggage_value = baggage_header[0]->value().getStringView();
const auto workload = Istio::Common::convertBaggageToWorkloadMetadata(baggage_value);
if (workload) {
return *workload;
}
return {};
}

FilterConfig::FilterConfig(const io::istio::http::peer_metadata::Config& config,
Server::Configuration::FactoryContext& factory_context)
: shared_with_upstream_(config.shared_with_upstream()),
Expand Down Expand Up @@ -273,6 +290,13 @@ std::vector<DiscoveryMethodPtr> FilterConfig::buildDiscoveryMethods(
methods.push_back(std::make_unique<MXMethod>(downstream, additional_labels,
factory_context.serverFactoryContext()));
break;
case io::istio::http::peer_metadata::Config::DiscoveryMethod::MethodSpecifierCase::kBaggage:
if (downstream) {
methods.push_back(std::make_unique<BaggageDiscoveryMethod>());
} else {
ENVOY_LOG(warn, "BaggageDiscovery peer metadata discovery option is only available for "
"downstream peer discovery");
}
case io::istio::http::peer_metadata::Config::DiscoveryMethod::MethodSpecifierCase::
kUpstreamFilterState:
if (!downstream) {
Expand Down
9 changes: 8 additions & 1 deletion source/extensions/filters/http/peer_metadata/filter.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,13 @@ class BaggagePropagationMethod : public PropagationMethod {
const std::string value_;
};

class BaggageDiscoveryMethod : public DiscoveryMethod, public Logger::Loggable<Logger::Id::filter> {
public:
BaggageDiscoveryMethod();
absl::optional<PeerInfo> derivePeerInfo(const StreamInfo::StreamInfo&, Http::HeaderMap&,
Context&) const override;
};

class FilterConfig : public Logger::Loggable<Logger::Id::filter> {
public:
FilterConfig(const io::istio::http::peer_metadata::Config&,
Expand Down Expand Up @@ -155,7 +162,7 @@ class FilterConfig : public Logger::Loggable<Logger::Id::filter> {

using FilterConfigSharedPtr = std::shared_ptr<FilterConfig>;

class Filter : public Http::PassThroughFilter {
class Filter : public Http::PassThroughFilter, public Logger::Loggable<Logger::Id::filter> {
public:
Filter(const FilterConfigSharedPtr& config) : config_(config) {}
Http::FilterHeadersStatus decodeHeaders(Http::RequestHeaderMap&, bool) override;
Expand Down
Loading