AWS Cloudwatch sink missing region error when role_arn is used with Credentials File Authentication

[cahartma]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

When we try and authenticate our cloudwatch logs sink using auth.credentials_file, if the file contains role_arn and web_identity_token_file (to authenticate via AssumeRoleWithWebIdentity), there is an Invalid Configuration Error in the log, and the sink fails to authenticate. STS returned an error assuming web identity role, Missing Region

By default, the region should fallback to the service region (which is additionally specified in the config)

Tried setting env var AWS_REGION, AWS_DEFAULT_REGION, neither resolved the issue. When trying to set auth.region in the vector config, the field is invalid for cloudwatch credentials file auth: error=data did not match any variant of untagged enum AwsAuthentication

There are no 'Missing Region' errors when using long-lived credentials in the file. (aws_access_key_id, aws_secret_access_key)

Configuration

[sinks.output_my_cw]
type = "aws_cloudwatch_logs"
inputs = ["in"]
region = "us-east-2"
group_name = "group-test"
stream_name = "my-stream"
auth.credentials_file = "/root/.aws/credentials"
#auth.profile = "default"
encoding.codec = "json"

--------------------------
Sample aws credentials file:
[default]
web_identity_token_file=/var/run/secrets/serviceaccount/token
role_arn=arn:aws:iam::123456789012:role/logging-role-for-sts

Version

v0.45

Debug Output

`WARN sink{component_kind="sink" component_id=output_my_cw component_type=aws_cloudwatch_logs}:request{request_id=1}:invoke{service=cloudwatchlogs operation=DescribeLogStreams}:try_op:try_attempt:lazy_load_identity:load_base_credentials: aws_config::web_identity_token: STS returned an error assuming web identity role error=dispatch failure: other: Invalid Configuration: Missing Region (DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: ResolveEndpointError { message: "Invalid Configuration: Missing Region", source: None }, connection: Unknown } }))`

Example Data

No response

Additional Context

I've verified a simple fix by setting the region to the service_region in file: /src/aws/auth.rs.

===================================================================
diff --git a/src/aws/auth.rs b/src/aws/auth.rs
--- a/src/aws/auth.rs	
+++ b/src/aws/auth.rs	
@@ -306,7 +306,9 @@
                     .with_file(ProfileFileKind::Credentials, credentials_file)
                     .build();
 
-                let provider_config = ProviderConfig::empty().with_http_client(connector);
+                let provider_config = ProviderConfig::empty()
+                    .with_region(Option::from(service_region))
+                    .with_http_client(connector);
 
                 let profile_provider = ProfileFileCredentialsProvider::builder()
                     .profile_files(profile_files)

I believe the correct fix is to add the auth.region config to the cw logs credentials_file auth type. I will submit a PR with the new field added.

References

Proposed Fix:

• fix(auth): #22827 missing region error when using AWS credentials file auth #22831