Skip to content

AWS Web Identity / IRSA Support #10541

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pjain1 merged 5 commits into from
Jan 25, 2021
Merged

AWS Web Identity / IRSA Support #10541

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4e28d9d
AWS Web Identity Support
himadrisingh Oct 30, 2020
0c75e2a
Update kinesis-ingestion.md
himadrisingh Nov 6, 2020
f1cd7ae
disabling coverage tests
himadrisingh Jan 22, 2021
1ca8dce
exclude coverage
himadrisingh Jan 22, 2021
f1d3a56
Update licenses.yaml
himadrisingh Jan 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions cloud/aws-common/src/main/java/org/apache/druid/common/aws/AWSCredentialsUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import com.amazonaws.auth.EnvironmentVariableCredentialsProvider;
import com.amazonaws.auth.InstanceProfileCredentialsProvider;
import com.amazonaws.auth.SystemPropertiesCredentialsProvider;
import com.amazonaws.auth.WebIdentityTokenCredentialsProvider;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;

public class AWSCredentialsUtils
Expand All @@ -35,6 +36,7 @@ public static AWSCredentialsProviderChain defaultAWSCredentialsProviderChain(fin
new LazyFileSessionCredentialsProvider(config),
new EnvironmentVariableCredentialsProvider(),
new SystemPropertiesCredentialsProvider(),
WebIdentityTokenCredentialsProvider.create(),
new ProfileCredentialsProvider(),
new EC2ContainerCredentialsProviderWrapper(),
InstanceProfileCredentialsProvider.getInstance());
Expand Down
2 changes: 1 addition & 1 deletion docs/development/extensions-core/kinesis-ingestion.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ To authenticate with AWS, you must provide your AWS access key and AWS secret ke
-Ddruid.kinesis.accessKey=123 -Ddruid.kinesis.secretKey=456
```
The AWS access key ID and secret access key are used for Kinesis API requests. If this is not provided, the service will
look for credentials set in environment variables, in the default profile configuration file, and from the EC2 instance
look for credentials set in environment variables, via [Web Identity Token](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_oidc.html), in the default profile configuration file, and from the EC2 instance
profile provider (in this order).

### Getting Supervisor Status Report
Expand Down
5 changes: 5 additions & 0 deletions extensions-core/s3-extensions/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,11 @@
<artifactId>aws-java-sdk-s3</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-sts</artifactId>
<scope>provided</scope>
</dependency>
<!-- Tests -->
<dependency>
<groupId>org.apache.druid</groupId>
Expand Down
5 changes: 3 additions & 2 deletions licenses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ name: AWS RDS SDK for Java
license_category: source
module: extensions/druid-aws-rds-extensions
license_name: Apache License version 2.0
version: 1.11.199
version: 1.11.884
libraries:
- com.amazonaws: aws-java-sdk-rds

Expand Down Expand Up @@ -3583,10 +3583,11 @@ name: AWS SDK for Java
license_category: binary
module: extensions/druid-kinesis-indexing-service
license_name: Apache License version 2.0
version: 1.11.199
version: 1.11.884
libraries:
- com.amazonaws: aws-java-sdk-kinesis
- com.amazonaws: aws-java-sdk-sts
- com.amazonaws: jmespath-java

---

Expand Down
11 changes: 9 additions & 2 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@
<hadoop.compile.version>2.8.5</hadoop.compile.version>
<mockito.version>3.2.4</mockito.version>
<powermock.version>2.0.2</powermock.version>
<aws.sdk.version>1.11.199</aws.sdk.version>
<aws.sdk.version>1.11.884</aws.sdk.version>
<caffeine.version>2.8.0</caffeine.version>
<jacoco.version>0.8.5</jacoco.version>
<!-- Curator requires 3.4.x ZooKeeper clients to maintain compatibility with 3.4.x ZooKeeper servers,
Expand Down Expand Up @@ -264,6 +264,11 @@
<artifactId>aws-java-sdk-s3</artifactId>
<version>${aws.sdk.version}</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-sts</artifactId>
<version>${aws.sdk.version}</version>
</dependency>
<dependency>
<groupId>com.ning</groupId>
<artifactId>compress-lzf</artifactId>
Expand Down Expand Up @@ -1293,7 +1298,9 @@

<!-- Exceptions -->
<exclude>org/apache/druid/query/TruncatedResponseContextException.class</exclude>
</excludes>

<exclude>org/apache/druid/common/aws/AWSCredentials*</exclude>
</excludes>
</configuration>
<executions>
<execution>
Expand Down