HDDS-14150. [STS] Connect STS Endpoint to Backend Processing #9654

fmorg-git · 2026-01-21T04:35:18Z

Please describe your PR in detail:

Connect S3 Gateway STS Endpoint to Backend OzoneManager Processing
Add configuration flag check in OzoneManager so STS assume role call is disallowed if configuration flag is not true. Also ensure non-native authorizer is being used as well.
A future PR will move duplicate constants and STS validation methods from the endpoint and backend OzoneManager to a shared location.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14150

How was this patch tested?

unit tests and smoke testing

ChenSammi · 2026-01-23T09:05:52Z

...one-manager/src/test/java/org/apache/hadoop/ozone/om/ratis/TestOzoneManagerRatisRequest.java

+  }
+
+  @Test
+  public void testAssumeRoleRejectedWhenStsEnabledAndNativeAuthorizerNotUsed() {


testAssumeRoleRejectedWhenStsEnabledAndNativeAuthorizerNotUsed ->

testAssumeRoleAllowedWhenStsEnabledAndNativeAuthorizerNotUsed

updated 21381a9

ChenSammi · 2026-01-23T09:14:26Z

...zone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/signature/AWSSignatureProcessor.java

+    if (queryParams == null) {
+      return null;
+    }
+    final String stsQueryParam = queryParams.getFirst("X-Amz-Security-Token");


Since "X-Amz-Security-Token" is case insensitive, it's better to loop the queryParams, and use the compareToIgnoreCase to compare the parameter name.

updated 21381a9

ChenSammi · 2026-01-23T10:13:41Z

hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3sts/S3STSEndpoint.java

-    Random random = new Random();
-    for (int i = 0; i < length; i++) {
-      sb.append(chars.charAt(random.nextInt(chars.length())));
+    final String requestId = UUID.randomUUID().toString();


Shall we return the requestId from OM, and add the requestId in assumeRole audit log?

ChenSammi · 2026-01-23T10:19:26Z

hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3sts/S3STSEndpoint.java

+          .build();
+    }

+    final AssumeRoleResponseInfo responseInfo = getClient()


It's better we catch the exception from assumeRole() and generateAssumeRoleResponse(), wrap it as OS3Exception to return.

It looks like audit log is not supported in S3AssumeRoleRequest currently.

I believe in case of signature mismatch, we should catch OMException too

updated - per internal communication, I identified that the endpoint was returning plain text instead of XML, so I revamped the error handling to use a new OSTSException that conforms to AWS XML structure. It distinguishes between signature mismatch and access denied similar to AWS bdb33a4

It looks like audit log is not supported in S3AssumeRoleRequest currently.

updated - per internal communication, I mentioned with @len548 that I'll add the audit log here 85c38d0

ChenSammi · 2026-01-23T10:25:59Z

hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3sts/S3STSEndpoint.java

+    final String accountId = parts[4];
+    final String resource = parts[5]; // role/<name>
+
+    if (accountId == null || accountId.isEmpty() || resource == null || !resource.startsWith("role/") ||


You can leverage Strings.isNullOrEmpty() for string null and empty check. Strings.isNullOrEmpty is wildly used in Ozone.

updated 21381a9

len548 · 2026-01-26T14:51:53Z

I think we can remove all the appearance of OS3Exception in S3STSEndpoint class. While developing in the previous PR, I added them to handle errors in endpoint. Later I replaced the way to handle errors with XML format instead of OS3Exception and I forgot to remove them. Can you remove them please?

… conform to AWS

fmorg-git · 2026-01-27T03:40:39Z

I think we can remove all the appearance of OS3Exception in S3STSEndpoint class. While developing in the previous PR, I added them to handle errors in endpoint. Later I replaced the way to handle errors with XML format instead of OS3Exception and I forgot to remove them. Can you remove them please?

Per internal communication, I noticed recently the endpoint was returning plain text for errors when it should have been returning XML. The OS3Exception has an error XML format, but testing against AWS STS shows the error response XML structure for STS errors is slightly different than for S3 errors, so I revamped the error handling in this commit bdb33a4

fmorg-git · 2026-01-27T03:58:29Z

This PR is getting a little big and unwieldy - I will close this and break it up into multiple PRs.

Just connect the endpoint and Ozone manager and a few small PR comment updates (i.e. ce3e85c and 21381a9)
Revamp the error handling in the endpoint to conform the AWS XML (i.e. bdb33a4)
Add audit log to S3G and OM and correlate requestID between them (i.e. 85c38d0 and future work to correlate requestID)

fmorg-git changed the base branch from master to HDDS-13323-sts January 21, 2026 04:35

ChenSammi added the sts Changes for Ozone's S3 Security Token Service label Jan 21, 2026

fmorg-git force-pushed the HDDS-14150 branch from 364aeaa to 7e981db Compare January 21, 2026 17:27

connect STS endpoint to backend processing

ce3e85c

fmorg-git force-pushed the HDDS-14150 branch from 7e981db to ce3e85c Compare January 21, 2026 18:02

fmorg-git mentioned this pull request Jan 22, 2026

HDDS-14472. [STS] Refactor constants and validation methods to shared location #9658

Draft

ChenSammi reviewed Jan 23, 2026

View reviewed changes

pr review updates for ChenSammi

21381a9

Fabian Morgan added 2 commits January 26, 2026 15:39

pr review updates for ChenSammi - better error handling and revamp to…

bdb33a4

… conform to AWS

pr updates for ChenSammi - add audit logs for assumeRole to S3G and OM

85c38d0

fmorg-git closed this Jan 27, 2026

fmorg-git mentioned this pull request Jan 27, 2026

HDDS-14150. [STS] Connect STS Endpoint to Backend Processing #9673

Merged

HDDS-14150. [STS] Connect STS Endpoint to Backend Processing #9654

HDDS-14150. [STS] Connect STS Endpoint to Backend Processing #9654

Conversation

fmorg-git commented Jan 21, 2026

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fmorg-git Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

len548 commented Jan 26, 2026

Uh oh!

fmorg-git commented Jan 27, 2026

Uh oh!

fmorg-git commented Jan 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fmorg-git Jan 27, 2026 •

edited

Loading