Add IAM Auth Method by pglass · Pull Request #12583 · hashicorp/consul

pglass · 2022-03-21T19:13:55Z

This adds an IAM auth method to Consul to support logging in with AWS IAM identities.

Overview

This PR uses a similar approach as the Vault IAM auth method. Vault's login token format is used here, but with a unique change to avoid the need for config the auth method with AWS credentials.

The "bearer token" contains one or two signed AWS API requests, which are JSON encoded.

A client-signed sts:GetCallerIdentity request (same as Vault).
- This is generated by consul login -aws-auto-bearer-token ....
- The auth method must be configured with EnableIAMEntityDetails=false
A client-signed sts:GetCallerIdentity request, with a signed iam:GetRole or iam:GetUser request embedded in headers (unique to Consul).
- This is generated by consul login -aws-auto-bearer-token -aws-include-entity ...
- The auth method must be configured with EnableIAMEntityDetails=true

Because the requests are signed with the client credentials, the IAM auth method does not need to be configured with its own AWS credentials.

Code

command/login/* - updates consul login with additional flags to support logging into the AWS auth method
- This calls GenerateLoginData() in the internal iamauth package to do the work of generating the bearer token
- Then, the existing logic is used to login with the bearer token
agent/consul/authmethod/awsauth/* - implements the auth method
- This implements the existing authmethod.Validator interface
- This calls ValidateLogin() in the internal iamauth package to do the work of validating the identity of a client attempting to login
internal/iamauth/* - an internal package which implements the core logic. This is a mix of new code and code borrowed/modified from the Vault IAM auth method
- Authenticator.ValidateLogin() validates a client's identity to determine if they are permitted to login
- The BearerToken type parses the login token and does some validation of the token
- The GenerateLoginData function is used to generate the login data that forms the bearer token, which involves formatting signed AWS API requests

Usage

Configuring the auth method (example):

consul acl auth-method create -type aws-iam \
  -name <name> \
  -config='{
    "BoundIAMPrincipalARNs": [],
    "EnableIAMEntityDetails": false,
    "IAMEntityTags": []
    "MaxRetries": -1,
    "IAMEndpoint": "",
    "STSEndpoint": "",
    "STSRegion": "",
    "ServerIDHeaderValue": "",
    "AllowedSTSHeaderValues": [],
}'

Login

consul login -type aws-iam -method <name> -aws-auto-bearer-token -token-sink-file token.txt

rboyer · 2022-03-21T19:42:07Z

Probably good to in the PR somewhere and likely also in a comment in one of the packages indicating which version/sha of various vault libraries the various AWS bits were lifted from. That way there's a paper trail to figure out which possible future patches on the vault side need to be re-replicated here.

pglass · 2022-03-21T20:32:23Z

+		creds = sess.Config.Credentials
+	}
+
+	loginData, err := iamauth.GenerateLoginData(&iamauth.LoginInput{


Vault calls into a GenerateLoginData function, which I didn't end up re-using because (1) it doesn't support encoding the "get entity" request (GetRole or GetUser) in the login data, which is unique to the Consul implementation, and (2) there's a hardcoded vault server id header name, which is also why there are a bunch of header names passed into GenerateLoginData here, so that we're using X-Consul* header names.

pglass · 2022-03-21T20:39:24Z

+		)
+	} else {
+		// Session loads creds from standard sources (env vars, file, EC2 metadata, ...)
+		sess, err := session.NewSessionWithOptions(session.Options{Config: cfg})


This relies the AWS SDK's standard credential chain, whereas Vault credential discovery calls into GenerateCredentialChain, which does some custom things. I haven't deeply compared these. Probably, there's a good reason for what Vault does, and we should adopt their implementation, which I can do now or in a follow up.

It might be worth figuring out what value GenerateCredentialChain is bringing and erring on the side of using it over not.

I've dug through the credential loading a good bit in the past couple days. I time boxed this, but there's nothing I've found that Vault supports in its custom credential chain that the SDK doesn't also support out of the box.

Users have also noticed differences between Vault's credential chain and the SDK, and Vault has generally moved toward matching the SDK behavior where possible (such as: hashicorp/vault#7738 and hashicorp/vault#13807).

So I think we should rely on the SDK's credential discovery in favor of being more standardized.

pglass · 2022-03-21T20:52:44Z

+	return identityDetails, nil
+}
+
+// https://github.com/hashicorp/vault/blob/ba533d006f2244103648785ebfe8a9a9763d2b6e/builtin/credential/aws/path_login.go#L1321-L1361


I referenced the corresponding Vault code, but this is simpler since it doesn't support the unique role/user id check.

Note: I kept the links to Vault code for comparison during this initial code review, but I'm planning on removing those from the source code. I'll add permalinks to the Vault source in the README for this internal module instead.

pglass · 2022-03-21T21:31:36Z

+// For backwards compatibility, even if you request a region other than us-east-1, it'll still sign for us-east-1.
+// See, e.g., https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_enable-regions.html#id_credentials_temp_enable-regions_writing_code
+// So we have to shim in this EndpointResolver to force it to sign for the right region
+func stsSigningResolver(service, region string, optFns ...func(*endpoints.Options)) (endpoints.ResolvedEndpoint, error) {


TODO: Figure out if the STSRegionalEndpoint config option takes care of this for us.

rboyer · 2022-03-24T19:17:17Z

+
+	// ServerIDHeaderValue adds a X-Consul-IAM-ServerID header to each AWS API request.
+	// This helps protect against replay attacks.
+	ServerIDHeaderValue string `json:",omitempty"`


Might want to carry more detail over from the vault equivalent field, or do so on the website pages describing this field: https://www.vaultproject.io/api-docs/auth/aws#iam_server_id_header_value

Yeah, I'll detail this in the docs on consul.io.

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

pglass · 2022-03-30T18:46:37Z

Created the following couple of issues to follow up on before 1.12 GA:

hc-github-team-consul-core · 2022-03-31T15:19:20Z

🍒 If backport labels were added before merging, cherry-picking will start automatically.

To retroactively trigger a backport after merging, add backport labels and re-run https://circleci.com/gh/hashicorp/consul/619124.

pglass requested review from a team, erichaberkorn and rboyer and removed request for a team March 21, 2022 19:13

github-actions Bot added pr/dependencies PR specifically updates dependencies of project theme/cli Flags and documentation for the CLI interface labels Mar 21, 2022

pglass commented Mar 21, 2022

View reviewed changes

Comment thread go.sum Outdated

rboyer reviewed Mar 21, 2022

View reviewed changes

Comment thread internal/iamauth/auth.go Outdated

rboyer reviewed Mar 21, 2022

View reviewed changes

Comment thread command/login/login.go Outdated

pglass commented Mar 21, 2022

View reviewed changes

vercel Bot temporarily deployed to Preview – consul March 22, 2022 23:51 Inactive

vercel Bot temporarily deployed to Preview – consul-ui-staging March 22, 2022 23:51 Inactive

vercel Bot temporarily deployed to Preview – consul-ui-staging March 22, 2022 23:54 Inactive

vercel Bot temporarily deployed to Preview – consul March 22, 2022 23:54 Inactive

vercel Bot temporarily deployed to Preview – consul-ui-staging March 23, 2022 21:10 Inactive

vercel Bot temporarily deployed to Preview – consul March 23, 2022 21:10 Inactive

vercel Bot temporarily deployed to Preview – consul March 23, 2022 21:25 Inactive

vercel Bot temporarily deployed to Preview – consul-ui-staging March 23, 2022 21:25 Inactive

pglass requested a review from rboyer March 23, 2022 21:34

Paul Glass added 6 commits March 24, 2022 10:45

Add IAM Auth Method

97d411b

Tidy IAM auth method

323149a

Add lib/glob.go for IAM auth method

423a836

Unit tests for IAM auth method BearerToken

ad1353c

fix check for unique id in IAM auth method

82b6abe

Add changelog

f0af5b1

rboyer reviewed Mar 24, 2022

View reviewed changes

Comment thread .changelog/12583.txt Outdated

rboyer reviewed Mar 24, 2022

View reviewed changes

Comment thread command/login/login_test.go Outdated

rboyer reviewed Mar 24, 2022

View reviewed changes

Comment thread command/login/aws.go

rboyer reviewed Mar 24, 2022

View reviewed changes

Remove aws_oss.go, add aws_test.go for IAM auth method

495d3b6

pglass force-pushed the pglass/iam-auth-method branch from c6febdc to 495d3b6 Compare March 25, 2022 20:47

vercel Bot temporarily deployed to Preview – consul March 25, 2022 20:47 Inactive

vercel Bot temporarily deployed to Preview – consul-ui-staging March 25, 2022 20:47 Inactive

rboyer added this to the 1.12.0-beta1 milestone Mar 30, 2022

Update .changelog/12583.txt

25fddbc

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

vercel Bot temporarily deployed to Preview – consul-ui-staging March 30, 2022 16:56 Inactive

vercel Bot temporarily deployed to Preview – consul March 30, 2022 16:56 Inactive

Tweak IAM auth method cred loading

4b2d3b5

vercel Bot temporarily deployed to Preview – consul March 30, 2022 18:01 Inactive

vercel Bot temporarily deployed to Preview – consul-ui-staging March 30, 2022 18:01 Inactive

This was referenced Mar 30, 2022

IAM Auth Method: Remove stsSigningResolver? #12661

Closed

IAM Auth Method: Documentation #12662

Closed

rboyer reviewed Mar 30, 2022

View reviewed changes

Comment thread internal/iamauth/auth.go Outdated

IAM auth: info -> debug log

3715803

vercel Bot temporarily deployed to Preview – consul March 30, 2022 21:53 Inactive

vercel Bot temporarily deployed to Preview – consul-ui-staging March 30, 2022 21:53 Inactive

rboyer added the pr/no-metrics-test label Mar 30, 2022

rboyer approved these changes Mar 30, 2022

View reviewed changes

pglass merged commit 706c844 into main Mar 31, 2022

pglass deleted the pglass/iam-auth-method branch March 31, 2022 15:18

Conversation

pglass commented Mar 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Code

Usage

Uh oh!

Uh oh!

rboyer commented Mar 21, 2022

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pglass commented Mar 30, 2022

Uh oh!

Uh oh!

hc-github-team-consul-core commented Mar 31, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pglass commented Mar 21, 2022 •

edited

Loading