AWS Role Based credentials Assume role in child accounts

[chemapolo]

Version:

4.5.8

$ openshift-install version
./openshift-install 4.5.8
built from commit 0d5c871ce7d03f3d03ab4371dc39916a5415cf5c
release image quay.io/openshift-release-dev/ocp-release@sha256:ae61753ad8c8a26ed67fa233eea578194600d6c72622edab2516879cfbf019fd

Platform:

aws

• UPI (semi-manual installation on customised infrastructure)

What happened?

Using export $AWS_CONFIG_FILE, $AWS_DEFAULT_PROFILE, $AWS_SDK_LOAD_CONFIG and $AWS_SHARED_CREDENTIALS_FILE env vars to assume role while creating manifest for installation I retrieve:

level=debug msg= Generating Master Ignition Config...
level=debug msg=Generating Master Machines...
level=fatal msg=failed to fetch Master Machines: failed to generate asset "Master Machines": creating AWS session: fetching availability zones: UnauthorizedOperation: You are not authorized to perform this operation.
level=fatal msg= status code: 403, request id: b5cdf37d-bdab-4a01-8d83-ffeb71b5d01e

What you expected to happen?

It works perfectly with a user created in the account with KEy and secret credentials but unfortunatelly we use child accounts in our org and I expected to assume the role specified in the profile, child accounts in aws need to be role based accesed

How to reproduce it (as minimally and precisely as possible)?

simply configure env vars to force the profile and config files and run openshift-install create manifest with an already created install-config.yaml

$ export AWS_CONFIG_FILE=config location
$ export AWS_DEFAULT_PROFILE=desired profile
$ export AWS_SDK_LOAD_CONFIG=1 (to force sdk to gather those values)
$ export AWS_SHARED_CREDENTIALS_FILE= cred file location

$ openshift-install create manifest

$ openshift-install create cluster

Anything else we need to know?

I've been debuging the code and as you can see here https://github.com/Burnerator/installer/tree/aws-role-based-credentials it works to create manifests, however while creating cluster it retrieves the next error in platformpermscheck.go file while calling cloud credential operator package

FATAL failed to fetch Cluster: failed to fetch dependency of "Cluster": failed to generate asset "Platform Permissions Check": validate AWS credentials: checking install permissions: error gathering AWS credentials details: error querying username: ValidationError: Must specify userName when calling with non-User credentials
FATAL status code: 400, request id: 7c5c4dad-e594-4ef3-a456-4e06c82a3f18

basically it fails but now in pkg github.com/openshift/cloud-credential-operator/pkg/aws/utils.go I've been checking it and it uses Get UserID what is valid for non arn role users, for arn roled based user the id needs to be gathered from sts get-caller-identity.

References

https://github.com/openshift/cloud-credential-operator/pkg/aws/utils.go

https://github.com/Burnerator/installer/tree/aws-role-based-credentials

I was able to modify the installer to assume the role and pass the checks, but unfortunatelly cloud-credential-operator is going to be installed just without assumerole support that's why i didn't do the pull request to the installer.

I got an approach that works with CCO and assume role but i think that they are having strong dependency, I've reviewing how to contribute but unfortunatelly i dont see any information about how to create a pull request which depends on other pr for another project. I mean the installer PR will make no sense if the cloud-credential-operator PR is not approved.

Could you please kindly let me know how to proceed?

Thanks inadvance and kind regards

[dgoodwin]

In this case it would probably make sense to first lay the groundwork in the CCO before posting the installer PR, but that would need to be done carefully and in a backward compatible way to be able to roll this out.

As it happens we have substantial work underway as we speak for some form of STS support across CCO, the installer, and the various operators that use our creds. We should have an enhancement open sometime in the next couple weeks I will link here for you to review. Meanwhile work is starting to land in CCO (openshift/cloud-credential-operator#264) and some of the other operators.

Feel free to open your CCO PR and we'll take a look and see what we can use.

[chemapolo]

Thanks a lot @dgoodwin I'll test it to ensure backward compatibility and open the PR. Looking forward for that link ;-)

[glennodickson]

Version: openshift-client-linux-4.5.0-0.okd-2020-10-15-235428.tar.gz
Cloud: AWS

Running Openshift installer from EC2 with ~/.asw/credentials file loaded with access key, secret however, I get the same error as you have described above:

FATAL failed to fetch Cluster: failed to fetch dependency of "Cluster": failed to generate asset "Platform Permissions Check": validate AWS credentials: checking install permissions: error gathering AWS credentials details: error querying username: ValidationError: Must specify userName when calling with non-User credentials FATAL status code: 400, request id: 7c5c4dad-e594-4ef3-a456-4e06c82a3f18

Are all the exports you mentioned above required?

I ran the following command with the output which looks similar:

      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                <not set>             None    None
access_key     ****************4JDF shared-credentials-file
secret_key     ****************Memb shared-credentials-file
    region           ap-southeast-2      config-file    ~/.aws/config

AWS commands works, like:
AWS s3 ls

Not sure what Openshift requires to resolve this, any ideas would be very welcome.

Thanks.

[chemapolo]

Hi @glennodickson with shared credentials and the user created inside the child account it works for me fine. as far as i know the problem with get user is just for assume-role credentials.

Could you please kindly run the next commands and paste the output here?

$ aws iam get-user

$ aws sts get-caller-identity

Regards

Chema

[chemapolo]

Hi, thanks a lot i'll wait until a new release fix this. regards