feat(awsutil-v2): implement awsutil for aws-sdk-go-v2

[ddebko]

Summary

This PR is adding a new major version of awsutil. The breaking change that requires a major version release is using the latest aws sdk: aws-sdk-go-v2.

Disclaimer:

Feel free to update to this major release change if you are migrating to aws-sdk-go-v2. The Vault team should be cautious of updating to this version due to potential breaking changes from behavior differences between the aws-sdk-go libraries.

Note:

• This PR does not replicate the exact behaviors of the original v1 awsutil package. For example, environment variables are always loaded by default in v2. The aws-sdk-go-v2 also doesn't support RemoteCredProviders. GetRegion no longer can get a default region value from ec2metadata because it does not exist and the replacement imds pkg does not support the same feature.
• Custom endpoint resolvers are now associated with a client object (ex: iam, sts) rather than a aws.Config
• withStsEndpoint type was changed from a string to sts.EndpointResolverV2. This option flag was relabeled to withStsEndpointResolver.
• withIamEndpoint type was changed from a string to iam.EndpointResovlerV2. This option flag was relabeled to withIamEndpointResolver.

The benefits of changing the endpoint options from a string to their respective EndpointResolverV2 definition type is that now we can enable complex routing to a set of different endpoints based on fields defined in the resolverV2 struct. Please follow this link to read more about the EndpointResolverV2 option. Example:

type resolverV2 struct {
    // you could inject additional application context here as well
    region string
}

func (*resolverV2) ResolveEndpoint(ctx context.Context, params s3.EndpointParameters) (
        smithyendpoints.Endpoint, error,
    ) {
    if region == "A" {
        return smithyEndpoints.Endpoint{
            URI: url.Parse("https://custom.service.endpoint/"),
        }
    }
    if region == "B" {...}

    // delegate back to the default v2 resolver otherwise
    return s3.NewDefaultEndpointResolverV2().ResolveEndpoint(ctx, params)
}

[jefferai]

@ddebko rather than making this awsutilv2 just make this v2 of the awsutil package. That will make it a separate Go module that one must explicitly import and the current version can still be modified if desired. But the general line of development is clearly to catch up with the current, supported version of the AWS SDK.

[sgmiller]

Jeff's suggestion might make it easier to review too.

[ddebko]

@jefferai @sgmiller I have updated the PR to move the code into a subdirectory/package called v2 in awsutil. I have 2 TODO comments for the custom endpoint resolvers and I was hoping to get some feedback on the behavior of the resolvers.

UPDATE: I believe I was confused by Jeff's feedback on making the awsutilv2 into the v2 of the awsutil pacakge. Originally I thought that meant that I needed to use subdirectory for major version changes, which is a thing in go. But now I believe his
intentions were to actually just rename the module in the go.mod file.

[jefferai]

Just a note to please move the files back to the main directory and update go.mod before tagging as v2.

Arguably it'd be more useful to have them there now as we could see the differences instead of it all being new code.

[sgmiller]

Afaict it looks good to me.

[jefferai]

Please make sure to make the necessary changes to go.mod given the version bump.

Unless the plan is to mark this as v1.0.0 since the current tags are still in v0?

[ddebko]

The current code will be kept as v0 and this new code will be v2. To continue working on v0, the branch awsutil/v0 was created. The module name for awsutil was relabeled to module github.com/hashicorp/go-secure-stdlib/awsutil/v2

[jefferai]

Approving to unblock merge access

[benashz]

For some reason the changes from 1a4b955 were not ported over. The PR description does not seem to mention why they were dropped, nor provide a migration path forward.