Skip to content

OSDOCS#10947: Support for CCO in HCP#77653

Merged
kcarmichael08 merged 1 commit intoopenshift:mainfrom
xenolinux:cco-hcp
Aug 26, 2024
Merged

OSDOCS#10947: Support for CCO in HCP#77653
kcarmichael08 merged 1 commit intoopenshift:mainfrom
xenolinux:cco-hcp

Conversation

@xenolinux
Copy link
Copy Markdown
Contributor

@xenolinux xenolinux commented Jun 18, 2024

Version(s): 4.15+

Issue: https://issues.redhat.com/browse/OSDOCS-10947

Link to docs preview:

QE review:

  • QE has approved this change.

SME review:

  • SME has approved this change.

Additional information:

@openshift-ci openshift-ci Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 18, 2024
@ocpdocs-previewbot
Copy link
Copy Markdown

ocpdocs-previewbot commented Jun 18, 2024

Copy link
Copy Markdown
Contributor

@jeana-redhat jeana-redhat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work navigating the STS conditional spaghetti 😅 I've made a few suggestions here to catch a couple loose ends and match the structure of this in the OCP install books (since that's a clear workflow). I'll catch up with you on Slack about placement considerations for this content 🤓

[role="_additional-resources"]
.Additional resources

* To know about the Cloud Credential Operator, see xref:../operators/operator-reference.adoc#cloud-credential-operator_cluster-operators-ref[Cluster Operators reference page for the Cloud Credential Operator].
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add'l resources should just have the link text :)

Suggested change
* To know about the Cloud Credential Operator, see xref:../operators/operator-reference.adoc#cloud-credential-operator_cluster-operators-ref[Cluster Operators reference page for the Cloud Credential Operator].
* xref:../operators/operator-reference.adoc#cloud-credential-operator_cluster-operators-ref[Cluster Operators reference page for the Cloud Credential Operator]


* To know more about supported identity providers, see xref:../authentication/understanding-identity-provider.adoc#understanding-identity-provider["Understanding identity provider configuration"] in _Authentication and authorization_.

include::modules/cco-ccoctl-configuring.adoc[leveloffset=+3]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might consider bumping this to a +2? Right now it looks like it's part of "Configuring the OAuth server", but IIUC it's not really part of that

Suggested change
include::modules/cco-ccoctl-configuring.adoc[leveloffset=+3]
include::modules/cco-ccoctl-configuring.adoc[leveloffset=+2]

Alternatively, you could make a subheading at the == level to put all the CCO stuff inside to more closely match the structure of these tasks in the install books.

So, something like:

  • Configuring the OAuth server for a hosted cluster by using the CLI
  • Configuring the OAuth server for a hosted cluster by using the web console
  • Configuring an {hcp} cluster to use the AWS Security Token Service
    • Configuring the Cloud Credential Operator utility
    • Creating AWS resources with the Cloud Credential Operator utility
      • Creating AWS resources with a single command
      • Creating AWS resources individually
    • Incorporating the Cloud Credential Operator utility manifests

This was a bit harder to show with a PR review diff, but I sketched it out in my editor here if you want to see what I'm thinking 🙂

Local render TOC looks like this:
image


include::modules/cco-ccoctl-install-creating-manifests.adoc[leveloffset=+3]

include::modules/cco-mode-sts.adoc[leveloffset=+1]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't seem to exist

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in addition to this one, the module cco-ccoctl-creating-individually.adoc needs to have an hcp-cco condition included anywhere there's aws-sts for it to render right (assuming all STS steps apply to HCP 🙂)

Comment on lines +27 to +29
* You can use the `ccoctl aws create-all` command to create the AWS resources automatically. This is the quickest way to create the resources. See xref:../../../installing/installing_aws/ipi/installing-aws-network-customizations.adoc#cco-ccoctl-creating-at-once_installing-aws-network-customizations[Creating AWS resources with a single command].

* If you need to review the JSON files that the `ccoctl` tool creates before modifying AWS resources, or if the process the `ccoctl` tool uses to create AWS resources automatically does not meet the requirements of your organization, you can create the AWS resources individually. See xref:../../../installing/installing_aws/ipi/installing-aws-network-customizations.adoc#cco-ccoctl-creating-individually_installing-aws-network-customizations[Creating AWS resources individually].
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the links in these items, you probably want to point them to the instance of these modules on this page rather than off in IPI install

Suggested change
* You can use the `ccoctl aws create-all` command to create the AWS resources automatically. This is the quickest way to create the resources. See xref:../../../installing/installing_aws/ipi/installing-aws-network-customizations.adoc#cco-ccoctl-creating-at-once_installing-aws-network-customizations[Creating AWS resources with a single command].
* If you need to review the JSON files that the `ccoctl` tool creates before modifying AWS resources, or if the process the `ccoctl` tool uses to create AWS resources automatically does not meet the requirements of your organization, you can create the AWS resources individually. See xref:../../../installing/installing_aws/ipi/installing-aws-network-customizations.adoc#cco-ccoctl-creating-individually_installing-aws-network-customizations[Creating AWS resources individually].
* You can use the `ccoctl aws create-all` command to create the AWS resources automatically. This is the quickest way to create the resources. See xref:../hosted_control_planes/hcp-authentication-authorization.adoc#cco-ccoctl-creating-at-once_hcp-authentication-authorization[Creating AWS resources with a single command].
* If you need to review the JSON files that the `ccoctl` tool creates before modifying AWS resources, or if the process the `ccoctl` tool uses to create AWS resources automatically does not meet the requirements of your organization, you can create the AWS resources individually. See xref:../hosted_control_planes/hcp-authentication-authorization.adoc#cco-ccoctl-creating-individually_hcp-authentication-authorization[Creating AWS resources individually].

include::modules/cco-ccoctl-configuring.adoc[leveloffset=+3]

[id="hcp-sts-mode-ccoctl_{context}"]
== Using Credential Operator utility for {hcp}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
== Using Credential Operator utility for {hcp}
== Creating {aws-short} resources with the Cloud Credential Operator utility

@xenolinux xenolinux force-pushed the cco-hcp branch 2 times, most recently from 4389f5a to 218c9e9 Compare June 27, 2024 14:06
@xenolinux xenolinux changed the title HCP CCO OSDOCS#10947: HCP CCO Jun 27, 2024
@xenolinux xenolinux changed the title OSDOCS#10947: HCP CCO OSDOCS#10947: Support for CCO in HCP Jun 27, 2024
@zanetworker
Copy link
Copy Markdown

looks good. Thanks @xenolinux

@huangmingxia
Copy link
Copy Markdown

Hi @xenolinux Please hold off on merging this documentation PR for now. I’ve just reviewed the current doc and noticed that some issues are not clearly explained. I am in the process of reviewing it and will add my comments once I’m finished.
Thank you!

@xenolinux
Copy link
Copy Markdown
Contributor Author

@huangmingxia Sure. Thanks.

Copy link
Copy Markdown

@huangmingxia huangmingxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As @jeana-redhat suggested, the document structure is now very clear.

However, there are differences in how the Cloud Credential Operator works for HyperShift compared to OpenShift:

  1. For HyperShift on AWS, the CCO only supports Manual mode.
  2. In OpenShift, when creating a cluster in Manual mode, ccoctl is typically used to generate manifests and OIDC resources. However, this is not the case for HyperShift. HyperShift Hosted Clusters on AWS are always set to Manual mode by default, and there's no need to use ccoctl to generate manifests or OIDC resources.

For HyperShift, since the CCO only supports AWS in Manual mode and doesn’t require ccoctl to create other resources, given these points, there’s no need to include ccoctl usage in the document, we can remove that section from the doc.

Based on above background, we need to update the following content:

  1. Remove these sections(HyperShift Hosted Clusters on AWS do not use ccoctl. If ccoctl is needed in the future, we can add it then.).
  • Configuring the Cloud Credential Operator utility
  • Creating AWS resources with the Cloud Credential Operator utility
  • Incorporating the Cloud Credential Operator utility manifests
  1. Clarify the steps for creating a Hosted Cluster in Manual mode:
    Follow the same steps as in the hcp-getting-started-aws guide.

  2. Add below verification steps after installation:

  • Verify Hosted Cluster in Manual Mode, check that the Cloud Credential Operator is set to Manual mode by running:
    $oc get cloudcredentials cluster -o=jsonpath={.spec.credentialsMode}
    The output should be Manual:
    Manual

  • Verify Service Account Issuer is not empty, ensure the serviceAccountIssuer is not empty by executing:
    $oc get authentication cluster -o jsonpath --template '{.spec.serviceAccountIssuer }'
    The output like:
    https://aos-hypershift-ci-oidc-26499.s3.us-east-2.amazonaws.com/hypershift-ci-26499


You can configure the Cloud Credential Operator (CCO) utility (`ccoctl`) on hosted clusters to assign components IAM roles that provide short-term, limited-privilege security credentials.

To configure the CCO utility for a hosted cluster, your hosted cluster must use the {aws-short} {sts-short}.
Copy link
Copy Markdown

@huangmingxia huangmingxia Jul 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change the sentence to this note:
Note:
The Cloud Credential Operator (CCO) currently supports only Manual mode for Hosted Clusters on AWS. All Hosted Clusters are automatically configured to Manual mode by default, regardless of the management cluster's mode. At present, there is no need to generate additional resources using ccoctl.
In the future, if there are scenarios where ccoctl is added, we can update accordingly.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my education: are hosted clusters using AWS STS, or just manual CCO by itself? I ask because I wasn't aware that we could have AWS STS without the ccoctl 🙂

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me try to explain in my comment next.

@huangmingxia
Copy link
Copy Markdown

@jianping-shu @heliubj18 @YuLi517 Just want you to know this specific change.
@jianping-shu @heliubj18 Please also help review this change when you have time, if you have any questions or other suggestions, please feel free to raise them here. Thanks.

@xenolinux xenolinux force-pushed the cco-hcp branch 2 times, most recently from 65c6af1 to 3a20f6c Compare July 11, 2024 12:05
@openshift-ci openshift-ci Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 11, 2024
@xenolinux xenolinux force-pushed the cco-hcp branch 2 times, most recently from d417263 to 4cb0476 Compare July 11, 2024 12:11
@openshift-ci openshift-ci Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jul 11, 2024
@xenolinux
Copy link
Copy Markdown
Contributor Author

xenolinux commented Jul 11, 2024

@huangmingxia I appreciate your review. Thank you!!

I updated this PR as per your suggestions. Can you please re-review?

3. Add below verification steps after installation:
* Verify Hosted Cluster in Manual Mode, check that the Cloud Credential Operator is set to Manual mode by running:
  $oc get cloudcredentials cluster -o=jsonpath={.spec.credentialsMode}
  The output should be Manual:
  Manual

In this command (and the next command) cluster is the hosted cluster name, right? I updated this command as oc get cloudcredentials <hosted_cluster_name> -n <hosted_cluster_namespace> -o=jsonpath={.spec.credentialsMode} -- Please let me know if it needs to be corrected.

* Verify Service Account Issuer is not empty, ensure the `serviceAccountIssuer` is not empty by executing:
  $oc get authentication cluster -o jsonpath --template '{.spec.serviceAccountIssuer }'
  The output like:
  https://aos-hypershift-ci-oidc-26499.s3.us-east-2.amazonaws.com/hypershift-ci-26499

This link https://aos-hypershift-ci-oidc-26499.s3.us-east-2.amazonaws.com/hypershift-ci-26499 is giving the [1] output. Is it the expected output? If not, can you please help with the expected command output?

[1]

<Message>The specified bucket does not exist</Message>
<BucketName>aos-hypershift-ci-oidc-26499</BucketName>
<RequestId>3H472PZ2T22NS3Q9</RequestId>
<HostId>
9I0VrG2h+ZOYvFZMvwsv88WYSbEr+/HTvEpP3p8cuDF+CvXCQKYw/3Tk54cIQ0mqG9HTQd2JWq3BUALblTlR5+M/JWzScfTnhj8Yv/lEW5k=

@huangmingxia
Copy link
Copy Markdown

Hi @xenolinux Thank you for the update! I am working on some blocker tasks today and expect to have time to review either tomorrow or next Monday. I will address all the issues you mentioned during my review.
By the way, could you please share the docs preview link again? (It looks like the link is invalid.) Thanks. If you have any other questions, feel free to contact me.

@xenolinux
Copy link
Copy Markdown
Contributor Author

Hi @xenolinux Thank you for the update! I am working on some blocker tasks today and expect to have time to review either tomorrow or next Monday. I will address all the issues you mentioned during my review. By the way, could you please share the docs preview link again? (It looks like the link is invalid.) Thanks. If you have any other questions, feel free to contact me.

@huangmingxia Sure. No rush. :) Thanks!

It takes a couple of mins to get the preview updated again. Here's the updated preview link to the particular sections:

@jianping-shu
Copy link
Copy Markdown

jianping-shu commented Jul 15, 2024

Sorry for late review and feedback!
I'd like to add some background firstly. Let's assume that cloud is aws for next.
Hypershift hosted cluster only supports STS , and its management cluster can be non-STS with CCO running as normal.
Previously Hypershift hosted cluster is created by Hypershift operator without utilizing ccoctl, all logic is done by Hypershift own. There's no CCO running on the hosted cluster.
Feature OCPSTRAT-171 introduced a function that add awsSTSIAMRoleARN in CredentialsRequest file and CCO will generate its corresponding credential secret automatically. To implement the similar feature for the hosted cluster, that's OCPSTRAT-110 which the current doc PR is tied up to.
With OCPSTRAT-110, CCO is running on control plane for the hosted cluster and CCO mode is hard code to Manual.
My comment:
To address OCPSTRAT-110 feature usage, it is better to have a link to the procedure of Add token auth via CCO for Operator authors (AWS STS), introduced by the following PR. I don't know if any difference to add operator for ocp and hosted clusters, I assume the procedures are the same or very close.
#64808

@xenolinux xenolinux force-pushed the cco-hcp branch 2 times, most recently from 3f37009 to 2cc4d84 Compare July 22, 2024 06:53
@xenolinux
Copy link
Copy Markdown
Contributor Author

/label peer-review-needed

@openshift-ci openshift-ci Bot added the peer-review-needed Signifies that the peer review team needs to review this PR label Aug 16, 2024
@lahinson lahinson added peer-review-in-progress Signifies that the peer review team is reviewing this PR and removed peer-review-needed Signifies that the peer review team needs to review this PR labels Aug 16, 2024
Copy link
Copy Markdown
Contributor

@lahinson lahinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! I had a couple suggestions for your consideration, but overall, LGTM.

* To know more about supported identity providers, see xref:../authentication/understanding-identity-provider.adoc#understanding-identity-provider["Understanding identity provider configuration"] in _Authentication and authorization_.

[id="hcp-cco-aws-sts_{context}"]
== Using the CCO in a hosted cluster on {aws-short}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
== Using the CCO in a hosted cluster on {aws-short}
== Assigning components IAM roles by using the CCO in a hosted cluster on {aws-short}

Just a thought. Is the goal of this task to use the CCO in a hosted cluster, or is it to assign components IAM roles by using the CCO in a hosted cluster?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per the text Assigning components IAM... fits better. Addressed the suggestion.


[NOTE]
====
The CCO only supports a manual mode for hosted clusters on {aws-short}. By default, hosted clusters are configured in a manual mode. The management cluster might use modes other than manual.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The CCO only supports a manual mode for hosted clusters on {aws-short}. By default, hosted clusters are configured in a manual mode. The management cluster might use modes other than manual.
The CCO supports a manual mode only for hosted clusters on {aws-short}. By default, hosted clusters are configured in a manual mode. The management cluster might use modes other than manual.

I think "only" is modifying "a manual mode", so I moved "only" closer to that phrase. If "only" is actually modifying "supports", feel free to ignore this suggestion :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack'ed. Addressed

Comment thread modules/hcp-cco-verify-aws-sts.adoc Outdated
[id="hcp-cco-verify-aws-sts_{context}"]
= Verifying the CCO installation in a hosted cluster on {aws-short}

You can verify if the Cloud Credential Operator (CCO) is running correctly in your hosted control plane.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You can verify if the Cloud Credential Operator (CCO) is running correctly in your hosted control plane.
You can verify that the Cloud Credential Operator (CCO) is running correctly in your hosted control plane.

[role="_additional-resources"]
.Additional resources

* xref:../operators/operator-reference.adoc#cloud-credential-operator_cluster-operators-ref[Cluster Operators reference page for the Cloud Credential Operator].
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* xref:../operators/operator-reference.adoc#cloud-credential-operator_cluster-operators-ref[Cluster Operators reference page for the Cloud Credential Operator].
* xref:../operators/operator-reference.adoc#cloud-credential-operator_cluster-operators-ref[Cluster Operators reference page for the Cloud Credential Operator]

@lahinson lahinson added peer-review-done Signifies that the peer review team has reviewed this PR and removed peer-review-in-progress Signifies that the peer review team is reviewing this PR labels Aug 16, 2024
@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Aug 26, 2024

@xenolinux: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@xenolinux
Copy link
Copy Markdown
Contributor Author

/label merge-review-needed

@openshift-ci openshift-ci Bot added the merge-review-needed Signifies that the merge review team needs to review this PR label Aug 26, 2024
@kcarmichael08 kcarmichael08 added merge-review-in-progress Signifies that the merge review team is reviewing this PR branch/enterprise-4.15 branch/enterprise-4.16 branch/enterprise-4.17 and removed merge-review-needed Signifies that the merge review team needs to review this PR labels Aug 26, 2024
@kcarmichael08 kcarmichael08 added this to the Continuous Release milestone Aug 26, 2024
@kcarmichael08 kcarmichael08 merged commit dd1a0f3 into openshift:main Aug 26, 2024
@kcarmichael08
Copy link
Copy Markdown
Contributor

/cherrypick enterprise-4.15

@kcarmichael08
Copy link
Copy Markdown
Contributor

/cherrypick enterprise-4.16

@kcarmichael08
Copy link
Copy Markdown
Contributor

/cherrypick enterprise-4.17

@openshift-cherrypick-robot
Copy link
Copy Markdown

@kcarmichael08: new pull request created: #80906

Details

In response to this:

/cherrypick enterprise-4.15

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-cherrypick-robot
Copy link
Copy Markdown

@kcarmichael08: new pull request created: #80907

Details

In response to this:

/cherrypick enterprise-4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-cherrypick-robot
Copy link
Copy Markdown

@kcarmichael08: new pull request created: #80908

Details

In response to this:

/cherrypick enterprise-4.17

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@kcarmichael08 kcarmichael08 removed the merge-review-in-progress Signifies that the merge review team is reviewing this PR label Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

branch/enterprise-4.15 branch/enterprise-4.16 branch/enterprise-4.17 peer-review-done Signifies that the peer review team has reviewed this PR size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.