Skip to content

Add nutanix credentials handling with manual mode #450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-robot merged 3 commits into from
Mar 16, 2022
Merged

Add nutanix credentials handling with manual mode #450

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3956c2a
Update openshift/api dependency in go.mod
thunderboltsid Feb 7, 2022
ac7e44c
Update docs/ccoctl.md
thunderboltsid Feb 14, 2022
bdd8533
Add nutanix credential handling with manual mode
thunderboltsid Feb 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,7 @@ Azure | N<sup>1</sup> | N | Y | Y | N
GCP | Y | 4.7+ | Y | Y | 4.10+
IBMCloud | N | N | N | Y | N
KubeVirt | N | N | Y | N | N
Nutanix | N | N | N | Y | N
OpenStack | N | N | Y | N | N
oVirt | N | N | Y | N | N
VMWare | N | N | Y | N | N
Expand Down
2 changes: 2 additions & 0 deletions cmd/ccoctl/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
"github.com/openshift/cloud-credential-operator/pkg/cmd/provisioning/aws"
"github.com/openshift/cloud-credential-operator/pkg/cmd/provisioning/gcp"
"github.com/openshift/cloud-credential-operator/pkg/cmd/provisioning/ibmcloud"
"github.com/openshift/cloud-credential-operator/pkg/cmd/provisioning/nutanix"
)

func main() {
Expand All @@ -21,6 +22,7 @@ func main() {
rootCmd.AddCommand(gcp.NewGCPCmd())
rootCmd.AddCommand(ibmcloud.NewIBMCloudCmd())
rootCmd.AddCommand(alibabacloud.NewAliababaCloudCmd())
rootCmd.AddCommand(nutanix.NewNutanixCmd())

if err := rootCmd.Execute(); err != nil {
log.Fatal(err)
Expand Down
92 changes: 91 additions & 1 deletion docs/ccoctl.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -335,4 +335,94 @@ For alibaba cloud, the CCO utility (`ccoctl`) binary will create credentials Se
```
where:
- `name` is the name used to tag any cloud resources that are created for tracking.
- `region` is the Alibaba Cloud region in which cloud resources will be created.
- `region` is the Alibaba Cloud region in which cloud resources will be created.

## Nutanix

This is a guide for using manual mode on Nutanix, for more info about manual mode, please refer to [cco-mode-manual](https://github.com/openshift/cloud-credential-operator/blob/master/docs/mode-manual-creds.md).

For Nutanix, the CCO utility (`ccoctl`) binary will create credentials Secret manifests for the OpenShift installer.


### Prerequisite

1. Extract and prepare the ccoctl binary from the release image.

2. Ensure the following environment variables are set:
- NUTANIX_ENDPOINT=<endpoint_for_prism_central>
- NUTANIX_PORT=<port_for_prism_central>
- NUTANIX_USER=<username_for_prism_central>
- NUTANIX_PASSWORD=<password_for_prism_central>

### Procedure
Copy link
Copy Markdown

@jeana-redhat jeana-redhat Mar 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before doing all this, I assume the same prereq steps of getting the release image and setting up the ccoctl. Do they also need to do anything on the cloud provider end (such as in Alibaba procedure)?

Copy link
Copy Markdown
Contributor Author

@thunderboltsid thunderboltsid Mar 4, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to look further into Alibaba and how they're going about it. We are basically taking credentials requests and creating secret manifests with the same secret for all the corresponding credentials requests.

Copy link
Copy Markdown

@jeana-redhat jeana-redhat Mar 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No rush from me, but adding this comment for tracking (and to remind myself where we are 🙂):

Can add docs-approved label once this question is resolved

Copy link
Copy Markdown
Contributor Author

@thunderboltsid thunderboltsid Mar 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is in the sense that the workflow is manual entirely via ccoctl to extract credential requests via oc and generate corresponding secret manifests. We have added a prerequisite step to ensure user prepares a ccoctl binary https://github.com/openshift/cloud-credential-operator/pull/450/files#diff-2480a11ca4927139d6eaa9883946b6f4cb38358cd98def8c57dd73e9319dbc9cR349

Copy link
Copy Markdown

@rvanderp3 rvanderp3 Mar 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thunderboltsid i think the question is are there any pre-reqs that a customer has to complete prior to running ccoutl. each cloud has it's own cloud specific pre-reqs that have to be completed prior to generating the manifests.

Copy link
Copy Markdown

@jeana-redhat jeana-redhat Mar 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rvanderp3 yes, exactly what I am looking for.

@thunderboltsid is there anything they need to do analogous to items 2&3 here? Or anything else the user has to do like creating specific accounts etc. (especially on the Nutanix side) to get this ready aside from the usual ccoctl binary set up?

Copy link
Copy Markdown
Contributor Author

@thunderboltsid thunderboltsid Mar 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jeana-redhat @rvanderp3 I see. No, there are no extra steps that need to be performed other than the ones already mentioned in pre-requisites.

Copy link
Copy Markdown

@jeana-redhat jeana-redhat Mar 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! thanks :)
/docs-approved


1. Extract the list of CredentialsRequest custom resources (CRs) from the OpenShift Container Platform release image:

```bash
$ oc adm release extract --credentials-requests --cloud=nutanix --to=<path_to_directory_with_list_of_credentials_requests>/credrequests quay.io/<path_to>/ocp-release:<version>
```

> step 2&3 are only needed when preparing for upgrading clusters with manually maintained credentials. When doing a fresh installation please skip step 2&3**

2. For each CredentialsRequest CR in the release image, ensure that a namespace that matches the text in the spec.secretRef.namespace field exists in the cluster. You can check the list of namespaces on the cluster by running `oc get namespace`. This field is where the generated secrets that hold the credentials configuration are stored.

Sample Nutanix CredentialsRequest object
```yaml
apiVersion: cloudcredential.openshift.io/v1
kind: CredentialsRequest
metadata:
annotations:
include.release.openshift.io/self-managed-high-availability: "true"
labels:
controller-tools.k8s.io: "1.0"
name: openshift-machine-api-nutanix
namespace: openshift-cloud-credential-operator
spec:
providerSpec:
apiVersion: cloudcredential.openshift.io/v1
kind: NutanixProviderSpec
secretRef:
name: nutanix-credentials
namespace: openshift-machine-api
```

3. For any `CredentialsRequest` CR for which the cluster does not already have a namespace with the name specified in `spec.secretRef.namespace`, create the namespace:

```bash
$ oc create namespace <component_namespace>
```

4. Use the `ccoctl` tool to process all `CredentialsRequest` objects in the `credrequests` directory:

```bash
$ ccoctl nutanix create-shared-secrets --credentials-requests-dir=<path_to_directory_with_list_of_credentials_requests>/credrequests --output-dir=xxxxxx
```

where:

- `credentials-requests-dir` is the directory containing files of component CredentialsRequests.
- `output-dir` is the directory containing files of component credentials secret under the `manifests` directory.

5. Prepare to run the OpenShift Container Platform installer:

a. Create the install-config.yaml file:
```bash
$ openshift-install create install-config --dir ./path/to/installation/dir
```
b. Configure the cluster to install with the CCO in manual mode:

```bash
$ echo "credentialsMode: Manual" >> ./path/to/installation/dir/install-config.yaml
```

c. Create install manifests:

```bash
$ openshift-install create manifests --dir ./path/to/installation/dir
```

d. Copy the generated credential files to the target manifests directory:

```bash
$ cp <output_dir>/manifests/*credentials.yaml ./path/to/installation/dir/manifests/
```
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ require (
github.com/hashicorp/go-retryablehttp v0.7.0 // indirect
github.com/leodido/go-urn v1.2.1 // indirect
github.com/mitchellh/mapstructure v1.4.1 // indirect
github.com/openshift/api v0.0.0-20211209135129-c58d9f695577
github.com/openshift/api v0.0.0-20220203140920-bfe251c51d2d
github.com/openshift/build-machinery-go v0.0.0-20211213093930-7e33a7eb4ce3
github.com/openshift/library-go v0.0.0-20220121154930-b7889002d63e
github.com/pkg/errors v0.9.1
Expand Down
3 changes: 2 additions & 1 deletion go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -646,8 +646,9 @@ github.com/onsi/gomega v1.17.0 h1:9Luw4uT5HTjHTN8+aNcSThgH1vdXnmdJ8xIfZ4wyTRE=
github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY=
github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/openshift/api v0.0.0-20211209135129-c58d9f695577 h1:NUe82M8wMYXbd5s+WBAJ2QAZZivs+nhZ3zYgZFwKfqw=
github.com/openshift/api v0.0.0-20211209135129-c58d9f695577/go.mod h1:DoslCwtqUpr3d/gsbq4ZlkaMEdYqKxuypsDjorcHhME=
github.com/openshift/api v0.0.0-20220203140920-bfe251c51d2d h1:WuD14VS4SFKKH5hKeYiHTswlEByICzMNvaZrDXUjZiY=
github.com/openshift/api v0.0.0-20220203140920-bfe251c51d2d/go.mod h1:F/eU6jgr6Q2VhMu1mSpMmygxAELd7+BUxs3NHZ25jV4=
github.com/openshift/build-machinery-go v0.0.0-20210712174854-1bb7fd1518d3/go.mod h1:b1BuldmJlbA/xYtdZvKi+7j5YGB44qJUJDZ9zwiNCfE=
github.com/openshift/build-machinery-go v0.0.0-20211213093930-7e33a7eb4ce3 h1:65oBhJYHzYK5VL0gF1eiYY37lLzyLZ47b9y5Kib1nf8=
github.com/openshift/build-machinery-go v0.0.0-20211213093930-7e33a7eb4ce3/go.mod h1:b1BuldmJlbA/xYtdZvKi+7j5YGB44qJUJDZ9zwiNCfE=
Expand Down
Loading