-
Notifications
You must be signed in to change notification settings - Fork 152
[disconnected] Create hook to configure disconnected cluster #3814
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
drosenfe
wants to merge
1
commit into
openstack-k8s-operators:main
Choose a base branch
from
drosenfe:disconnectedhook
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
246 changes: 246 additions & 0 deletions
246
hooks/playbooks/config_cluster_for_disconnected_deployment.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,246 @@ | ||
| --- | ||
| - name: Update cluster for disconnected deployment | ||
| hosts: "{{ cifmw_target_host | default('localhost') }}" | ||
| vars: | ||
| oc_mirror_download_url: "{{ cifmw_disconnected_mirror_url | default('https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/latest/oc-mirror.rhel9.tar.gz') }}" | ||
| mirror_registry_url: "{{ cifmw_disconnected_registry_url | default('https://mirror.openshift.com/pub/cgw/mirror-registry/latest/mirror-registry-amd64.tar.gz') }}" | ||
| openstack_namespace: "{{ cifmw_openstack_namespace | default('openstack') }}" | ||
| disconnect_working_dir: "{{ cifmw_disconnected_working_dir | default('/home/zuul/disconnect_working_dir') }}" | ||
| mirror_location: "{{ disconnect_working_dir }}/mirror_location" | ||
| local_registry: "{{ disconnect_working_dir }}/local_registry" | ||
| mirror_registry_password: "JbmsjFR0yf6SNxKhk185BOVX2Dv39T74" # notsecret | ||
| oc_mirror_catalog_url: "{{ cifmw_ci_gen_kustomize_values_ooi_image | default('registry-proxy.engineering.redhat.com/rh-osbs/iib:1125611') }}" | ||
| oc_mirror_cert_manager_catalog_url: "{{ cifmw_cert_manager_catalog_url | default('registry.redhat.io/redhat/redhat-operator-index:v4.18') }}" | ||
| tasks: | ||
| - name: Create disconnected working directories | ||
| ansible.builtin.file: | ||
| path: "{{ item }}" | ||
| state: directory | ||
| mode: '0777' | ||
| loop: | ||
| - "{{ disconnect_working_dir }}" | ||
| - "{{ mirror_location }}" | ||
| - "{{ local_registry }}" | ||
|
|
||
| - name: Download oc mirror image to controller | ||
| ansible.builtin.get_url: | ||
| url: "{{ oc_mirror_download_url }}" | ||
| dest: "{{disconnect_working_dir}}/oc-mirror.rhel9.tar.gz" | ||
| mode: '0644' | ||
|
|
||
| - name: Install oc mirror | ||
| ansible.builtin.shell: | | ||
| set -eux | ||
| tar xvf {{disconnect_working_dir}}/oc-mirror.rhel9.tar.gz -C {{disconnect_working_dir}} && | ||
| chmod +x {{disconnect_working_dir}}/oc-mirror && | ||
| sudo mv {{disconnect_working_dir}}/oc-mirror /usr/local/bin/. | ||
|
|
||
| - name: Get host FQDN | ||
| ansible.builtin.command: hostname -f | ||
| register: host_fqdn | ||
|
|
||
| - name: Create mirror location file | ||
| become: true | ||
| ansible.builtin.shell: | | ||
| cat <<EOF > /etc/containers/registries.conf.d/010-stage.conf | ||
| unqualified-search-registries = ["registry.access.redhat.com", "docker.io"] | ||
| short-name-mode = "" | ||
|
|
||
| [[registry]] | ||
| prefix = "" | ||
| location = "registry.redhat.io" | ||
|
|
||
| [[registry.mirror]] | ||
| location = "registry.stage.redhat.io" | ||
| pull-from-mirror = "digest-only" | ||
| EOF | ||
|
|
||
| - name: Create update service namespace | ||
| cifmw.general.ci_script: | ||
| output_dir: "{{ cifmw_basedir }}/artifacts" | ||
| script: | | ||
| oc apply -f - <<EOF | ||
| apiVersion: v1 | ||
| kind: Namespace | ||
| metadata: | ||
| name: openshift-update-service | ||
| annotations: | ||
| openshift.io/node-selector: "" | ||
| labels: | ||
| openshift.io/cluster-monitoring: "true" | ||
| EOF | ||
|
|
||
| - name: Create update service operator group | ||
| cifmw.general.ci_script: | ||
| output_dir: "{{ cifmw_basedir }}/artifacts" | ||
| script: | | ||
| oc apply -f - <<EOF | ||
| apiVersion: operators.coreos.com/v1 | ||
| kind: OperatorGroup | ||
| metadata: | ||
| name: update-service-operator-group | ||
| namespace: openshift-update-service | ||
| spec: | ||
| targetNamespaces: | ||
| - openshift-update-service | ||
| EOF | ||
|
|
||
| - name: Create subscription service | ||
| cifmw.general.ci_script: | ||
| output_dir: "{{ cifmw_basedir }}/artifacts" | ||
| script: | | ||
| oc apply -f - <<EOF | ||
| apiVersion: operators.coreos.com/v1alpha1 | ||
| kind: Subscription | ||
| metadata: | ||
| name: update-service-subscription | ||
| namespace: openshift-update-service | ||
| spec: | ||
| channel: v1 | ||
| installPlanApproval: "Automatic" | ||
| source: "redhat-operators" | ||
| sourceNamespace: "openshift-marketplace" | ||
| name: "cincinnati-operator" | ||
| EOF | ||
|
|
||
| - name: Wait for update service operator to be installed | ||
| ansible.builtin.shell: | | ||
| set -eux | ||
| oc get crd | grep -i updateservice.operator.openshift.io | ||
| register: crd_out | ||
| until: "'updateservice.operator.openshift.io' in crd_out.stdout" | ||
| retries: 10 | ||
| delay: 30 | ||
|
|
||
| - name: Create Image Set yaml | ||
| ansible.builtin.shell: | | ||
| cat <<EOF >{{ disconnect_working_dir }}/imageset-config-v2.yaml | ||
| kind: ImageSetConfiguration | ||
| apiVersion: mirror.openshift.io/v2alpha1 | ||
| mirror: | ||
| operators: | ||
| - catalog: {{ oc_mirror_catalog_url }} | ||
| packages: | ||
| - name: openstack-operator | ||
| - name: local-storage-operator | ||
| - catalog: {{ oc_mirror_cert_manager_catalog_url }} | ||
| packages: | ||
| - name: kubernetes-nmstate-operator | ||
| - name: openshift-cert-manager-operator | ||
| - name: metallb-operator | ||
| - name: lvms-operator | ||
| - name: cluster-observability-operator | ||
| additionalImages: | ||
| - name: registry.redhat.io/ubi8/ubi:latest | ||
| - name: registry.redhat.io/ubi9/ubi@sha256:20f695d2a91352d4eaa25107535126727b5945bff38ed36a3e59590f495046f0 | ||
| EOF | ||
|
|
||
| - name: Get registry.redhat.io username and password from pull secret | ||
| ansible.builtin.shell: | | ||
| set -eux | ||
| oc get -n openshift-config secret pull-secret -o json | jq '.data[".dockerconfigjson"]' -r | base64 -d | jq '.auths["registry.redhat.io"].auth' -r | base64 -d | ||
| register: pull_secret_user_pass | ||
|
|
||
| - name: Login to registry.redhat.io | ||
| containers.podman.podman_login: | ||
| username: "{{ pull_secret_user_pass.stdout.split(':')[0] }}" | ||
| password: "{{ pull_secret_user_pass.stdout.split(':')[1] }}" | ||
| registry: "registry.redhat.io" | ||
|
|
||
| - name: Login to registry.stage.redhat.io | ||
| containers.podman.podman_login: | ||
| username: "{{ cifmw_registry_token.credentials.username }}" | ||
| password: "{{ cifmw_registry_token.credentials.password }}" | ||
| registry: "registry.stage.redhat.io" | ||
|
|
||
| - name: Mirror specified image set configuration to disk | ||
| ansible.builtin.shell: | | ||
| oc mirror --v2 --config {{ disconnect_working_dir }}/imageset-config-v2.yaml file://{{ mirror_location }} >>{{ disconnect_working_dir }}/mirror_images.log | ||
| register: mirror_image_result | ||
| until: mirror_image_result is not failed | ||
| retries: 1 | ||
|
|
||
| - name: Download mirror registry to controller | ||
| ansible.builtin.get_url: | ||
| url: "{{ mirror_registry_url }}" | ||
| dest: "{{disconnect_working_dir}}/mirror-registry-amd64.tar.gz" | ||
| mode: '0644' | ||
|
|
||
| - name: Install mirror registry | ||
| ansible.builtin.shell: | | ||
| set -eux | ||
| tar xvf {{disconnect_working_dir}}/mirror-registry-amd64.tar.gz -C {{disconnect_working_dir}} | ||
| {{disconnect_working_dir}}/mirror-registry install --quayHostname {{ host_fqdn.stdout }} --quayRoot \ | ||
| {{ local_registry }} --initPassword {{ mirror_registry_password }} >{{disconnect_working_dir}}/registry_install.log | ||
|
|
||
| - name: Increase gunicorn-web timeout in quay-app container | ||
| ansible.builtin.shell: | ||
| podman exec -it quay-app {% raw %}sed -i '/command=gunicorn -c %(ENV_QUAYCONF)s\/gunicorn_web.py web:application/c\command=gunicorn --timeout 900 -c %(ENV_QUAYCONF)s\/gunicorn_web.py web:application' /quay-registry/conf/supervisord.conf{% endraw %} | ||
|
|
||
| - name: Restart gunicorn-web quay-app container with new timeout value | ||
| ansible.builtin.shell: | ||
| set -eux | ||
| podman exec -it quay-app supervisorctl -c /quay-registry/conf/supervisord.conf help reread | ||
| podman exec -it quay-app supervisorctl -c /quay-registry/conf/supervisord.conf restart gunicorn-web | ||
|
|
||
| - name: Configure system to trust mirror registry root ca | ||
| become: true | ||
| ansible.builtin.shell: | | ||
| set -eux | ||
| cp {{ local_registry }}/quay-rootCA/rootCA.pem /etc/pki/ca-trust/source/anchors/ | ||
| update-ca-trust extract | ||
|
|
||
| - name: Login to mirror registry | ||
| ansible.builtin.shell: | | ||
| podman login -u init -p {{ mirror_registry_password }} {{ host_fqdn.stdout }}:8443 | ||
|
|
||
| - name: Configure cluster to trust mirror registry root ca | ||
| ansible.builtin.shell: | | ||
| set -eux | ||
| oc create configmap registry-cas -n openshift-config --from-file={{ host_fqdn.stdout }}..8443={{ local_registry }}/quay-rootCA/rootCA.pem | ||
| oc patch image.config.openshift.io/cluster --patch '{"spec":{"additionalTrustedCA":{"name":"registry-cas"}}}' --type=merge | ||
|
|
||
| - name: Get cluster's current pull secret | ||
| ansible.builtin.shell: | | ||
| oc get secret {% raw %}pull-secret -n openshift-config -o template='{{index .data ".dockerconfigjson" | base64decode}}'{% endraw %} > {{ disconnect_working_dir }}/pull-secret.json | ||
|
|
||
| - name: Configure cluster to use pull secret from mirror registry | ||
| ansible.builtin.shell: | | ||
| set -eux | ||
| oc registry login --registry {{ host_fqdn.stdout }}:8443 --auth-basic=init:{{ mirror_registry_password }} --to={{ disconnect_working_dir }}/pull-secret.json | ||
| oc set data secret/pull-secret -n openshift-config --from-file=.dockerconfigjson={{ disconnect_working_dir }}/pull-secret.json | ||
|
|
||
| - name: Mirror contents of generated image set to target mirror registry | ||
| ansible.builtin.shell: | | ||
| oc mirror --v2 --config {{ disconnect_working_dir }}/imageset-config-v2.yaml --from file://{{ mirror_location }} docker://{{ host_fqdn.stdout }}:8443 >>{{ disconnect_working_dir }}/mirror_contents.log | ||
| register: mirror_contents_result | ||
| until: mirror_contents_result is not failed | ||
| retries: 1 | ||
|
|
||
| - name: Disable catalog source | ||
| ansible.builtin.shell: | | ||
| oc patch OperatorHub cluster --type json -p '[{"op": "add", "path": "/spec/disableAllDefaultSources", "value": true}]' | ||
|
|
||
| - name: Extract image name and tag from catalog source | ||
| ansible.builtin.set_fact: | ||
| index_image_name_tag: "{{ oc_mirror_cert_manager_catalog_url.split('/') | last | replace(':', '-') | replace('.', '-') }}" | ||
|
|
||
| - name: Prepare catalog source for environment | ||
| ansible.builtin.shell: | | ||
| sed -i 's/cs-{{ index_image_name_tag | quote }}/redhat-operators/g' {{ mirror_location }}/working-dir/cluster-resources/cs-{{ index_image_name_tag }}.yaml | ||
|
|
||
| - name: Apply yaml files from results directory to cluster | ||
| ansible.builtin.shell: | | ||
| oc apply -f {{ mirror_location }}/working-dir/cluster-resources | ||
|
|
||
| - name: Wait for mirrored operators to be available | ||
| ansible.builtin.shell: | | ||
| oc get packagemanifests.packages.operators.coreos.com | ||
| register: packagemanifest_out | ||
| until: "'openstack-operator' and 'kubernetes-nmstate-operator' in packagemanifest_out.stdout" | ||
| retries: 10 | ||
| delay: 30 | ||
|
|
||
| - name: Wait until the OpenShift cluster is stable | ||
| ansible.builtin.command: | ||
| oc adm wait-for-stable-cluster --minimum-stable-period=5s --timeout=30m | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you only want this to apply to cs-redhat-operator-index-v4-18.yaml
cifmw will create it's own CatalogSource called openstack-operator-index that will be configured to use
cifmw_ci_gen_kustomize_values_ooi_image, and then the mirror configuration will make sure the mirror gets used.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe this will only apply to: cs-redhat-operator-index-v4-18.yaml. The previous set fact statement uses variable: oc_mirror_cert_manager_catalog_url to extract: redhat-operator-index-v4-18. I could hardcode the file name, but think this gives a little more flexibility.