Azure CCM assets #62
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| kind: Deployment | ||
| apiVersion: apps/v1 | ||
| metadata: | ||
| name: azure-cloud-controller-manager | ||
| namespace: openshift-cloud-controller-manager | ||
| labels: | ||
| app: azure-cloud-controller-manager | ||
| spec: | ||
| selector: | ||
| matchLabels: | ||
| app: azure-cloud-controller-manager | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: azure-cloud-controller-manager | ||
| spec: | ||
| hostNetwork: true | ||
| serviceAccountName: cloud-controller-manager | ||
| priorityClassName: system-cluster-critical | ||
| nodeSelector: | ||
| node-role.kubernetes.io/master: "" | ||
| affinity: | ||
| podAntiAffinity: | ||
| requiredDuringSchedulingIgnoredDuringExecution: | ||
| - topologyKey: "kubernetes.io/hostname" | ||
| labelSelector: | ||
| matchLabels: | ||
| app: azure-cloud-controller-manager | ||
| tolerations: | ||
|
Contributor
There was a problem hiding this comment. does this also need to tolerate seems like aws does, https://github.com/openshift/cluster-cloud-controller-manager-operator/blob/master/pkg/cloud/aws/assets/deployment.yaml#L55
Contributor
There was a problem hiding this comment. None of our components should be tolerating that taint, if we do, we will schedule onto a node without networking, which will likely cause our pods to crash loop as they can't contact the APIs they need to contact
Contributor
There was a problem hiding this comment. It will now need to tolerate the not-ready based on the additions made in #76 |
||
| - effect: NoSchedule | ||
| key: node-role.kubernetes.io/master | ||
| operator: Exists | ||
| - effect: NoExecute | ||
| key: node.kubernetes.io/unreachable | ||
| operator: Exists | ||
| tolerationSeconds: 120 | ||
| - effect: NoExecute | ||
| key: node.kubernetes.io/not-ready | ||
| operator: Exists | ||
| tolerationSeconds: 120 | ||
| - effect: NoSchedule | ||
| key: node.cloudprovider.kubernetes.io/uninitialized | ||
| operator: Exists | ||
| - effect: NoSchedule | ||
| key: node.kubernetes.io/not-ready | ||
| operator: Exists | ||
| containers: | ||
| - name: azure-cloud-controller-manager | ||
| image: quay.io/openshift/origin-azure-cloud-controller-manager:4.8.0 | ||
| imagePullPolicy: "IfNotPresent" | ||
| env: | ||
| - name: CLOUD_CONFIG | ||
| value: /etc/kubernetes-cloud-config/cloud.conf | ||
| - name: OCP_INFRASTRUCTURE_NAME | ||
| value: kubernetes # default cluster name in ccm | ||
| resources: | ||
| requests: | ||
| cpu: 200m | ||
| memory: 50Mi | ||
| command: | ||
| - /bin/bash | ||
| - -c | ||
| - | | ||
| #!/bin/bash | ||
| set -o allexport | ||
| if [[ -f /etc/kubernetes/apiserver-url.env ]]; then | ||
| source /etc/kubernetes/apiserver-url.env | ||
| fi | ||
| exec /bin/azure-cloud-controller-manager \ | ||
| --v=3 \ | ||
| --cloud-config=$(CLOUD_CONFIG) \ | ||
| --cloud-provider=azure \ | ||
| --controllers=*,-cloud-node,-route \ | ||
| --allocate-node-cidrs=false \ | ||
| --configure-cloud-routes=false \ | ||
| --use-service-account-credentials=true \ | ||
| --bind-address=127.0.0.1 \ | ||
| --cluster-name=$(OCP_INFRASTRUCTURE_NAME) \ | ||
| --leader-elect-resource-namespace=openshift-cloud-controller-manager | ||
| volumeMounts: | ||
| - name: host-etc-kube | ||
| mountPath: /etc/kubernetes | ||
| readOnly: true | ||
| - name: config-accm | ||
| mountPath: /etc/kubernetes-cloud-config | ||
| readOnly: true | ||
| volumes: | ||
| - name: config-accm | ||
| configMap: | ||
| name: cloud-conf | ||
| items: | ||
| - key: cloud.conf | ||
| path: cloud.conf | ||
| - name: host-etc-kube | ||
| hostPath: | ||
| path: /etc/kubernetes | ||
| type: Directory | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| apiVersion: apps/v1 | ||
| kind: DaemonSet | ||
| metadata: | ||
| name: azure-cloud-node-manager | ||
| namespace: openshift-cloud-controller-manager | ||
| labels: | ||
| component: azure-cloud-node-manager | ||
| kubernetes.io/cluster-service: "true" | ||
| spec: | ||
| selector: | ||
| matchLabels: | ||
| app: azure-cloud-node-manager | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: azure-cloud-node-manager | ||
| annotations: | ||
| cluster-autoscaler.kubernetes.io/daemonset-pod: "true" | ||
| spec: | ||
| priorityClassName: system-node-critical | ||
| serviceAccountName: cloud-node-manager | ||
| hostNetwork: true | ||
| nodeSelector: | ||
| kubernetes.io/os: linux | ||
| tolerations: | ||
|
Contributor
There was a problem hiding this comment. same question here, i'm guessing since this uses the host network that it will need to tolerate
Contributor
There was a problem hiding this comment. Host network maybe means we can allow it to tolerate the taint, though IMO, we should only add the toleration if we are certain we need it. I'd suggest being conservative initially |
||
| - effect: NoSchedule | ||
| key: node-role.kubernetes.io/master | ||
| operator: Exists | ||
| - effect: NoExecute | ||
| key: node.kubernetes.io/unreachable | ||
| operator: Exists | ||
| tolerationSeconds: 120 | ||
| - effect: NoExecute | ||
| key: node.kubernetes.io/not-ready | ||
| operator: Exists | ||
| tolerationSeconds: 120 | ||
| - effect: NoSchedule | ||
| key: node.cloudprovider.kubernetes.io/uninitialized | ||
| operator: Exists | ||
| - effect: NoSchedule | ||
| key: node.kubernetes.io/not-ready | ||
| operator: Exists | ||
| containers: | ||
| - name: azure-cloud-node-manager | ||
| image: quay.io/openshift/origin-azure-cloud-node-manager:4.8.0 | ||
| imagePullPolicy: IfNotPresent | ||
| command: | ||
| - /bin/bash | ||
| - -c | ||
| - | | ||
| #!/bin/bash | ||
| set -o allexport | ||
| if [[ -f /etc/kubernetes/apiserver-url.env ]]; then | ||
| source /etc/kubernetes/apiserver-url.env | ||
| fi | ||
| exec /bin/azure-cloud-node-manager \ | ||
| --node-name=$(NODE_NAME) \ | ||
| --wait-routes=false | ||
| env: | ||
| - name: NODE_NAME | ||
| valueFrom: | ||
| fieldRef: | ||
| fieldPath: spec.nodeName | ||
| volumeMounts: | ||
| - name: host-etc-kube | ||
| mountPath: /etc/kubernetes | ||
| readOnly: true | ||
| resources: | ||
| requests: | ||
| cpu: 50m | ||
| memory: 50Mi | ||
| volumes: | ||
| - name: host-etc-kube | ||
| hostPath: | ||
| path: /etc/kubernetes | ||
| type: Directory | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,38 @@ | ||
| package azure | ||
|
|
||
| import ( | ||
| "embed" | ||
|
|
||
| "github.com/openshift/cluster-cloud-controller-manager-operator/pkg/cloud/common" | ||
| appsv1 "k8s.io/api/apps/v1" | ||
| utilruntime "k8s.io/apimachinery/pkg/util/runtime" | ||
| "sigs.k8s.io/controller-runtime/pkg/client" | ||
| ) | ||
|
|
||
| var ( | ||
| //go:embed assets/* | ||
| azureFs embed.FS | ||
|
|
||
| azureResources []client.Object | ||
|
|
||
| azureSources = []common.ObjectSource{ | ||
| {Object: &appsv1.DaemonSet{}, Path: "assets/cloud-node-manager-daemonset.yaml"}, | ||
| {Object: &appsv1.Deployment{}, Path: "assets/cloud-controller-manager-deployment.yaml"}, | ||
| } | ||
| ) | ||
|
|
||
| func init() { | ||
| var err error | ||
| azureResources, err = common.ReadResources(azureFs, azureSources) | ||
| utilruntime.Must(err) | ||
| } | ||
|
|
||
| // GetResources returns a list of AWS resources for provisioning CCM in running cluster | ||
| func GetResources() []client.Object { | ||
| resources := make([]client.Object, len(azureResources)) | ||
| for i := range azureResources { | ||
| resources[i] = azureResources[i].DeepCopyObject().(client.Object) | ||
| } | ||
|
|
||
| return resources | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,24 @@ | ||
| package azure | ||
|
|
||
| import ( | ||
| "testing" | ||
|
|
||
| "github.com/stretchr/testify/assert" | ||
| ) | ||
|
|
||
| func TestGetResources(t *testing.T) { | ||
| resources := GetResources() | ||
| assert.Len(t, resources, 2) | ||
|
|
||
| var names, kinds []string | ||
| for _, r := range resources { | ||
| names = append(names, r.GetName()) | ||
| kinds = append(kinds, r.GetObjectKind().GroupVersionKind().Kind) | ||
| } | ||
|
|
||
| assert.Contains(t, names, "azure-cloud-controller-manager") | ||
| assert.Contains(t, kinds, "Deployment") | ||
|
|
||
| assert.Contains(t, names, "azure-cloud-node-manager") | ||
| assert.Contains(t, kinds, "DaemonSet") | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suppose hostNetwork is not required for other controllers except
cloud-node