This repository was archived by the owner on Oct 24, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 513
This repository was archived by the owner on Oct 24, 2023. It is now read-only.
KMS encryption produces zero byte azure.json #49
Copy link
Copy link
Closed
Azure/acs-engine
#4279Labels
bugSomething isn't workingSomething isn't working
Description
On a master immediately after provisioning:
$ sudo systemctl status kubelet
● kubelet.service - Kubelet
Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Active: failed (Result: start-limit-hit) since Wed 2018-11-21 00:43:09 UTC; 11min ago
Main PID: 9924 (code=exited, status=255)
Nov 21 00:43:09 k8s-master-16337524-0 kubelet[9924]: I1121 00:43:08.985421 9924 mount_linux.go:211] Detected OS with systemd
Nov 21 00:43:09 k8s-master-16337524-0 kubelet[9924]: I1121 00:43:08.991898 9924 server.go:376] Version: v1.10.9
Nov 21 00:43:09 k8s-master-16337524-0 kubelet[9924]: I1121 00:43:08.991942 9924 feature_gate.go:226] feature gates: &{{} map[PodPriority:true]}
Nov 21 00:43:09 k8s-master-16337524-0 kubelet[9924]: F1121 00:43:08.992081 9924 server.go:233] failed to run Kubelet: could not init cloud provider "azure": No credentials provided for AAD ap
Nov 21 00:43:09 k8s-master-16337524-0 systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Nov 21 00:43:09 k8s-master-16337524-0 systemd[1]: Stopped Kubelet.
Nov 21 00:43:09 k8s-master-16337524-0 systemd[1]: kubelet.service: Start request repeated too quickly.
Nov 21 00:43:09 k8s-master-16337524-0 systemd[1]: Failed to start Kubelet.
Nov 21 00:43:09 k8s-master-16337524-0 systemd[1]: kubelet.service: Unit entered failed state.
Nov 21 00:43:09 k8s-master-16337524-0 systemd[1]: kubelet.service: Failed with result 'start-limit-hit'.
$ ls -la /etc/kubernetes/azure.json
-r-------- 1 root root 0 Nov 21 00:42 /etc/kubernetes/azure.json
$ cat /etc/kubernetes/encryption-config.yaml
kind: EncryptionConfig
apiVersion: v1
resources:
- resources:
- secrets
providers:
- kms:
name: azurekmsprovider
endpoint: unix:///opt/azurekms.socket
cachesize: 0
- identity: {}
$ cat /etc/systemd/system/kubelet.service
[Unit]
Description=Kubelet
ConditionPathExists=/usr/local/bin/kubelet
Requires=kms.service
[Service]
Restart=always
EnvironmentFile=/etc/default/kubelet
SuccessExitStatus=143
ExecStartPre=/bin/bash /opt/azure/containers/kubelet.sh
ExecStartPre=/bin/mkdir -p /var/lib/kubelet
ExecStartPre=/bin/mkdir -p /var/lib/cni
ExecStartPre=/bin/bash -c "if [ $(mount | grep \"/var/lib/kubelet\" | wc -l) -le 0 ] ; then /bin/mount --bind /var/lib/kubelet /var/lib/kubelet ; fi"
ExecStartPre=/bin/mount --make-shared /var/lib/kubelet
# This is a partial workaround to this upstream Kubernetes issue:
# https://github.com/kubernetes/kubernetes/issues/41916#issuecomment-312428731
ExecStartPre=/sbin/sysctl -w net.ipv4.tcp_retries2=8
ExecStartPre=-/sbin/ebtables -t nat --list
ExecStartPre=-/sbin/iptables -t nat --list
ExecStart=/usr/local/bin/kubelet \
--enable-server \
--node-labels="${KUBELET_NODE_LABELS}" \
--v=2 \
--volume-plugin-dir=/etc/kubernetes/volumeplugins \
$KUBELET_CONFIG $KUBELET_OPTS \
$KUBELET_REGISTER_NODE $KUBELET_REGISTER_WITH_TAINTS
[Install]
WantedBy=multi-user.target
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working