customize node config

[unacao]

Is there any way to customize the kubelet config such as KUBELET_HARD_EVICTION_THRESHOLD in the AKS cluster?

[slack]

Right now no, beyond EVICTION_THRESHOLD what would you like to configure?

[unacao]

possibly the configuration mentioned here #245

[sfrolich]

I have some rather large images in ACR that timeout pulling the images. Some info here:
https://blog.openai.com/scaling-kubernetes-to-2500-nodes/
#245

changing to KUBELET_OPTS=--image-pull-progress-deadline=10m
on all my agent nodes solved my problem.

Being able to edit these should be made easier as I'm sure as more and more customers use AKS, each customer's scenario will require changing kubelet options.

[kautsig]

This seems like a duplicate of #245

[doubleyewdee]

I'm actually curious about whether and how we could use a different option (--cpu-manager-policy) passed in to KUBELET_OPTS as part of our node provisioning. Is there a generic way to do this or is the plan to expose specific functionality through other AKS knobs and have those turn on the appropriate flags indirectly for users?

[jnoller]

@doubleyewdee It would be the latter - depending on the complexity of the option (some aren't well suited for AKS as a managed solution). AKS would expose the knobs and then handle application behind the scenes.

[AlverezYari]

+1 to need to set my eviction limits

[zerda]

How about Dynamic Kubelet Configuration ?

I'm using --authentication-token-webhook=true to enable service account tokens to be used to authenticate to the kubelet.

It's required by kube-prometheus to scrape the kubelet metrics.

[henrylilei]

@jnoller I understand that certain configs are not suitable for managed AKS, but is it possible to have a blacklist/whitelist on the dynamic configurations? So that we can at least setup the cluster and VMSS better suited to the latency sensitive production traffic.

Some of the latency and perf related settings for AKS that I wish to be able to set when I have Cluster AutoScaler enabled,

• cpu-manager-policy
• cpu-cfs-quota
• pod-max-pids

Thank you for your consideration.

[faustos]

I would need to set ConfigMapAndSecretChangeDetectionStrategy

[jayush]

What is the latest status of this ask?

[jeff1985]

Custom kubelet config would also allow to solve #686 which is failing an important deployment on my cluster