Toleration support in Controller helm chart and Site resource for router pod

[jlo-norlysenergytrading]

Summary
In clusters where nodes are tainted, Skupper’s router pods created from a Site resource remain in Pending because the generated Deployment does not include tolerations. Similarly, the Helm chart for the controller does not expose tolerations. This makes dynamic deployment via CI/CD difficult in environments where taints are standard.
Proposal

Add tolerations to:

The Helm chart for the controller.
The Site CR spec so the controller can apply them to the router Deployment.

Why
Without this, Skupper cannot run on clusters where all nodes have NoSchedule taints without manual patching or admission webhooks.
Example

YAMLspec:  
  tolerations:
      - key: "dedicated"
         operator: "Equal"     
         value: "networking"
         effect: "NoSchedule"

Questions for maintainers

Is this something you’d like to support in Skupper?
Would you prefer a minimal tolerations field or a full podTemplate approach for future flexibility?

Contribution offer
This would be my first contribution to any open source project. I’m happy to give this a go if you think it’s a good fit and can point me to any development guidelines

[c-kruse]

Hi @jlo-norlysenergytrading. Thank you for the issue and for the interest in contributing!

Sounds like you could already be ahead of me here, but I'll state that there is a workaround for users finding this Issue. You should be able to patch the deployments after the fact: we intentionally used server-side apply in the skupper controller so that we would not conflict with external controllers and manually interventions like this (at least for the skupper router Deployments.)

I'll split my thoughts into two areas here: the controller deployment (helm/manifest yaml/kustomize) and the skupper-router deployment (Site CR).

Controller Deployment

I agree that this would be a nice thing to have. In an ideal world we'd support a Helm chart that exposes some of the deployment configurations like this along with several of our controller-specific configurations that are mostly left up to the user to discover and configure today.

You'll probably notice looking at our skupper chart that we have not fully embraced Helm. This mainly comes down to a lack of resources on our end - we wanted to avoid dual maintenance between deployment resource templates (helm) and a kustomize-style layout we use for generating the static manifest yamls and the OLM operator.

For now, I'd welcome one (or both) of these approaches:

• A new community maintained Helm chart outside the skupper repository to evaluate interest.
• Better documentation on installing our manifests with custom overlays using Kustomize.

Router Deployment (Site CR)

This is something we are broadly struggling with, and are open to proposals.

We are hesitant to explore extending the Site CRD with deployment configurations like tolerations, and have taken indirect approaches in the past with varying success (ex #2268). Our primary concern is that Skupper's API already has a steep learning curve, and that adding every reasonable configuration accommodation to the API will bury the core resource configurations in dozens of operational concerns.

[eryalito]

Hey!

I have encountered a similar scenario, when trying to deploy skupper using the helm chart on a disconnected environment, where imagePullSecrets is not configurable via values on the helm chart. Also the CR for sites doesn't allow adding imagePullSecrets, I worked around it by pre-creating a SA and attaching the pull secret to it.

Regarding the controller I found no way to actually perform this action. As I noticed what @c-kruse says about not fully embracing helm I decided to create a custom chart based on the manifest of the official one but with some parameters, if that might help you: https://github.com/eryalito/skupper-chart

[jlo-norlysenergytrading]

Hi everyone,

Thanks a lot for the detailed responses — and sorry for my slow reply.
We ended up taking a different architectural approach, which means we no longer need Skupper in our setup. Up until that decision was made, I had actually forked the project and added the toleration/metadata support I needed.
I completely understand and respect the idea of keeping the Skupper CRD clean, especially given the learning curve. At the same time, I still believe that exposing common Kubernetes features — such as labels, tolerations, and other basic metadata fields — would be valuable for many users.
Unfortunately, since our company ultimately chose another direction, I no longer have the dedicated time to contribute the changes properly. I just wanted to close the loop and explain the context behind the request.
Thanks again for the thoughtful feedback and the conversation — it’s much appreciated.

I’m not entirely sure whether this issue should be closed or kept open — on one hand, my own need for the feature has gone away, but on the other hand, others might still run into the same roadblock. I’m happy to leave it up to the maintainers to decide what’s most helpful for the community.