[feature][autoscaling] add Spot scheduling feature and update RBAC by AlexanderYastrebov · Pull Request #2957 · DataDog/datadog-operator

AlexanderYastrebov · 2026-04-29T10:02:54Z

What does this PR do?

Adds features.autoscaling.cluster.spot.enabled to the DatadogAgent API, which grants the cluster-agent permission to patch Deployments/StatefulSets with the spot-disabled-until annotation and evict pending spot pods during on-demand fallback. Spot is enforced as a sub-feature of cluster autoscaling (requires cluster.enabled=true).

Motivation

Support spot instance scheduling in the operator so that users can enable the spot scheduler via the DatadogAgent CR without manually managing RBAC.

Additional Notes

Spot scheduler implementation: DataDog/datadog-agent#47429

Updates https://datadoghq.atlassian.net/browse/CASCL-1312

Minimum Agent Versions

Cluster Agent: v7.79.0

Describe your test plan

Unit tests cover the new RBAC rules for spot scheduling. E2E tests in DataDog/datadog-agent validate spot scheduling behaviour end-to-end.

Checklist

PR has at least one valid label: bug, enhancement, refactoring, documentation, tooling, and/or dependencies
PR has a milestone or the qa/skip-qa label
All commits are signed (see: signing commits)

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b87a451c73

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-04-29T10:05:17Z

 	if clusterEnabled {
 		pr = append(pr, []rbacv1.PolicyRule{
+			{
+				// Patch workloads to write spot-disabled-until annotation during on-demand fallback
+				APIGroups: []string{rbac.AppsAPIGroup},


Scope spot RBAC rule behind spot feature flag

This rule is added whenever clusterEnabled is true, so users who enable only features.autoscaling.cluster.enabled now get extra apps patch access even though spot scheduling is documented as a separate sub-feature. That broadens cluster-agent privileges for non-spot deployments and breaks the intended least-privilege gating of cluster.spot.enabled; moving this rule to the clusterSpotEnabled block would keep permissions aligned with the enabled feature set.

Useful? React with 👍 / 👎.

codecov-commenter · 2026-04-29T10:24:42Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 41.40%. Comparing base (099f33f) to head (58dacda).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2957      +/-   ##
==========================================
+ Coverage   41.38%   41.40%   +0.02%     
==========================================
  Files         327      327              
  Lines       28952    28962      +10     
==========================================
+ Hits        11982    11992      +10     
  Misses      16109    16109              
  Partials      861      861

Flag	Coverage Δ
unittests	`41.40% <100.00%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...roller/datadogagent/feature/autoscaling/feature.go	`85.54% <100.00%> (+1.33%)`	⬆️
...ontroller/datadogagent/feature/autoscaling/rbac.go	`100.00% <100.00%> (ø)`

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 099f33f...58dacda. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

AlexanderYastrebov · 2026-04-29T11:44:45Z

 	Enabled *bool `json:"enabled,omitempty"`
+
+	// Spot contains the configuration for the spot instance scheduling sub-feature.
+	// Requires cluster autoscaling to be enabled.


Do we want this condition? In the agent spot scheduling does not depend on cluster scaling although its a nested feature.

What does this PR do? --------------------- Adds `features.autoscaling.cluster.spot.enabled` to the DatadogAgent API, which grants the cluster-agent permission to patch Deployments/StatefulSets with the `spot-disabled-until` annotation and evict pending spot pods during on-demand fallback. Spot is enforced as a sub-feature of cluster autoscaling (requires `cluster.enabled=true`). Motivation ---------- Support spot instance scheduling in the operator so that users can enable the spot scheduler via the DatadogAgent CR without manually managing RBAC. Additional Notes ---------------- Spot scheduler implementation: DataDog/datadog-agent#47429 Updates https://datadoghq.atlassian.net/browse/CASCL-1312 Minimum Agent Versions ---------------------- * Cluster Agent: v7.79.0 Describe your test plan ----------------------- Unit tests cover the new RBAC rules for spot scheduling. E2E tests in DataDog/datadog-agent validate spot scheduling behaviour end-to-end. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

AlexanderYastrebov · 2026-04-30T09:49:21Z

 	}

-	if clusterEnabled {
+	if f.workloadEnabled || f.clusterSpotEnabled {


We can split this condition and have duplicate rules as k8s merges them.

clamoriniere

:lgtm 👍

AlexanderYastrebov requested a review from a team April 29, 2026 10:02

AlexanderYastrebov requested review from a team as code owners April 29, 2026 10:02

github-actions Bot added team/container-platform team/container-autoscaling team/documentation labels Apr 29, 2026

AlexanderYastrebov added qa/skip-qa documentation Improvements or additions to documentation labels Apr 29, 2026

chatgpt-codex-connector Bot reviewed Apr 29, 2026

View reviewed changes

AlexanderYastrebov force-pushed the alexander.yastrebov/spot-statefulset-patch-rbac branch from b87a451 to 7852bcf Compare April 29, 2026 10:11

This comment has been minimized.

Sign in to view

AlexanderYastrebov marked this pull request as draft April 29, 2026 10:30

AlexanderYastrebov force-pushed the alexander.yastrebov/spot-statefulset-patch-rbac branch 3 times, most recently from 207ec23 to 17f0f7e Compare April 29, 2026 10:59

AlexanderYastrebov marked this pull request as ready for review April 29, 2026 11:01

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: waiting mergequeue-status: removed and removed mergequeue-status: waiting labels Apr 29, 2026

AlexanderYastrebov mentioned this pull request Apr 29, 2026

[autoscaling] add spot scheduling DataDog/helm-charts#2621

Draft

8 tasks

AlexanderYastrebov commented Apr 29, 2026

View reviewed changes

domalessi approved these changes Apr 29, 2026

View reviewed changes

AlexanderYastrebov force-pushed the alexander.yastrebov/spot-statefulset-patch-rbac branch from 17f0f7e to 58dacda Compare April 30, 2026 09:47

AlexanderYastrebov commented Apr 30, 2026

View reviewed changes

clamoriniere approved these changes Apr 30, 2026

View reviewed changes

Mathew-Estafanous approved these changes Apr 30, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feature][autoscaling] add Spot scheduling feature and update RBAC#2957

[feature][autoscaling] add Spot scheduling feature and update RBAC#2957
AlexanderYastrebov wants to merge 1 commit intomainfrom
alexander.yastrebov/spot-statefulset-patch-rbac

AlexanderYastrebov commented Apr 29, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Apr 29, 2026

Uh oh!

codecov-commenter commented Apr 29, 2026 •

edited

Loading

Uh oh!

This comment has been minimized.

AlexanderYastrebov Apr 29, 2026

Uh oh!

AlexanderYastrebov Apr 30, 2026

Uh oh!

clamoriniere left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

AlexanderYastrebov commented Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Additional Notes

Minimum Agent Versions

Describe your test plan

Checklist

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment has been minimized.

AlexanderYastrebov Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

AlexanderYastrebov Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

clamoriniere left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

AlexanderYastrebov commented Apr 29, 2026 •

edited

Loading

codecov-commenter commented Apr 29, 2026 •

edited

Loading