fix: enqueue a single request for all resources

[dboslee]

What this PR does / why we need it:

The current behavior of the controller is to run Reconcile for each resource that is updated. This gets quite expensive when you have 100s-1000s of resources.

This change allows for multiple resource updates to result in a single run of Reconcile.

This greatly reduces the cpu/memory consumption at startup and during periodic re-syncs of all resources.

Which issue(s) this PR fixes:

Fixes #1797

[arkodg]

can we also do this for L1318 ?

[arkodg]

can we add a comment / doc string mentioning that all watched resources trigger this single Reconcile . Also can we discard _ reconcile.Request with a comment mentioning that the requests content dont map to a specific resource.

[codecov]

Codecov Report

Merging #1813 (aaa25af) into main (4285f4e) will increase coverage by 0.10%.
The diff coverage is 90.32%.

@@            Coverage Diff             @@
##             main    #1813      +/-   ##
==========================================
+ Coverage   65.11%   65.21%   +0.10%     
==========================================
  Files          86       86              
  Lines       12316    12318       +2     
==========================================
+ Hits         8019     8033      +14     
+ Misses       3782     3774       -8     
+ Partials      515      511       -4     

Files Changed	Coverage Δ
internal/provider/kubernetes/controller.go	48.84% <90.32%> (+1.20%)	⬆️

... and 2 files with indirect coverage changes

[arkodg]

this is a neat idea, thanks for implementing this @dboslee, trying to understand this better, by using EnqueueRequestsFromMapFunc we've managed to create a specific Reconcile request with the same name for all Watch cases, which get coalesced / rate limited by the controller (since the name is same ) so we end up getting 1 reconcile request within a specific duration ?

[dboslee]

this is a neat idea, thanks for implementing this @dboslee, trying to understand this better, by using EnqueueRequestsFromMapFunc we've managed to create a specific Reconcile request with the same name for all Watch cases, which get coalesced / rate limited by the controller (since the name is same ) so we end up getting 1 reconcile request within a specific duration ?

I think you've got it more or less. There isn't a specific duration here other than the rate limiting provided by the controller. Here are a few examples describing the behavior.

At startup a single call to reconcile is executed since the controller will wait for all caches to initialized and events are coalesced.

In general, during a re-sync of many resources, any events that are processed before Reconcile is called are coalesced. If additional resource events arrive while the call to Reconcile is still running they will be coalesced in the next call to Reconcile.

The time in between calls to Reconcile is enforced by the rate limiters but its pretty quick by default if not immediate when events occur.

[arkodg]

LGTM, thanks for introducing this design pattern to improve performance at scale !

[arkodg]

this is a neat idea, thanks for implementing this @dboslee, trying to understand this better, by using EnqueueRequestsFromMapFunc we've managed to create a specific Reconcile request with the same name for all Watch cases, which get coalesced / rate limited by the controller (since the name is same ) so we end up getting 1 reconcile request within a specific duration ?

I think you've got it more or less. There isn't a specific duration here other than the rate limiting provided by the controller. Here are a few examples describing the behavior.

At startup a single call to reconcile is executed since the controller will wait for all caches to initialized and events are coalesced.

In general, during a re-sync of many resources, any events that are processed before Reconcile is called are coalesced. If additional resource events arrive while the call to Reconcile is still running they will be coalesced in the next call to Reconcile.

The time in between calls to Reconcile is enforced by the rate limiters but its pretty quick by default if not immediate when events occur.

ah makes sense, thanks for the explanation !

[zirain]

Thanks!