BroadcastDistributionRules to load segments on whole tiers rather than making server-by-server decisions

[leventov]

Broadcast rules which load segments on servers with colocated data sources (server-by-server decision) are very hard or impossible to implement correctly. I think the problems of non-atomic moves during balancing are underestimated.

When rules are loaded from the metadata store, Coordinator can figure out load rules for data sources and maps them to "colocated data sources", obtaining a list of tiers on which broadcasted segments should be effectively loaded. Then it loads broadcasted segments on all servers in the needed tiers, as described here: #4077 (comment)

FYI @jihoonson @gianm

[leventov]

It probably makes BroadcastDistributionRules's timing (interval, period, forever) irrelevant too. All timing can be figured out from the load rules applying to the colocated data sources.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.

[stale]

This issue is no longer marked as stale.