diff --git a/pip/pip_380.md b/pip/pip_380.md new file mode 100644 index 0000000000000..f03b255eb2416 --- /dev/null +++ b/pip/pip_380.md @@ -0,0 +1,80 @@ +# PIP-380: Support setting up specific namespaces to skipping the load-shedding + +# Background knowledge + +Some of the Pulsar protocols, for example the KoP will use a system topic to store the metadata, eg: the topic's offset, +when the bundle owner changes the topic's data will need to be full recovered to the broker, and when unloaded, +it needs to remove the broker local memory, if the topic's data were huge, it may cause slow loading. + +# Motivation + +Support setting up specific namespaces to skipping the load-shedding, for the system topics namespace, +or for benchmark-relevant namespaces we better be skipping the load shedding because for system topics, +if the ownership transfers to another broker, it may need to recover the data by replaying the messages, +it will cause message pub/sub temporary unavailable. + +# Goals + +## In Scope + +- Support setting up specific namespaces to skipping the load-shedding +- Use `RoundRobinBrokerSelector` to assign the skipped namespaces bundle. + +# High Level Design + +Add new configuration `loadBalancerSheddingExcludedNamespaces` to the `ServiceConfiguration`. +While doing the load-shedding, filter out the configuration namespaces. + +For shedding excluded namespaces, use `RoundRobinBrokerSelector` to assign the ownership, it can make the assignment +more average because these will not automatically rebalance to another broker unless manually unloaded it. + +# Detailed Design + +## Design & Implementation Details + +## Public-facing Changes + +### Configuration + +Add new configuration `loadBalancerSheddingExcludedNamespaces` to the `ServiceConfiguration`. + +```java + @FieldContext( + dynamic = true, + category = CATEGORY_LOAD_BALANCER, + doc = "The namespaces skip for load shedding" + ) + private Set loadBalancerSheddingExcludedNamespaces = new TreeSet<>(); +``` + +# Monitoring + +No new metrics are added in this proposal. + +# Security Considerations + +No new security considerations are added in this proposal. + +# Backward & Forward Compatibility + +## Revert + +No changes are needed to revert to the previous version. + +## Upgrade + +No other changes are needed to upgrade to the new version. + +# Alternatives + +None + +# General Notes + +# Links + + +* Mailing List discussion thread: https://lists.apache.org/thread/316nmco393tzv30gwqdz53rwpj7mzfx8 +* Mailing List voting thread: https://lists.apache.org/thread/yd645km49z5x6t2c0215moncymb1r19n