Unlimited number of nominators eligible for Rewards payout (rebased)

[Ank4n]

closes paritytech/polkadot-sdk#439 and paritytech/polkadot-sdk#473.
polkadot companion: paritytech/polkadot#6765.

Context

Rewards payout is processed today in a single block and limited to MaxNominatorRewardedPerValidator. This number is currently 512 on both Kusama and Polkadot.

This PR tries to scale the nominators payout to an unlimited count in a multi-block fashion. Exposures are stored in pages, with each page capped to a certain number. Practically, this number would be the same as MaxNominatorRewardedPerValidator or lower.

The PR also intends to be mostly backward compatible.

Note about the release

This PR introduces a new storage item MaxExposurePageCount which defaults to minimum value of 1 when not set. This means functionally, the pallet will continue to work exactly as before after this release. To enable paged rewards payout, MaxExposurePageCount can be increased to a bigger number by OpenGov for each runtime separately.

How payouts would look like after this change

Staking exposes two calls, 1) the existing payout_stakers and 2) payout_stakers_by_page.

payout_stakers

This remains backward compatible with no signature change. If for a given era a validator has multiple pages, they can call payout_stakers multiple times. The pages are executed in an ascending sequence and the runtime takes care of preventing double claims.

payout_stakers_by_page

Very similar to payout_stakers but also accepts an extra param page_index. An account can choose to payout rewards only for an explicitly passed page.

Lets look at an example scenario.
Given an active validator on Kusama had 1100 nominators, MaxExposurePageCount set to 10 and MaxExposurePageSize set to 512 for Era e. In order to pay out rewards to all nominators, the caller would need to call payout_stakers 3 times.

• payout_stakers(origin, stash, e) => will pay the first 512 nominators.
• payout_stakers(origin, stash, e) => will pay the second set of 512 nominators.
• payout_stakers(origin, stash, e) => will pay the last set of 76 nominators.
  ...
• payout_stakers(origin, stash, e) => calling it the 4th time would return an error InvalidPage.

The above calls can also be replaced by payout_stakers_by_page and passing a page explicitly.

Important to note

• Validator commission is paid out in chunks across all the pages where each commission chunk is proportional to the total stake of the current page. This encourages the validator to payout all the exposure pages.

Storage Changes

Added

• ErasStakersOverview
• ClaimedRewards
• ErasStakersPaged

Deprecated

The following can be cleaned up after 84 eras. Tracker issue for cleaning up.

• ErasStakers.
• ErasStakersClipped.
• StakingLedger.claimed_rewards, renamed to StakingLedger.legacy_claimed_rewards.

Config Changes

• Renamed MaxNominatorRewardedPerValidator to MaxExposurePageSize.
• New item MaxExposurePageCount

TODO

• Tracker ticket for cleaning up the old code after 84 eras.
• Add companion.
• Redo benchmarks in kusama and polkadot.
• Add Changelog for pallet_staking.
• Pallet should be configurable to enable/disable paged rewards.
• Commission payouts are distributed across pages.
• Review documentation thoroughly.
• Rename MaxNominatorRewardedPerValidator -> MaxExposurePageSize.
• NMap for ErasStakersPaged.
• Deprecate ErasStakers.
• Integrity tests

Follow up issues

• Staking runtime apis to help with paged reward payouts polkadot-sdk#426

[Ank4n]

bot rebase

[paritytech-processbot]

Branch is already up-to-date

[Ank4n]

Seems something weird happened while rebasing and the bot thinks the history of master and this branch is different. Squashing my commits and adding these changes on top of fresh master in a new PR #13498.