[Staking update] Paged staking reward to avoid validator oversubscription issue is live on Westend

1

Posting this here for wider visibility.

Problem

If you are staking as a nominator on Polkadot, you might have seen a value shown as the minimum DOT you need to stake to earn rewards (~502 DOT currently). If you have staked more than this value, you might assume that you will earn rewards, but this is not the full truth. It might turn out that the validator you are exposed to on a given day (era in relay chain terms) is oversubscribed and any nominator outside of the top 512 nominators by stake would not receive any rewards. You would be understandably pissed off if it happened to you.

The reason for this limitation is, any work chain does in a block needs to be a bounded value and we bound how many payouts we can do while paying out staking rewards to 512 per validator. Any nominator outside this bound is ignored.

Solution

We can payout unbounded nominators in bounded chunks (called page) of 512 nominators. For example, if a validator has 1000 nominators, it will have two pages of rewards to payout, first page with 512 nominators and the second page with 488 nominators.

This is what this merged PR does which is now live on Westend and if all goes well, will be released on Kusama and Polkadot in the coming months. This change would also help with our ongoing effort to pagify various parts of validator election process and get the staking system ready for being moved to its own system parachain.

What you need to do

Nominator

Hopefully not much but you will not have to worry about oversubscription issue once these changes are live. As long as you have more DOTs staked than the minimum needed, you should be good. Otherwise, you can always stake via Nomination Pool.

Validator

If you have more than 512 pages, you will need to payout multiple pages. This can be done simply by calling the permissionless extrinsic payout_stakers in the Staking module multiple times (once for each page). There is also a new extrinsic payout_stakers_by_page if for some reason you want to invoke paying out only a specific page (might be useful for nominators if they want to invoke the reward payout and only care about the page they belong in).

The number of pages for an era can be queried by looking at the new storage item ErasStakersOverview. If you use bots to claim payouts, these changes would need to be integrated in the bot expecially if your bot needs to read era exposure which is now in a new storage. These changes are only partially backward compatible as in, the extrinsic payout_stakers if called once, would continue to payout page 0. But if you have more than 1 page of nominators, then you might miss out on paying your other nominators.

Dapp builders

The old storages for exposure has now been deprecated and stored in new storage keys in a paged manner. We took the decision to lazy migrate the exposure since we save 84 eras of historical exposure and it would not be possible to migrate them in one block.

What this means is, the old storage items will keep all the values, but once paged rewards changes are applied to runtime, any future era exposures will be written in the new storage items.

Following are the storage items that are deprecated along with their corresponding new storage item:

  • ErasStakers and ErasStakersClipped will be deprecated. Two new storage item are introduced for storing exposures by era and page, (1) ErasStakersOverview which contains metadata about all the pages (such as total stake and own stake of the validator), and (2) ErasStakersPaged which is very similar to ErasStakers but is now a triple key map of era, validator stash and page index.
  • Data for eras for which rewards are already claimed was earlier stored as one of the fields in storage Ledger, which is deprecated and a new storage map is introduced ClaimedRewards.

You can find more details in the description of the original PR as well as refer to the changes in the Staking Dashboard PR to integrate with these changes. I am happy to answer any questions or concerns you may have.

5 Likes
2

To expand on what @ankan explained, this is quite a large staking pallet update, and DApps will need to migrate to the new storage items and calls.

I’d like to document some implementation details that were picked up when supporting paged rewards on staking dashboard (the full PR can be referred to here: Add paged rewards support by rossbulat · Pull Request #1678 · paritytech/polkadot-staking-dashboard · GitHub)

maxNominatorRewardedPerValidator is now maxExposurePageSize

The constant maxNominatorRewardedPerValidator no longer exists, and is replaced by maxExposurePageSize. Make sure you query consts.staking.maxExposurePageSize - and this should just warrant variable name changes in your code.

Hard-coded paged rewards networks & start era

I opted to hardcode which networks have paged rewards active, and which era paged rewards started. This looks like this in code:

// DEPRECATION: Paged Rewards
//
// Temporary until paged rewards migration has completed on all networks.
export const NetworksWithPagedRewards: NetworkName[] = ['westend'];
export const PagedRewardsStartEra: Record<NetworkName, number | null> = {
  polkadot: null,
  kusama: null,
  westend: 7167,
};

You can then create a function that determines if paged rewards is active for a particular era:

// Given an era, determine whether paged rewards are active.
const isPagedRewardsActive = (era: BigNumber): boolean => {
    const networkStartEra = PagedRewardsStartEra[network];
    if (!networkStartEra) return false;

    return (
      NetworksWithPagedRewards.includes(network) &&
      era.isGreaterThanOrEqualTo(networkStartEra)
    );
 }

If paged rewards are active, make sure to query eras stakers data from erasStakersOverview and erasStakersPaged, from the era paged rewards started.

If the current era is less than the paged rewards start era, or the network does not yet support paged rewards, query from the deprecated erasStakers storage item.

ErasStakers Changes: total and value in Overview, others in Paged

DApps can still adhere to the previous erasStakers structure whereby the total, value and others properties were fetched altogether. The difference now is that total and value are fetched from staking.erasStakersOverview, and others is fetched from staking.erasStakersPaged.

The only way to determine a nominator’s page is by iterating erasStakersPaged.

There is currently no shortcut for DApps to determine a nominator’s page in storage. I would suggest spinning up a service worker in your DApp that iterates erasStakersPaged others until the staker is found, and then return the page number.

Keep track of claimed paged in ClaimedRewards storage item

A new CliamedRewards storage item keeps track of what pages have been claimed for an era, mapping era => number[].

Use the new payout_stakers_paged call

To claim rewards for a particular page, a new payout_stakers_paged call can be used. supply the page number to payout in this call. payout_stakers still exists, but will pay out the earliest unclaimed page.

Happy to answer any questions relating to DApps supporting this new feature if I missed anything.

3 Likes