Economic Model for System Para Collators

There is afaik no one type of system parathread:

An NPoS parathread would be spawned by whatever chain holds the staking data, basically by cloning some of its state. All parablocks are a proposed nominator-validator assignment, so block validation checks the PJR property and that the block improves upon the previous block. Any polkadot validator could make blocks whenever they liked, assuming they posses the staking chain, maybe non-validators too within reason.

In this, we have validators make blocks because there is no state on this system parathread, just an immutable reference to the staking chain’s state, so anybody with this state can make blocks, an this includes all validators if the relay chain is the staking chain.

At the opposite extreme is moving the staking data off the relay chain. It’s a low throughput chain, with a sizable state, so validators cannot do this work for us.

Instead, we should do checkpoints that erasure code the entire state across the validators once or so per epoch, so that new nodes can sync the entire recent staking chain using only data stored by validators.

You’d still need some archive node if you want historical balances, but that’s outside the chain’s threat model.

We’d incentivize these extremes differently too of course.

A simple solution that we already have the infrastructure for is using approval stakes, so kind alike DPoS.

We deploy:

  • pallet staking.
  • one instance of pallet bags-list that keeps all collator-wanna-bees sorted based on approval stakes.

Winning validators are the top N items of this sorted list.

Do we want to reward delegators as well? That would be the tricky part. If we want to reuse the old existing pallet, we would need to fill all the Exposures, which will be tricky in a para-friendly way. If we don’t care about rewards, this is more or less solved.

That’s an interesting question. I would actually say no - the main goal is to ensure that someone* has the incentive to produce blocks. The motivation for nominators in staking is to provide security, and having more DOT at stake (i.e. liable for slashing) provides more security for the network.

In this role, having more DOT behind a collator set does not produce more security for the network. Also one entity running multiple nodes does not provide more security, as they would presumably censor the same transactions if they did engage in censorship. So the “small self-stake with multiple nominated nodes” strategy is not so relevant here.

As we add more system parachains, having nominators/delegators on each one seems like a complexity nightmare. Imagine unbonding on Statemint and going to nominate on Collectives, etc. That doesn’t make sense to me. I’d expect collators to have some stake and be competitive in getting elected and earning points to be rewarded, but beyond that I think nominators should be staking to secure the network.

*Well, for each transaction, at least one collator who would not censor it.


@burdges I generally agree with your post, at least from a research/conceptual point of view. But the fact is that right now there is a single type of system chain. We should factor your comments into the roadmap and long-term vision, especially when moving staking into a parachain. But right now we need a way to make sure that collators exist for Statemint, Collectives, and the Bridge Hub.

We’d ideally make sure that whatever we do with staking on system parathreads does not mess up moving NPoS onto a system parathread, which primarily means:

When the staking chain exports the tree for use in the election, who receives this tree?

Sure, but that seems like an implementation issue with moving staking into a parachain. How does that relate to this thread?

A couple small points first…

In general, parachains will work best when the collator-selection mechanism is plausibly unique, i.e. for a parachain block at any given position there are typically not forks. For that reason the ‘free-for-all’ approach is definitely not ideal.

This might be of relevance to Allow "Restaking" of relay-chain stake through custom slashing privileges .

More to the main point…

Materially, it doesn’t seem to me that there is much difference in computing this on the front-end of inflation or on the back-end of the treasury. Couldn’t we also consider alternative models where the treasury votes to send DOT to system parachains, to be distributed as rewards to the collators of those parachains?

The reason I bring this up is that, as you point out, parachain collators don’t provide any service except for liveness & censorship resistance. As such, I believe that the amount of rewards for system parachain collators should be relatively low - 20% of inflation seems quite high to me. Furthermore, since opportunity costs are less of a factor without a need for slashing, it seems to me that the rewards per system chain should be based on the real costs operational nodes for system chains undertake. This will vary heavily from one chain to the next - something like Statemint is extremely lightweight compared to e.g. a storage chain and I think we’d want to give the treasury the ability to budget accordingly.

1 Like

That’s a good point that I had not considered. I thought about using Treasury proposals as an interim solution for the several months it will take to design/implement/audit/release a new system, but perhaps it actually is the best solution.

There are two main points that I think need to be solved to make it sustainable, though:

  1. Autonomousness: At least today, Treasury proposals imply some manual action. Of course triggering staking payouts do as well, but they are lazier as all the actual calculation happens within the protocol. Using the Treasury either needs some autonomous function to schedule payouts or some group (it’d very much be in the collators’ interest) needs to provide some service to make the proposals on a regular basis.
  2. Costs vs. Contribution: Let’s say someone runs their node all the time, but is only elected as a collator 50% of the time. Would the proposal cover their full costs? Or what if their quality of service is low, e.g. they miss many of their slots? For this I still think we need some way to evaluate the contribution of a particular node and convert it to DOT terms. Basically we need some standard on what is being compensated from the Treasury.

This starts to feel very similar to Treasury proposals for teams providing public infrastructure (e.g. On Finality, Dwellr, etc.). The question here is should each team make proposals arguing their contribution vs. their costs (which could vary based on node hardware, team location, etc.) or do we have Treasury proposals that give some DOT per parachain, where some parachains could be deemed more resource-intensive than others. The former is more trust-based (need to trust that each provider is honest about their costs), while the latter could create incentives to run lower quality services in order to capture more profit.

related: A Better Treasury System

We definitely do need some way of evaluating validators’ performance so they can be paid accordingly. It seems like the system parachain itself is the best place to evaluate and reward that.

Couldn’t the relay-chain treasury send to the system chain and have the system chain handle the rest, determining cost vs. contribution? The top-level budget of the relay chain would then only have to focus on the overall cost of the system chain, whereas the specifics will be handled by the system chain’s own logic. We could even make the reward mechanism on the system chain adaptive to different sizes of funding tranches from the relay chain, so that an X% change in the budget automatically maps onto an X% change in rewards for participants of the system chain.

I guess a few questions / points of disucssion:

  • What is the baseline level of hardware / performance needed for system / common good chains? Is it expected that they also meet the cpu/mem/disk performance of Standard Hardware that relay chain validators require? The recommendations for this affect what the estimated cost of running these chains would be.
  • how big should the collator set be for a system / common good chain? Overall (since they don’t provide security), it’s not entirely necessary to have a large collator set. What I’m most interested in is having them be run by a majority that is not Parity, w3f, or a small amount of single entities, and that they are more decentralized / running on uncorrelated infrastructure (say not all on hetzner for instance). Parachains seem to work fine being run with just a few nodes, but ideally, if we want to have them be more antifragile, what might that look like?
2 Likes

A few of us (@kianenigma @burdges @giulia @bkontur & Roman) discussed this topic and how to go about implementing something on a call today and wanted to sum up the discussion/first steps we plan to take.

First, we generally agreed with @rphmeier ‘s point that different chains could have different requirements, and that some Treasury process that roughly covers collators’ expenses is better than using a fixed percentage of inflation. This plays nice with the idea that one parachain might require more funding than another, so the Treasury proposals can account for that.

Regarding the specific mechanisms by which a chain splits up the DOT (or, once the Treasury holds stablecoins, perhaps those) allocated to it, we briefly discussed it but didn’t make any implementation decisions. There are also a few other services that collators can provide, like public RPC endpoints for the parachain and/or the Relay Chain that could be factored in if the particular node isn’t elected all the time. However, I think a good starting point is a pallet that pays from a single pallet-controlled account to some set of accounts with various strategy impls (e.g. equally, pro-rata based on number of blocks authored, etc.).

Second, we discussed election mechanisms. We were trying to balance two main aspects: (1) we want to ensure a diverse set of entities collating, and (2) we want most of the stake to be on validators, so we expect collator bonds to be low.

The problem with (2) is that low bonds makes it easy for one relatively small entity to take over collation on a system chain. Our opinion was that we could balance them with some nodes selected by governance. This is quite similar to the Invulnerables system that exists today, except that rather than having those nodes run by Parity (as they were for Statemint/Statemine at genesis), there would be some conventions like having a verified identity, running a validator node, etc.

These governance selected collators would account for some percentage of the collator set (let’s say, arbitrarily, 30%), while the remainder of the set would be chosen by order of self-stake. This combination means that anyone can become a collator on a system chain by placing a bond higher than the nth highest existing collator on a chain, but ensures some diversity of entities (assuming governance has chosen wisely) so that no amount of stake can fully take over collation (and kill liveness) of a system chain.

Those are all things we can do in the short term via small modifications to the Collator Selection pallet and some Treasury proposals/precendent.

Longer term, an election parachain or staking solution can pass the nomination info to parachains, which can apply approval backing to select collators that correspond to more approval behind the address that is also a validator (i.e. we’d expect the same AccountId to be used on all chains). Jeff and Kian had more concrete ideas so probably better for one of them to respond to this.

I don’t think there is a single standard, but the invariant is that the Relay Chain validators must be able to validate whatever the collator passes them. So off the top of my head, I’d say validator benchmark specs are actually a ceiling of the max needed. That’s speaking very generally though, parachains might have higher storage or other special requirements on a per-para basis.

I also think it depends on the parachain. Some parathread that makes one block a day might be fine with a single primary collator with some fallback in case it goes down, while a storage or fast payments chain might want/need a lot to ensure the decentralization you mentioned.

2 Likes

Should we try to figure it out together?

3 Likes

Hey everyone, first of all, I appreciate that @joepetrowski has started this discussion since I also started to wonder about this recently.

First a bit of background, I am the founder of Polkadotters - half of our “business” is to run a media house that publishes news, tutorials and reviews from the Dotsama ecosystem and the other half is actually running validators and collators. We have been on Statemine from the very beginning and now we have joined Statemint as well :slight_smile:

I’m not sure how far you are with implementing or re-thinking the reward system, but I just wanted to add my 2 cents. There are already some parachains that have successfully created their collator set by governance votes of their respective councils (Centrifuge or Calamari for instance) and this approach makes actually quite a lot of sense

  • Since collators don’t provide security, parachains don’t really have to take into account the size of the stake (just a size of the initial bond to keep “skin in the game”)
  • It kinda equates the opportunity to get into the set for both big players (staking companies) and community collators like we are (if it’s desired to have a diverse collator set - which should be)
  • Community collators also tend to perform more scrutiny and take their job very seriously - since Dotsama is our only focus, our reputation is everything. Obviously, this doesn’t apply to just Polkadotters
  • Slots for CGPs were given to them through the governance, CGPs also serve as a system chain and extend the functionality of the DOT token itself - it makes perfect sense to let the governance decide about the collators being in the set based on the preferences of the given CGP. Honestly, I am not sure why should be there a need to create a free market that would make collators lock an unreasonable amount of DOT tokens and compete with each other. It also makes implementation more difficult in my point of view.

Based on our experience, we have seen quite a lot of “collator wars” on various chains where some sort of PoS is implemented. If the reward is good enough, it is always going to bring a “mercenary” capital that will run the collators for the yield only and without the required technical experience or participation in governance or discussions about the future of the chain itself.

This brings me to another thought, I think it makes sense to have variable rewards for each respective CGP - the operational costs can vary greatly based on the performance needs. The way I would structure it is to have a proposal for expected rewards where all the collators can present their case and formulate their operational costs and then it will be up to the council to decide the “one-size-fits-all” reward that will be paid to all the collators evenly. Seems fair enough and I would expect collators on any CGP to be involved and take an active part in such a discussion - it shouldn’t be just about “bond whatever they say and forget” :slight_smile:

1 Like

lately I have been wondering if payments should be done in Stable coins so collators don’t get reck with the fluctuations of the currency. I know is not an easy change but worth exploring.

@pmensik I agree with a lot of what you are saying. A lot of those reasons are why we have been gravitating toward this solution where some subset of collators are chosen by governance. I still think that there should be some way to collate most chains by placing a bond and being elected.

The ratio of governance selected to bonded collators could vary by chain. A parathread that executes once or twice per day might be fine with a handful of collators (maybe 3, all chosen by governance), while some “fast, optimistic” chain might needs hundreds of collators (and could introduce other roles like fishermen). It would be impossible for governance to select 100+ collators and do any kind of evaluation of them, so in that case the split might be like 10/90.

1 Like

That’s a general issue with Treasury diversification but we’re working on it and I agree.

I think parachains needing 100+ collators is something that will probably materialize in the distant future since having a lot of collators can have a performance impact on the parachain itself. Few parachains are already close to this number (having around 70-80 collators, for instance Astar or Moonbeam) but they are heading for a rather large set to achieve better decentralization and liveness, definitely not to have a performance benefit.

But yeah, I think it’s useful to think about such situations beforehand and have the tools ready when such a thing occurs. On the other hand, it won’t happen anytime soon, any current or upcoming CGP I know ( be it Statemint, Collectives, Snowbridge or Encointer) can be run by just a handful of collators so it’s a question of whether should we “overengineer” for something that most probably won’t come in a foreseeable future.

While the issue of missing rewards for existing collators is becoming more pressing since Statemine went live in March (or April?) and all collators are basically running it for free since then. So it would rather focus on solving the elections to collator set by governance and re-routing part of the rewards from inflation or treasury than trying to incorporate different voting mechanisms and delay the final solution significantly because of that.

Yeah, I mean I would already support a Treasury proposal for people running collators on Statemine.

A small collator pool can be targeted, collators can be attacked, don’t know if easy or not. but if that is the case some collators in stand-by should also be rewarded for just that.

A disruption on Statemine parachain once we have all parachains in full swing can be problematic, for the whole ecosystem.

I think the answer to that is having a properly diversified (by node operators, data centers and geolocation) collator set - determining the right size of it is another topic but in my point of view, anything above 30-40 should be sufficiently safe for the sake of decentralization and liveness. Don’t forget that only one online collator is enough for parachain to produce blocks.

I really appreciate this initiative. Encointer currently still operates its 5 collators by itself. We considered paying other entities to do so (with Encointer Association being the client). While this would distribute, it would not really decentralize anything. It is not unstoppable if collation stops because EA doesn’t pay its bills.

We would likely opt for small collator sets. I personally just fail to see the benefit of large collator sets.

On the matter of selecting a diverse set of collators: If you want to do that automatically, you’ll be looking for sybil resistance. Sybil resistance is the job Encointer aims to do for our ecosystem. Happy to discuss this further.

However: a diverse, sybil-resistant set of operators could still instantiate all their collators within the same datacenter by accident or ignorance. So Encointer can’t do this job alone. But it may enable an automated solution.

3 Likes