Parachain Consensus Updates: Coretime, asynchronous backing, scalability

[eskimor]

Quick update on parachain consensus related ongoing work.

Just recently a new runtime got deployed on Rococo, which enabled on-demand relay chain functionality. You can play around with it now! @antonva will provide instructions how this works soon.

We expect on-demand relay chain functionality on Kusama and Polkadot by the end of the year, when we have more layers of sandboxing ready for PVF execution.

Another big change we plan to have ready by the end of the year is the new agile coretime. This will replace the parachain auction model by a more agile core acquisition method, that works on a monthly basis. This and on-demand aim at lowering barrier to entry and optimizing block space utilization. In other words: More block space/core time for more people!

When I was talking about on-demand, I was very specific by saying “relay chain functionality”. The reason is, that realistically for a production chain people will want more. A way of having core orders automated and decentralized. We assume, even without this, that on-demand will already be useful for startups and early projects.

We plan on having an implementation for the above design ready by Q1/Q2 2024. That being said, the design is there and we definitely welcome contributions, which might make it feasible to have this delivered even sooner.

Asynchronous Backing

Another big change, that just made it to Rococo, is asynchronous backing. Enabling parachains to have bigger blocks (in terms of execution time) while at the same time also enables them to shorten block times from twelve to six seconds. Reducing latency and multiplying possible throughput at the same time!

Performance

A lot of effort is also put into improving performance of our system. E.g. here - planned, or already implemented here, here and here to name just a few recent improvements, about to land soon.

More work is happening behind the scenes, when it comes to disputes hardening, slashing, determinism, validator disabling strategies, … but I think the above are the most exiting/user facing developments.

Really looking forward seeing projects making use of asynchronous backing capabilities, authoring their first on-demand block and later on having first projects acquiring new bulk block space via agile coretime!

[bill_w3f]

We expect on-demand relay chain functionality on Kusama and Polkadot by the end of the year, when we have more layers of sandboxing ready for PVF execution.

When I was talking about on-demand, I was very specific by saying “relay chain functionality”. The reason is, that realistically for a production chain people will want more . A way of having core orders automated and decentralized.

Could you explain a bit more in-depth on what this means? I read the linked PR, but I’m still not clear on what is available now on Rococo (which I assume is also approximately what should be available on Kusama/Polkadot by EOY?) and what is future (Q1/Q2 2024) work.

[wil]

@eskimor

Another big change we plan to have ready by the end of the year is the new agile coretime .

“have ready” meaning the PR ready and then start working on rolling out to Rococo then Kusama then Polkadot in Q1/Q2 of 2024 right?

[GuinnessStache]

@eskimor

Another big change we plan to have ready by the end of the year is the new agile coretime

Does this mean live on Polkadot by end of year?

[eskimor]

@bill_w3f What is there on Rococo and will be available by EOY is that you can place an order for a core. Then you will get scheduled, your para will be assigned to a core and your parachain will be able to produce a block.

What is not yet available, is any logic that places the order in an automated and decentralized fashion. This can of course be built by parachain teams themselves. The presented design is for the all-in fully decentralized/everybody is untrusted design space.

Placing the order itself is easy, just send an extrinsic to the relay chain. E.g. one could just setup a funded machine, monitoring the mempool and sending orders as need be. Could be a simple script.

It gets more hairy if you want order placement itself being decentralized. Then you need to setup incentivization schemes, which requires proofs and on-chain logic. This is the part that has not yet been implemented.

[eskimor]

I would say, ready meaning rolled out on Rococo. Kusama and Polkadot following quickly afterwards (Q1).

[bill_w3f]

Thanks! That clarifies things. Much appreciated!

[antonva]

Here are the instructions:

In order to try out on demand parachains on Rococo, first make sure you’ve grabbed some ROC from the testnet faucet https://paritytech.github.io/polkadot-testnet-faucet.

If you have a runtime on hand you can register it on a new ParaId and onboard it as a Parathread just like usual but do not request a parachain slot. After onboarding, starting a collator and syncing you can go straight on to the placing an order section.

If you don’t have a runtime ready but would like to try it anyway, you can read up on the setup for the adder collator section at the end of this post to get started with a ready to go runtime.

Placing an order

In Polkadot.js:

1. Navigate to Developer → Extrinsics.
2. Select onDemandAssignmentProvider from the “submit the following extrinsic” field.
3. Choose either placeOrderAllowDeath or placeOrderKeepAlive extrinsic.
4. The maxAmount field must be somewhere between 10_000_000 and the maximum you would be willing to pay for a single on demand block. The paraId field should be set to the ParaId you registered and onboarded.
   
   

If the extrinsic was accepted and the collator is working, a new parachain block should be included within a few relay chain blocks depending on congestion.

In the case of an adder-collator runtime, the block head will increase by 1 for each block added.

Looking up on demand price and other data

• The price of an on demand block is calculated by multiplying configuration.on_demand_base_fee with SpotTraffic, which is the output of a price controller.

• The base fee and parameters to the price controller are all exposed to governance. Most of these parameters can be located in the configuration pallet under on_demand_*.

• The actual fee charged for an order is found in the SpotOrderPlaced event which is emitted on every placed order.

• The number of on demand parachains that can be processed in the same relay chain block is determined by the configuration.on_demand_cores value.

• A parachain cannot have parallel blocks on 2 or more different cores on the same relay chain block. Multiple orders for the same parachain will be processed sequentially.

• The onDemandAssignmentProvider pallet stores the order queue under onDemandQueue and the spot traffic multiplier can be found under spotTraffic.

• The lowest amount that maxAmount can be set as is determined by configuration.on_demand.base_fee, which by default is 10_000_000.

Setup for the adder collator

The following instructions will go through the steps needed to get an instance of adder-collator up and running on Rococo as an on-demand parachain.

Compile the adder-collator:

# In the polkadot-sdk repository
cargo build -r -p test-parachain-adder-collator 

Export the genesis state for the adder-collator:

# Still in the polkadot-sdk repository
./target/release/adder-collator export-genesis-state \
  --chain=rococo \
  $SOME_PATH/rococo-adder-genesis-state

In Polkadot.js:

1. Register a new ParaId: Network → Parachains → Parathreads → ParaId. Note down the ParaId.
2. On the same page, register a ParaThread. The ParaId should be the one used in step 1. The code is obtained from ./target/release/wbuild/test-parachain-adder/test_parachain_adder.compact.compressed.wasm and the initial_state is obtained from $SOME_PATH/rococo-adder-genesis-state that was generated earlier.

While waiting for onboarding to finish, launch the collator to start syncing with Rococo.

# You guessed it, polkadot-sdk repository
./target/release/adder-collator \
  --parachain-id=$YOUR_PARA_ID \
  --chain=rococo 

If you don’t have a backup available on hand, this might take a while (a couple of days in my experience).

[xiaojie_PolkaWorld]

Are you referring to parathreads when you mention the on-demand relay chain functionality?

[eskimor]

Yes, indeed. Naming is hard and evolving. parathreads and on-demand parachains is the same thing. I am a bit reluctant to mention that there was even another name created just recently. Sorry for the confusion, I hope we will be able to straighten up our communication a bit soon.

[filippoweb3]

How are coretime sales handled on Rococo? Is there any implementation or will this be decided later on only on Kusama and Polkadot?

[alejandro]

I don’t fully understand what you are exactly asking for @filippoweb3

If you are talking about the exact implementation for the on demand assignment provider that @antonva showcases in the above example, you can find its implementation in this pallet - assigner_on_demand.

[seraya_w3f]

@eskimor what’s the timeline on Asynchronous Backing?

[eskimor]

Goes live on Kusama with the next release.