Elastic Scaling MVP launched

Tech Talk
1

The latest release stable2407 of the Cumulus node contains the support for Elastic Scaling, so parachain teams can start experimenting locally (using Zombienet), on Westend and very soon on Kusama.

Current status

This MVP release enables parachains to use up to 3 cores per relay chain block allowing 3x more throughput and block times of 2 seconds.

Relay chain

The functionality is fully implemented from the perspective of the relay chain and currently enabled on Westend/Rococo.

For security reasons, all nodes (collators) of an elastic parachain need to be trusted. This means collators must announce/gossip blocks only to trusted nodes. Typically this would imply some sort of private network where the collators are deployed. This is only a temporary requirement until RFC 103 is implemented.

Collators

Elastic scaling support can be enabled on collators only by using the new and experimental slot based collator . This is exposed in polkadot-parachain as a node CLI switch: --experimental-use-slot-based.

The cumulus block authorship and import pipelines are not yet optimised for elastic scaling so maximum achievable throughput depends on the number of collators and their networking latency.

Maximum throughput (2s execution) is only possible if the collator set has just 1 single collator. Adding more than one collator in the set reduces the maximum usable execution time per block.

With more than 1 collator, testing so far has showed that the maximum throughput can be achieved with 2 cores. That gives 1.3s of execution and 5MB PoV per parachain block, leading to 2.6s of execution for each relay chain block. More details can be found in this ticket.

We plan to optimise cumulus block authorship in the near future as described in Elastic Scaling: Streamlined Block Production · Issue #5190 · paritytech/polkadot-sdk · GitHub . This should alleviate or even remove any throughput limitations of the current implementation.

Documentation

More details about elastic scaling and how to enable it in the parachain runtime and node side can be found in this guide

Zombienet quick start

Pre-requisites

Sample test

Use the example test available in this branch: polkadot-sdk/polkadot/zombienet_tests/elastic_scaling/0000-cumulus-3cores.zndsl at sandreim/elastic-scaling-test · paritytech/polkadot-sdk · GitHub . You need to rebuild polkadot-parachain to include the runtime changes for elastic scaling.

The test will spin up a Rococo development relay chain with 3 validators and one parachain using polakdot-parachain and the rococo-parachain-runtime test runtime. The runtime has already been updated to support the elastic as instructed by the documentation, see commit

The zombienet test makes use of a JS script to assign cores to the parachain.

Run the test with node ./packages/cli/dist/cli.js -p native test ~/polkadot-sdk/polkadot/zombienet_tests/elastic_scaling/0000-cumulus-3cores.zndsl

Polkadot node template

If you are using the polkadot parachain node template, you additionally need to add the elastic scaling node support by following the instructions here and build the node binary.

18 Likes
2

More should also be possible with more performant hardware setups. We have not tested that yet, but welcome everybody to try it out and tell us how far you were able to push it. :rocket:

3

Hi - so what is the current estimate for Elastic Scaling to come to Kusama and Polkadot?

4

On Kusama very soon, end of September or early October.

What’s missing is a referenda to bump the async backing parameters so that parachains can use up to 3 cores, otherwise Elastic Scaling MVP is deployed in the current runtime and node.

On Polkadot it isn not clear at this time, but will happen this year. The list of things still to be done include:

6 Likes
5

Quick update: [Whitelisted Caller] Set AsyncBackingParams.max_candidate_depth to 6 was executed and elastic scaling MVP is supported on Kusama.

7 Likes