alejandro
Thu, 09 Mar 2023 19:30:15 +0000
Ecosystem

Polkadot Release Analysis v0.9.39

The following report is not an exhaustive list of changes included in the release, but a set of changes that we felt deserved to be highlighted due to their impact on the Polkadot ecosystem builders.

Please do not stop reading the release notes v0.9.39. This report is complementary to the changelogs, not a replacement.

Highlights are categorized into High Impact, Medium Impact, and Low Impact for ease of navigation.
Also there is a section related to all note worthy changes related to tooling integration.

Help us improve the release analysis by completing this 6 question survey.

Summary

This release includes some interesting feautures such as adding metadata to a referendum proposal, the chance to parametrize the executor environment as well as features added to the new NFT pallet. Also there is a new pallet called Glutton that makes it possible to test the boundaries of a chain.

Runtimes built on release v0.9.39

Rococo v9390

Westend v9390

Kusama v9390

Polkadot v9390

Tooling Section

TxWrapper Core - v5.0.0
Substrate Sidecar API - v14.4.0
Polkadot JS API - v9.14.2

Medium Impact

Referendum proposal’s metadata

PR: https://github.com/paritytech/substrate/pull/12568

Why is this important?

This PR permits to attach metadata for a proposal/referendum on the referenda and democracy pallets. Both pallets just store the hash of the preimage through a preimage provider, which in the case of Polkadot/Kusama is the pallet_preimage.

Implementation rules (as stated in the PR description):

In the democracy pallet the metadata can be set only for a proposal (public or external). If the proposal evolves to the referendum the metadata. The metadata of a referendum can be cleared only by root origin (due the absence of the submitter information, and it fits to the overall idea).

Why is this change interesting for builders?

The democracy pallet is a pallet widely used within most parachains and having proposal/referendum metadata is a feature that has been sought after by the community.

[NFTs] Offchain mint

PR: https://github.com/paritytech/substrate/pull/13158

Why is this important?

With this PR, a collection owner can sign mint data offchain and pass the signed mint data to the enduser who in turn can submit it onchain via the new mint_pre_signed extrinsic. The pre-signed mint data consists of the following:

pub struct PreSignedMint<CollectionId, ItemId, AccountId, Deadline> {
	/// A collection of the item to be minted.
	pub(super) collection: CollectionId,
	/// Item's id.
	pub(super) item: ItemId,
	/// Additional item's key-value attributes.
	pub(super) attributes: Vec<(Vec<u8>, Vec<u8>)>,
	/// Additional item's metadata.
	pub(super) metadata: Vec<u8>,
	/// Restrict the claim to a particular account.
	pub(super) only_account: Option<AccountId>,
	/// A deadline for the signature.
	pub(super) deadline: Deadline,
}

Note that you can have None as a value for only_account. This would allow anyone to claim the NFT.

How does this impacts the Polkadot builders?

Offchain minting of NFTs is a powerful feature which is now available in the NFT pallet. If you are not familiar with the NFT pallet, you can start here.

Why is this change interesting for builders?

Offchain NFT minting can offer several benefits in terms of cost, speed, flexibility, scalability, privacy, and user experience.

Glutton pallet

PR: https://github.com/paritytech/substrate/pull/12833

Why is this important?

This pallet adds the functionality to use all the weight, both time and space, remaining in each produced block.

With this pallet, we can perform practical tests and check the theoretical limits of how many parachains can run in parallel without destabilizing the Relay chain.

Why is this change interesting for builders?

You can read the conversations prior to its development in the following forum post and github issue:

polkadot forum post

github issue

The idea is to create several runtimes with this pallet and bootstrap a number of “Glutton Parachains”, and connect them to Kusama so we can have a clearer picture of the real number of parachains a Relay chain can handle.

The pallet exposes three extrinsics that Root origin will be able to successfully call, these are:

pub fn initialize_pallet(
    origin: OriginFor<T>,
	new_count: u32,
	witness_count: Option<u32>,
) -> DispatchResult { // -- snip -- }

pub fn set_compute(
    origin: OriginFor<T>,
    compute: Perbill
) -> DispatchResult { // -- snip -- }

pub fn set_storage(
    origin: OriginFor<T>,
    storage: Perbill
) -> DispatchResult { // -- snip -- }

With these extrinsics, we can initialize the pallet, set the PoV ref_time and size usage for every block.

Low Impact

add warp to target block for parachains

PR: https://github.com/paritytech/substrate/pull/12761

Why is this change interesting for builders?

Substrate provides a few sync mechanisms like full sync, fork sync, gap sync, warp sync, state sync, etc. warp sync is much faster than the others. It initially downloads the best finalized block and the whole state at that block, and finality proofs (currently from GRANDPA) for all validator set transitions from genesis up until the best finalized block. After the initial warp sync is done, the node is ready to import the latest blocks and it will start downloading all previous blocks in the background.

Prior to this PR, warp sync was not supported in parachains. This PR provides warp sync implementation for parachains as well.

Please visit this section to see the changes, which you have to make in your service.rs to enable warp sync support. You can find more details in companion PRs and related issue.

Does this change have a related companion PR?
Polkadot companion PR #6334
Cumulus companion PR #1909

Related Issues: Parachain Warp Sync · Issue #1619 · paritytech/cumulus · GitHub

sc-client-db: Fix PruningMode::ArchiveCanonical

PR: https://github.com/paritytech/substrate/pull/13361

Why is this change interesting for builders?

In a Substrate node, we can set state pruning to archive-canonical, which would only archive the finalized chain and keep only the state of finalized blocks. In Polkadot-v0.9.30, archive-canonical parameter was added to the --block-prunning and --state-prunning parameters, which permits to keep the blocks that are only finalized as well as only the state related to finalized blocks.

At the moment, archive-canonical flag is broken and is not working properly. This PR fixes the issues related to archive-canonical.

Related Issues:

github.com/paritytech/substrate

Node doesn't start with `archive-canonical` flag

opened 05:00PM - 01 Dec 22 UTC

closed 11:39AM - 13 Feb 23 UTC

vedhavyas

J2-unconfirmed

### Is there an existing issue? - [X] I have searched the existing issues …### Experiencing problems? Have you tried our Stack Exchange first? - [X] This is not a support question. ### Description of bug The node errors out when I start with `archive-canonical`. Our substrate fork is not too old and be found here - https://github.com/subspace/substrate. ``` subspace-node-1 | 2022-12-01 16:52:02.890 INFO main sc_cli::runner: Subspace subspace-node-1 | 2022-12-01 16:52:02.891 INFO main sc_cli::runner: ✌️ version 0.1.0-unknown subspace-node-1 | 2022-12-01 16:52:02.891 INFO main sc_cli::runner: ❤️ by Subspace Labs <https://subspace.network>, 2021-2022 subspace-node-1 | 2022-12-01 16:52:02.891 INFO main sc_cli::runner: 📋 Chain specification: Subspace Gemini 3a subspace-node-1 | 2022-12-01 16:52:02.891 INFO main sc_cli::runner: 🏷 Node name: ved_local subspace-node-1 | 2022-12-01 16:52:02.891 INFO main sc_cli::runner: 👤 Role: AUTHORITY subspace-node-1 | 2022-12-01 16:52:02.891 INFO main sc_cli::runner: 💾 Database: ParityDb at /var/subspace/chains/subspace_gemini_3a/paritydb/full subspace-node-1 | 2022-12-01 16:52:02.891 INFO main sc_cli::runner: ⛓ Native runtime: subspace-0 (subspace-0.tx0.au0) subspace-node-1 | 2022-12-01 16:52:03.288 INFO main sc_service::client::client: [PrimaryChain] 🔨 Initializing Genesis block/state (state: 0x1c3c…f9bb, header-hash: 0x8797…5b2f) subspace-node-1 | Error: SubstrateService(Other("Failed to build a full subspace node: Sub(Client(RuntimeApiError(Application(UnknownBlock(\"State already discarded for 0x8797298eb8cd5c3b3a08b6c1a6ed7809ee40e7f80fc0acf3cdb3fde00a435b2f\")))))")) ``` If I run the node with `archive` or other pruning method. it works. The Node is completely pruned everytime. Also note that `hash` that is missing is the Genesis block. ### Steps to reproduce This is the docker-compose I used. ``` services: node: image: ghcr.io/subspace/node:gemini-3a-2022-dec-01-aarch64 volumes: - node:/var/subspace:rw ports: - "0.0.0.0:30333:30333" restart: unless-stopped command: [ "--chain", "gemini-3a", "--base-path", "/var/subspace", "--execution", "wasm", "--state-pruning", "archive-canonical", "--blocks-pruning", "archive-canonical", "--port", "30333", "--rpc-cors", "all", "--rpc-methods", "safe", "--unsafe-ws-external", "--validator", "--log=subspace=debug", "--name", "node_local" ] healthcheck: timeout: 5s interval: 30s retries: 5 ``` Executables are also fond here - https://github.com/subspace/subspace/releases/tag/gemini-3a-2022-dec-01

github.com/paritytech/substrate

`--blocks-pruning archive-canonical` broken

opened 10:15PM - 29 Jan 23 UTC

closed 11:39AM - 13 Feb 23 UTC

jasl

### Is there an existing issue? - [X] I have searched the existing issues …### Experiencing problems? Have you tried our Stack Exchange first? - [X] This is not a support question. ### Description of bug ``` ./polkadot --tmp --chain kusama --state-pruning archive-canonical --no-hardware-benchmarks --no-telemetry 2023-01-29 22:14:30 ---------------------------- 2023-01-29 22:14:30 This chain is not in any way 2023-01-29 22:14:30 endorsed by the 2023-01-29 22:14:30 KUSAMA FOUNDATION 2023-01-29 22:14:30 ---------------------------- 2023-01-29 22:14:30 Parity Polkadot 2023-01-29 22:14:30 ✌️ version 0.9.37-645723987cf 2023-01-29 22:14:30 ❤️ by Parity Technologies <admin@parity.io>, 2017-2023 2023-01-29 22:14:30 📋 Chain specification: Kusama 2023-01-29 22:14:30 🏷 Node name: diligent-orange-7774 2023-01-29 22:14:30 👤 Role: FULL 2023-01-29 22:14:30 💾 Database: RocksDb at /tmp/substratePJsmzv/chains/ksmcc3/db/full 2023-01-29 22:14:30 ⛓ Native runtime: kusama-9370 (parity-kusama-0.tx19.au2) 2023-01-29 22:14:30 🔨 Initializing Genesis block/state (state: 0xb000…ef6b, header-hash: 0xb0a8…dafe) Error: 0: UnknownBlock: State already discarded for 0xb0a8d493285c2df73290dfb7e61f870f17b41801197a149ca93654499ea3dafe Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it. Run with RUST_BACKTRACE=full to include source snippets. 2023-01-29 22:14:30 👴 Loading GRANDPA authority set from genesis on what appears to be first startup. ``` `--state-pruning archive` works ### Steps to reproduce `./polkadot --tmp --chain kusama --state-pruning archive-canonical --no-hardware-benchmarks --no-telemetry` or other Substrate-based chains including parachains

[Feature] Introduce storage_alias for CountedStorageMap

PR: https://github.com/paritytech/substrate/pull/13366

Why is this important?

This PR introduces a storage alias for CountedStorageMap. If you are not familiar with CountedStorageMap, you can read about it here.

How does this impacts the Polkadot builders?

You can now interact with the CountedStorageMap via the storage alias:

type ExampleCountedMap = CountedStorageMap<Prefix, Twox64Concat, u16, u32>;
assert_eq!(ExampleCountedMap::count(), 0);
ExampleCountedMap::insert(3, 10);

Why is this change interesting for builders?

Better developer ergonomics.

try-runtime::fast-forward

PR: https://github.com/paritytech/substrate/pull/12896

Why is this important?

The try-runtime feature fast-forward in conjunction with RemoteExternalities scrapes live chain state, inserts it into a TestExternality and produces N empty blocks or indefinitely creates empty blocks.

How does this impacts the Polkadot builders?

This is one additional feature to utilize in the arsenal of try-runtime. If you’re not familiar with try-runtime, there is an excellent resource here.

Example Usage:

./target/release/node-template try-runtime --runtime existing fast-forward --n-blocks 2 live --uri ws://127.0.0.1:9944

Why is this change interesting for builders?

Try-runtime allows the developer to run tests off of live chain state.

Staking and nomination pools runtime API improvements

PR: https://github.com/paritytech/substrate/pull/13119
Polkadot companion PR: Companion PR for PR#13119 by gpestana · Pull Request #6683 · paritytech/polkadot · GitHub

Why is this important?

This PR adds new runtime API calls for staking and nomination pools that can be called through state_call rpc to fetch:

Nominations quota (StakingApi_nominations_quota)

Returns the current nominations quota for nominators. For now, this api runtime call will always return value of T::MaxNominations and thus it is redundant. However, with the upcoming changes in: Implements dynamic number of nominators #12970 the nominations quota will change depending on the nominators balance. We’re introducing this runtime API now to prepare the community to use it before rolling it out in PR#12970.

Points to balance conversion for pools (NominationPoolsApi_points_to_balance)

Returns the points to balance conversion for a specified pool.

Balance to point conversion for pools (NominationPoolsApi_balance_to_points)

Returns the equivalent new_funds balance to point conversion for a specified pool.

Executor Environment parameterization

PR: https://github.com/paritytech/polkadot/pull/6161

Why is this important?

This PR helps with the following issue:

Currently there is no explicit versioning on the PVF execution environment (EE), so once a client release includes a different EE there will be a window of time where the previous and last version of EE will be available at the same time. This implies that there might exist a candidate which would be valid on the newer version and invalid on the previous one, leading to disputes.

The changes included in this PR make the execution environment parametrized, hence bounding these parameters to the session, so when a candidate produced in an older session needs to be a candidate in a newer session the EE will use the parameters that existed in that older session.

To be noted, as per this change executor environment parameters may only be changed with runtime upgrades, this could be changed in favor of being able to change the parameters on the fly.

Additionally, this PR only takes care of parametrizing the EE, this is not versioning, however it is one step in that direction.

Does this change have a related companion PR?
Cumulus companion PR: companion for paritytech/polkadot#6161 by s0me0ne-unkn0wn · Pull Request #2151 · paritytech/cumulus · GitHub

Your friendly neighbourhood Polkadot Release Analysis team,
@hectorb @alejandro @ayush.mishra @bruno

2 posts - 1 participant