Polkadot Release Analysis v1.4.0

Ecosystem
1

Polkadot Release Analysis v1.4.0

:warning: The following report is not an exhaustive list of changes included in the release, but a set of changes that we felt deserved to be highlighted due to their impact on the Polkadot ecosystem builders.

Please do not stop reading the release notes v1.4.0. This report is complementary to the changelogs, not a replacement.

Highlights are categorized into High Impact, Medium Impact, and Low Impact for ease of navigation.
Also there is a section related to all note worthy changes related to tooling integration.

Help us improve the release analysis by filling out this 6 question survey.

Summary

In the following report we are featuring some of the changes included in the polkadot node release 1.4.0. All the runtime changes that are part of this release will be included in the corresponding runtime release done by the polkadot technical fellowship.

Migrations in this release:

:hammer_and_wrench: Tooling Section

:exclamation: High Impact

[S/P/C] Use Message Queue as DMP and XCMP dispatch queue

PR: https://github.com/paritytech/polkadot-sdk/pull/1246

Why is this important?

This MR refactores the XCMP, Parachains System and DMP pallets to use the MessageQueue for delayed execution of incoming messages. The DMP pallet is entirely replaced by the MQ and thereby removed. This allows for PoV-bounded execution and resolves a number of issues that stem from the current work-around.

How does this impact the Polkadot builders?

  • DMP Queue Pallet

The pallet got removed and its logic refactored into parachain-system. Overweight message management can be done directly through the MQ pallet.

Final undeployment migrations are provided by cumulus_pallet_dmp_queue::UndeployDmpQueue and DeleteDmpQueue that can be configured with an aux config trait like:

parameter_types! {
	pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME;
	pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent;
}

impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime {
	type PalletName = DmpQueuePalletName;
	type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>;
	type DbWeight = <Runtime as frame_system::Config>::DbWeight;
}

// And adding them to your Migrations tuple:
pub type Migrations = (
	...
	cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>,
	cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>,
);

How to configure those:

// Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since
// the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s.
type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>;

// Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched
// with origin `Junction::Sibling(…)`.
type XcmpProcessor = ProcessFromSibling<
	ProcessXcmMessage<
		AggregateMessageOrigin,
		xcm_executor::XcmExecutor<xcm_config::XcmConfig>,
		RuntimeCall,
	>,
>;

// Not really important what to choose here. Just something larger than the maximal number of channels.
type MaxInboundSuspended = sp_core::ConstU32<1_000>;

The InboundXcmpStatus storage item was replaced by InboundXcmpSuspended since it now only tracks inbound queue suspension and no message indices anymore.

Now only sends the most recent channel Signals, as all prio ones are out-dated anyway.

  • XCMP Queue pallet

Removed all dispatch queue functionality. Incoming XCMP messages are now either: Immediately handled if they are Signals, enqueued into the MQ pallet otherwise.

New config items for the XCMP queue pallet:

/// The actual queue implementation that retains the messages for later processing.
type XcmpQueue: EnqueueMessage<ParaId>;

/// How a XCM over HRMP from a sibling parachain should be processed.
type XcmpProcessor: ProcessMessage<Origin = ParaId>;

/// The maximal number of suspended XCMP channels at the same time.
#[pallet::constant]
type MaxInboundSuspended: Get<u32>;
  • Parachain System pallet

For DMP messages instead of forwarding them to the DMP pallet, it now pushes them to the configured DmpQueue. The message processing which was triggered in set_validation_data is now being done by the MQ pallet on_initialize.

XCMP messages are still handed off to the XcmpMessageHandler (XCMP-Queue pallet) - no change here.

New config items for the parachain system pallet:

/// Queues inbound downward messages for delayed processing. 
///
/// Analogous to the `XcmpQueue` of the XCMP queue pallet.
type DmpQueue: EnqueueMessage<AggregateMessageOrigin>;

How to configure:

/// Use the MQ pallet to store DMP messages for delayed processing.
type DmpQueue = MessageQueue;

:warning: Medium Impact

[S] Expose collection attributes from Inspect trait

PR: Expose collection attributes from `Inspect` trait by dastansam · Pull Request #1914 · paritytech/polkadot-sdk · GitHub

Why is this important?

Prior to this PR, it was not possible to read the collection_attribute with <Nfts as Inspect>::collection_attribute() if different namespace values were used. For example, setting the attribute with AttributeNamespace::Pallet, while the inspect uses AttributeNamespace::CollectionOwner when reading.

This PR makes it possible by making item an optional parameter in Inspect::system_attribute(). If item is not specified then system_attribute() will return the collection attributes.

However, this is a breaking change for those using NftsApi because this changes the item param in NftsApi::system_attribute().

[S] Contracts expose pallet-xcm

PR: https://github.com/paritytech/polkadot-sdk/pull/1248

Why is this important?

A new type has been included into pallet-contract’s Config:

#[pallet::config]
pub trait Config: frame_system::Config {
// -- snip --
+    /// A type that exposes XCM APIs, allowing contracts to interact with other parachains, and
+    /// execute XCM programs.
+    type Xcm: xcm_builder::Controller<
+        OriginFor<Self>,
+        <Self as frame_system::Config>::RuntimeCall,
+        BlockNumberFor<Self>,
+    >;
}

Together with a new error:

#[pallet::error]
pub enum Error<T> {
// -- snip --
+    /// Failed to decode the XCM program.
+    XCMDecodeFailed,
// -- snap --
}

Watch out for index changes within the Error enum.

How does this impact the Polkadot builders?

With the inclusion of xcm_send and xcm_execute, pallet-contracts can now proxy these function to, for instance, pallet-xcm if the new trait is configured to do so.

Using these is equivalent to dispatching pallet_xcm::send and pallet_xcm::execute through call_runtime, except that the functions are called directly instead of being dispatched.

[S] Identity pallet improvements

PR: https://github.com/paritytech/polkadot-sdk/pull/2048

Why is this important?

The following rename has been done in pallet-identity’s Config:

#[pallet::config]
pub trait Config: frame_system::Config {
// -- snip --
    #[pallet::constant]
-    type FieldDeposit: Get<BalanceOf<Self>>;
+    type ByteDeposit: Get<BalanceOf<Self>>;
// -- snap --
}

At the same time, its configuration in the different relay runtimes has been changed so it accounts for a price per encoded byte.

set_fieldsextrinsic changes its function signature as follows:

pub fn set_fields(
    origin: OriginFor<T>,
    #[pallet::compact] index: RegistrarIndex,
-   fields: IdentityFields<
-       <T::IdentityInformation as IdentityInformationProvider>::IdentityField,
-	>,
+   fields: <T::IdentityInformation as IdentityInformationProvider>::FieldsIdentifier,
) -> DispatchResultWithPostInfo {

[S] [NPoS] Paging reward payouts in order to scale rewardable nominators

PR: Pull requests · paritytech/substrate · GitHub

Why is this important?

Rewards payout is processed today in a single block and limited to MaxNominatorRewardedPerValidator. This number is currently 512 on both Kusama and Polkadot.

This PR tries to scale the nominators payout to an unlimited count in a multi-block fashion. Exposures are stored in pages, with each page capped to a certain number (MaxExposurePageSize). Starting out, this number would be the same as MaxNominatorRewardedPerValidator, but eventually, this number can be lowered through new runtime upgrades to limit the rewardeable nominators per dispatched call instruction.

The changes in the PR are backward compatible.

How does this impact the Polkadot builders?

  • Migration Note

Strictly speaking, there was no need to bump our storage version since there is no migration of storage in this PR. But it is still useful to mark a storage upgrade, being some of the reasons:

  • New storage items are introduced in this PR while some older storage items are deprecated.
  • For the next HistoryDepth eras, the exposure would be incrementally migrated to its corresponding paged storage item.

To see the detailed list of reasons that behind this migration and othger storage changes, please read the description of this change.

  • How payouts would work like after this change

payout_stakers remains backward compatible with no signature change. If for a given era a validator has multiple pages, they can call payout_stakers multiple times. The pages are executed in an ascending sequence and the runtime takes care of preventing double claims.

New extrinsic payout_stakers_by_page. Very similar to payout_stakers but also accepts an extra param page_index. An account can choose to payout rewards only for an explicitly passed page_index.

Please, visit the description of the PR for if you would like to see a example scenario.

[P] pallet-xcm: enhance reserve_transfer_assets to support remote reserves

PR: https://github.com/paritytech/polkadot-sdk/pull/1672

Why is this important?

So far pallet-xcm only allowed for sending reserve-based assets to a remote destination and beneficiary when the reserve is the local chain.

With these changes pallet_xcm::(limited_)reserve_withdraw_assets now supports transfers when reserves are other chains.
This will allow complete, bi-directional reserve-based asset transfers user stories using pallet-xcm.

Why is this change interesting for builders?

Enables following scenarios:

  • transferring assets with local reserve (was previously supported iff asset used as fee also had local reserve - now it works in all cases),
  • transferring assets with reserve on destination,
  • transferring assets with reserve on remote/third-party chain (iff assets and fees have same remote reserve),
  • transferring assets with reserve different than the reserve of the asset to be used as fees - meaning can be used to transfer random asset with local/dest reserve while using DOT for fees on all involved chains, even if DOT local/dest reserve doesn’t match asset reserve,
  • transferring assets with any type of local/dest reserve while using fees which can be teleported between involved chains.

All of the above is done by pallet inner logic without the user having to specify which scenario/reserves/teleports/etc. The correct scenario and corresponding XCM programs are identified, and respectively, built automatically based on runtime configuration of trusted teleporters and trusted reserves.

:information_source: Low Impact

[S] pallet-grandpa: Remove GRANDPA_AUTHORITIES_KEY

PR: pallet-grandpa: Remove `GRANDPA_AUTHORITIES_KEY` by bkchr · Pull Request #2181 · paritytech/polkadot-sdk · GitHub

Why is this important?

Removes GRANDPA_AUTHORITIES_KEY key and its usage. Now there is a runtime api that takes care of communicating grandpa authorities to the node.

This MR includes a migration to doing so in a running chain.

[S] Refactor transaction storage pallet to use fungible traits

PR: https://github.com/paritytech/polkadot-sdk/pull/1800

Why is this important?
This is a refactor of the transaction storage pallet to use the fungible trait in place of the deprecated currency trait. Part of this refactor add an additional type to the config: RuntimeHoldReason - which will need to be implemented in the runtime.

The following will need to be added to your runtime:

+ type RuntimeHoldReason = RuntimeHoldReason;

This is part of a larger tracking issue:
- FRAME: Move pallets over to use fungible traits

[S] feat: FRAME umbrella crate.

PR: https://github.com/paritytech/polkadot-sdk/pull/1337

Why is this important?

In an effort to have a cleaner set of FRAME APIs, this PR is a step in that direction. This is still a WIP and under the experimental flag, however, this was merged in order to have the work done in the developer hub docs reference the new re-exported APIs.

The developer hub docs is an initiative to have a holistic, minimal documentation portal for Polkadot developers.

Related
#14137

[S] Adds syntax for marking calls feeless

PR: https://github.com/paritytech/polkadot-sdk/pull/1926

Why is this change interesting for builders?

This MR introduces a new attribute pallet::feeless_if:

#[pallet::feeless_if(|_origin: &OriginFor<T>, something: &u32| -> bool {
	*something == 0
})]
pub fn do_something(origin: OriginFor<T>, something: u32) -> DispatchResult {
     // doing something
}

The closure passed accepts references to arguments as specified in the call fn. It returns a boolean that denotes the conditions required for this call to be “feeless”.

The changes also include a new signed extensions SkipCheckIfFeeless<T: SignedExtension> wrapping a transaction payment processor like pallet_transaction_payment::ChargeTransactionPayment.
It checks for all calls annotated with pallet::feeless_if to see if the conditions are met. If so, the wrapped signed extension is not called, essentially making the call feeless.

Following the author notes, you can replace your existing signed extension that manages transaction payment like so:

- pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
+ pallet_skip_feeless_payment::SkipCheckIfFeeless<
+	Runtime,
+	pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
+ >,

[S] Add sudo::remove_key

PR: https://github.com/paritytech/polkadot-sdk/pull/2165

Why is this important?

Adds a new extrinsic to pallet-sudo, remove_key with call index 4.
This new extrinsic will remove the sudo key permanently. Being a less-invasive way of rendering the sudo pallet useless without needing a code upgrade.

[S] sc-block-builder: Remove BlockBuilderProvider

PR: https://github.com/paritytech/polkadot-sdk/pull/2099

How does this impact the Polkadot builders?

This change is removing the BlockBuilderProvider trait. And introduces a builder for creating a BlockBuilder. This builder is currently named BlockBuilderBuilder.

Why is this change interesting for builders?

If you used new_block or new_block_at before you now need to switch it over to the new BlockBuilderBuilder pattern:

// `new` requires a type that implements `CallApiAt`. 
let mut block_builder = BlockBuilderBuilder::new(client)
    // Then you need to specify the hash of the parent block the block will be build on top of
    .on_parent_block(at)
    // The block builder also needs the block number of the parent block. 
    // Here it is fetched from the given `client` using the `HeaderBackend`
    // However, there also exists `with_parent_block_number` for directly passing the number
    .fetch_parent_block_number(client)
    .unwrap()
    // Enable proof recording if required. This call is optional.
    .enable_proof_recording()
    // Pass the digests. This call is optional.
    .with_inherent_digests(digests)
    .build()
    .expect("Creates new block builder");

[S/P] Identity Deposits Relay to Parachain Migration

PR: https://github.com/paritytech/polkadot-sdk/pull/1814

Why is this important?

This PR is part of the migration of the identities from the Identities pallet on the Relay chain to the new People Chain.

Interestingly, it uses two extrinsics and XCM to migrate the identity deposits.

For the migration, a new IdentityMigrator pallet is introduced:

This pallet is designed to go into a source chain and destination chain to migrate data. The design motivations are:

  • Call some function on the source chain that executes some migration (clearing state, forwarding an XCM program).
  • Call some function (probably from an XCM program) on the destination chain.
  • Avoid cluttering the source pallet with new dispatchables that are unrelated to its functionality and only used for migration.

Two new extrinsics: reap_identity and poke_depoist are used to perform the migration along with a ReapIdentityHandler. The PR does a nice job of describing their interactions.

After the migration is complete, the IdentityMigrator pallet may be removed from both chains’ runtimes as it has served its purpose.

[P] XCM builder pattern

PR: https://github.com/paritytech/polkadot-sdk/pull/2107

Why is this change interesting for builders?
Adds a procedural macro to be able to write XCMs using the builder pattern.

This means we go from having to do this:

let message: Xcm<()> = Xcm(vec![
  WithdrawAsset(assets),
  BuyExecution { fees: asset, weight_limit: Unlimited },
  DepositAsset { assets, beneficiary },
]);

to this:

let message: Xcm<()> = Xcm::builder()
  .withdraw_asset(assets)
  .buy_execution(asset, Unlimited),
  .deposit_asset(assets, beneficiary)
  .build();

Your friendly neighborhood Polkadot Release Analysis team,
@alejandro @ayush.mishra @bruno

2 Likes