Polkadot Release Analysis v0.9.38
The following report is not an exhaustive list of changes included in the release, but a set of changes that we felt deserved to be highlighted due to their impact on the Polkadot ecosystem builders.
Please do not stop reading the release notes v0.9.38. This report is complementary to the changelogs, not a replacement.
Highlights are categorized into High Impact, Medium Impact, and Low Impact for ease of navigation.
Also there is a section related to all note worthy changes related to tooling integration.
Help us improve the release analysis by completing this 6 question survey.
Summary
XCM v3 is included in this release 
!
We highly encourage you to look at the actual PR #4097 for details on XCM v3.
This is an important release since XCM v3 will land on each respective relay chain with their corresponding runtime upgrades.
Another important change is the inclusion of the proof_size field calculation during the pallet benchmarking process. This is basically making effective weights v2 for FRAME pallets and for any pallet migrating to use benchmarking v2 macros.
As usual, new features as well as some bug fixes are being added as part of the release.
Runtimes built on release v0.9.38
- Rococo v0.9.38
- Westend v0.9.38
- Kusama v0.9.38
- Polkadot v0.9.38
Tooling Section
TxWrapper Core - v5.0.0
Substrate Sidecar API - v14.4.0
Polkadot JS API - v9.14.2
As a result of the new XCM version, the transaction_version was bumped.
High Impact
XCM v3
PR: https://github.com/paritytech/polkadot/pull/4097
This PR is merging most of the work related to the new version of XCM v3. As this change is massive and there are several improvements and design changes. The highlights for this new version can be simplified around Non-Fungibles (NFT) support, enabling XCM for bridges and multiple enhancements and new features regarding the programability of this messaging format like AssetLock, QueryResponseInfo, an implementable trait in XcmExecutor::Config for the ExchangeAsset instruction to use, etc.
In regards XCM v2 <> XCM v3 compatibility and new scenarios, we are working on this living document that we will continue to update:
The documentation for this new verison can be found mostly in the PR description. It explains the changes in details and for some of them the corresponding PR is linked. Also, the migration from v2 to v3 is included under the section Code migration considerations v2 -> v3.
How does this impact the Polkadot builders?
Very unlikely, but in case your parachain runtime is still using v0 or v1 versions, you must upgrade at least to v2 since v0 and v1 will no longer be supported.
This PR as well as its companion introduces transaction_version breaking changes, meaning that applying these changes is enough for transaction_version to be bumped.
Why is this change interesting for builders?
This new version of XCM brings a lot of new possibilites for interoperatbility and remote programmability between different consensus systems. This is a major step forward to have a more powerful and performant ecosystem.
XCM: ExpectTransactStatus instruction
Why is this important?
With XCM v3 a new register has been added to xcm-executor, Transact Status Register, it stores the status of the most recent attempt to use Transact instruction. The Transact Status Register is set according to the result of dispatching the call. The possible values are:
#[derive(Clone, Eq, PartialEq, Encode, Decode, Debug, TypeInfo, MaxEncodedLen)]
pub enum MaybeErrorCode {
Success,
Error(BoundedVec<u8, MaxDispatchErrorLen>),
TruncatedError(BoundedVec<u8, MaxDispatchErrorLen>),
}
This new instruction will compare the given status with the one held in the registry, returning an error, XcmError::ExpectationFalse, in case these two are not the same.
Why is this change interesting for builders?
Builders now can leverage this instruction to check if transact calls have been dispatched correctly.
Does this change have a related companion PR?
Cumulus companion PR: Companion for paritytech/polkadot#6578 by KiChjang · Pull Request #2112 · paritytech/cumulus · GitHub
[Fix] CountedMap::set now takes Counter into account
Why is this change interesting for builders?
A CountedStorageMap is a wrapper around StorageMap and has a counter of type StorageValue<Value=u32>, which tracks how many items are present in map. When we insert or remove any item from map, this counter is updated.
At the moment, when we set the item in map, this counter is not being updated.
// Current Behaviour
#[pallet::storage]
pub type CountedMap<T> = CountedStorageMap<_, Blake2_128Concat, u8, u16>;
#[test]
fn counted_map_works() {
new_test_ext().execute_with(|| {
CountedMap::<Test>::set(3, Some(100));
assert_eq!(CountedMap::<Test>::count(), 1);
CountedMap::<Test>::set(3, None);
// This will fail.
assert_eq!(CountedMap::<Test>::count(), 0);
})
}
This PR fixes this issue. Counter will be updated when we set an item.
How does this impacts the Polkadot builders?
This PR does not provide any migration. There is a separate issue provided for this. If you are using CountedStorageMap::set, your stored count and the actual number of elements in the map will differ. It is recommended to call initialize_counter.
Related Issues: CountedMap isn't updating the count on `set`. · Issue #13212 · paritytech/substrate · GitHub
Add Proof Size to Weight Output
As stated in the PR: This updates the frame-benchmarking-cli to output estimated Proof-of-Validity sizes which are needed for chromatic weights.
Integration guide (this can happen at your own pace):
- If you have a custom weight template; update it according to the changes in this MR.
- Re-benchmark all your pallets to generate new weights.
- Ensure that the outputted proof sizes will not overflow your block limit for important tasks like democracy.
THIS IS IMPORTANT TO NOT STALL YOUR CHAIN
Runtime upgrades itself should still work since they run in on_initialize which is mandatory and thereby not weight limited.
One of the most remarkable features described in this change is the chance to influence the (benchmark, storage-key) level of the PoV calculation with the new pov_mode attribute. This new attribute can be set to Meassured, MaxEncodedLen or Ignored:
#[pov_mode = Measured]
my_benchmark_with_length_witness {
}
#[pov_mode = MaxEncodedLen {
// Use MaxEncodedLen for all except Preimage::PreimageFor which uses Measured instead.
Preimage::PreimageFor: Measured
// ...
}]
service_task_fetched {}
Medium Impact
Scheduler is already at V4
PR: Scheduler is already at V4 by ggwpez · Pull Request #13105 · paritytech/substrate · GitHub
Why is this change interesting for builders?
As a part of this PR, some refactoring and new changes were introduced in Scheduler pallet along with migrations. However scheduler code version was not changed. This PR changes the version from V3 to V4.
How does this impacts the Polkadot builders?
If you have configured and deployed scheduler genesis without this patch, you need to set scheduler code version manually from 3 to 4 and not to use v3::MigrateToV4.
Fix the storage_size/state_getStorageSize RPC call
Why is this important?
This PR introduces a breaking change to storage_size RPC call. Allowing unbound execution time only when unsafe RPCs is enabled, otherwise a 30 second timeout would prevent this from running forever. If the storage_size call doesn’t return a result within this time an error is returned.
Does this change have a related companion PR?
Polkadot companion PR #6570
Correct arithmetical semantic of PerDispatchClass
Why is this important?
This PR introcues breaking changes to PerDispatchClass implementation. The following public functions have been renamed:
-
addrenamed toaccrue -
checked_addrenamed tochecked_accrue -
subrenamed toreduce
The following is introduced to sp_weights::Weight:
/// Construct [`Weight`] from the same weight for all parts.
pub const fn from_all(value: u64) -> Self {
Self { ref_time: value, proof_size: value }
}
/// Reduce [`Weight`] by `amount` via saturating subtraction.
pub fn saturating_reduce(&mut self, amount: Self) {
*self = self.saturating_sub(amount);
}
/// Try to increase `self` by `amount` via checked addition.
pub fn checked_accrue(&mut self, amount: Self) -> Option<()> {
self.checked_add(&amount).map(|new_self| *self = new_self)
}
/// Try to reduce `self` by `amount` via checked subtraction.
pub fn checked_reduce(&mut self, amount: Self) -> Option<()> {
self.checked_sub(&amount).map(|new_self| *self = new_self)
}
And implements From<u64> and From<(u64, u64)> for Weight in testing environments, meaning that the implementations are decorated with:
#[cfg(any(test, feature = "std", feature = "runtime-benchmarks"))]
How does this impacts the Polkadot builders?
In case you or your team are using the renamed functions of PerDispatchClass implementation, please apply the relevant changes to avoid errors.
XCM: Add HRMP to SafeCallFilter
PR: https://github.com/paritytech/polkadot/pull/6606
Associated issue: NoPermission · Issue #6602 · paritytech/polkadot · GitHub
This change is whitelisting the HRMP and Registrar pallets on the SafeCallFilter set on the relay chains. These new rules let the whitelisted calls from both pallets to be executed on the relay chains through the Transact XCM instruction.
In case you need more information about the SafeCallFilter you can refer to this PR Introduce whitelist for Transact and limit UMP processing to 10 messages per block by KiChjang · Pull Request #6280 · paritytech/polkadot · GitHub.
Low Impact
NIS should retain funds in reserve
PR: https://github.com/paritytech/substrate/pull/12928
Why is this change interesting for builders?
NIS pallet allows for accounts to auction for being frozen and receive open-ended inflation-protection in return. Learn about the pallet here. This PR introduces the following changes:
- Rename
thawtothaw_private - Add new dispatchable functions
communifyandprivatizeto create/burn fungible counterparts - Add new dispatchable function
thaw_communalto reduce or remove an outstanding receipt - A new config type
ReserveId
How does this impacts the Polkadot builders?
If you are using NIS pallet, you may have to add this new config type in runtime. For example:
parameter_types! {
pub const NisReserveId: [u8; 8] = *b"py/nis ";
}
impl pallet_nis::Config for Runtime {
--- Other Config types ---
type ReserveId = NisReserveId;
}
Does this change have a related companion PR?
Polkadot companion PR #6569
[NFTs] Track item’s metadata depositor
Why is this change interesting for builders?
In the previous release-v0.9.37, a new pallet (pallet-nfts) has been introduced with some advanced NFT features.
As a part of this PR, changes have been made regarding the reserved item’s metadata deposit to track whether the deposit has been made by the owner or a different account.
Enable treasury.spend by Root origin for Polkadot network before Gov2
Why is this important?
There could be scenarios where it is not feasible for anyone to pay the proposal deposit for a treasury expense. This PR allows Root to call treasury.spend and avoid such a situation by changing the definition of SpendOrigin in the Polkadot runtime to:
type SpendOrigin = frame_system::EnsureRootWithSuccess<AccountId, RootSpendOriginMaxAmount>;
Add WeightToFee and LengthToFee impls to transaction-payment Runtime API
Why is this change interesting for builders?
Transaction Payment pallet has two methods length_to_fee to compute the length portion of a fee and weight_to_fee to compute the unadjusted portion of the weight fee. At the moment, these functions are not available in the RuntimeApi so they can not be accessed. As a part of this PR, these two methods have been added to the runtime API.
How does this impacts the Polkadot builders?
If you are using pallet_transaction_payment_rpc_runtime_api, you will face below compilation error:
error[E0046]: not all trait items implemented, missing: `query_weight_to_fee`, `query_length_to_fee`
You have to implement both functions in your runtime:
impl pallet_transaction_payment_rpc_runtime_api::TransactionPaymentApi<Block, Balance> for Runtime {
--- Other Code ---
fn query_weight_to_fee(weight: Weight) -> Balance {
TransactionPayment::weight_to_fee(weight)
}
fn query_length_to_fee(length: u32) -> Balance {
TransactionPayment::length_to_fee(length)
}
}
impl pallet_transaction_payment_rpc_runtime_api::TransactionPaymentCallApi<Block, Balance, RuntimeCall> for Runtime {
--- Other Code ---
fn query_weight_to_fee(weight: Weight) -> Balance {
TransactionPayment::weight_to_fee(weight)
}
fn query_length_to_fee(length: u32) -> Balance {
TransactionPayment::length_to_fee(length)
}
}
For more details, please refer to companion PRs.
Does this change have a related companion PR?
Polkadot companion PR #6536
Cumulus companion PR #2073
migrate new benchmarking syntax from frame_support::benchmarking to frame_benchmarking::v2
PR: https://github.com/paritytech/substrate/pull/13235
This migrates all of the new proc-macro-based benchmarking code to live under frame_benchmarking::v2.
This is a non-breaking change and is part of a larger umbrella issue #13132 to bring about benchmarking v2.
This also migrates the old decl-macro-based benchmarking syntax from frame_benchmarking to frame_benchmarking::v1, but keeps a public export of v1::* at the root of frame_benchmarking, allowing existing benchmarking code to still compile without any changes. Sometime in the distant future we would remove this export when the v1 syntax is officially deprecated.
v3::Junction supports small (32-byte max) “vecs”
This change prevents that the junction GeneralKey does not exceed the 32 bytes and keeps the compatibility for chains using this junction with different XCM versions. In case it exceeds, it returns an error.
Your friendly neighbourhood Polkadot Release Analysis team,
@hectorb @alejandro @ayush.mishra @bruno