Weight V2 Discussion and Updates

Tech Talk
1

Hey all,

Wanted to create this thread to publicly track and discuss the ongoing effort to migrate to a multi-dimensional weight, which we have named “Weight v2”.

At a high level, Weight is supposed to represent the amount of resources consumed by the blockchain when executing state transitions. Every blockchain is limited on a few factors:

  • Computational Time
  • Memory Usage
  • Storage Usage

At a high level, our current weight system is a single u64 value which measures the computational time to execute some runtime logic, in the Wasm environment, using some reference hardware. These weights are calculated using the benchmarking system.

You can watch this video which is a deep dive into weights and benchmarking for v1: Substrate Seminar: Benchmarking for your Substrate Pallet - YouTube

Weight v1 worked fine for solo-chains, where the only practical limitation was the block time. However, with the parachains protocol, another limit which has been introduced is the proof size, which is needed to execute the proof of validity function by validators on Polkadot.

To satisfy this, we need to update the Weight type to not only represent computational time, but also proof size, to ensure that parachains do not produce blocks which are invalid in the eyes of Polkadot.

pub struct Weight {
	/// The computational time used to execute some logic based on reference hardware.
	ref_time: u64,
	/// The size of the proof needed to execute some logic.
	proof_size: u64,
}

The first step will be merged soon which simply updates Weight from a type alias to a single field struct:

As you can see, this will be a painful breaking change, but quite easy to fix, with little chance of making errors as the changes to fix this logic cannot really introduce logical changes.

There will be a few more intermediate steps required to get to the final Weight v2, and I will try my best to keep everyone updated here, as well as on GitHub.

9 Likes
2

Thank you for sharing. Weight V2 will definitely introduce some challenges in Frontier.

Currently users sending ethereum-like transactions provide a gas limit, and we have a conversion GasToWeight to provide the Weight V1 hint to the extrinsic. But because the number of storage reads / writes in the EVM cannot be known in advance, is not possible to calculate the proof size for it.

Some ideas that came up in our team while briefly talking about it:

  • A rough conversion of highest possible proof size for the provided gas limit. Meaning there is only a number of storage writes one could possibly do for a given gas limit. This would be a overestimation of the proof size.
  • Record an “estimate gas” result for a ethereum-like transaction before submitting it to the transaction pool. This might be not realistic at all, as any estimation will be done over a state in which it’s not guaranteed to follow the same EVM code path than when applying the actual extrinsic, so most likely not the same storage access and different proof size at the end.

Can you think of any alternative workarounds for this problem?

Also, unrelated to this I have a question, how the new Weight V2 will be integrated in XCM Transact instructions.

3

Generally, Weight v2 will be very challenging for EVM because EVM itself has no concept of multi-dimensional gas. It is just computation time.

This is honestly one of the reasons why bootstrapping EVM onto Substrate is more of a “fun and temporary” solution than anything that will last for many generations. If we really want EVM to be compatible with Substrate and Polkadot, we probably need to push multi-dimensional gas to the EVM itself, and have it be part of the official vm.

Beyond that, probably you can make some kind of side-crate which keeps metadata about the size of each storage item in the EVM, and you check that before actually reading the storage item… but I am just spitballing ideas.

1 Like
4

The current work for Weights V2 can be tracked with this tracking issue:

This does not encapsulate all the work, but will be most of the steps we need to take before everyone can start migrating their code in parallel.

5

Acala EVM does have built in mandatory storage limit field for all the EVM transactions so we can somewhat support this.

However, the current storage limit is meant to be used for deposit fee for storage writes, not exactly the PoV size. They are somewhat linked as any storage write will result increase of PoV. However storage delete will result a storage deposit refund so which is not a thing in PoV case so there are still some considerations required on this area.

My main issue with combination of XCM + Weight V2 + Acala EVM contracts call is that we need to charge compute fee (gas), storage deposit and later PoV fee for all smart contract calls in Acala. XCM only offer the ability to buy weight (compute fee + PoV fee). There is no instruction about storage deposit. Ideally, we want to use 3D weight that’s compute + PoV + net storage writes for XCM fee. Looking at current Weight V2 implementation, I don’t think that’s possible.

6

The ask here seems reasonable to me at a high level.

I will state that Weight != Fee. Weight should be used as a representation of a blockchains limits. Generally speaking, blockchains are not yet limited by the amount of data they write, but perhaps this should be a constraint.

If we should add this as a third dimension to weight (and perhaps memory usage as a 4th), it should be easy enough given the way that weight is structured.