Upcoming Metadata V16 - Features to include in V16

Metadata in Substrate is a scale-encoded blob containing essential information to interact with Substrate nodes generically.

Metadata V15

Metadata V15 was stabilized nearly a year ago. This version includes support for Runtime APIs, pallet documentation, and more. Additionally, it provides a mechanism to upgrade metadata in the future through the following runtime APIs:

An upgrade to Metadata V16 is planned for the upcoming quarters.

Requested Features for V16

This is the tracking issue for requested features: Metadata V16: Features to include in V16:

We would love to hear feedback from the ecosystem on use cases and features to be included in Metadata V16. Please share any specific metadata features you anticipate, issues with the current metadata or improvements. Any suggestions are welcomed :smiley:

6 Likes

Not sure if it’s within the scope of what the metadata could expose, but at KILT we have been wondering about how we can re-use more code between our node and our clients (our SDK). Runtime APIs solve this halfway, as they allow code to be implemented on the node and being invoked by clients. Would there be a possibility for the metadata to expose WASM blobs that can be executed locally by a client whenever needed, without needing to involve at least one round-trip and load the server with something that can be done locally by the client? If there’s an existing solution to this problem, I’d be more than happy to know about them!

The metadata is currently exposing only type information. This is something we have thought about but not articulated in an issue so far.

Having the metadata express code, even as a WASM blob, would be beneficial. For example, in subxt we rely on the Config::Hashing type to determine the hashing used to communicate with the substrate-based node. We’ll be able to be generic over this with the addition of the associated types of config traits in the metadata. However, if the chain uses a custom in-house built hasher, we won’t be able to generate the code for that.

I think expressing WASM code like fn sing(message: &[u8], raw_private_key: &[u8]) -> Vec<u8> becomes a bit problematic since there is no way to know programmatically that the WASM blob won’t expose in the output the private key. This needs a bit more thought indeed, but generally I think we could expose eventually a WASM blob.

One workaround that I can think of would be to express the WASM blob as a CustomValueMetadata in the current metadata V15. Then, on the client side decode and instantiate a WASM instance from that blob of data.

I’m curious, what type of functionality would you like to share between nodes and clients? :smiley:

I think expressing WASM code like fn sing(message: &[u8], raw_private_key: &[u8]) -> Vec<u8> becomes a bit problematic since there is no way to know programmatically that the WASM blob won’t expose in the output the private key. This needs a bit more thought indeed, but generally I think we could expose eventually a WASM blob.

So we can’t be sure the node serving the WASM blob is trusted not to serve malicious blobs? Or more of a doubt as to what should be allowed to be in a WASM blob?
Can I/we do anything to start the conversation and investigate if other people have similar needs? It looks that subxt could already benefit from such a solution, so it would already be a good argument to take some steps into trying to see how this could look like.

One workaround that I can think of would be to express the WASM blob as a CustomValueMetadata in the current metadata V15. Then, on the client side decode and instantiate a WASM instance from that blob of data.

Ok I was not aware this was a possibility. We will explore what we can do it with, thank you!

I’m curious, what type of functionality would you like to share between nodes and clients? :smiley:

A very stupid example is here for the chain part (kilt-node/runtimes/spiritnet/src/lib.rs at b0e6bf240db5137efdd58c8557f9ac6044d3ea06 · KILTprotocol/kilt-node · GitHub) and here for the SDK part (sdk-js/packages/did/src/DidDetails/FullDidDetails.ts at 23c5b045dc7d3d587aeca5f6650736bb1218130b · KILTprotocol/sdk-js · GitHub). We want to understand what DID key is required to authorise what extrinsic. So every time we add a new extrinsic or pallet, we potentially have to release a new version of our SDK. Having the possibility to fetch the latest implementation from the metadata would greatly reduce maintenance efforts.

I’ve created this issue in substrate to continue the discussions around the WASM blobs: https://github.com/paritytech/polkadot-sdk/issues/4714

So we can’t be sure the node serving the WASM blob is trusted not to serve malicious blobs? Or more of a doubt as to what should be allowed to be in a WASM blob?

The node could in theory alter the WASM blob to generate malicious behaviors. For example, if the metadata contains at some point the fn hash(private_key: &[u8]) -> Vec<u8> a malicious node could modify the code to pass through the private_key and leak it in the returned vector. Then, when the user submits the hash to the node, the node will gain access to the user’s private key.