Metadata in Substrate is a scale-encoded blob containing essential information to interact with Substrate nodes generically.
Metadata V15
Metadata V15 was stabilized nearly a year ago. This version includes support for Runtime APIs, pallet documentation, and more. Additionally, it provides a mechanism to upgrade metadata in the future through the following runtime APIs:
Enrich metadata with associated types of config traits - Expose the associated types of config traits for each pallet generically. This is particularly useful for tools like subxt to eliminate the need for hard-coded configurations when communicating with chains.
Extend metadata with deprecation - Introduce a deprecation flag, potentially including the date and reason for deprecation, to announce when a particular call, storage item, or constant is deprecated
Subscribe-able runtime apis - Enable subscription to accessed storage items and prefixes during runtime execution, which might necessitate metadata indicating which runtime calls are subscribable
We would love to hear feedback from the ecosystem on use cases and features to be included in Metadata V16. Please share any specific metadata features you anticipate, issues with the current metadata or improvements. Any suggestions are welcomed
Not sure if it’s within the scope of what the metadata could expose, but at KILT we have been wondering about how we can re-use more code between our node and our clients (our SDK). Runtime APIs solve this halfway, as they allow code to be implemented on the node and being invoked by clients. Would there be a possibility for the metadata to expose WASM blobs that can be executed locally by a client whenever needed, without needing to involve at least one round-trip and load the server with something that can be done locally by the client? If there’s an existing solution to this problem, I’d be more than happy to know about them!
The metadata is currently exposing only type information. This is something we have thought about but not articulated in an issue so far.
Having the metadata express code, even as a WASM blob, would be beneficial. For example, in subxt we rely on the Config::Hashing type to determine the hashing used to communicate with the substrate-based node. We’ll be able to be generic over this with the addition of the associated types of config traits in the metadata. However, if the chain uses a custom in-house built hasher, we won’t be able to generate the code for that.
I think expressing WASM code like fn sing(message: &[u8], raw_private_key: &[u8]) -> Vec<u8> becomes a bit problematic since there is no way to know programmatically that the WASM blob won’t expose in the output the private key. This needs a bit more thought indeed, but generally I think we could expose eventually a WASM blob.
One workaround that I can think of would be to express the WASM blob as a CustomValueMetadata in the current metadata V15. Then, on the client side decode and instantiate a WASM instance from that blob of data.
I’m curious, what type of functionality would you like to share between nodes and clients?
I think expressing WASM code like fn sing(message: &[u8], raw_private_key: &[u8]) -> Vec<u8> becomes a bit problematic since there is no way to know programmatically that the WASM blob won’t expose in the output the private key. This needs a bit more thought indeed, but generally I think we could expose eventually a WASM blob.
So we can’t be sure the node serving the WASM blob is trusted not to serve malicious blobs? Or more of a doubt as to what should be allowed to be in a WASM blob?
Can I/we do anything to start the conversation and investigate if other people have similar needs? It looks that subxt could already benefit from such a solution, so it would already be a good argument to take some steps into trying to see how this could look like.
One workaround that I can think of would be to express the WASM blob as a CustomValueMetadata in the current metadata V15. Then, on the client side decode and instantiate a WASM instance from that blob of data.
Ok I was not aware this was a possibility. We will explore what we can do it with, thank you!
I’m curious, what type of functionality would you like to share between nodes and clients?
So we can’t be sure the node serving the WASM blob is trusted not to serve malicious blobs? Or more of a doubt as to what should be allowed to be in a WASM blob?
The node could in theory alter the WASM blob to generate malicious behaviors. For example, if the metadata contains at some point the fn hash(private_key: &[u8]) -> Vec<u8> a malicious node could modify the code to pass through the private_key and leak it in the returned vector. Then, when the user submits the hash to the node, the node will gain access to the user’s private key.
