Announcing the Versioning of Pallet Revive’s Runtime API

Tech Talk
1

Over the past few months, we have been working on versioning pallet-revive’s runtime API to give clients stronger guarantees for backward and forward compatibility across runtime upgrades. pallet-revive exposes this interface through the ReviveApi trait. A runtime upgrade should not force every client to coordinate an update alongside it.

Before this work, changing a function’s arguments or returned data could break clients after a runtime upgrade. A client built against the old signature might no longer be able to encode the arguments expected by the runtime or decode the data it returns. And adding another function avoided that breakage, but left the interface with names such as eth_transact_with_config as well as an ever expanding set of runtime API functions.

We have now added a versioned counterpart ending in _versioned for every unversioned runtime API function. Each versioned function pairs one input payload enum with one output payload enum. Both enums use consecutive variants beginning at V1; below, Vn means variant n. The input variant determines the request format and, when the call succeeds, the matching output format.

Guarantees for clients

When a payload version is released, we make the following guarantees:

  • The version_declarations() runtime API function on ReviveApi reports a maximum N for each versioned function. This advertises payload versions 1 through N. For any n from 1 through N, a successful call with a Vn input is guaranteed to return a Vn output. This remains true after a later runtime upgrade whenever that runtime reports a maximum of at least n.
    • The version_declarations runtime API function returns a map where the key is the versioned runtime API function name (e.g., eth_transact_versioned) and the value is a u8 of the currently maximum supported payload version. When using subxt’s type generation, the type of this would be a Vec<(String, u8)> which users can then collect into a HashMap<String, u8>.
  • The SCALE schema of a released payload variant is fixed. Its fields keep the same order and types. Their meaning is also fixed. The same commitment applies to the pallet-revive-owned types nested inside that payload.
  • Input and output variants always advance as a pair. When the client-facing format or intended behavior changes, we add the next matching pair, even if only one side needs a new shape. Correctness and security fixes may still repair the behavior of an existing version.
  • Clients choose a payload version they understand within the target runtime’s advertised range. A runtime upgrade that adds a newer version does not force a client to adopt it.

An existing V1 client can keep sending V1 after a runtime adds V2. For a historical call, read the declarations at the older block and choose an understood version, provided that block exposes the versioned interface.

The versioned payload enums and pallet-revive-owned types nested inside them live in the pallet-revive-types crate, separate from pallet-revive’s internal execution types. This lets us change the internals without changing a released payload format.

Note that at present the error types returned by the runtime API functions are not versioned. However, all payloads for the success case are fully versioned.

What shipped

The versioned interface is part of ReviveApi version 2. version_declarations() reports the maximum payload version supported by each versioned function in the runtime at that block.

Most names map one-to-one where the versioned variants have the same name with a _versioned postfix. Below are the only exceptions to this rule:

  • eth_transact and eth_transact_with_config have been merged into eth_transact_versioned.
  • trace_call and trace_call_with_config have been merged into trace_call_versioned.
  • get_storage and get_storage_var_key have been merged into get_storage_versioned.

Each V1 input/output pair preserves the behavior and returned data of the unversioned path it represents. The unversioned functions remain available and are marked as deprecated.

The implementation is merged in polkadot-sdk. A network gets the versioned interface when it ships a runtime containing these changes, so support depends on the block being queried.

Calling the API

  1. Get the ReviveApi version from the metadata at the block you are calling.
  2. Below version 2, use the unversioned functions. At version 2 or later, call version_declarations() at that block.
  3. Choose a payload version your client understands that is no higher than the declared maximum. Call the function’s _versioned counterpart with that input variant. When the call succeeds, a Vn input produces a Vn output.

The pseudo-code below shows how the versioned runtime API functions can be called.

let version_declarations = runtime_api
    .call(version_declarations_payload)
    .await?;
let rtn = match version_declarations.get("function_we_want_to_call_versioned") {
    // This example client doesn't care about the version, it just wants
    // to use v1 regardless of what the current highest version declared is.
    _ => {
        let rtn = runtime_api.call(
            // Notice we used a V1 input here.
            FunctionWeWantToCallVersionedInputPayload::V1(
                FunctionWeWantToCallInputPayloadV1 { arg1: val, arg2: val }
            )
        )
        .await?;
				
        // Notice that we unwrap the output as a v1 output since it must be a
        // v1 output due to the version of the input
        let FunctionWeWantToCallVersionedOutputPayload::V1(rtn) = rtn
        else {
            unreachable!("Impossible to use v1 input and not get v1 output; qed")
        }
        rtn
    }
};

Quick FAQ

  • What changes for existing unversioned clients?

    Existing clients continue to function as normal since no changes or removals were made to pallet-revive’s runtime API. Additionally, after this release clients using the unversioned runtime API functions will get the same stability guarantees we offer to the versioned variants since all unversioned functions are internally routed to the versioned functions.

  • How does V1 compare with an unversioned call?

    The operation keeps the same behavior and returned data. The client encodes the arguments in the V1 input enum variant and, when the call succeeds, decodes the V1 output enum variant.

  • How will an existing versioned function evolve?

    When its client-facing payload format or intended behavior changes, we add matching input and output variants. Earlier variants remain unchanged. Clients may keep using any version they understand that the target runtime supports.

  • How do I know what set of functionality the team has shipped with a new version of a specific runtime API function?

    We are currently working on getting appropriate documentation for this and deciding on discoverability and will share more details on this when it’s ready.

Implementation links

For a concrete client implementation, see version_aware_runtime_api.rs. We should note that this implementation was ideal for the eth-rpc but might be too much or too little for other clients. The requirements of your client dictates how much of the logic in this reference implementation you may want to implement.

The ReviveApi trait and implementation define the interface. Its versioned payload enums and nested pallet-revive-owned types are in the pallet-revive-types crate source.

Implementation PRs: #12244, #12037, #11957, #12447, #12522, #12536, and #12548.

5 Likes