When I started building PAPI, one of my key goals was to make it easy for DApp developers to build applications that could seamlessly survive runtime upgrades.
To achieve this, I studied how other libraries like Subxt and Polkadot-JS approached the problem. What I found was… far from ideal. These tools just weren’t built to handle the complexity of runtime upgrades in a robust way.
I kept iterating, trying different approaches, but I hit several dead end… until @voliva joined the team! With his invaluable help, we made some critical breakthroughs that now make PAPI the only library capable of properly handling runtime upgrades. The most mind-blowing of these is the ability to detect fine-grained compatibility changes between upgrades. We presented this at Polkadot Decoded 2024.
However, we made a critical mistake: we assumed that solving the problem was enough. We thought others would naturally build on top of this work. But it’s clear now: we didn’t communicate the significance of our breakthroughs well enough. So in this post, I want to walk you through the problem, what we solved, and how PAPI can help DApps thrive in a world of frequent runtime upgrades.
Understanding Runtime Upgrades: Breaking vs Compatible Changes
Many runtime developers don’t realize what kinds of changes can break DApps. So let’s unpack it.
First, all on-chain interactions happen via SCALE-encoded data, which is not self-describing. That means consumers need metadata to interpret this data correctly.
Since Metadata v14, it’s possible to rely entirely on FRAME metadata for performing chain interactions. But there’s a catch: this metadata is auto-generated from the runtime code, meaning seemingly “harmless” changes (from the runtime dev’s perspective) can cause real issues for DApps.
Example: Imagine a runtime developer renames a struct field. The SCALE data stays the same, but the metadata changes… and that can break DApps relying on the old naming, because decoding logic now interprets it differently.
On the flip side, a change like upgrading a storage value from u64 to u128, despite changing the on-chain SCALE encoded values, will not be a breaking change for a DApp built with PAPI, since both are handled as bigint in TypeScript.
The key insight? It’s the metadata that determines DApp compatibility, not necessarily the raw SCALE encoding.
How PAPI Changes the Game
If you’re familiar with FRAME metadata, you might ask: “So, does any metadata change break compatibility?”
Not with PAPI.
Types can often be expressed differently in metadata but still produce identical TypeScript representations. For example:
Array<2, bool>andTuple<bool, bool>- Nested 1 element tuples that ended up being simplified as one type.
u16andu32produce the same TypeScript type (number)- Many other obscure cases…
PAPI can detect these as equivalent, even if the metadata differs drastically. As long as they produce the same TypeScript interfaces, PAPI can dynamically detect if the types are equivalent.
So, when comparing two runtimes, PAPI knows when APIs are actually identical, even if the metadata looks different.
Beyond Identical: Detecting Backward Compatibility
Things get trickier when dealing with compatible changes.
Take adding a field to a struct. Is that backwards-compatible?
Answer: It depends.
- If it’s an output field (e.g., from a storage read value), then yes, it’s fine.
- But if it’s an input field (e.g., part of a tx input), then no, unless it’s optional (i.e., an
Option). Otherwise, the encoder won’t know how to construct valid SCALE data.
PAPI accounts for all of this. It checks compatibility against the actual code the DApp developer used when deploying their DApp.
Yes, PAPI will even detect that an added Option field in a struct is backwards-compatible for inputs.
Enums: Where things get tricky
Enum changes can also have nuanced effects:
- Adding a variant to an input enum: backward-compatible. Old inputs still work.
- Removing a variant from an output enum: also backward-compatible. The consumer just won’t receive the removed variant.
But:
- Adding a variant to an output or removing a variant from an input can be risky.
These are what we call partially breaking changes.
PAPI handles these gracefully: it allows the interaction, but will throw an error only if the new variant is actually encountered at runtime, thus avoiding false negatives.
And Finally: Real Breaking Changes
Some changes are truly breaking:
- Renaming struct fields
- Changing the shape of types (e.g., tuple → struct, or struct → tuple)
- Changing a
u16to au128(TypeScript seesnumbervsbigint) - etc, etc
These will definitely cause problems if not handled correctly. So, we consider them breaking changes.
PAPI is built to detect all of these, providing devs with clear insight into what is or isn’t compatible. Our upgrade recipes and compatibility API are designed to help devs prepare for these changes before they break things in production.
Introducing: @polkadot-api/compare-runtimes and https://diff.papi.how
Even with PAPI, we noticed that runtime upgrades still catch people off guard, mainly because there’s no easy way to see what a new WASM actually changes. So, DApp developers are not aware that they have to prepare their DApps for an incoming runtime upgrade.
That why this last weekend, I built @polkadot-api/compare-runtimes: a simple but powerful tool to compare any two runtimes. It’s built on top of the lower-level @polkadot-api/metadata-compatibility, authored by @voliva and used by PAPI’s compatibility logic.
To demonstrate this, I also created a UI for comparing runtimes. It lets you:
- See what changes a new WASM will introduce
- Compare runtimes from two different chains (e.g., Polkadot vs Kusama)
- Compare how a runtime has evolved over time (e.g., between blocks)
- …and more
Note: This first version of the UI does not yet explain why an interaction is marked as incompatible or partially compatible, only that it is. We will work on enhancing the tool so that in the future it will also provide detailed reasoning for each incompatibility, helping developers quickly understand what exactly changed and why it matters.
It’s just an MVP for now, but it already shows how powerful this kind of tooling can be. Our hope is to collaborate with Parity and the Polkadot Fellowship to integrate this into ecosystem release workflows.
Wrapping Up
PAPI is built with DApp resilience in mind. We’ve put an enormous amount of effort into helping developers build applications that survive runtime upgrades—not just theoretically, but practically.
If you’re building on Polkadot, now’s the time to start thinking about upgrade resilience. And PAPI is here to help.
Let’s keep the conversation going, and if you’re interested in integrating this tooling or want to collaborate, don’t hesitate to reach out.
