PVF executor environment versioning for dummies
To execute a PVF, we’re currently using Wasmtime, which is a compiler and a runtime in one bottle. When a new version of Wasmtime is released, we update the Wasmtime version in our node implementation. Unfortunately, that is not a deterministic behavior: if a PVF is successfully executed with Wasmtime version N, nobody guarantees it can be successfully executed with Wasmtime version N+1. At the moment, we’re lucky. Nothing broke up horribly because of those upgrades. But we cannot just hope to be infinitely fortunate, so we want to introduce PVF executor environment versioning. That is, we want to carry several versions of Wasmtime along with the Polkadot node and to be able to execute a candidate produced in some session with the exact version of Wasmtime that was effective in that session. That should make things deterministic and infallible.
Github issue for those who want to go beyond tl;dr: Explicit versioning of PVF execution environment · Issue #4212 · paritytech/polkadot · GitHub
The problem with the UX
I always considered the versioning issue a purely technical problem, but in a discussion with @rphmeier, some UX concerns were brought up. If we want to carry several versions of Wasmtime with our Polkadot node implementation, it basically means “several files”. Not talking right now about whether they would be binaries of dynamically-linked libraries, but generally, the Polkadot node ceases to be a single-binary distribution. So we should carry and manage some auxiliary files to make the versioning happen. That would affect the node operator’s user experience significantly. So I’m creating this topic to discuss the implications and possible ways to handle them.
From my perspective, there are several ways to proceed:
- Create a distribution as an archive with several files inside. They may reside in a single directory or a directory tree. Documentation should contain obligations for the node owner to preserve the file structure/directory tree. It’s the easiest choice for us but the most annoying for node owners.
- Create a single-binary installer that properly lays out the needed structure and upgrades an old version to a new one. The most common way in the industry, still prone to errors due to space management procedures, etc.
- Put some effort into keeping the distribution single-binary. That is technically possible. ELF binaries may contain sections not loaded by default; we can load them on demand and link against them in runtime, they may represent different executor environment versions, and everyone would be happy. But that requires some currently unknown amount of work. It’s the best choice for node owners but the most demanding for us.