I haven’t posted in a while, but there has been are ton of new changes to PolkaVM, so I thought I’ll update you guys on what’s been going on! Here are some of the highlights.
It can run DOOM!
Yes, we now have a DOOM port which runs fully within the VM at full speed!
(This is an actual screenshot I took of DOOM running under PolkaVM right now.)
You can play it yourself here. It even runs with full audio support, and you can finish the whole game like this!
As a bonus, I also hooked it up to our CI for testing, so now we’re probably the only VM in existence which runs DOOM on its CI. Gotta get rid of all of those demons bugs!
A generic sandbox
Our Linux sandbox is blazingly fast and very secure. Unfortunately it’s not portable, and not everyone uses Linux. So now we also have a generic wasmtime-like sandbox which should be portable across all operating system. It is, of course, not as secure, but at least should run almost at full speed!
This is currently only supported on macOS.
A benchmarking framework
I’ve added a benchmarking framework where we can trivially add new benchmarks and new virtual machines to cross-compare the performance.
We currently have two benchmarks: Pinky, which is my own NES emulator, and prime sieve. I will be adding more (suggestions welcome!).
We have support for benchmarking the following VMs:
- PolkaVM (obviously)
- Wasmtime
- Wasmer
- Wasmi
- Wazero
- Wasm3
- CKB VM (This is another blockchain RISC-V VM!)
- PVF Executor
- Native (runs the benchmark as native code)
Major performance improvements
I’ve also done a ton of performance improvements. I’ll let the benchmark results speak for themselves here (sorted from the fastest to the slowest; this is the Pinky benchmark that you can find in the repository, but the other benchmark gives similar results):
- Native: 3.774ms
- PolkaVM: 6.033ms
- Wasmtime: 6.054ms
- Wasmer (singlepass): 14.307ms
- Wazero: 21.621ms
- Wasm3: 109ms
- Wasmi (register): 114ms
- CKB VM: 140ms
- Wasmi (stack): 212ms
Yes, we are as fast as wasmtime now, while still being being orders of magnitude simpler, more sandboxed, faster to compile, and with guaranteed linear time compilation. And there are still plenty of avenues left to optimize, so I believe I might be able to improve upon this even further.
Gas metering
Gas metering was implemented and is working! Here are some performance numbers:
- PolkaVM (no gas): 6033us
- PolkaVM (async gas): 6350us
- PolkaVM (sync gas): 7276us
- Wasmtime (no gas): 6054us
- Wasmtime (sync gas): 8624us
- Wasmtime (epoch interruption): 7633us
The asynchronous gas metering is almost free and only slows down this benchmark by ~5%, while synchronous gas metering slows it down by ~20%.
For comparison, for Wasmtime its synchronous gas metering slows the benchmark down by ~42% (so twice as much as our implementation!), and even its epoch interruption based metering (which is closer to our asynchronous gas metering in how it works, although not exactly) slows the program down by ~26%, more than even our fully synchronous metering!
You might ask here - what’s the difference between synchronous and asynchronous gas metering? Well, there’s essentially only a single difference: when exactly the gas is checked. Let me explain.
Asynchronous gas metering will only periodically and asynchronously check whether we still have gas remaining while the program is running, so the program can run slightly longer than it would otherwise, and the exact point when it is interrupted is not deterministic, but whether the computation as a whole finishes under a given gas limit will still be strictly enforced and deterministic. On the other hand the synchronous gas metering will always check the remaining gas immediately, and interrupt the execution right away.
So if what you’re doing is transactional and doesn’t have any effect unless it finishes whole anyway - that’s great, now you can use the faster asynchronous metering! And in fact most use cases (like smart contracts!) don’t actually need full synchronous gas metering and can get away with the cheaper asynchronous one.
Thanks for coming to my TED talk
And that’s all of the exciting highlights!
So what happens now? Well, considering we’re now as fast as wasmtime (of course remember that this is not going to be true in every case, depending on the exact benchmark and the hardware, at least not yet) we’re now considering moving everything to PolkaVM, including the runtimes and the PVFs!
Although this is of course not going to happen tomorrow nor next month, and it still requires a ton of work, we now do have a lot of evidence that this can be done with little to no compromises. But more on that in a future Tales from the PolkaVM post! Stay tuned, same time, same channel!