This is a constant problem with your benchmark. You’re always trying to cheat your way in the benchmark for PolkaVM. The recompiler has a bytecode throughput. If you tweak the LLVM inline to get a vastly different blob size, the recompile process will of course be vast different. We can also tweak our LLVM inline parameter (and in PVM2 it’s much much easier) and get smaller sizes.
If you tweaked inline to get half the blob size (which we can also do) and still 40% slower, that means we’re 3x faster than PolkaVM. But we won’t claim those premature data.
It’s interesting that several months ago you told us that your interpreter is “unoptimized”. And now you claim your recompiler is “not fully optimized”.
The “simplified model” you mentioned is also incorrect. The current implemented “simplified model” in PolkaVM is the “naive” gas metering, which is far from a production-grade spec. You’re welcome to adopt the single-pass pipeline simulation gas metering we developed for JAR Chain.
But this is missing the tree for the forest. If you actually read the top forum post, what our benchmark primarily shows is not about PolkaVM vs JAVM. It’s true of course that JAVM is 2x faster than PolkaVM, and this has been an established facts for over a month. In fact, PolkaVM is so slow that we aren’t even using it as a target any more. (I’m sure if you adopt the optimizations in our old code pre-PVM2-migration, you can catch up on the single-vm benchmarks, but you’ll never catch up on our large multi-vm benchmarks – we have a much more optimized capability-based architecture that it’s not possible for you to match unless you’re open to a full overhaul of the JAM design.)
What we’re showing is, rather, the entire PVM ISA is unnecessary. If you read our benchmark data, we mainly compare in-house – our own JAVM (legacy PVM, but already 2x faster than PolkaVM) vs JAVM (PVM2). We get competitive results for JAVM (PVM2). There’s no significant benefits to the legacy PVM ISA. We can get comparable performance and binary size using mostly standard RISC-V ISA. There are enormous benefits following a standard instead of handroll things, and that’s the direction (PVM2) we’ll go with in JAR Chain.
This is honestly a trend we’ve been repeatedly seeing in Polkadot development. A lot of over-engineering without sufficient data to back it. Last year’s EVM debate was another example. Now in retrospect everyone will just look at the real-world data and laugh.