"Almost compatibility" is unsafe

Tech Talk
1

First of all, I do want to congrats the Polkadot community and especially Parity for the significant progress in the Polkadot Hub plan. A major part of this plan is the new Ethereum compatibility layer revive / pallet-revive, running Ethereum-alike contracts on PolkaVM.

With the recent discussions on the issues of Ethereum and especially Ethereum compatibility, I do want to again provide my perspective on this. I want to first establish two important premises, and then I’ll explain in more detail what I mean by the title that “almost compatibility” is unsafe.

Premises

Audit and testing are the most significant cost of Web3 development

The part that takes most time in Web3 development is not the coding itself, but the significant amount of audit and testing that follows it to make sure things are safe. It may take only a week or two to write an EVM engine, but it takes a year or two before one is confident to use it in production for Ethereum mainnet. Polkadot makes things “upgradable”. For many parachain launches, “sudo” is also available. But this does not change the situation much, because an adversary can always act faster than you can react.

Safety is important. Millions of dollars of real values are at stake.

EVM is subtle

EVM is the Javascript of Web3. It’s not a well-designed language, but it’s used everywhere. Its stack-based design may be simple, but its complexity in terms of the many ways that you can write unsafe things should not be underestimated.

The important message I want to convey here is that its unsafety is usually really subtle, especially if one tries to make changes to the specification of EVM. You may already have a million “happy paths”, but it only takes one “sad path” for a disaster to happen.

As an exercise for the reader and as an example for what I mean above, take a look at EIP-1283, and try to figure it out yourself why the specification is unsafe, and why eventually EIP-2200 was instead deployed. This is only gas changes. Issues like this happen all the time in Ethereum history.

Full compatibility, almost compatibility, some compatibility

When dealing with Ethereum compatibility, there are three approaches
you can take:

  • Full compatibility: replicate everything in EVM, make sure things always run the same. All code run.
  • Almost compatibility: try to replicate everything in EVM, make subtle changes when it’s not possible. Most code run.
  • Some compatibility: replicate EVM “in spirit”, but generally require developers to rewrite their contracts. Off-chain dapps can be re-used by RPC compatibility layer.

Polkadot’s pallet-evm, Solana’s Neom EVM are examples of the “full compatibility” approach. revive is an example of the “almost compatibility” approach. solang is an example of the “some compatibility” approach.

Audits and testings are the most significant cost, but “almost compatibility” means it must start again from scratch

If everything is the same (“full compatibility”), then past audits and testings can generally be reused. However, because EVM is subtle, “most things are the same” is no better than “some things are the same”. The audits will have to be re-done again. The significant difference in gas cost schema, the changed semantics of CALL and CREATE, and the new design of precompiles, mean that there are a few more attack vectors opened (while a few old attack vectors closed).

Of course, the first implications that every reader can get from the above, is that “full compatibility” has an advantage on this over “almost compatibility” and “some compatibility”. But the more important message I want to convey here is that “almost compatibility” has no advantage over “some compatibility”. If audits and testings are the most significant cost, then for both cases this cost will have to be paid again in full. “Almost compatibility” may help a few new developers get up to speed quicker and may get us a few more hackathons, but it does not help with real-world production-ready deployments.

EVM is subtle, therefore some “forced” rewrite is necessary

Because of the many ways that EVM can fail and all the subtle details in the EVM specification, in the case of “almost compatibility” and “some compatibility”, you want to force the smart contract developers to pay attention. If not everything is the same (not “full compatibility”), allowing developers to deploy contracts unchanged is generally not a good idea. A contract may work perfectly fine on Ethereum, but it may be another multisig hack on Polkadot, and if that happens, core devs will be blamed (because it’ll be benchmarked against Ethereum). It’s therefore important to “force” the developer to pay some attention to properly “adopt” their contract for Polkadot.

The sweet spot is to have something that is of familiar syntax, but also different enough that nothing should directly compile. The smart contract itself is usually a tiny part of the overall dapps, and with a familiar syntax, a rewrite is not hard to do (and as discussed above, the audit and testing has to be re-done, anyway). “Almost compatibility” has no advantage over “some compatibility”, and one could argue that it’s rather worse because it encourages bad practices (blindly deploy) and makes things unsafe.

Ethereum is a moving target, and thus the maintenance burden of “almost compatibility” is particularly high

You may or may not like the way Ethereum AllCoreDevs governance is structured, and you may think things are moving slow, but hard fork improvements are still deployed every year. New Solidity/YUL versions generally follow Ethereum upgrades because it’s developed for it. The maintenance burden of “almost compatibility” is thus particularly high.

For “full compatibility”, we can generally take the code already written for Ethereum. For “some compatibility”, we are on our own schedule and because we don’t expect things on Ethereum to directly compile, we can choose to take whatever features we like or not. However, for “almost compatibility”, we can only choose one of the two hard choices – either we don’t follow Ethereum upgrades, in this case “almost compatibility” will gradually become “some compatibility” and new Solidity contracts will not compile on Polkadot any more, or we follow Ethereum upgrades, in this case we must try to adopt our existing changes. If this touches anything we have already changed (gas schemas, CALL/CREATE, etc), the adoption may become challenging.

Either “full compatibility” or “some compatibilty” is better than “almost compatibility”

In conclusion, “full compatibility” or “some compatibility” is generally better choices than “almost compatibility”.

  • “Almost compatibility” still requires full re-auditing and re-testing, and therefore it has no advantage over “some compatibility” in this aspect.
  • “Almost compatibility” is unsafe because it encourages blind deployment. On the other hand, the on-chain code is generally a small part of the dapp codebase that a re-write may always be preferable. Therefore it may be a worse choice over “some compatibility” in this aspect.
  • “Almost compatibility” has a higher maintenance burden because Ethereum is a moving target and direct code reuse is no longer possible.
6 Likes
2

Thanks for the write up! it’s very important to truly assimilate and understand the points you raise, I feel most people might be of the naive assumption we are getting full compatibility which is dangerous and I’m not sure we’ll ever get(?).

I have immense respect for the work done with revive but sorry for the rant, I’m very skeptical about the whole AH strategy we are pushing for with what I assume will be a half-backed EVM support, a.k.a. “almost compatibility”. Do it Moonbeam style or don’t do it, ideally don’t do it and instead support the specialist but I guess it’s already too late for that, I don’t think “some compatibility” is good either, if we are going require people to learn new things it’s going to be hard to attract anyone, so be ready to throw a lot of money at developers to make them look our way as they’ll have better alternatives, but in that case I would rather focus on promoting the good think! we have.
In general I feel this whole strategy a bit forced trying to showcase PVM but I don’t think we are even doing that, as far as I understand PVM on revive compiled to WASM runs in interpreted mode so it might not even run better than WASMI(pallet-contracts)? I guess with the transition of parachains to JAM services revive’s PVM can go native, but by then there will be many better ways for bigger audiences of developers on mainstream languages to deploy simple to develop programs.

I wish us the best luck with the Hub roll out! :crossed_fingers: Although I’ll still express my discomfort with a sad lonely nay when the runtime upgrade I didn’t ask for arrives to Kusama governance :slight_smile:

2 Likes
3

It’s possible to accomplish full compatibility via EVM-to-RISCV transpilation, but this was unfortunately not the approach that revive has chosen.

In my view, the Moonbeam style interpretation will probably also be the go-to choice if Ethereum core devs were to design our compatibility layer. Historically, EVM JITs (EVM-to-x86 transpilation) were not favored due to several factors:

  1. Storage access was the true bottleneck, but not VM performance.
  2. Transpilation with good compatibility will mean four times the storage requirement for codeHash to code mapping.
  3. The safety issue. The difficulty of audit and testing for JITs and transpilers.

The landscape is changing, but it’s still premature:

  1. VM performance does recently become important in the case of ZK rollups, but we do not want to deploy Ethereum rollup solutions on Polkadot Hub.
  2. Contract size can be reduced with EVM64 and further optimizations of the compiler, but this probably only works for fully rewritten contracts that take advantage of 64-bit width, not any existing Solidity contracts.
2 Likes
4

No. It is not. By definition. There are multiple examples for this. To give you a very obvious one: Deploy code which EXTCODECOPY another contract and manipulates its code. It will assume EVM bytecode.

Wrong. Frontier is not “fully compatible”. There is no such thing as EVM equivalence or “Full compatibility”. It is not only about the bytecode. You can look it up, there were security problems on Ethereum L2s what you would (incorrectly) categorize as “fully compatible” caused by the most minuscule differences (for example hard coded gas stipends working well on one network but not the other). Frontier falls into “almost compatible” and is arguable more EVM compatible than revive. Even if we would deploy frontier instead of revive, just because frontier uses EVM bytecode, other incompatibilities, notable incompatibilities like different currency denominations, don’t go away. So by your very own analysis, your very own project, is even more unsafe than revive. :person_facepalming:

It’s not all or nothing as the title suggests. We are not doomed because we try to innovate and gain a unique position in the market. Min-maxing exists. All the EVM compat stuff is optional anyways, in fact allowing for much more secure dApp deployments not dealing with anything EVM at all. It also means we can deploy an EVM interpreter, like frontier, maybe a better one than frontier. I so we can directly support EVM bytecode. Nothing suggests or indicates that we are going or not going to do this: Anyone can.

I have to admit, I am a bit confused if not disappointed. You are a highly respected person, here since a long time and you made very valuable contributions. But not only should you be able to assess better on technical levels, this post reads like any other cheap FUD:

  • Vastly ex-aggregated FUD title
  • Nonsensical conclusions
  • Attacking Parity, saying anything other than what we currently do would be better
  • At least a year too late
  • You, the author, evidently contribute to improve EVM. Ethereum with an EVM L1 is in that sense a direct competitor to a Polkadot PVM L1.

After all, you make a valid point for what I am trying hard to point out too: EVM compatibility does not equal EVM equivalence. And we should be careful. Builders should be careful. I still like the post, because despite being mostly nonsense, it helps us raise awareness.

3 Likes
5

Untrue statement without any citations. It is obvious that this varies highly case by case: Your dogs memecoin project has 0 audit costs. World class cross chain protocols might have audit costs of on-chain code exceeding the development costs. Or not.

Utter nonsense. For obvious reasons, the more compatible a solution is to EVM the less potential attack vectors caused by differences there are. The more tooling is re-usable. The less development costs there are. Less differences and customization auditors must be aware of.

Re-developing + full re-auditing and re-testing is is obviously more costly. Obviously there are crystal clear advantages of what you consider “Almost compatible”.

Code re-use is, in fact, possible. You are saying that even less compatibility decreases maintenance burden. Like what? This statement, literally, makes ZERO SENSE.

Any serious dApp or protocol does not blindly deploy anything just because they can.

Strawman: That risk is acceptable for solc but not for the PVM JIT (the PVM JIT is even much simpler than a full blown compiler like solc, lol).

Again. Correct overall promise of the post: Builders should always understand what they are deploying, where they are deploying, audit and test properly. Yes, our solution, obviously and by nature will introduce new attack vectors. A lot of them. The same goes for Astar, Moonbeam, Ethereum L2s, any other network that is not Ethereum L1 obviously has different attack vectors than Ethereum L1. Instead of spreading FUD I invite you to contribute sensible suggestions as to how we can improve in that regarding.

Let’s recap. You are saying that what we build is inherently insecure and has not a single advantage over anything else that exist. Backed by a bunch of made up brain farts. Conveniently leaving out that we have “no compat” already. Conveniently leaving out that EVM can run within PVM. Conveniently leaving out that deploying EVM interpreters are a non-goal and people can just go to say Moonbeam if they want. Without leaving any suggestion on how we could actually improve the situation. I view this as a hostile action against our entire ecosystem and will refuse to engage further here.

2 Likes
6

This is only temporary; the plan is to either switch to a recompiler on the current Polkadot stack (with the addition of a new hostcall) or to migrate to a recompiler during the JAM migration, whichever comes first.

The interpreter is essentially only temporary while all of the kinks are ironed out/the spec for PVM hits 1.0. It would be silly to start with the recompiler right of the bat as once we introduce the recompiler hostcall we’ll have to support it forever and won’t be able to change it anymore.

And what makes you think that devs will actually want to use those “many better ways”, or just by having support for general purpose languages will make the “bigger audiences” want to develop on a blockchain? I don’t doubt some will do, but I suspect this will be more of a small, gradual trickle, rather than a sudden big flood.

It’s still useful to have some degree of Solidity/Ethereum tech stack compatibility. Whether you like it or not the vast majority of developers which deploy on blockchains today are using Solidity. We have tried the strategy of “let’s build better tech and get the devs to come to us” in the past, and it should be obvious at this point that it has pretty much failed.

Nevertheless, we can have both. Ink already has experimental support for running under pallet-revive, you can also use bare Rust to write pallet-revive-compatible programs, and JAM will bring more general-purpose programmability.

5 Likes
7

Funnily Frontier has not been audited until very recently and was in production for years. Audits are expensive (when we do them).

Testing is also relatively easier now that there are lots of test suites from Ethereum centric companies that increases the confidence that the current runtime is working as expected.

Overall I agree with you, 100% compatible or not at all is likely best. But we tried both without much success (Ink! and Moonbean). At the end of the day, technicalities matter less that what i would like. Liquidity, #users and fame are as important if not more.

8 Likes
8

Afaik solidity and EVM could never be “safe” per se. We have two primary concerns here:

  • Can someone port Solidity code to run on Polkadot?

There is a long history of porting C code between architectures, so the concerns here look more like the blockchain specific tooling. As an example, if the user does not supply enough gas then does the RPC node catch this and report it in a similar enough way to how the EVM ecosystem handles this? Afaik yes

  • W3F and treasury must give grants for Solidity development work. Are those teams going to get pissed off at Polkadot and switch to ETH or similar?

Again, the issue looks like the blockchain specific tooloing. As another example, there exists EVM in the browser tooling, probably not 100% EVM compatible either, but does revive match this browser tooling? If we hit difficulties matching the browser tooling with revive, then should we pay someone to improve revive compatiblity for the browser tooling?

Also we really want people to add their own rust for improved functionality over native EVM. In theory, grants could prioritize people doing some rust, in part making this list longer, but also just linking rust into their combiled solidity.

As a specific example, you cannot efficently implement a drand timelock bridge in solidity using EVM precompiles, but you can do one in polkadot, and it’d be efficent if we deploy the ECC hostcalls (onegoing goals). See

Anyways..

We should catalog and prioritize the specific incompatiblities that look problematic, not just in revive but in all the tooling.

We could also quietly identify incompatiblities that exist between other parties EVMs and provide compiler flags in revive for improved compatiblity with those, which then maybe presents revive as a possibly better option for wider “EVM compatiblity” than EVM itself.

3 Likes
9

For things like EVM with code and gas introspection you’ll have to also store the original EVM bytecode and correlate PCs. This is generally a standard practice for EVM JITs.

There’s a really simple test you can do to properly distinguish “full compatibility” and “almost compatibility” – whether, with some reasonably minimum changes, that the EVM engine can be pluged in to sync Ethereum mainnet.

Frontier never received direct audit, at least I’m not aware of any. There just wasn’t any resources. Beforehand Ethereum compatibility wasn’t any priority at all.

Only the underlying rust-evm engine may be considered to have some “proofs of production-ready” because it synced mainnet just fine, it has been around for a really long time and has been used, forked in several large production blockchains. It probably also gained a few more testing and audits there. In the old days when the blockchain space was small it was still permissible to initially gain the “production-ready” tag this way, but I don’t think it will be a good idea nowadays especially for anything EVM related.

Generally those browser toolings are full compatible because it’s simply an EVM interpreter running under the hood.

I have no interest in arguing over directions because this has been “decided”, but it’ll be my responsibility to point it out if I see anything potentially unsafe. On the other hand, I don’t think it’s reasonable to say “full compatibility” was tried “without much success”. Frontier was never properly “tried” and was never a priority in the development, yet Moonbeam, Acala EVM+, etc are some of the largest Polkadot parachains.

1 Like
10

I am afraid that this, next to being impractical, does not solve the problem of achieving full EVM bytecode compatibility. Not only because of the (non-negligible) storage overhead. Dynamic jumps are a thing in EVM bytecode and very effectively prevent optimizations. The control flow in EVM bytecode is lost and thus the compiler can’t effectively employ optimizations. I suspect that even with ignoring dynamic jumps, this is a hindrance. CFG reconstruction, for example via the relooper algorithm is in theory possible but: The constructor emitting EVM bytecode implies compilation happening on-chain. Not just JIT compilation but also AOT. Which is, well.. I am not going into that. We can’t do this. It is the reason Wasm sucks for our use case and PVM was invented anyways.

I would like point out again: Compatibility on the bytecode level is not the end of the story. Even if with enough stunts to make EVM bytecode at least somewhat efficient. Differences in gas metering, currency denomination and many more subtle things will stay different anyways.

Thank you. I appreciate that.

11

As I said above, I think incompatiblies in revive vs ETH would not matter much, except for a few specific projects like hyperbridge. Indeed they maybe a distraction. Instead we sould worry about the wider class of “tooling” problems, whatever that means.

In particular, we could create some retroactive treasury bounty for “ETH compatiblie tooling” (broadly interpreted) that awards limited compensation for “non-trivial porting of popular tooling from the ETH world” which occured as part of a development project for polkadot.

After a team ports some tooling, then they could report the hours this required, the resulting git commits, and a polkadot specific documentation outlike. Next the custodians propose some compensation. If the compensation is judges acceptable by the team, then the team provides the actual polkadot specific documentation and a blog post on the forum. If the custodians underpay then the team never writes the polkadot specific documentation or keeps it as their own in-house advantage.

In essence, we’d directly pay for people for tooling development in exchange for them explaining qwhat the did, why, etc. There should be some larger polakdot project for which the work was done, if only to prove utility (porting say whisper is not useful), but that larger project maybe supported by a W3F or treasury grant, so long as the proposed tooling was not expresses as part of a deliverable there. This bounty program should not prejudice W3F or treasury grants against explicit larger grants for porting more complex tooling.

12

If you take spec conformation and safety (but not speed) with higher priority, then those are not problems. There is a limited number of JUMPDEST, but of course more optimizations of the control flow will not be possible. EVM JIT is a thing that sounds attractive in surface, but not so much when all the details are considered.

In revive you are already making contract code 4x the sizes, which is a significant overhead. If you haven’t cared about making the situation 4x worse then another 1x to store the EVM bytecode wouldn’t be that of a big deal.

Just two years ago you would have been arguing with me why Solidity sucks.

Initially we thought EVM sucks so we went with WASM. Then we figured out WASM sucks and went with RISCV. At this moment we have a few “small” things we didn’t like about RISCV so PVM and RISCV are not exactly the same any more.

It’s honestly not a problem that we always chase the next best solutions. This is how we make progress in blockchain. But we have an important constraint, called “safety”, that will always be the first priority over everything else.

If there are no guardrails and if the risks of “almost compatibility” is not explained clearly to the community, someone is going to compile their Solidity contracts, unmodified, to production. It only takes one hack, if we advertise “compatibility” but do not treat compatibility serious enough, before all the Ethereum momentum on Polkadot is killed. That is why the “almost compatibility” approach is a lot more unsafe than the other two approaches “full compatibility” or “some compatibility”.

We had the multisig hack, and it’s our duty to be more careful. If we prioritize speed over safety, we may get none. The speed improvements will probably not be felt by everyone until a long time – the Ethereum mainnet hardly get so congested nowadays.

1 Like
13

Excuse me but I care a lot about code sizes. Compiling Solidity to a general purpose 64bit little endian RISC bytecode, for a number of reasons, naturally introduces code size overhead against the EVM, which it is very tightly coupled to.

The revive is in a MVP phase, didn’t even have it’s first non-preview release. More optimizations than the low hanging fruits already picked up will be implemented.

You are criticizing us for having factor k times n EVM byte code size overhead and at the same time your suggested solution (EVM JIT) is not only impossible but would introduce a code size overhead of k * n + n. I don’t get it.

So what? Everything I’ve said there is valid and still applies. The Ethereum Foundation dropped the ball on Solidity. Yet it does somehow not stop people from wanting to use it. I just develop a compiler for a language people want to use. I detect problems and formulate critics around them.

So what? Stupid people are waking up today to do stupid things today. It does not mean that secure deployments of dApps are impossible on pallet-revive.

Again: By nature, we do introduce many new potential vulnerabilities. We do acknowledge and ask builders to be careful. People should understand the differences and re-audit their project deployment, if they care enough. Would having compatibility on the byte code level introduce fewer attack vectors? Maybe, probably. Would it introduce zero new attack vectors? Of course not. Does it stop stupid people from doing stupid things? No. Does it stop from reasonable people doing reasonable things? No.

Ridiculous. What narrow minded Ethereum maxis always say. Sell Solidity and EVM as the guardrails of blockchain security. Classical Eth maxi bullshit TED talk right here. The Ethereum Foundation dropping the ball on Solidity developments and EVM improvements is the reason Solidity and EVM are the source many of blockchain security problems. EVM bytecode as the guard rails of blockchain Security, seriously? Ethereum maxis run around non stop and use it as killer argument in any discussion as an attempt to shove the PROBLEMS of the EVM down everyone else’s throat. They, wrongly, accuse Software like LLVM “cAn NoT bE trUstEd aNd iS tOo cOmplEx fOr tHe FuTurE of FinAnCe”. Going at length with false claims to make their own steaming pile of crap Software look good.

Not only was the multi sig hack directly tied to your bespoke “security guard rails” of Ethereum. It has nothing at all to do with whether we deploy an EVM interpreter to a system chain or not.

For people who actually care about security, the revive pallet not only eliminates some security problems of Frontier and unlike Frontier will be audited. We also try to eliminate EVM security problems. For example, it forbids re-entrant calls by default. It is objectively the better choice than EVM which allows it by default. Now, your brainless EVM “security guardrails” requires the compiler to enable re-entrant calls again by default. Think about this for a moment.

EVM bytecode is not the guardrails of blockchain security. Shipping an EVM bytecode interpreter does neither stop stupid people from doing stupid things nor does it eliminate other incompatibilities and thus the need to understand and re-audit contract deployments. I don’t have the time nor the energy to engage in such unproductive discussions.

Thank you for raising the awareness. Builders are advised to read and understand our docs clearly and, especially multi-chain protocols, are advised to security audit their Polkadot Hub deployments.

14

The revive compiler preserves not all but many EVM semantics by compiling YUL emitted by solc. To target “some compatibility”, Solang is required to deliberately violate many more EVM invariants and semantics than “almost compatibility” of revive. What it led to: People were still assuming it anyways. Not only because they were lazy but sometimes they needed to assume it because not assuming it prevents them from using any third part library. Which makes the whole thing useless entirely. Your claim that “some compatibility” is more secure is by logic and by evidence false.

How many more false claims and misleading statements are you going to drop?