WrappedEVM Eth RPC compatibility layer

[bernardoaraujor]

Here is a scheme for categorizing Ethereum compatibility for some given blockchain. The key is to assess the layers of Program Equivalence:

• Level 0: Full compatibility (Essentially a chain code fork)
• Level 1: Bytecode level compatibility
• Level 2: Code compile-level language compatibility (Eg. Solang)
• Level 3: RPC/endpoint extrinsic compatibility (Eg. Ability to sign using Metamask / Ability to read data using the same block explorers)

While Frontier’s pallet-evm and pallet-ethereum already provide Level 0 compatibility to Substrate, there’s instances where we might prefer native Substrate execution over Frontier’s SputnikVM Runner, as a workaround to its inherent performance penalties. So we want to be able to navigate this scheme as a spectrum of compatibility with legacy EVM standards.

The easiest place to start is at Level 3. From there, climbing up the compatibility levels implies that things can go in a couple different directions, namely:

1. EVMless Execution: Re-routing Eth RPCs for native Subtrate execution under multiple FRAME pallets.
2. Wasm Contract Execution: Re-routing Eth RPCs for execution under pallet-contracts.

I recently learned about a community effort that is apparently being called WrappedEVM, including teams like Interlay, Centrifuge and HydraDX. So I’m starting this thread as a way to coordinate those efforts.

EVMless Execution (pallet-evmless)

The dispatch precompile is made available by pallet-evm. It an obvious initial step in this direction, as it opens up the possibility of triggering FRAME execution via Eth RPCs. It is already available and a really low-hanging-fruit for teams aiming to go down this route.

It is still a bit cumbersome, however, because the RPC side is still partially “contaminated” with Substrate, as it requires the actual FRAME calls to be SCALE encoded.

For some cases, we might want the Eth RPCs to believe they’re interacting entirely with an EVM ABI, while staying completely agnostic in regards to the Substrate realm. This starts from the assumption that there are already pallets deployed into the Runtime that are able to cover for all the expected functionality that the ABIs expose.

This would require some mechanism for the Runtime to receive the EVM dispatchables and re-route them for native Substrate execution. This can be achieved with a stripped down version of pallet-evm (let’s call it pallet-evmless), that provides a simplified EVM shell Runner. Then, for each Solidity use-case, different precompiles would be written ad-hoc, either tighly or loosely coupling with the pallets responsible for execution. Unique’s evm-coder crate can be leveraged here so that developers can focus on writing their precompile logic in Rust without worrying too much about the low-level ABI details.

Ideally, we should standardize pallet-evmless as a new upstream feature on Frontier.

Note: I’m using the term EVMless to describe this specific path of action, while WrappedEVM is being used as the umbrella term for the overall initiative (which also includes the pallet-contracts path described below).

Wasm Contract Execution (pallet-contracts)

While still a WIP, there’s a few tools in our community that have great potential to onboard Solidity devs into Substrate:

evm-coder

Maintained by Unique Network, evm-coder is a Rust library for seamless call translation between Rust and Solidity code. Not only it can be quite helpful writing precompiles based on Solidity contracts (on the EVMless context described above), but it can also be used while writing ink! contracts as well.

Sol2Ink

Sol2ink is able to parse compilable Solidity interfaces into ink! traits and compilable Solidity contracts into ink! contracts, while leveraging the power of OpenBrush. The effort is led by SuperColony, but the repository seems stale for the past few months at the time of writing.

Solang

Maintained by the HyperLedger Foundation, Solang is a Solidity compiler for Solana (which we don’t really care about here) and Substrate. It is able to compile Solidity contracts for execution under pallet-contracts. The project is under active development, with frequent contributions from Parity’s Core Engineering.

pallet-contracts

Be it some Solidity contract:

• transpiled to ink! and then compiled into Wasm via Sol2ink or evm-coder
• compiled straight into Wasm via Solang

While deploying them into pallet-contracts, we might want to still interact with them via Eth RPCs. This would imply new features being added into pallet-contracts. While theoretically feasible, there might be a few challenges to be tackled along the way:

• ABI compatibility: currently, ink! and Solang are not following the EVM ABI calling conventions. Making this happen (if even possible) would imply changing the ink! and Solang calling conventions, which opens a pandora box of breaking changes between different versions and can be seen as tabu. There’s also the problem that the Wasm contracts use types as arguments and return values that don’t exist in the EVM ABI. In this case we would probably just specify them as as bytes and live with the fact that a Dapp needs custom code to encode/decode them. Bottom line is: this topic needs more research.

Edit: as added by @lach in comment below , evm-coder can potentially help solve some of these issues of ABI compatibility for ink!.

• EVM tooling compatibility: Tooling might inspect the code. It might be as little as checking for some magic byte and as sophisticated as generating constructor code. It doesn’t matter if it is just a trivial check. That tool will not work as we will be feeding it wasm code. We would need to try to upstream compatibility fixes for the tooling projects.
• Contract creation: A contract running on pallet-contracts contains a dedicated “construction” entry point. When creating a new contract, input data to is passed to this constructor as data in the transaction. This is different from EVM, where the code passed as data is the constructor itself. There are certainly some ways so work around this but it will require Solidity developers to do things slightly different from what they are used to.

So there will likely be no perfect on-size-fits-all solution, and we will have to deal with fragmentation at some point. Which is arguably unavoidable, as we can already see in some Eth L2s breaking EVM bytecode and RPC compatibility (e.g.: zkSync and ERC1167).

Nevertheless, this is still a path worth pursuing, as onboarding Ethereum dApps and devs will be really important for the vision of Hybrid Chains in our Ecosystem.

[lach]

ABI compatibility: currently, ink! and Solang are not following the EVM ABI calling conventions. Making this happen (if even possible) would imply changing the ink! and Solang calling conventions, which opens a pandora box of breaking changes between different versions and can be seen as tabu.

Regarding mentioned evm-coder - it is not dependent on substrate code, so it can be used inside of ink! contracts as well. I.e (example code, implemented using not-yet-existing evm_coder_ink adapter):

struct Contract;

/// Ink calls
impl Contract {
    #[ink(message)]
    pub fn hello_ink(&self) -> String {
        "Hello from ink! side of a contract!".into()
    }

    // Forward ethereum calls to evm_coder
    #[ink(message)]
    pub fn evm_call(&mut self, calldata: Vec<u8>) -> evm_coder_ink::Result {
        let call = MyContractCall::parse(&calldata)?;
        let value = self.env().transferred_balance();
        let caller = self.env().caller();
        self.call(evm_coder::Msg {
            call,
            value,
            caller,
        }).into()
    }
}

/// Evm calls
#[evm_coder::solidity_interface(name = MyContract)]
impl Contract {
    pub fn hello_evm(&self) -> String {
        "Hello from evm side of a contract!".into()
    }
}

[bernardoaraujor]

that’s awesome, thanks for this info @lach
I edited the OP to make sure I’m covering these aspects of evm-coder.

[bernardoaraujor]

Here’s a question that has been lingering in my mind:

Most parachains use AccountId32 (H256), but the Eth RPCs only understand H160.

While Frontier provides a translation scheme for H256 → H160 (simply drop the last 96 bits LE), there’s no deterministic way to convert H160 → H256 at execution time. So when some incoming RPC carries an H160 address as argument, how do we convert that into H256 on the fly?

The only solution I can think for this problem is to come up with some kind of on-chain mapping for H160 → H256. That does result in some extra burden on the UX (e.g.: the user needs to keep track of 2 addresses for each account) and some storage bloat.

Are there any alternative solutions for this problem that I’m not seeing here?

p.s.: we will carry a session during the Parachain Summit in Copenhagen on this WrappedEVM topic. Hopefully that will be an opportunity to move this discussion forward. I’ll try to collect some insights and share them here afterwards.

[xlc]

For Acala EVM+, we developed a pallet to handle the mapping and allow binding EVM address and Substrate address. The built-in MultiAddress from Substrate also makes the integration more smooth. For example, it is possible to use balances.transfer to send native token to EVM address in Acala.

[lach]

While Frontier provides a translation scheme for H256 → H160 (simply drop the last 96 bits LE), there’s no deterministic way to convert H160 → H256 at execution time. So when some incoming RPC carries an H160 address as argument, how do we convert that into H256 on the fly?

Not quite right, Frontier provides translation to the other side: H160 -> H256, trait AddressMapping.
The default implementation for the Substrate is HashedAddressMapping: hash("evm:" ++ address).
This mapping determines which Substrate address should hold the Ethereum account balance and nonce.

H256 -> H160 mapping (trait EnsureAddressOrigin) is not exposed directly; there is only a trait provided to check if H160 address provided by the user is allowed to be used for evm_call/evm_withdraw for the specified H256.
The default implementation for the Substrate is EnsureAddressTruncated: origin[0..20] == address.

While it is possible to create on-chain mapping between H160 and H256 addresses, this approach is not perfect, as it requires some form of registration.
This registration procedure may break some Ethereum RPC compatibility, i.e. Acala requires a custom web3.js provider to work.
Without such a form of compatibility layer, you can’t reliably share any resources between substrate and Ethereum accounts.

As the Ethereum account linking may be added at an arbitrary time, and you can’t know the mapped address ahead of time, you still can’t use a single address for everything.
Example:

• Alice has linked Ethereum address Alith on chain A.
• Someone transfers an NFT to Alith on the chain B. Chain B doesn’t yet know which substrate address Alith will have, so it still needs to have the original Ethereum address (Alith) stored.
• When Alice registers Alith’s mapping on chain B, assets should be either transferred to Alice (And this may not be possible if there is a large number of assets) or kept on Alith address (Then why do we need the mapping in the first place?).

At Unique, we have embraced the asymmetrical mapping for Ethereum (hashing when converting H160 -> H256 and truncation when converting H256 -> H160 (TruncateAddress)).
There were a couple of challenges to solve, but in the end, we believe that our solution is pretty good for the general usage.

For cases when we need to return the resource owner to the Ethereum side, e.g NFT ownerOf, we keep an original user address in enum CrossAccount {Substrate(AccountId), Ethereum(H160)}: https://github.com/UniqueNetwork/unique-frontier/blob/b1ff36163b15a36c1f73950c722cb1f3939d287d/frame/evm/src/account.rs#L53
What problem does it solve?

• Let Alice be the Substrate account.
• Let Alith be TruncateAddress(Alice).
• We mint NFT to Alith’s account.
• If we only store the substrate address as an NFT owner, then the resulting NFT owner will be, say, Alyssa = HashedAddressMapping(TruncateAddress(Alice)).
• And then, ownerOf will return TruncateAddress(Alyssa), which is not the same as TruncateAddress(Alice), thus the result of ownerOf is useless.

Instead, we store CrossAccount::Ethereum(Alith), so ownerOf will return the correct address: Alith.
The same applies when the token transfer target is Alice. In this case, we store CrossAccount::Substrate(Alice), and ownerOf will return TruncateAddress(Alice), which is precisely Alith.

The other problem, which arises when no bidirectional translation is available, is Ethereum transaction fees/payment. Our solution is to delay any address conversions and use CrossAccountId in Frontier everywhere it is possible: feat: move address conversions to the lower level · UniqueNetwork/unique-frontier@f814b3f · GitHub
This way, when the user tries to call an Ethereum contract using pallet_evm.call method, payment (Both fee and value) is done from the substrate address, instead of using HashedAddressMapping(TruncateAddress(origin)), as in vanilla frontier.