Ethereum RPC Compatibility for Polkadot Smart Contracts

The idea of making pallet-contracts Ethereum RPC compatible has been around for a while. I did some research on the topic and built a working prototype which can be found in this repo. This post describes briefly the approach taken to the problem, the results so far and possible vector of further development. Your feedback is very welcomed!

Rationale

We have the most technically developed blockchain platform, but lack of real users. The largest user base stays in Ethereum.
So the ultimate objective of this project is to empower onboarding these users to our ecosystem.
One obvious way of doing so is to migrate the beloved and popular Ethereum dapps onto Polkadot. This basically means to give the same user interfaces and UX, while providing cheaper and faster execution layer, with further possibly unlocked richer functionality, thanks to Polkadot’s unique cross-chain interactions like XCM and bridges.

Technically-wise, there is nothing special in Ethereum what can’t be built on Polkadot, which provides way more features and agility to developers. Therefore, in essence there is (theoretically) no insurmountable constraints for porting Ethereum dapps (read: their users) to our ecosystem. What we need to deal with here is user inertia. The less user has to change in her basic journey experience, the better for getting her come here.

At this point a meticulous reader may say: well, we’ve already been through this. We even have fully compatible EVM parachains. Where users?

Well, true, there is an EVM-based parachain. Still it looks like just porting EVM to Polkadot (with performance downsides) is not really enough to make a breakthrough. Our contracts engine had proven to be faster, and going to become ultra faster with coming support for the new ISA. We want to leverage this advantage to attract more users. Yes, we also got a parachain having both pallet-contracts and pallet-evm aboard. But while it tends to provide interoperability between contracts running on these separate engines, from the user perspective this is not going to bring more performance. And from the developer perspective it makes apps much more complicated to build.

The idea of the project introduced here is to make Polkadot dapps Ethereum RPC compatible without taking the EVM part of the deal. Let’s have a look on how’s that possible.

Compatibility Decoded

Consider a typical dapp tech stack:

Simply put, there are the following layers of it:

  • UI built with web3js libraries,

    which speaks to a node via its RPC;

  • RPC exposed by a chain node,

    which speaks to node’s runtime via its API;

  • EXEC Contracts execution engine,

    implemented as a runtime module;

  • CHAIN protocol logic,

    implemented in other runtime modules.

When we want to port an Ethereum dapp to our Substrate-based chain, we need to make sure that:

  1. We expose the RPC endpoint which speaks to UI in compliance with the Ethereum RPC spec.

  2. On the Execution layer, this implies that our engine should allow contracts to implement the same business logic as in EVM contracts,
    while keeping the same calling conventions between caller and callee (more on that in the next section).

  3. Aside from input and output of the contract being called, a dapp might rely on the underlying chain protocol data,
    such as block and transaction data, gas prices, storage state, etc.

    For our RPC to provide such data, our runtime should have logic for translating Substrate chain data to Ethereum chain data. (Right away this point could sound like a dealbreaker. But hold on, things might not be so bad. Keep reading to the next section.)

Now that we have pointed out what has to be done, let’s dive into how we try to achieve this.

Solving The Puzzle

As we’ve pointed out, just exposing the Ethereum-alike looking RPC is not enough. The underlying logic should comply with it as well (at least to a certain degree).

For that the system must be able to be compatible to how things work in EVM on the layers below the RPC. First of which is the execution engine.

Execution Layer

Luckily for us, pallet-contracts module of Substrate had been designed to be in feature-parity with EVM. Both are stack-based machines with gas metering, contract accounts are of the same nature as external (user) accounts (from the chain protocol perspective), there are constructors and call messages, payable and non-payable messages, as well as support for cross-contracts calls (both context-switching and delegated ones aka libraries), and overall its API allows implementing pretty much everything one would expect from an Ethereum smart contract.

Still, there is one quite a sound discrepancy, which albeit do not really relate to the engine level. It is the way contract input data is encoded: Solidity contracts use ABI encoding, whereas ink! contracts use SCALE. But this is a language level difference. On the lower, execution level, our contracts (both Wasm and RISC-V) accept input just as a byte sequence. How it’s being treated\decoded is determined by contract intrinsic logic, meaning contract developer can implement it in the way so that it works with ABI encoded data. Of course this would not be a very handy thing to do from the developer point of view, given no special tooling provided for that. But what’s important is that it is not an insurmountable obstacle, and we will talk a bit on how to deal with that further in this post.

Next compatibility things refer to chain-specific data.

Chain Layer

Here it comes to the most noticeable difference between Polkadot and Ethereum building blocks: account format and crypto primitives:

  • Normally in Polkadot we use 32-bytes-long AccountId accompanied with Schnorrkel/Ristretto sr25519 algorithm for keys and signing, and blake2 for hashing.
  • Ethereum standard is 20-bytes-long AccountId and ECDSA secp256k1 key pairs in combination with keccak256 hash function.

There are more other discrepancies like e.g. the fact that extrinsic Id is not guaranteed to be unique between blocks in Substrate-based chains, while for Ethereum transactions that’s the fact on which a good part of business logic relies upon. Also, Ethereum has just gas for measuring computational effort, whereas in Polkadot we have two-dimensional weight, counting for execution time and for implied size of the proof data used by PVF. There are more other issues, but all of those seem to be surmountable (at least of this point of the research).

As per AccountId, well… Polkadot was designed so that you can customize everything, and that unlocks some opportunities. First, nothing restricts your parachain from using whatever AccountId and elliptic curves you want in your runtime’s business logic. More to that, polkadot-sdk provides ready to use crypto primitives for this.

Second, here we have to turn back to the objective we set at the beginning of this post, which is to get existing Ethereum users aboard. And then we have to admit that users are not going to switch to other wallets/signing extensions right away, overnight. That means that for account format and keypairs we have to allow them keeping what they currently use.

What we have listed in this section is not an exhaustive set of possible compatibility issues on the chain level, there might be others. For now in ethink! some of RPC methods related to chain data are mocked, others return Substrate chain data. With the contracts tested so far, this looks like not a problem. In general, it should not be a showstopper, as in the worst case we could construct and store a fake Ethereum block for every Substrate block, this approach seems to work fine in Frontier-based chains. But again, the research is ongoing, and this would need to be tested in practice with porting real Ethereum dapps onto this solution.

Finally, we can’t just wave a magic wand and solve the problem for all parties (users and developers) at once. Then let’s break it into steps, the roadmap of which could look like the one presented in the following section.

The Road To All-Hands Compatibility

As a first step, in short-term we may try to change as little as possible for the user, which naturally comes with the cost of additional work to be done by the dapp developer.
In particular, to deal with ABI/SCALE encoding mismatch, as well as different message selectors, the frontend piece of the dapp would have to be modified to work with metadata of a_contract.ink instead of a_contract.sol one. Or, the contract piece of the dapp would have to be taught to deal with ABI encoding so that it takes the same input from the caller and returns the same output as the original Solidity contract does. In any case, unless we have a tooling for those things, the contract developer would have to re-write his Solidity contract in ink! by hand, which is totally feasible thing to do, as ink! was designed to have similar contract layout to Solidity, and if you look at its basic examples, you find a number of such ported contracts.

Good news are that the tools for automation of such a translation are in active development: there is solang compiler. I’m not sure if we are there yet, but once it’s ready, we take the second step, and let dapp developer just re-compile a Solidity contract source to be deployed to pallet-contracts.

Last but not the least, some helpful tooling for developers, either for the ease of porting the frontend piece of dapp, or for making a contract deal with ABI-encoded input, could also possibly be provided. (At the end of the day, ink! contracts are basically Rust code, so chances are you could possibly use some existing crates for that).

What’s being said could be summarized to the table:

Currently, we are in the first column, and it feels like starting entering to the second one (need to sync with @Cyrill and do some experiments with solang).

Following this narrative, we may envision the target scenario as follows:

Target Scenario:

  1. Take a successful Ethereum dapp.

  2. Transcompile its contract from Solidity to Wasm/PolkaVM’s RISC-V.

  3. [might not be needed] Port dapp frontend so that it deals with new metadata.

    From [ABI encoded input + selector] to [SCALE encoded input + selector].

    (As mentioned before, this step might not be needed, ideally could be solved on the previous step.)

  4. Deploy the contract to a parachain having ethink! aboard.

  5. Profit!!

Now that we explained how we deal with particular compatibility issues, let’s have a look on how we bring all the pieces together to a running chain node which has pallet-contracts and works with Metamask.

High-level Design

In a nutshell, ethink! is like Frontier, but for pallet-contracts instead of pallet-evm.

Here is the simplified (non-exhaustive) components map to the compatibility layers introduced in the beginning of this post:

There are 3 main pieces of it:

  • RPC The RPC “frontier”, exposing the RPC endpoint which looks just like a normal Ethereum RPC.

    Its function is to accept requests and decode them, then call an appropriate method of the API exposed by our Substrate runtime, and response back to the caller.
    By adding this piece to our node, we make it look like an Ethereum node to the caller, which is normally a dapp’s frontend.

  • GLUE The Runtime of our node, which provides special methods in its exposed API, gets calls from the RPC “frontier” and routes them further.

    Some of them, like e.g. account balance checks or contract “dry-run” calls, does not bring any state changes. For such cases it just calls the corresponding pallet’s API method (like pallet_balances::free_balance()).

    For the ones that do alter state (called transactions in Ethereum and extrinsics in Polkadot), the mechanics is as follows. The incoming Ethereum transaction is being wrapped into an UncheckedExtrinsic with the call to pallet-ethink::transact(), which decodes passed-in Ethereum transaction, and routes the call further (based on type of the destination account) to a specific destination pallet. Specifically, for the calls addressed to an account which belongs to a contract, the destination module is pallet-contracts. If the callee address is a user account, the destination module is pallet-balances, as the call is considered to be just a balance transfer. (This logic is inherited from Ethereum).

    The wrapper extrinsic is being first put to the transaction pool. Just like any other extrinsic, it has its special logic for checking its validity. Upon such validation, the Ethereum signature is being checked, and the caller account is being extracted from it. The further way for the extrinsic is no different from any other extrinsic on its way to execution and inclusion into a block.

  • EXEC Execution of the contract happens in pallet-contracts module as usual.

No Upstream Changes

For all this stuff to work, no customization is required neither for pallet-contracts nor for ink!. Polkadot-sdk overall, and its smart contracts stuff in particular, was designed with quite a good level of abstraction, allowing building such compatibility layers on top of it, with no tightly coupling to a particular execution environment. (And this is what makes ethink! different to Frontier, which is tightly coupled to pallet-evm.)

When it comes to tooling, in particular cargo-contract and subxt have got some customization which makes it work with Ethereum accounts and signing, as well as to speak with ethink! node in the same language (for that chain metadata was updated).

Worth mentioning here, as “ethink!” name could be a little confusing, that this solution is not solely for ink! contracts.
It is basically language-agnostic, and low coupled with the executor (meaning it should work both with Wasm and RISC-V -flavored pallet-contracts, and could be adopted to other embedders\executors if someday we will have new options for that). Still the most mature contracts language in our ecosystem is ink!, and it’s basically the only feasible one for production use so far. That’s why the project naming was made with an accent on it.

Current State

The Prototype includes a Substrate node with pallet-contracts aboard and exposed Ethereum RPC. You can write your contracts in ink!, build them and deploy to the node using the fork of cargo-contract tool. Then you can interact with the contracts via Metamask. Follow the guidelines written in the project README file to try it out.

ink! contracts have to use custom environment with 20-bytes-long account addresses, see the example contract on how to do that.
Currently, there are two working examples, Flipper and ERC20, with more to be added in the near future.

There is also a little end-to-end testing framework, and the examples are covered with integration tests. Having a look at them might help to understand how input data is expected to be encoded on the caller side, and shed some light on the overall workflow of communicating with the node.

Please feel free to play with the prototype, and give your feedback here to the comments, or to the project repository issues.

What’s Next

The possible next steps are (if you have better ideas, please tell so in the comments):

  1. Update dependencies to recent polkadot-sdk.

    Currently it uses polkadot-v1.1.0, which is a bit outdated.
    There is no reason not to update to the recent release (just didn’t get around to it yet).

  2. Update examples to ink! v5.

  3. Deploy this prototype to a testnet.

    Most likely this will be Yerba Network.

  4. Choose a good Ethereum dapp and try to port it to the testnet.

    The goal is to make a showcase, and a reality test for this prototype. Expect chaos (c).

    Would it be really possible? Would it be faster? And seamless from the user experience perspective?
    Well, I’m looking forward to figuring this out.

Shy Request to Community

I would like to ask the community to propose some candidate dapps for migration from Ethereum to Polkadot, which would be simple enough to start with, and popular enough at the same time to make it the right target. We will take one and try to port it to a parachain with ethink! aboard.

I would also kindly ask for feedback from Polkadot core developers on the prototype introduced hereby, as I know this problem is being thought about for a while, yet I’m not aware of any other attempts of implementing this.

Many thanks for reading all through to the end of this post =).

8 Likes

Thanks for the update @sasha, this is fantastic!

1 Like

Just one thing to keep in mind so such work can be compatible with light clients, Chopsticks, and the upcoming omni node:

It needs to be built with a clear isolation with the node itself and ideally only communicate with the node via standard RPC. In that way, we can replace the node with a light client or Chopsticks and enjoy all the benefits.

Otherwise you will end up with the same situation as Frontier that it wont’ work with Chopsticks or light clients.

Let’s build modular and compostable software together.

Also while I won’t have time to try to build some PoC, I suspect someone can reuse 95% of the code at bodhi.js/packages/eth-rpc-adapter at master · AcalaNetwork/bodhi.js · GitHub and have a working solution done in less than a week

2 Likes

Good point, thanks Bryan! Just checked, so far it looks like to be the case.

So the idea is that with a custom fork of that eth-rpc-adapter (re-implementing the same logic of passing Ethereum RPC requests into runtime API calls as in ethink-rpc crate), one could use Chopsticks with it to enjoy a lightweight replica of any ethink-powered parachain, right?

Good stuff, we definitively want to have full compatibility, including ETH RPC and compatibility layers :slight_smile:

For the runtime, contract languages should be fully opaque. I think that ink! would benefit from having contracts to optionally support Solidity ABI encoding too. An obvious example that comes to my mind for ethink! would be some ERC20 implemented in ink! but it looks like an actual ERC20 to users.
W.R.T. to Solidity: Solang at the moment only supports SCALE (Solidity ABI could be implemented though if people are strongly interested in it); re-compiled YUL/EVM contracts on the other hand obviously will only support native Solidity ABI.

Actually we might think about having some way to be able to distinguish between contracts using SCALE encoding, it might come beneficial once people start deploying contracts written in either Rust or Solidity that require different ABI encodings. A non-invasive way (not bothering the runtime at all) would be to just agree to an (optional) interface that contracts can implement, so they can be queried against their supported encodings. Or by storing a flag for each contract (basically when deploying a contract you’d specify what encodings it supports) and making this information available via a runtime API.

Concerning ink! again, in theory we could have ink! contracts that support both encodings, i.e. they are callable from other ink! contracts using SCALE or as well Solidity contracts using Solidity ABI.

4 Likes

Yes. eth-rpc-adapter only uses a few runtime API to communicate with Acala runtime so it should be easy to just replace it with corresponding ink! runtime API and it should work. Then update the subql project to support pallet-contracts and the historical API should also work. Feel free to open issue on the repo if you want to give it a go and have more technical questions.

2 Likes

Thanks @sasha. Great work.

Do you mean host functions here instead of “RPC”? I can’t quite follow otherwise. Can you elaborate a bit more what the problem is in this regard with frontier? Does it expect the node to implement additional host functions?

I mean RPC. Frontier is part of the node and that’s the problem.

Okay got think. We don‘t get away with zero code in the client, right? You are suggesting to have a minimal forwarder and do most of the processing in the runtime?

I don’t think you get it. I mean having all the ETH RPC logic on a separate binary, which communicate with the Substrate node via Substrate RPC. The Substrate node is just a node implements Substrate RPC. e.g. the Omni node or Chopsticks.

The ETH RPC binary will forward some work to be processed by runtime but it also handles a lot of the RPC process such as historical transaction.

You are right. I didn’t get it so far. Now it makes sense to me. As a separate binary we wouldn’t need to add support to each existing node. However, do you think this is feasible? For example our submit_extrinsic won’t understand an Ethereum transaction. This means the extra binary would need to somehow create a new transaction. How would this binary sign this transaction?

Or do I have the wrong idea here?

It is already working for Acala so it is 100% feasible.

There are few ways to deal with the tx format.

Substrate is super generic so you can make it accept eth tx format and the RPC should just take it.

For Acala EVM+, the eth-rpc-adapter repackage the eth tx format to Substrate format, and during tx verification, we reassemble the eth tx payload in the runtime for signature verification.

Thanks for explaining. I wasn’t aware how much of the transaction is opaque to the node. Yes, this sounds like the ideal solution. Not adding another burden for node implementors.