Python smart contracts

Tech Talk
1

A year ago, I was with friends, casually joking about Solidity and what mainstream languages eventually will be used to write smart contracts. Lua? Python? It sowed a seed into my head. Soon after, on a rainy Saturday afternoon, I sat down and ported MicroPython to rv32emc, the PVM target architecture. The interpreter eventually compiled down to a 180kb large object file. This could definitively be deployed as a contract. In my head, things were coming together: Implement and register a custom system call module for interacting with the chain. Add helper libraries to let contracts speak Solidity ABI and simplify function dispatching (maybe similar to http path routers via annotations in flask). Alex had another brilliant idea: For interpreted byte code, we can delegate call into the interpreter contract (or a pre-compile, but micro python is a C code base), allowing to upgrade the interpreter or merkleize the contract code more easily. Solidity interopable Python smart contracts seemed actually very doable!

Momentum

A year ago, revive was roughly estimated to launch in Q3 this year. In other words, we were still busy with implementing the very fundamentals and it wasn’t the time to go after intriguing but uncertain ventures like this. However, I’d like to discuss this now in public. Solidity support is shaping up, and soon we will be busy no longer implementing but optimizing it, leaving room for more. Beyond Solidity, what else could help bringing mass-adoption for Polkadot?

The Python programming language is gaining momentum fast. It arguably is the most popular programming language on earth. Numbers vary by data source, but there seem to be more than 10 million python developers world wide, which is more than Solidity or Rust by large factors.

What helped Python gain popularity in the data analysis and AI field (Python scripts in the front, fast C++ libraries in the back) works with pallet-revive too: Where performance becomes a limiting factor, libraries can just be implemented in Rust using the UAPI instead.

New developments like the Mojo language inspire high confidence in the popularity of Python. Lately, Google wants to launch an L1 with Python contracts. I think that strong enough momentum formed to seriously consider Python smart contracts on Polkadot.

Implementation

What follows is a short brain-dump about various implementation aspects.

Language support

There are two different approaches for Python language support.

a) Use an existing python implementation

Implement a pallet-python using an existing Python implementation (compiler and VM) as a dependency. MicroPython is a bit limited in language support and also not a Rust codebase, so the RustPython project could work better for us. There seem to be successful efforts porting it tono_std - unfortunately it’s not very up to date with the base branch, needs nightly and I faced some compilation problems but overall looks promising.

The python interpreter functionality of pallet-python is exposed via a pre-compile address. The Python source code would then be compiled into byte code and stored in contract storage under a designated, specific storage keys (inaccessible in Solidity). Python contracts would actually be small PVM stubs which delegate call into the interpreter pre-compile.

I haven’t finalized my thoughts around how exactly the runtime handles the abstract concept of Contract instances (create semantics) and module imports. An obvious thing is statically link modules (“frozen” modules). As a future optimization, a JIT compiler could be employed, compiling some modules or maybe even hot paths to PVM byte code.

b) Compile Python byte code directly to PVM

Akin to the Mojo and revive compiler approaches. Results in faster contract code but potentiallly at the expense of larger contract code blobs. I think this is more work than the interpreter route, even when we could work with some existing Python compiler to spare us some scaffolding. In the case of Python however, I intuitively value getting support quickly and investing time into developer experience over having optimized runtime efficiency.

Developer experience

To me the most important part: Making the developer experience absolutely stellar. Short-term, we can implement a tool to convert Python source code into contract blobs. Maybe even integrate it with Hardhat and Foundry. The revive compiler infrastructure could serve as a basis for this. But that’s just the MVP. Ultimately, we make sure that Python on Polkadot makes developers fall into the pit of success:

  1. Full interoperability with Solidity contracts by default. We need to generate Solidity contracts compatible metadata. dApp End-users must not notice that their dApp was in fact not written in Solidity. We also provide audited versions of common libraries like ERC token standards (and work closely with partners).

  2. ink! has shown to me that we should take care to meet non-blockchain-native developers where they are. Writing basic contracts should be very easy and intuitive. We need to provide intuitive and easy to use modules, frameworks and tooling. Things like reading and writing files should just work. What about for example prototyping your smart contract in a Jupyter notebook in the browser? How to do package management in a secure way?

This represents a lot of work (more than the language supporting pallet or compiler). Now you might think, why would it be justified when it didn’t work out with ink!? I see three compelling reasons:

  1. ink! was neither Solidity nor Ethereum compatible. Trying to build a siloed, isolated ecosystem is harder and more effort.
  2. Python has significantly more market share than Rust.
  3. While Rust and Python are both general purpose languages, Rust is more tailored towards low-level and systems programming, where as Python is arguably more tailored towards application development and scripting. dApps fall into the applications category and it follows that Python is the better fit for finding traction in this domain.

Limitations, risks and open questions

Not all existing Python libraries will work well (especially those calling into native code via FFI). We also introduce additional technical depth and attack surfaces. I’m sure there are many more obstacles which I didn’t think of yet.

Alternatives considered

A valuable alternative to me seems to be Javascript (or Typescript for that matter). Arguably also very popular, it seems slightly harder to integrate or to write a working compiler, at least I could not find a viable no_std Rust runtime implementation. I also remember the ask! language framework targetting pallet-contracts.

Closing thoughts and next steps

Supporting one of if not the world’s most popular programming language on Polkadot gets us ahead the curve and could bring us a lot of traction. While we are busy polishing Solidity support, I am curious what all stakeholders think about this. Once we find consensus about if and how exactly we want to do this, resources can be allocated to develop a proof of concept.

Thanks for reading, looking forward to discuss!

19 Likes
2

Cyrill you’re amazing, this is so exciting!

1 Like
3

Very cool! This will also be helpful for writing JAM Services in Python.
I was evaluating writing them in C, which works, and can then be used as a basis to also port micro-python and similar.

But that uses RV64, probably only slightly different for the configs.

1 Like
4

Nice - yeah ideally we are able to provide a unified development stack for contracts and JAM services. Didn’t give it too much thinking yet TBH but I think it’s a very good point: With ink! and FRAME I remember the idea of “the merge” which never came to fruition. So maybe we should think about this from the beginning on :slight_smile: Then there’s also coreplay - which could be used effectively with asyncio? Even though w.r.t. to JAM I thought it might even be even easier to use polkaports and give CPython a shot.

1 Like
5

Very cool work ! I’ve seen Python every time more related with Quant Trading, which would be a very valuable market if crypto markets could outcompete with HFT Quant, an area which some interested groups are developing.

In this thread of thought, something to resolve is the opportunity cost of openly sharing a smart contract for quant competitiveness, but Python surely is an invitation driver for this business approach.

Thank you for sharing.

6

Yes I am using the Polkaports repo to get a Musl standard lib and Picoalloc for memory. Both thanks to Jan Bujak.

I guess SCs and Services could use a shared code-base. Only the entry points would be different. Maybe CPython would be better then, I can try it.

I have not looked into Coreplay yet, but into CoreVM. CoreVM makes a host-call available to the guest program that emulates the Linux Syscall interface. This allows the musl standard lib to do syscalls without realizing that it is not run by a Linux Kernel but in a PVM.
This was a bit too much for me; instead I opted to do the Syscall emulation for the most important calls directly in the service SDK.

7

Thanks for the writeup! I am confused by the two statements above. Your initial idea (first quote) was to deploy the python interpreter as contract. But then in your list of possible implementations it does not appear anymore. The pallet-python approach would not entail compiling an interpreter to PolkaVM, right? Its just part of the runtime.

Or am I missing something?