Multichain XCM integration test

The last runtime upgrade of Statemine increases tx fee by 10x, and it breaks XCM transfer from Karura to Statemine due to the configured fee is no longer enough to buy execution and ends up assets trapped.

Karura have now updated the tx fee config and should be able to claim the trapped assets back to rescue the funds in a future governance proposal.

This is not the first time Statemine upgrade breaks parachain integration. And we know there will be a big breaking change coming later this year: XCM V3. No doubt one chain will upgrade before others and there is a good chance we have some compatibility issue during the upgrade. Therefore we need to have a solution to test XCM compatibility between runtime versions.

Shaun from our team have developed XCM emulator (in addition to XCM simulator, which is contributed into Polkadot now) with the goal to be able to write unit tests to test XCM related code and that works fine. However, we can’t use this to test compatibility between versions. This is because at this stage, it is not possible to compile two runtimes using different version of Substrate in a single Rust binary. So we can’t test Statemine with XCM v3 with old Karura with XCM v2. We need a different solution.

Chopsticks have recently implemented the ability to launch multiple networks and transport XCM between them. Combined with the ability to override Wasm and perform runtime upgrade, we should be able to create a tool to leverage Chopsticks for e2e tests and ensure future runtime upgrade is compatible with other networks.

However, this must be a mutual effort between all the parachains connected with XCM to ensure the tests are up-to-date and run the tests with new runtime and notify others on breaking changes.

I have some vague idea on how this could look like but I am refraining propose it until it is getting more clear and meanwhile see what’s everyone’s opinion about this.

Please comment to signal if you are willing to commit to this effort to maintain compatibility between your parachain and all other connected parachains.


This tool: GitHub - paritytech/parachains-integration-tests: Tool to make Parachains integrations tests development easier and faster was created with the goal of testing XCM integration between chains, and has been successfully used to prevent unexpected results from different Polkadot/Substrate versions between the System Parachains and Kusama/Polkadot (you can find the tests under the examples folder). You can also find it as npm package @parity/parachains-integration-tests - npm

It’s been almost a kind of a solo effort, and unfortunately lately I didn’t find as much time as I’d like to keep improving the tool. I invite the community and Parity to participate on it as I strongly believe it can be very useful and can help many Parachains teams to test interactions with each other.