How to handle not-withdrawable assets properly with XCM?

Tech Talk
1

I am currently working on the transfer of Moonbeam ERC20 tokens through XCM, and in this context I have read all the resources I could find on XCM to make sure that my implement is compliant with the XCM standards and expectations, but I can’t find a not-convoluted way.

The idea of this post is not to talk about ERC20 in particular, nor about Moonbeam, but to answer the more general question: how to manage properly through XCM an asset which is not burnable by the XCVM?".

Following the reading of xcm format, these kind of assets should never be placed in the Holding Register as they still exist in the on-chain persistent storage (on our example, in the contract storage).

That’s why I went with an implementation of the AssetTransactor that returns the error XcmError::NotWithdrawable for method withdraw_asset.

To send a not-withdrawable asset to another chain it works, because the TransferReserveAsset instruction does not need to go through the Holding, but the problem arises to send back this asset to the reserve chain, because the InitiateReserveWithdraw instruction forces to send a message like [WithdrawAsset, .., DepositAsset].

A reserve asset never needs to be burned and then minted, it just needs to be transferred from/to the sovereign account, which is why I think there should be a way to properly manage these kinds of assets.

Proposals

Solution 1: Morph incoming messages (my current workaround)

Create a specific Barrier that morph incoming messages that match the exact pattern [WithdrawAsset, ClearOrigin, BuyExecution, DepositAsset] into [WithdrawAsset, BuyExecution, TransferAsset, DepositAsset].
Of course, only the not-withdrawable assets should be moved to TransferAsset, the withdrawable one should still in the initial WithdrawAsset instruction to be able to pay fees.

It seems to work well but it’s convoluted, and limited to specific cases.

Solution 2: Rework instruction InitiateReserveWithdraw to handle properly not-withdrawable assets

Ideally, the chain that runs InitiateReserveWithdraw doesn’t have to worry about whether the asset is withdrawable or not on the reserve chain, it should just have to say what it wants to do with this asset (deposit on a recipient account or manipulate it for other things).

We could for example add the parameter assets_to_transfer: Option<(MultiAssetFilter, Multilocation)> to be able to express which assets just need to be transferred, the new implementation of InitiateReserveWithdraw instruction could look something like this:

InitiateReserveWithdraw { assets, assets_to_transfer, reserve, xcm } => {
	let assets = Self::reanchored(
		self.holding.saturating_take(assets),
		&reserve,
		Some(&mut self.holding),
	);
	let mut message = if let Some((filter, beneficiary)) = assets_to_transfer {
		vec![WithdrawAsset(assets), TransferAsset {
			assets: assets.filter(|asset| filter.matches(asset)).collect(),
			beneficiary
		}, ClearOrigin]
	} else {
		vec![WithdrawAsset(assets), ClearOrigin]
	};
	message.extend(xcm.0.into_iter());
	self.send(reserve, Xcm(message), FeeReason::InitiateReserveWithdraw)?;
	Ok(())
},

For backward compatibility, we can just set the new parameter to None when we convert the instruction from v3 to v4.

In the case of a user/dapp that just wants to send 2 assets (with one to pay the fees), it would submit the following message:

Xcm(vec![
	WithdrawAsset(assets.clone()),
	InitiateReserveWithdraw {
		assets: All.into(),
		assets_to_transfer: Option(Definite(assets_to_transfer), beneficiary),
		reserve: reserve.clone(),
		xcm: Xcm(vec![
			BuyExecution { fees, weight_limit },
			DepositAsset { assets: All.into(), beneficiary }
		]),
	},
])

Solution 3: Create a new dedicated instruction

For example, we could create a new instruction TransferToReserve with the following parameters:

TransferToReserve {
  assets: MultiAssetsFilter,
  fees: Option<(MultiAsset, WeightLimit)>,
  reserve:: Multilocation
  beneficiary: Multilocation
  xcm: Xcm,
}

This new instruction would send a message like (just pseudo-code):

	let mut message = if let Some((fees, weight_limit)) = fees {
		vec![WithdrawAsset(fees.into()), BuyExecution { .. },
            TransferAsset {
			assets: assets.filter(|asset| !filter.matches(&fees)).collect(),
			beneficiary
		}, ClearOrigin]
	} else {
		vec![TransferAsset(assets), ClearOrigin]
	};

What bothers me with this solution is that every user/dapp dev has to ask himself if he should use the InitiateReserveWithdraw or TransferToReserve instruction, which makes understanding and using XCM more complex. I think it is better to modify the existing instruction rather than adding a new one.

Of course all this are only proposals, maybe there is a simpler solution that I have not seen, if so I would be happy to learn it :slight_smile:

2 Likes
2

First things first, since the name of the instruction is InitiateReserveWithdraw, any solution that doesn’t respect the “withdraw” part should not be considered. Thus, solution 1 and 2 are both automatically eliminated from consideration.

We now have solution 3 as the only sensible one to consider. I would probably call the new instruction InitiateReserveTransfer instead of TransferToReserve, as it should hint to people that it performs almost the same operation as InitiateReserveWithdraw, albeit using a TransferAsset instruction rather than a WithdrawAsset instruction.

If the issue here is that it increases complexity, then the real question to ask is why the asset being transferred is not withdrawable, since all the new instruction is doing is surfacing the complexity behind the distinction between a withdrawable asset and a non-withdrawable asset.

Alternatively, there is an easier solution. The entire reasoning behind the holding register to not contain any asset that still exist in on-chain storage is so that total issuance is maintained. The “owner” of the asset is technically the XCM executor, and so as long as the non-withdrawable asset is transferred to an account that can only be accessed by the XCM executor, you would not need to burn anything. Such an account could also double as an asset trap.

3

Because the ERC20 specifications do not support it, we have no way to burn/mint a token managed by a contract that is only required to be compliant with the ERC20 specifications.
And we can’t change that, each contract may have its own logic for burn or mint, or simply have no code for that.
We want to allow the transfer via XCM of ERC20 tokens already deployed and whose contract code may not be upgradable.
The only thing that can be done in the AssetTransactor implementation is to transfer tokens from one on-chain account to another on-chain account.

I’ve thought about this solution before, but besides the fact that it doesn’t comply with the XCM specifications, it has many problems:

  1. This requires two transfers when receiving a token (sovereign->“holding account” then "“holding account”-> sovereign). This is much more expensive in resources and costs for nothing, especially since here it requires two separate evm calls, with their high basic costs by nature. This is not acceptable to us.
  2. This causes side effects and incompatibilities with some instructions that take assets on the holding, like BurnAsset for instance, the assets will still in the “on-chain holding account” even though XCVM believes they were burned. If a remote chain rely on the result of the XCM execution, the XCVM will report that the assets was burned even it’s not the case.
  3. If there are still assets in this “on-chain holding account” at the end of the execution, they must be transferred to a kind of “on-chain trap account”, which forces one more expensive evm call.

It definitely seems like a very bad idea.

A temporary solution to not having to wait for a new XCM instruction would be to add a call to the xcm pallet somewhat similar to the reserve_transfer_assets call but to return a foreign asset on its reserve chain to a specific account. This call could specifically send a message that only serves to transfer the asset to a “beneficiary” account on the target chain (and include a fee_asset for fees).

1 Like
4

I see now, I’ve always thought the ERC20 specification contains burn and mint functions – seems like my EVM knowledge needs an update. Oh well, c’est la vie, n’est pas?

Now, about the 3 problems that you mentioned – I can only say that the real issue is the expensiveness of the EVM call, but that too is solvable. However, let’s talk about why 2 and 3 are not actual issues: the act of “burning” can take on many forms, one of which can simply be a transfer to an inaccessible account, and this is not without precedence in the EVM world – tokens are regularly transferred to the zero account on Ethereum to achieve a burn, so if we abstract the principle behind the process, “burning” means “taking the asset out of circulation”. As long as you have a process that can do so, then it can be considered as a burn. The XCM executor certainly doesn’t care whether it’s done by dropping the assets (Substrate) or sending it to an inaccessible account (EVM).

As for 3, I think I’ve specifically mentioned that the on-chain “holding account” can serve as an asset trap, so I’m actually not sure why you’ve raised this as an issue again here.

Finally, to solve the expensiveness of an EVM call, what we can do is to implement transactional processing with the XCVM. We actually will implement such a feature in the XCM executor, and it is tracked in this issue. However, I believe that issue is attempting to implement per-instruction transactional processing, so it doesn’t exactly fit your needs as what you in fact want is to hold off “committing” the withdrawal of an asset, until your XCM executor sees and executes the DepositAsset instruction. This can also potentially solve the asset trap problem as well, as any failed transaction would simply not contain any actual transfers, and the assets are left untouched at the source.

What we can do then is to make the API of the per-instruction transactional processing extensible, so that it can support aggregation of multiple instructions in a transaction. That way, you can then customize it to your needs. This does indeed sound like a feature for XCM v4, so in the meantime, I’d suggest extending your XCM executor to allow transactional processing to solve your particular needs.

5

Yes no worries, I know that you already have so many technical subjects to follow, you can’t be experts in everything :sweat_smile:

I can affirm on the contrary that “the real issue is that XCM requires two operations (Withdraw then Deposit) to answer a need that can be answered in a single operation (Transfer)”.
It is not up to existing things (like EVM) to adapt to XCM, it is up to XCM to be flexible enough to support any type of “systems”, otherwise XCM cannot claim to be really agnostic.

It can’t, because between two messages the on-chain “holding account” must be emptied, otherwise we’ll end up in inconsistent cases where the content of the on-chain “holding account” won’t correspond to what is in the holding register. A following message execution can then consume assets that are supposed to be trapped (and therefore can never be Claimed in the future).

That’s pretty much what I did initially, I had an implementation of WithdrawAsset that just tracks in a new custom registry the ERC20 assets that are virtually sent and only really does the transfer in deposit_asset: https://github.com/PureStake/moonbeam/blob/a3721deb89efa7fb7633a403990e4796b87d5ef1/pallets/erc20-xcm-bridge/src/xcm_holding_ext.rs

But it is a complex implementation to maintain and poses problems: If between the Withdraw and the Deposit there are other instructions that use the assets in the holding (like BuyExecution or BurnAsset), how do we manage that?
Finally, for this solution to be viable we would have to use a polkadot fork that modifies the behavior of the XcmExecutor for many instructions, which is much more complex, difficult to maintain and risky.

1 Like
6

I think the core message that I wanted to put out is that each operation that the XCM executor performs doesn’t have to always match 1-to-1 to any on-chain operation. Applying this principle to the problems you’ve mentioned, the solution becomes simple:

Answer: sure, the XCM executor sometimes requires 2 instructions to accomplish a transfer, but that does not mean you need to map each XCM instruction to exactly 1 on-chain operation. This is in essence the same idea behind compilers, microcode and VM instructions – they all don’t have statements that correspond 1-to-1 with the machine code of the target architecture, and so do the XCM instructions vs on-chain operations.

Answer: store some metadata on the Substrate side for the ERC20 token for the on-chain “holding account”, detailing how much of the asset is actually in holding and how much of them is trapped, and do verification checks on every DepositAsset or ClaimAsset instruction to ensure you never overconsume assets that was not meant for the other.

Answer: first, blackbox the XCM executor, and think about what it ultimately does. It essentially consumes some encoded data, and then somehow understands that some funds need to be transferred from one account to the other. In other words, if you view one XCM as an entire transaction, then all that matters is that you gather which accounts need to be debited and which accounts need to be credited. The details of how the credit and debit is connected lies within the exact instruction used, and you can determine that by examining them.

7

Yeah XCM isn’t really EVM friendly. I also raised this issue before: XCM executor should use transfer over withdraw/deposit · Issue #6553 · paritytech/polkadot · GitHub

Our workaround is indeed create a temporary holding register account to simulate withdraw/deposit. However it still have some less than ideal side effects. Acala/lib.rs at eda23d91fe730681b56b362323928ffa7dcb9cf0 · AcalaNetwork/Acala · GitHub

1 Like
8

To quote the XCM design document:

Being Agnostic means that XCM is not simply for messages between parachain(s) and/or the Relay-chain, but rather that XCM is suitable for messages between disparate chains connected through one or more bridge(s) and even for messages between smart-contracts. Using XCM, all of the above may communicate with, or through, each other.

The intent is that XCM be general rather than be constrained to the lowest-common denominator which all systems could possibly support perfectly. Withdraw + Deposit is strictly more general than Transfer. You can accomplish everything with a combination of Withdraw & Deposit which you can with Transfer, but not the other way around. Certain cross-chain asset transfer models such as teleports essentially rely on withdraw and deposit to be cleanly implementable. Cross-chain exchanges make much more sense with the Withdraw-Exchange-Deposit model rather than faffing around with transfers in and out around the exchange operation. Execution purchasing makes much more sense if you first withdraw, purchase then deposit any remaining. Transfering is not a “basic operation”, it’s just a fairly common combined operation made more so by ERC20, practical because ERC20 tokens are almost never used to pay fees, an assumption we do not (and cannot) make in the XCVM model.

I would echo Keth’s point that the failing here is of EVM’s being so expensive - it’s perfectly reasonable to model withdraw and deposit as transfers to and from a special “holding account”.

A more complex implementation of the XCVM could do lazy transfers: withdraws would not actually touch state but some additional internal state of the XCVM could silently track the accounts which are holding the assets in the Holding Register and thus which assets should be transfered from when the asset is used (for Exchange, Deposit, BuyExecution, or any other of the instructions which mutate the Holding register). But this is a fairly complex task and not code I especially want to write or maintain.

3 Likes
9

Thank you guys for your detailed answers, I understand better why XCM works like this now!

Not only that, another problem that makes the “holding account” solution unsuitable for us is that it generates two ethereum logs Transfer instead of one, showing a temporary account for the explorer blocks that is not relevant, and pollutes all the tools that rely on these ethereum logs.

From what I understand from our discussion, the only solution for us is to maintain our own custom version of the XCM executor, the only solution for us is to maintain our own custom implementation of the XCM executor that performs a lazy transfer for ERC20 assets (and probably also ERC721 in the future).

An intermediate solution might be to make the XcmExecutor implementation a bit more customizable, so we (on the moonbeam side) could just code and maintain only a subpart and rely on the polkadot implementation for most parts.

I would be happy to research along this path and propose a PR on polkadot.

2 Likes
10

I spent some time looking for ideas, and one idea that came to me to achieve this goal is to move the process_instruction function into a new ProcessInstruction trait.
Thus, if a parachain (like us) needs to modify the behavior of some instructions, or/and add some pre/post hooks, they can create a wrapper that implement the ProcessInstruction trait and rely on the polkadot implementation as much as possible.

I just opened a PR: Refactor XCM Executor: move `process_instruction` to a trait by librelois · Pull Request #6922 · paritytech/polkadot · GitHub

If you have a simpler idea to achieve the same goal, I’d be happy to hear it :slight_smile:

1 Like