XCM Global Asset Registry (GAR): Call for contributors

Ecosystem
1

Background: This is a call for contributors to a XCM Global Asset Registry as a followup to Polkadot Summit: Barcamp on Nov 30/Dec 1, 2022

You can volunteer as a contributor by submitting a PR in this XCM Global Registry github repo !

Problem: Asset Registry

Substrate asset registry pallets (e.g assets, tokens, currencies pallets) are maximally flexible and are not designed with cross-chain functionality in mind - they does not require all parachains to use the same “local” ids (or symbols) to represent a cross-chain asset.

For example, KSM has 10+ different local representation among parachains:

Chain CurrencyID
Khala 0
Shiden 340282366920938463463374607431768211455
Moonriver 42259045809535163221576417993425387648
Robonomics 4294967295
Calamari 12
Parallel Heiko 100
Basilisk 1
Kiko 100
Mangata 4
Listen 2

Even if all parachains agree to use the same local symbol, the asset registry alone is not sufficient enough to link these xc assets together. Moreever, taking Statemine’s USDT as another example, the same local symbol “USDT” can be added (and actually have been added) by anyone to its asset registry to potentially fool the unsuspecting community:

Symbol Name CurrencyID
USDt Tether USDT 1984
USDt Tether USDT 19840
USDT USDT 11

XCM Multilocation

So how does polkadot unify the same asset across different parachains? Partial Answer : MultiLocation. Yet, multilocation are also designed to be maximally flexible - In xcmV2, cross-chain assets can be specified in a combination of {Parachain, PalletInstance, GeneralIndex, GeneralKey}, coupled inside x0/x1/x2/…/x7 structure. And each parachains has a different pallet for their xcm asset registry.

Lack of XCM Global Asset Registry (GAR)

Thus parachain engineers, dapp developers and analytics providers are currently faced with a “Tower of Babel” to align each parachain’s asset registry and XCM asset registry and associated weights/fees - multichain dapp developers (including multi-chain indexers like Polkaholic.io) are required to independently develop this mapping just to initiate seemly simple XCM tasks like transferring “KSM” or “USDT” from one chain to another or indexing XCM transfers. In our opinion, it’s counter-productive to require multichain app developers to independently piece this together, and futhermore, read fee constants in N parachains to support their multi-chain dapps and be faced with so much friction.

In our work in Polkaholic.io XCM Indexing, we have developed a useful API that attempts to address the Tower of Babel:

The above datasets are visualizable in the following UI:
Polkaholic.io Multilocation Tool

We believe this already covers as much as 90% of the cross-chain transferable assets and over 97% of the XCM Transfer USD volume in Polkadot + Kusama at present. However, we believe that the recipe for this dataset construction should be managed not by one “trusted” team but with

  • (A) Open Source Data Generation (automated Github Action) - Given Input: Polkadot + Kusama WSEndpoints from polkadot.js apps, augmented with a polkadot.toml file containing details of how to process the (xc)asset registry of each parachain

    • Step 1: Crawl assetRegistry + xcAssetRegistry and store it in JSON file
    • Step 2: Aggregate and publish registry keyed by XcmInteriorKey

  • (B) Joint Collaboration -

    • Having Open Source Data Generation managed by parachain teams who model their own (xcm) asset registry and fees accurately when working their parachain partners.

The data generation process is technically simple, but we cant stress enough the importance of joint collaboration - currently many parachain chains are largely building their own ‘xcm-tools’ independently of one another and only trying to cover a subset of the xcm global asset registry problem.

Key Use cases for the XCM Global Asset Registry (GAR):

  • Powering XCM Transfer dapps with Multilocation
  • Parachain Bridge Monitoring
  • Statemine DEX Aggregator
  • Cross-chain Price Quote Mechanism that have MultiLocation

Together, we can do much better, and be more reactive to any incompleteness and inaccuracy, because simply put, 90-97% is not good enough to Here the expectation is to collaborate for the common good/maximal impact:

  • When parachains or dapp developers see errors in step 1/2, they submit PRs because their community depends on the output
  • Parachain reviewers from the affected teams will approve the PR, with special attention to how the registry changes with any change.
  • Updates to the repo’s output dataset
  • Data pipelining with Github actions

We can only succeed if parachains possess high reactivity (< 12-24 hrs) here and are not bottlenecked by a central reviewer, and data quality is either at 100%, or the data contains 100% reliable social proof data that users can understand.

2023 Roadmap / Open Bounty Program

A partial list of what we can hope to achieve using GAR in 2023:

  • Rapidly adapt to XCMv3/… changes
  • Augment standardized/custom “weight/fee” information
  • Support MultiLocation users, e.g. DEXes, bridge monitors, APIs, Cross-Chain Indexers
  • Augment each asset in the result with a “pageRank” type reflecting degree of support by parachains, such that dapp developers can threshold on this attribute, where more parachains supporting an asset will apply
  • [something else you and your team think is important]

Funding / Call for Contributors

We (a collective of people working on this project) seek $125K/quarterly budget in 2023 to support:

  • 20% Colorful Notion (Q1/Q2 2023)
  • 50% 10-12 parachain developers + on a quarterly basis [below]
  • 10% Some other team (Q3 2023) [elected by 10-12 devs, with primary implementer as tie-breaker]
  • 10% Some other team (Q4 2023) [elected by 10-12 devs, with primary implementer as tie-breaker]
  • 10% Open bounty requests [proposed by 10-12 devs]

We seek 10-12 parachain developers + additional developers request to maintain this very simple but extensible XCM Global Asset registry. Ideally:

  • You should be involved in your parachains (xc)asset registry, have debugged XCM transfers with at least 2+ other parachains, and can speak to how you can represent your fees in 2023 if you’re going non-standard in some way.
  • You are comfortable signing up for debugging a pretty simple Javascript dataflow processing problem for your own parachain a couple of times in 2023.
  • You will contribute up to 100 hours/yr from each parachain to support this effort, specifically how things may change with:
    • XCMv3 changes – GlobalConsensus
    • ERC20/721/1155 + PSP22/34/37
    • Kusama/Polkadot bridges
    • Snowfork Ethereum bridge
    • Remote execution “Transact” support

Additional Reading:

Please volunteer as a paid/non-paid volunteer by submitting a PR in this repo:

Thank you and Happy Holidays, we look forward to great 2023 together!

5 Likes
2

Happy to see more effort on this area.

I also drafted a global asset registry design while ago but never actually implement it.

Instead, we build local asset registry and suits our short term needs very well.

Now we have more parachains connected and lot more XCM transactions. I definitely see it is now a good time to resurface this and create a global asset registry together. This will be very useful for crosschain tooling, which are very lacking.

As a Polkadot/Kusama governance participant, I am fully supporting a bounty for this.

As an Acala maintainer, happy to contribute developer hours into this.

3 Likes
3

I believe this will allow us to effectively deploy what is described in XCM as a Standard for Reading And Interacting with Parachains as well, using XCM as the unified read and write language over balances.

4

Happy new year, and welcome back from holidays, everyone!

This was one of the particularly fruitful workshops at the Lisbon summit and I am very eager to get this some more attention and love.

I am going to be spending some more time on this in the coming weeks, and any ideas on how you think that we can get initiatives like this more exposure are greatly appreciated, it’s clearly defined and valuable as far as I can see.

5

I’d like to shout out the orml-asset-registry as well, although it’s more of a foreign asset registry, i.e. it keeps track of what asset from other parachains are supported. We could extend that registry to also store local currency info, like the name, decimals, and multilocation of native assets. That’d simplify data generation for anyone using this pallet.

Also related reading: polkawallet-bridge

6

Happy holidays, everyone!

My name is Michael. I’m the original implementer of polkaholic’s xcm indexer and the initial contributor of xcm-global-registry. Two months ago, we demoed at barcamp how Polkaholic match xcm using the traces/events/globalregistry. Since then, we have made enough progress to make this community project into a contributable form - all the xcmMultilocation standardization logics have been extracted away.

A brief summary of what we have accomplished so far:

  • 33 polkadot xcAssets in xcmRegistry
  • 45 Kusama xcAssets in xcmRegistry
  • Over 15 custom parachain parser written
  • Over 225+ local xcAsset correctly parsed among 50+ chains
  • support tokens and asset pallet parsing
  • support commonly used xcm registry parsing(including oralAssetRegistry)

We have modularized the registry parser in a way that a custom parser can be spin up under 30min (see tutorial [here]); Optional registry augmentation can be added to improve registry, although is not required to implement a custom parser

While I’m optimistic about the current result, the registry parser logic is not perfect and may contains bugs that I’m simply not aware of. Hopefully the assets directory(which the github action will auto publish regularly) would enables every parachain team to quickly inspect the erroneous registry about thier chains. And ideally, having parachain team/volunteers to submit a PR to fix it. For example, the Native asset’s are currently missing for Acala’s ERC20 assets and Interlay’s KBTC and etc

I’m looking for feedbacks/criticism/contributions to make this xcm-registry maximally useful. You can always find me on element @mkchung:matrix.org ; also feel free to reach out and we can do zoom call if you have any question about this repo

3 Likes
7

Also checkout our 2022 XCM Transfers public dataset - powered by this repo!