Generalized multichain monitoring solution

It is clear that an onchain data monitoring solution is super critical. I believe every parachains have built some in house monitoring solution. It will be great if we can work together to avoid duplicated work.

Especially with XCM and have all the parachains interconnected, it is not enough to just monitor data on a single parachain. All the connected parachain data are also important. For example, to ensure the reserve assets matches to XCM reserved transferred asset amount. For such monitoring, we will need to be able to pull data from multiple parachains.

In additional to that, each parachain would have its own way of storing data. For example, for token data, some parachains could use pallet-assets or orml-tokens or their own multi-token pallet. Therefore, it is hard to have a generalized way to process those data without getting to implementation details of each parachain.

It will be great if we can all work together to build a generalized multichain monitoring solution that allow anyone to easily create monitor for any parachains and reuse / composite / collaborate monitors together.

Some tech stack ideas:

  • Use smoldot to connect to all the chains at once
  • Use wasm view function to write accessors & assertions and have them compiled as wasm
  • Monitors are compiled as a wasm and we can have shared hosted service running the user provided wasm
2 Likes

I’ll share about our project that I think could be relevant. We have been working for a while on different ways to integrate Substrate and Matrix as there are lots of benefits the decentralized communications protocol can bring to the table. One such benefits being monitoring on-chain data where Matrix can take care of publishing insights in rooms that anyone can join to monitor and get notified automatically.

Our PoC is a container with a Substrate node, a lightweight homeserver like dendrite/conduit and a general purpose appservice that comes with its own WASM runtime to extend the homeserver functionality with plugins that can be composed. One such plugins is a Substrate client that talks to the node via RPC to let other plugins submit transactions, query or subscribe to state changes. The next stage of this project is to pack everything in a single lightweight and easy to deploy binary where the WASM runtime also hosts the homeserver and a smoldot light/full node as plugins. Chains will also have a pallet-summa(substrate meets matrix) that couples the two ecosystems further adding nice extra features.

Having a general purpose WASM runtime where plugins can subscribe to events happening in a chain(or multiple) and then be able to publish the result of a computation on a storage that anyone can replicate in a decentralized way is a great solution to this kind of problem.

1 Like

I briefly presented Moonbeam’s in-house monitor solution for this, and the way we set up our monitor to track Sovereign Account balances on each chain and total supply of the representation asset. You can find the slides here: 20221129 - Moonbeam XCM Sub0 [External] - Google Slides

We rely on public endpoints (with redundancy and we check for endpoint health), and programming how each asset is defined as a multilocation, and how to read the sovereign account’s balance on each specific chain.

The idea’s mentioned seem good but it might take some time to implement.

1 Like

Hi,

Wanted to let you know that we have built a multi-chain monitoring SDK that addresses some of the use cases that you’ve mentioned i.e. connection with Smoldot and pallet-specific modules. We also have an example application for monitoring sovereign account assets across parachains.

You can find all the details in our post, including the Github repo link and our future plans on building a potentially decentralised network of monitoring agents. Would love to know if what we are building is useful for you and what capabilities you would like to see in a multi-chain monitoring solution.

I see it is an SDK which could be a useful foundation component of the monitoring solution I want, but lot more works are still required for it to be useful for my use cases.

You’re exactly right that it’s simply a foundation piece of a bigger solution. We’re currently fleshing out a proposal for a follow-up grant to build the portal, where users could deploy and manage pre-defined configurable monitoring programs, which would be made available in a marketplace. These programs will be built with our SDK and will contain the complete logic for complex monitoring use cases. They can be instantiated as monitoring agents that will emit data streams that we plan to make it easy to integrate with existing monitoring, alerting and visualisation infrastructure e.g. Prometheus, Grafana

The proposal is still WIP but we already have some mockups of the portal available here:

It would be great to hear from you what kind of use cases you’re looking at so we can take it into consideration when designing the system.

Let me give you some use case:

  1. Ensure consistent balances of sovereign accounts. We are usually using reserved transfer to transfer assets between relay/parachains. This creates an invariant that the total issuance of the foreign asset on dest parachain should not be more than the reserved asset amount of the sovereign account of the dest chain on source chain. We should receive alerts if a difference is detected for more than a few minutes.

  2. Ensure other crosschain state consistency. Acala/Karura offers liquid staking solution and the parachain controls multiple staking accounts on relaychain. The invariant is that the total value of the liquid staking asset on parachain should match to the staking assets on relaychain.

1 Like

Thank you very much for the feedback!

We do have an example application for your first use case. You can check out the code here. Of course, it’s still missing the part of time series data and alerting which is what we plan to tackle in our next grant.

When we have the time series and alerting implemented, and the applications as one-click deploy agents, would this be something that you and your team would be interested in using?

Monitor balances on two networks is the easy part. I didn’t check all the features in your SDK but let me list some of my requirements:

  • Robust. Able to handle (temporary) network fork / rollback, RPC unavailability, service down, etc.
  • Easy to deploy & maintain / managed host.
  • Integration with alert/notification system.
  • Support all the commonly used networks.
  • Abstract difference between networks. (e.g. some chain use pallet-assets and some use orml-tokens and some use their own)
  • Easy to configure / not require a senior dev to configure
  • Support dashboard. e.g. easy integration with grafana

Thank you for taking the time to list your requirements, we will incorporate them in our development plan.

We’ve rolled out the first release of the XCM Monitoring Server that meets most of the points above. Also supports connecting with smoldot. We plan to add open observability in the future.

Let us know if you have any questions or feedback!

Visualisation is almost monitoring: https://dotsama.world (but this has no back end and is just to see what’s going on)