Redesign Event system

Tech Talk
1

The purpose of events is to allow clients (dApps, wallets, etc) to understand what happened within a block. The current event system of Substrate works but also have number of issues that make it not suitable for many use cases.

Problems

All the events in a block are stored in a single storage entry, which makes it not suitable for light clients and cross chain use cases. The size of an event proof unpredictable and can be easily manipulated.

Events are somewhat similar to logging/tracing that allow runtime to output something so others can observe the inner details of the execution. All the logging/tracing library have verbosity level configuration to allow the application to output the right amount of the details. The current Substrate event system is suitable for dApp use case however not very useful for tracing use case to analysis exact execution of the runtime logic. There is no easy way to have the runtime to output extra details. For example, it is currently very hard to determine which transaction caused a storage change in a block.

We don’t have enough built-in events which makes a lot of simple requirement very complicated to implement. For example, there are a lot of additional work required to distinguish events in a batch call. Impossible to associate events triggered in hooks with the actual calls. Very hard to determine what events are associated with transaction fee payments.

Requirements

We have to redesign the event system and here are some of the requirements I have so far:

  • Efficient to emit
    • Emit event should have minimal overhead, should not increase PoV size.
  • Efficient to proof
    • It should be possible to generate a proof of existent of some events with minimal proof size. Light client and runtime should be able to verify the proof very efficiently.
  • Efficient removal
    • We need to purge all events from previous block at beginning of block construction so remove old events should be as efficient as possible.
  • Easy to query
    • Clients and light clients should be able to query some particular events without iterating every single events in the block
    • Clients and light clients should be able to subscribe for some particular events by monitoring for storage changes for some storage keys
  • Structural
    • Events should be structural so it is possible to distinguish and associate events with corresponding calls to correctly handle some complicated calls such as multisig + proxy + batch. We need to know which events are related to the multisig call, proxy call, batch call, each batch item call, non call part (e.g. tx fee).

Ask

I would like to ask everyone to help create the new event system. Please help confirm/refine the requirements, propose data structure we should use, draft high level API, etc.

This is an essential step to allow light clients to handle a busy chain, for parachains to very events on other chain, for client developers to able accurately analysis onchain data and many more use cases.

Original Substrate issue: https://github.com/paritytech/substrate/issues/11216

7 Likes
2

@xlc

  • what is the reasoning for this requirement?
  • basically this means, that a proof of event can only happen with knowledge of the block it happened in and not canonically through the latest state-root, right?
3

This is already the case today. Events are deleted in System::initialize, aka when starting to build a new block. You don’t want that to be slow and slowing down your block production, so it needs to be fast. You also don’t want that the old events land the proof of the block you are just building, because that would be waste of POV space.

4

You also don’t want that the old events land the proof of the block you are just building, because that would be waste of POV space

when would this be the case, when keeping them with no differentiation at which block they where emmited?

I know, but why not rethink this?

  • What would be an advantage/use-case of allowing to prove events for any past extrinsics through the latest storage-root?

Maybe:

  • frame_system::Events = ChildStorage<_, Blake128, ExtrinsicIndex, Vec<Event>> → Of course heavier to delete during initialization of a block
  • System::finalize → Calculates child-storage-root, stores child storage root maybe with key=BlockNumber,value=RootOfChildStorageEvents
5

Events are only stored for one block, not for older blocks. In general I don’t get what you mean here.

You can just proof them by taking the block the extrinsic was applied. Not sure we need to do this in some descendant block.

6

You were referring to old events landing the POV. My question was when this would be the case for old events? The current events hit the POV anyways, right?

Yes, but then you need knowledge about past state. At least for other parachains this is rather complex to prove, while all parachains could prove events when they are provable by the latest head of that chain as they get them indirectly via the latest relay-chain head.