Unifying bulk and mid-term blockspace with strided regions

Tech Talk
1

In October, I wrote a post (Blockspace Products and Mechanisms in Polkadot) proposing a few different mechanisms looking forward.

Summing up a few key desiderata:

  • Bulk blockspace markets (parachain slot auctions)
  • Mid-term (1hr - 7wk) purchases
  • On-demand purchases (design writeup here: On-demand parachains)
  • Secondary blockspace markets (re-selling of blockspace, blockspace futures markets, etc.)
  • Sharing bulk blockspace purchases across several chains
  • Elastic scaling for parachains by purchasing parallel blockspace with some combination of the above (Parachain Scaling by Parablock splitting)

In particular, I see mid-term blockspace markets as a key mechanism for practical elastic scaling. Periods of peak demand are often short, so it’s the right mechanism for adapting to predictable-yet-short-lived load.

It’s not clear to me that bulk and mid-term blockspace actually require different mechanisms. Blockspace can enter the system in a bulk format, and markets living elsewhere can handle most of the work of assigning the bulk blockspace onto different chains.

Consider this post an early-stage proposal for new mechanisms to unite both bulk and mid-term blockspace while looking forward to elastic scaling and secondary blockspace markets.

Blockspace Region Subdivisions

Let’s imagine a mechanism that works like this:

  • The Relay Chain auctions off a bulk piece of blockspace, i.e. a dedicated execution core for 6 months. This creates a contiguous blockspace region

  • Every region is a [start, end, stride, offset] data structure. For a region with full access to a core, the stride is 0. For a region that has intermittent access to a core, the stride is nonzero.

  • The owner of a region can split it into pieces down to some minimum granularity in size and maximum stride. Every split comes with a corresponding deposit for the on-chain bookkeeping overhead.

  • Regions can be split in two ways: contiguously or strided. A contiguous split only affects the start and end, while a strided split alters the stride and offset.

  • Regions can be transferred by the owner.

  • Regions have a single ParaId assigned to them, which can be changed by the owner.

  • Regions can be recombined when they correspond to the same core and together form a well-formed region

  • The Relay Chain’s scheduler looks at the next upcoming regions for each core to determine the upcoming ClaimQueue that collators and validators watch.

Re-selling blockspace

A parachain looking to re-sell blockspace can simply split off a region and sell it (in its own runtime, or on a dedicated parachain for this purpose). This sale can be made atomic, and can be performed using XCM, and would allow parachains to monetize unneeded blockspace.

Bulk Slot Sharing

Strided regions would allow multiple chains to share a bulk slot in even or uneven proportions.

Elastic Scaling

This depends on other mechanisms for supporting chains to occupy multiple cores at the same time, but once that is completed, a parachain looking to scale in the near future would be able to purchase regions corresponding to one or multiple cores

A dedicated market for deferred fragmentation

One clear issue with this approach is that regions can become heavily fragmented and even though they can be combined, this could carry high overheads.

What makes most sense is to defer the splitting of regions to the latest point possible (i.e. delivery) by having a dedicated market on a parachain to trade potential regions which then finally resolve shortly before they are relevant, by issuing the appropriate parachain assignment and region splitting calls on the relay chain via XCM.

This is a notable project for R&D that should only rely on the right interfaces being set at the relay chain level.

Strides and Availability Delays

The main issue with scheduling on execution cores is that cores are occupied from the time a parablock is backed to the time that parablock’s data is available. Availability is dependent on the entire validator set, which means that blocks can occupy an availability core for a fairly long time, although the size of the data is bounded.

If we interpret strides as “turns” on an execution core, then we may need mechanisms for either truncating or extending regions as some turns may take longer than expected due to availability delays. Truncating regions has the effect that regions may not actually be as long as initially expected in the face of availability delays, whereas extending regions means that the point in time when the region is concluded is not as predictable. This is worth discussing further.

6 Likes
2

I’d been struggling to properly grok scheduling and how the blockspace market could work. This and the earlier posts [1] [2] were very helpful.

They coalesced my understanding and I think I’ve had my light bulb moment, the cloud/AWS analogy helped cement it.

I have a few thoughts / questions, apologies if this comes across as too scatter-brained:

  • I love the concept of mid-term auctions but wonder then what the point is of parathreads? If a parathread is a short term “spin up” ephemeral parachain and a user can purchase the same effect through a mid-term auction - what’s the difference?
  • You talk about the existing bulk blockspace auctions being the “seeding” market for execution cores/regions and that those can then be “resold” at will by parachain owners to form the mid-term market. What about the execution cores that are to be allocated to parathreads, how will they interface with this?
  • The market concept reminds me of ethereum gas tokens where users “purchased” gas by exploiting the gas refund mechanism of ethereum. This led to a market in gas/blockspace which has echoes of this. However, I’m conscious that was ultimately removed from Ethereum and considered harmful. Are there any market force parallels here that we should be wary of? E.g. is it desirable for blockspace hoarders to come along and buy up swathes of future space or would that be an afront? You could argue they become market makers, but it feels a bit icky.
  • This thread seems to suggest that mid-term auctions would actually be “bought” as opposed to the lease mechanism currently in place for bulk auctions. Would this be in DOT or is it the region holders prerogative as to what they sell them for?
  • If the execution cores for parathreads end up within this mid-term auction space, is there an opportunity to “burn” the DOT they generate and counter some of Polkadot’s inflationary rewards?
  • If the most successful parachains become the primary purchasers of blockspace initially to sustain their level of growth, do we risk a “winner takes all” dynamic where Polkadot gravitates towards serving a majority parachain which operates most of the execution cores at any given time and prices new projects out of the market?

Really looking forward to seeing this develop. It does feel like a break from the mould and creation of something new. Like you say… “the most effective blockspace producer in the world.”

3

the point is of parathreads? If a parathread is a short term “spin up” ephemeral parachain and a user can purchase the same effect through a mid-term auction - what’s the difference?

Parathreads (or as I call them, on-demand parachains) are still useful for intermittent block authoring and serve as a useful on-ramp. Use-cases that make 1 block every 10 minutes, for example, like early-stage pre-PMF products, might not need more than this. I imagine that on-demand orders would have dedicated execution cores. It’s useful as an on-ramp but not as efficient for scaling throughput as products grow.

The market concept reminds me of ethereum gas tokens where users “purchased” gas by exploiting the gas refund mechanism of ethereum. This led to a market in gas/blockspace which has echoes of this. However, I’m conscious that was ultimately removed from Ethereum and considered harmful

GasToken was definitely on my mind while writing about a dedicated blockspace futures market. I do worry about hoarders cornering the market, although hopefully the sheer volume of blockspace would prevent them from doing so. Needs some more examination

This thread seems to suggest that mid-term auctions would actually be “bought” as opposed to the lease mechanism currently in place for bulk auctions. Would this be in DOT or is it the region holders prerogative as to what they sell them for?

With this proposal, it’d be the region holders’ choice how to split and re-sell bulk regions into smaller ones.

If the execution cores for parathreads end up within this mid-term auction space, is there an opportunity to “burn” the DOT they generate and counter some of Polkadot’s inflationary rewards?

I’d propose that on-demand cores be separated economically from the “bulk” cores, and although off-topic, I do think that some amount of burning is reasonable for on-demand order fees.

If the most successful parachains become the primary purchasers of blockspace initially to sustain their level of growth, do we risk a “winner takes all” dynamic where Polkadot gravitates towards serving a majority parachain which operates most of the execution cores at any given time and prices new projects out of the market?

I suspect not, at least if there are enough cores to service demand. If we assume that each core provides 1,000tx/s of throughput, with 100 cores a chain would have to consume 100,000tx/s in a sustained way in order to economically consume all the blockspace of Polkadot. For reference, Twitter experiences 6,000 tweets per second and Google processes around 100,000 searches per second.

I do think it’s likely that in any case where demand > supply that new projects would be pushed outwards to the frontiers of cheaper blockspace, regardless of whether that demand is driven by a single or several main consumers. Scaling further will be important in that respect. I’d also expect that at least some of the main consumers of blockspace would be smart contract platforms, which also provide on-ramps for new projects to enter the system.

2 Likes
4

Thanks Rob, some really interesting responses and definitely helped my thinking, greatly appreciate you taking the time to respond so thoughtfully.

I now get the on-demand nature of parathreads vs continuous (but less frequent) nature of mid-term blockspace. That is a useful distinction worth maintaining for different use cases.

Agree on the hoarding point, I’m eager to see other’s thoughts on the topic. Early on in the launch of scheduling, I doubt that it will be apparent to many the potential value of blockspace so people could pick up swathes for next to nothing. I guess a key limiter here will be the terms available, if you can only buy a “future” 1 week or 1 month in advance then the impact long term is muted.

On the topic of it being the holder’s choice as to how they sell their blockspace, I wonder if we could mandate a percentage of the sale that is burned or similar. The reason I say this is that they are selling something that they acquired from the network for “free” (well staking opportunity cost, but close to free). They should give back to the ecosystem, else suddenly per my point above, parachain auctions will become a squatters paradise and wouldn’t that be sad. People acquiring them with no desire to ever actually launch a parachain, but rather for the resale value when chunked up into regions.

Understood on the question of scale, but I suspect if Polkadot reaches the scale we all hope, we will bump into this sooner than we expect. There is always Kusama short term though!

Thanks again and I’m looking forward to reading the communities thoughts.

Can’t wait to see this implemented :grinning:

1 Like
5

Yeah, this is the key difference between this type of blockspace futures vs. GasToken. GasToken, until it was made obsolete, had an indefinite time horizon. If Polkadot produces blockspace with a 6 month or 1 year time horizon, that definitely changes any valuation calculation.

Still, I think that premature financialization of blockspace is probably not a good idea and this is more of an “endgame” than something to build out right away.

As far as how we get there, I’d propose to incrementally roll out this mechanism or something like it, where initially some blockspace would just allocated by governance to a “mid-term blockspace” pallet that parachains can use to buy “boosts” based on a market price.