In Part 1 I dissected how slot collisions create forks even in a perfect network. In Part 2 I replaced the global broadcast with real P2P gossip and showed how visibility lag makes those forks survive longer.
For Part 3 I wanted to go further I broke the network on purpose, waited, and measured exactly what happens when it heals.
This isn’t just a fun experiment. It’s the core reason GRANDPA exists in Substrate.
The Setup: A Controlled Partition
I kept the same 3-node line topology (node_0 ↔ node_1 ↔ node_2) and the same probabilistic BABE-lite model running for 20 discrete slots.
At slot 5 I severed the link between node_1 and node_2.
At slot 15 I restored it.
// In main.rs
if slot == 5 {
simulator_network.disconnect("node_1", "node_2");
}
if slot == 15 {
simulator_network.connect("node_1", "node_2");
}
During slots 5–15, node_2 was completely isolated. It couldn’t see any blocks from the other two nodes, and they couldn’t see it. Both sides continued producing blocks like nothing had happened.
The Paradox: Consensus Without Agreement
Here’s the crazy part: BABE doesn’t stop when the network splits. The slot clock is global, so every validator keeps trying to author blocks in its assigned slots.
- Partition A (
node_0+node_1): 2 authors → faster convergence and a longer chain - Partition B (
node_2): 1 isolated author → shorter but still cryptographically valid chain
Both chains are locally valid. Both sides believe they have the canonical head. The protocol has no idea the network is partitioned.
The Heal: Deep Re-org at Slot 15
When the connection came back at slot 15, the isolated blocks finally started propagating. As soon as node_2 saw the longer chain from Partition A, it triggered a full re-org. Every single block node_2 had authored in isolation got discarded.
The node that never went offline, never equivocated, and kept doing honest work still lost everything. That’s the partition paradox.
Quantified Protocol Implications
I added basic metrics tracking for this run. Here’s what came out:
========================================================
SUBSTRATE CONSENSUS LAB: RESEARCH REPORT
========================================================
MODEL DEFINITION:
- Slots Simulated: 20
- Validator Nodes: 3
- Model Type: Probabilistic BABE-lite
- Fork Choice: Recursive Longest-Chain
QUANTIFIED OBSERVATIONS:
- Total Blocks Authored: 25
- Max Chain Height: 15
- Slot Collisions (Forks): 8
- Forks Resolved: 1
PROTOCOL IMPLICATIONS:
- Chain Inefficiency: 66.67% (wasted work)
- Fork Density: 0.40 forks/slot
- Avg Convergence Latency: 1.00 slots/fork
- State Divergence: 2 nodes at max height
========================================================
Breaking these down:
- 25 blocks produced, only 15 canonical → 10 blocks (40% of total work) were permanently discarded during the re-org. This is expected behavior in a purely probabilistic longest-chain system under partition.
- Fork density of 0.40 → nearly every other slot produced competing heads because of the gossip latency.
- Avg convergence latency of 1.00 slot → normal forks (without partition) resolved quickly. The partition-induced divergence is fundamentally different.
- State divergence → even after the heal, only 2 of 3 nodes ended up at the same maximum height.
Comparative Scaling: Short vs Long Partition
To see how this scales, I also ran a 40-slot version and compared two different partition lengths. The result was clear: re-org depth scales proportionally with partition duration.
Experiment A: Short Partition (5 slots isolated)
Total Blocks Authored: 37
Max Chain Height: 21
Slot Collisions: 12
Re-org Events: 18
Chain Inefficiency: 76.19%
Max Re-org Depth: 21 blocks
Experiment B: Long Partition (15 slots isolated)
Total Blocks Authored: 37
Max Chain Height: 20
Slot Collisions: 12
Re-org Events: 16
Chain Inefficiency: 85.00%
Max Re-org Depth: 20 blocks
The longer the isolation, the deeper the eventual rollback.
Technical Deep-Dive: Tracking Real Rollback Depth
I refined the re-org metric to measure actual blocks rolled back (Old Tip Height − Common Ancestor Height) instead of just counting discarded blocks:
// In src/core/node.rs
fn reorg_chain(&mut self) -> Option<u64> {
let old_hash = self.best_head_hash;
let old_height = self.best_height();
// ...
if is_direct_extension {
None
} else {
let ancestor_hash = self.find_common_ancestor(old_hash, self.best_head_hash);
let ancestor_height = self.blocks.get(&ancestor_hash)
.map(|b| b.header.number)
.unwrap_or(0);
Some(old_height.saturating_sub(ancestor_height))
}
}
The Deeper Implication: Re-org Depth is Unbounded (without finality)
In a BABE-only world, post-partition re-org depth is bounded only by the length of the partition. A 100-slot partition can produce a 100-block re-org. An application that treated a block as “final” at slot 90 would be wrong.
This is exactly the attack surface GRANDPA is designed to close.
How GRANDPA Closes the Gap
GRANDPA runs as a separate finality gadget alongside BABE. Validators vote on chain prefixes, and when a supermajority (2/3+ stake) agrees on a block, it becomes finalized it can never be re-orged, no matter how long a partition lasts.
- BABE = liveness (the chain keeps growing)
- GRANDPA = safety (finalized blocks are irreversible)
In real Substrate (sc-consensus-babe), BABE’s Longest fork choice is always bounded below by GRANDPA’s finalized checkpoint. Without that bound, applications would need ever-growing probabilistic confirmations that scale with network diameter and partition risk.
What This Simulator Now Measures
I added convergence latency tracking and rollback depth metrics. The code is straightforward and lives in core/metrics.rs and core/node.rs. “Forks Resolved: 1” in the 20-slot run means only one normal fork fully converged the rest were absorbed into the partition divergence.
Model Limitations
To keep the research signal clean, it’s important to note what this simulation does not yet model:
- Stake Weighting: Leadership is probabilistic but uniform; all nodes have equal authority.
- Equivocation Handling: We don’t punish nodes for authoring on multiple forks simultaneously.
- Gossip Topology: A 3-node line is an extreme bottleneck; production Kademlia meshes are denser.
Open Research Questions
I’d love to hear from the community (especially anyone working on core consensus):
- How does Substrate handle partitions that last longer than the normal finality window?
- Are there empirical benchmarks for “acceptable” re-org depth before GRANDPA typically finalizes?
- Does the move to Asynchronous Backing change how visibility lag affects fork density in the authoring layer?
The full codebase is open here:
GitHub - Kanasjnr/substrate-consensus-lab · GitHub
Happy forking. 