The Road to JAM

November Updates

PolkaJAM:

Most Gray Paper 0.8.1 changes are now implemented in PolkaJAM, including:

  • a dedicated compile hostcall for inner PVM modules

  • bounded preimage requests

  • the new Thin VRF scheme

Please note that these changes will not be merged to the main branch and available in nightly builds until GP 0.8.1 has been released and all changes have been implemented. The GP 0.8.1 release should happen within the next month.

A series of fixes landed:

  • integer overflow protections, with new work-report limits and test vectors (some of these changes are strictly part of GP 0.8.1 – see Bound work-report values by work-package limits by davxy · Pull Request #563 · gavofyork/graypaper · GitHub – but have been merged to the main PolkaJAM branch already)

  • fixes to disputes, authorizer pool updates and ticket encoding

  • fixes to hostcall behaviour so it matches the spec

  • PolkaJAM now avoids building on rejected blocks

  • a memory leak in the auditing subsystem was fixed

  • guarantees are now properly filtered when blocks are built

  • PolkaJAM will now avoid guaranteeing two work-packages which consume the same authorizer

  • nodes will now download block history after a warp sync

The new auditing tranche mechanism was designed, implemented and opened for review. For details, see the Gray Paper PR: WiP Proposal for rework of auditing tranches by zdave-parity · Pull Request #552 · gavofyork/graypaper · GitHub

All conformance repositories moved to a dedicated GitHub organisation, jam-conformance.

A new jam-crypto repository provides test vectors for Bandersnatch, Ed25519 and Reed-Solomon. It includes a Python reference implementation compatible with reed-solomon-simd.

CoreVM and the JAM tooling will be moving to a public repository; watch this space!

CoreVM’s outer-VM gas cost model was merged. New tests check that the simulator and the node agree.

CoreVM video codec optimisations, with no loss in output quality:

Before After
Encoded gas ~70M ~14M
Decoded gas ~64M ~12M
Encoded size ~2.2MB ~1.5MB

Minimal Relay (AHM v2):

  • Migration framework: the migrator state machine and shared types are merged. The lockdown stage is merged, and the account migration stage is merged.

  • On-demand coretime: the new on-demand pallet on the Coretime chain is merged. The final piece is in review.

  • Parachain registration: parachain deregistration will run through the Coretime chain after migration. Force registration is ready to merge, and only a few registration flows remain.

  • HRMP channels: this workstream sped up noticeably. The HRMP pallet was refactored so channel deposits can be managed remotely by the Coretime chain. The async deposit provider is ready to merge, and PRs for the full HRMP implementation are open.

  • FAQ: We’ve created a FAQ answering questions that came in after the AMA. Here’s also a short presentation introducing the project and its scope. If you have any more questions, feel free to leave a message on the support chat or here on this thread.

Offchain messaging:

  • Design v0.6 published (#13622). It includes fixes for issues found while reviewing the primitives. One of them is an edge case where a message channel could stall permanently, which was caught and resolved at design stage.

  • Core primitives nearly merged (#12676). They went through further review with SnowFork and are one approval away.

  • Supporting pieces:

    • MMR ancestry-proof helpers were merged and released in the MMR crate.

    • New fuzzers cover the MMR primitives.

    • The design for the minimal Parachain Service features was updated to lower gas usage for message settlement.

  • JAM side: a proof of concept was published for discovering bootnodes over a DHT on JAM (#13523). Work has started on running a JAM testnet with a collator prototype.

Parachain Services:

Specifications:

  • A storage proof format for JAM was proposed (JIP-16), so parachains can get proofs of JAM state.

  • Proposals were opened so non-validator nodes can recover work packages (JIP-22 plus a networking spec change).

Design refinement: implementing the spec in Rust surfaced small issues, which were cleaned up in the design. The design for work-package resubmission was agreed.

Verification: the formal Quint specification is being fuzzed against the Rust implementation, to check they behave the same.

Upstreaming: the authorizer is being upstreamed into the Polkadot SDK (#13453).