Litep2p Network Backend Updates

Tech Talk
2

We’re excited to announce litep2p version 0.8.0, which introduces support for content provider advertisement and discovery in the Kademlia protocol, aligning with the libp2p spec. This enables nodes to publish and discover specific content providers on the network. Alongside this feature, the release brings notable improvements in stability, performance, and memory management.

For a full list of changes, refer to the litep2p changelog.

Content Provider Advertisement and Discovery

With this release, Litep2p now supports content provider advertisement and discovery using the Kademlia protocol, allowing content providers to publish records to the network, and enabling other nodes to locate and retrieve these records with the GET_PROVIDERS query. This feature is crucial for storing parachain bootnodes in the relay chain DHT.

    // Start providing a record to the network.
    // This stores the record in the local provider store and starts advertising it to the network.
    kad_handle.start_providing(key.clone());

    // Wait for some condition to stop providing...

    // Stop providing a record to the network.
    // The record is removed from the local provider store and stops advertising it to the network.
    // Please note that the record will be removed from the network after the TTL expires.
    kad_provider.stop_providing(key.clone());

    // Retrieve providers for a record from the network.
    // This returns a query ID that is later producing the result when polling the `Kademlia` instance.
    let query_id = kad_provider.get_providers(key.clone());

Connection Stability

The release includes several improvements to enhance the stability of connections in the litep2p library:

  • Connection Downgrading: Inactive connections are now downgraded only after extended inactivity, reducing interruptions and improving long-term stability.

  • Enhanced Peer State Management: A refactored state machine with smoother transitions enhances the management of peer connections, preventing issues like state mismatches that could lead to rejected connections.

  • Address Store Improvements: Address tracking is now more precise, with a new eviction algorithm to manage unreachable addresses and better control memory usage.

Optimizations

  • Improved Dialing Logic: Dialing across TCP, WebSocket, and Quic is now more resource-efficient, with canceled attempts immediately terminating to save resources.

  • Kademlia Data Handling: Data handling is now more efficient by replacing unnecessary data cloning with reference-based retrievals for Kademlia messages.

  • Memory Leak Fixes: Addressed memory leaks across TCP, WebSocket, and Quic transports, especially in canceled connections. Unremoved pending operations were resolved in both the ping and identify modules. See the relevant PRs: #272, #271, #274, #273.

I want to extend my thanks to everyone who contributed to making this release possible. Special thanks to Dimitry @dmitry-markin for his outstanding work on implementing the content provider advertisement and discovery feature, and to Alex @alexggh for his dedicated testing efforts and for detecting a high memory consumption. Thanks also to Andrei @sandreim for his valuable suggestions on investigating memory cloning, and to Andrei @AndreiEres for his ongoing commitment to enhancing benchmarking! :pray:

5 Likes