Hi everyone,
by chance, I just came across this thread and wanted to chime in. I want to bring to your attention a tool that we (ProbeLab formerly with Protocol Labs) have built:
The tool has support for Polkadot/Kusama/Westend for quite some time and we have been running a DHT crawl every 30m for the last 12 months for all three networks. So, we are sitting on quite a lot of data but just haven’t had the time to analyze it.
The tool is similar to the linked subp2p-explorer but instead of doing queries for random keys it structurally enumerates all peers in the DHT starting from a configurable set of bootstrap peers. A full Polkadot network crawl takes around ~4 minutes.
We gather the same information as in that “Experiment: P2P Network Visualization” and more (except the authority API parts - yet). (the visualization is great btw)
Nebula powers IPFS weekly reports like this recent one (discourse won’t let me post more links - see the end of this post):
github /plprobelab/network-measurements/blob/master/reports/2024/calendar-week-02/ipfs/README.md
We also produced one for Polkadot:
github /plprobelab/network-measurements/blob/master/reports/2023/calendar-week-41/polkadot/README.md
Which we discussed a few weeks back in this topic: /t/proposal-polkadot-p2p-network-health-weekly-reports/5094/6
For the IPFS network, we also discussed a /genesis/telemetry protocol a couple of times in the past and personally I’m totally in favour of something like this. Probably opt-in, but if it can be combined with some differential privacy concepts maybe even opt-out.
The ProbeLab team is keen to tailor Nebula more towards the Polkadot ecosystem. This could include Polkadot-specific logic like employing the AuthorityDiscoveryApi or trying to gather more information from, e.g., handshake data of another protocol.
Further, we could build an API to access the data and/or offer more sophisticated visualizations (the weekly reports were only an interim solution).
For that, we’re seeking funding and would like to submit a grant application to polkassembly.io. It would be great to get feedback on what other data the community would be interested in that can be extracted from the DHT/p2p layer to make the grant application as relevant as possible.
PS: This Discourse instance wouldn’t let me post more links that’s why I didn’t format them as such. I always got “An error occurred: Sorry you cannot post a link to that host.” which is super weird as I was only linking to GitHub or this forum. That’s why the links aren’t proper links.