Help with Accessing Validator Telemetry Data for Geographic Decentralization Dashboard

Tech Talk
1

Hi Polkadot Community,

I’ve been exploring Polkadot telemetry for a few years now. While I’ve made some progress, I’ve also hit roadblocks, which led me to pause the project. However, I’m giving it another shot and would love your guidance!

In Polkadot telemetry, there’s a map showing the locations of active validators, built as a TypeScript application. I’ve tried reverse engineering this map to understand how the data is retrieved and displayed, but so far, I haven’t had much luck.

Using the WebSocket feed (wss://feed.telemetry.polkadot.io/feed) provides some network data, but it doesn’t seem to include detailed validator telemetry information.

I’m wondering:

  1. Is there another WebSocket feed (or a similar endpoint) that streams validator telemetry data, including geolocation details like city and latitude/longitude?
  2. Are there tools or methods I could use to scrape or interact with the telemetry backend to extract validator-specific information (e.g., geolocations)?
  3. I am also running my own telemetry server instance. Is it possible to modify this instance to create a custom WebSocket feed that includes the information I need? Specifically, I’d like a telemetry WebSocket that provides validator details, including city and latitude/longitude data.

My goal is to start logging validator data across Polkadot and other Substrate networks. With this data, I’d like to analyze the geographic distribution of validators and build a dashboard to visualize how decentralized the network is geographically.

Assuming this goes well, I’m also curious if it might be possible to create a mechanism to influence validator nominations to favor geographic decentralization. Does this sound like a crazy idea?

Any assistance with accessing telemetry data, modifying my own instance, or alternative approaches would be greatly appreciated. I’m excited to contribute insights into the decentralization of the network!

Thanks in advance for your help, and I look forward to hearing your thoughts.

Best regards,
Rob

2

Hi Rob,

Welcome to the Polkadot Community!

You just touched on a topic that’s very dear to me and the whole Data team at Parity, as we collect this data mainly for debugging/statistical reasons.

For your points 1. (partly) & 2., check this project out: GitHub - dennis-tra/nebula: 🌌 A network agnostic DHT crawler, monitor, and measurement tool that exposes timely information about DHT networks. , it’s an open-source DHT crawler and monitor that works with Polkadot and is maintained by @dennis-tra.

It is relatively straightforward to setup as the team did a great effort with documentation. From the Readme: “It queries the GeoLite2 database from Maxmind to extract country information about the IP addresses and UdgerDB to detect datacenters.” so it should give you the data that you require.

The team behind it had a successful OpenGov proposal: Polkadot P2P Network Health Weekly Reports | Polkassembly and is providing weekly reports here: Week 2024-45 | ProbeLab , so you can see what more you can collect there.

For point 3., and the last question, I’m not certain, but maybe someone else can give you an answer on that (and the real-time part for nebula).

Looking forward to the dashboard :sunny:

All the best!

1 Like
3

This is a huge help, I am playing with the DHT tool and it seems like I might be able to get this all working. Thanks so much! I might have more questions so stay tuned!

1 Like