Experiment: P2P Network Visualization

1

:wave: Greetings,

We’ve been prototyping a P2P scanner for Substrate networks. Our little experiment involved scanning the Kusama network at the P2P-level and creating a geospatial visualization of the discovered nodes with enriched information.

:mag: Take a peek at the overall network of discovered nodes (~2k) in the image below.

scan-0
scan-01920×976 154 KB

And a screenshot showing some node information.

scan-1
scan-11920×976 82.5 KB

Curious about the data? Here’s a snippet of the information for each peer in JSON format:

{
  "_id": "12D3KooWSueCPH3puP2PcvqPJdNaDNF3jMZjtJtDiSy35pWrbt5h",
  "type": "Ed25519",
  "addresses": [
    {
      "family": 4,
      "host": "51.77.66.187",
      "transport": "tcp",
      "port": 30333
    },
    {
      "family": 4,
      "host": "51.77.66.187",
      "transport": "tcp",
      "port": 30334
    },
    {
      "family": 6,
      "host": "a29:a25:3520:6b75:7361:6d61:2d62:6f6f",
      "transport": "tcp",
      "port": 30334
    }
  ],
  "agent": {
    "version": "Parity Polkadot/v1.3.0-7c9fd83805c (kusama-bootnode-0)",
    "protocol": "/substrate/1.0"
  },
  "protocols": [
  "/b0a8d493285c2df73290dfb7e61f870f17b41801197a149ca93654499ea3dafe/beefy/2",
  "/b0a8d493285c2df73290dfb7e61f870f17b41801197a149ca93654499ea3dafe/beefy/justifications/1",
  "/b0a8d493285c2df73290dfb7e61f870f17b41801197a149ca93654499ea3dafe/block-announces/1",
  "/b0a8d493285c2df73290dfb7e61f870f17b41801197a149ca93654499ea3dafe/grandpa/1",
  ...more protocols
  ],
  "metadata": {
    "AgentVersion": {
      "$binary": {
        "base64": "UGFyaXR5IFBvbGthZG90L3YxLjMuMC03YzlmZDgzODA1YyAoa3VzYW1hLWJvb3Rub2RlLTAp",
        "subType": "00"
      }
    },
    "ProtocolVersion": {
      "$binary": {
        "base64": "L3N1YnN0cmF0ZS8xLjA=",
        "subType": "00"
      }
    }
  },
  "tags": {
    "bootstrap": {
      "value": 50,
      "expiry": 1699622067589
    }
  },
  "ipInfos": [
    {
      "ip": "51.77.66.187",
      "hostname": "ns3136711.ip-51-77-66.eu",
      "city": "Frankfurt am Main",
      "region": "Hesse",
      "country": "DE",
      "country_name": "Germany",
      "country_currency": {
        "code": "EUR",
        "symbol": "€"
      },
      "continent": {
        "code": "EU",
        "name": "Europe"
      },
      "isEU": true,
      "loc": "50.1155,8.6842",
      "org": "AS16276 OVH SAS",
      "postal": "60306",
      "timezone": "Europe/Berlin"
    }
  ]
}

Cheers!

2

As a heads up, you can also determine which nodes are validators using the so-called “authority discovery” system. Each validator uploads to the DHT its network identity.

I believe that a tool like that is much better than the existing telemetry system, as it is not centralized and uses the existing network protocol rather than a separate one.

3

Amazing interface!

  • Is the P2P network scanned entirely from the browser and is it written in Rust :smiley: ?
  • Did you find any reliable database for IP translation to geolocation?
  • Usually takes a bit of time to scan the network, did you implement any sort of backoff / heuristics to improve the time of data collection?

That reminds me of a little tool I wrote a while back that can scan any substrate-based p2p network and submit transactions on the low level protocol. The limitation atm is that is not wasm compatible.

@tomaka Could the “authority” also be derived while scanning the network from the NodeRole::authority part of the handshake? :thinking:

4

Seemingly what is being put into the DHT is the hash256 of the AuthorityId with the corresponding value of a SignedAuthorityRecord containing multi addresses and the signature. Unfortunately, the AuthorityId is a session key, so we cannot find the authority records by any derivation of the information contained in the node peer id. Are we overlooking something?

A potential solution is to utilize the light client protocol for querying the state to get the session.validators or even the staking.validators. Then we can try to map the addresses to the node by using session.nextKeys to obtain the authority id and subsequently query the DHT for the SignedAuthorityRecord. However, this approach feels somewhat overcomplicated, and not very convincing.

Using the AuthorityDiscoveryApi to query the runtime could be another way…

As for the sync handshake approach using the role field (@lexnv), the approach seems like abusing the protocol semantics and lacks security guarantees. But it has the upside that it is easy to implement.

5

Thanks for the feedback! :smiley_cat:

  • No it is not, we are using Node.js and libp2p. Running from the browser has serious limitations in relation to the supported protocols and secure origins. Since it is a scanner we want to support all the protocols possible.
  • We are using https://ipinfo.io/ that is pretty nice, but has limitations on the free-tier. We are thinking on integrating Shodan as well, and maybe a Substrate NSE lib for nmap.
  • Yes, we wrote a custom kad handler with focus on quick discovery instead of using the standard kademlia discovery implementation.
6

Yeah that’s how I had in mind you would do it. The AuthorityDiscoveryApi_authorities runtime call returns the list of all current and next authorities.

The other drawback of this approach is that knowing the role of a peer requires opening a gossip channel with it, and that peer could refuse the opening.

In practice, due to spaghetti code issues, Substrate never straight-up refuses opening a gossip channel and instead implements refusing by immediately closing any gossip channel that it doesn’t want. Consequently you’re always able to know the role of a peer.
But in theory, at least how the networking protocol has been conceived, an implementation could straight up deny the opening and not send back its role.

7

This will be fixed in the relatively-near future and I wouldn’t write any code that relies on that specific behavior.

8

We’ve added support to the prototype for authority discovery using the AuthorityDiscoveryApi to get the authority keys, querying the DHT to get the authority record and finally matching it with the peer id.

Indicating the authority nodes with a different color:

scan_disco
scan_disco1920×976 177 KB

1 Like