The ProbeLab team, which spun out of Protocol Labs (inventor and core sponsor of libp2p and IPFS), is focusing on measurement and benchmarking of protocols at **the P2P layer ** of Web3 platforms.
So far, the team has focused on developing metrics for IPFS (see https://probelab.io), but has looked into other libp2p-based networks and one of them has been Polkadot. We have been running our crawler and accompanying tooling for the P2P layer of Polkadot since the beginning of 2023.
After processing the raw data, we create weekly reports such as this one: https://github.com/plprobelab/network-measurements/blob/a7641238352c46b43f09fe72bf18c66af7faeb6d/reports/2023/calendar-week-41/polkadot/README.md. The corresponding reports for IPFS can be found here: https://github.com/plprobelab/network-measurements/tree/master/reports.
What do the reports show and why is it important
The reports show several general purpose metrics by crawling the P2P network more than 300 times a week. Apart from node counts in the network, the reports show statistics on the following:
- connectivity of nodes, which can indicate issues with regard to unavailable, overloaded or misconfigured nodes [link to plot],
- availability of nodes (i.e., whether they’re online or offline) together with classification of availability per parachain client [link to plot], which indicates network health,
- agent version breakdown and protocols, from which one can infer update rates to new releases, or when is a good time to break backward compatibility and save engineering resources (e.g., when very few nodes are left in an old version) [link to plot],
- errors connecting to nodes which can indicate faulty software or configuration, or other issues with the network [link to plot],
- overall peer churn, as well as per client, which can indicate issues with node availability or misconfiguration [link to plot],
- geolocation info [link to plot] together with the relevant classification of availability of nodes in each region [link to plot], as well as agent version per region [link to plot],
- Hosting infrastructure breakdown (datacenter vs custom) [link to plot], as well as detailed classification per group of nodes [link to plot] and agent version [link to plot].
Ask
Running the tools, processing and storing the data and analysing the results is clearly incurring infrastructure and engineering costs that the team has supported up to now.
Going forward, we can only continue to run the tools and produce these reports for a service fee of $2500 per month, plus a one-off, upfront cost for the initial setup of $5000. The service fee includes:
-
running and covering the cost of the infrastructure,
-
maintaining and updating the tools and infrastructure as needed (e.g., dependency updates and database maintenance),
-
have an expert from our group inspect the results nd flag any “notable” items that need further attention on a weekly basis.
We plan to submit a proposal under the “Small Spenders” track, but in the meantime, we’d love to get feedback on the content of this post, the weekly reports, as well as new metrics that are of interest to the community.
In the future we plan to integrate those reports at https://probelab.io to have that as a central point of reference, but this will come as a separate discussion item.
Feedback Request
With this post, we would like to invite the Polkadot community to give feedback on these reports, quantify the value that they bring to the ecosystem and ultimately vote on whether maintenance and publication of the reports should continue for the cost mentioned above.
We are also very keen to receive feedback and suggestions on other metrics that would be of interest to the community so that we can plan ahead and develop them.
