`subrpc`: manage multiple endpoint registries and quickly find endpoints

subrpc

I am happy to share a new cli tool: subrpc.

Context

I recently posted about the Polkadot Network Directory which helps find chains and informations related chains and parachains.

Here is the repo:

Many projects already contributed with PRs bringing their data to the directory.

One of the benefits of having this data is that we could also generate a registry: a list of available endpoints:
https://paritytech.github.io/polkadot_network_directory/registry.json

There is however no reason for this data to be centralized in one location and users may be interested in either subsets or augmented data set.

I am happy to share the first versions of subrpc. You can check its readme for the full details.

TLDR

The TLDR is:

  • subrpc is a:
    • cli written in Rust you can use in your terminal to find endpoints: subrpc ep get dot
    • a lib/crate you can use from your app to connect to the right endpoint without having to manage that part in depth
    • local database YOU build based on your favorite registries or endpoint providers
    • a bunch of commands to update and test your local data and the endpoints
    • knows about aliases, so if you are tired of typing polakdot :slight_smile: , dot will do as well
    • can tap into the registry automatically generated by the Polkadot Network Directory

subrpc makes it easy to find an endpoint but most importantly, it helps building locally a list that is tailored to you and built up from decentralized sources that each user can freely define.

subrpc also makes it simple to share the list of your internal/private/testing endpoints.

Sample use

The following is an extract of the readme:

We first install subrpc, it is written in Rust so we use cargo:

cargo install subrpc

and check that it works:

subrpc --help
`subrpc` allows managing a set of registry providing rpc nodes

Usage: subrpc <COMMAND>

Commands:
  registry   Manage your registries
  system     System
  endpoints  Endpoints
  config     Config
  help       Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version

We can then make it aware of one or more registries using the subrpc registry add <url> command

# `reg` is an alias for `registry`
subrpc reg add https://raw.githubusercontent.com/chevdor/subrpc/master/registry/subrpc.json
subrpc reg add https://paritytech.github.io/polkadot_network_directory/registry.json
subrpc reg add https://raw.githubusercontent.com/chevdor/subrpc/master/registry/parity.json

It is now time to refresh the registry data:

subrpc reg up
subrpc system info

The output will look like:

Running subrpc v0.0.3
chevdor <chevdor@gmail.com>
local data file: /Users/will/.subrpc/data.json
- [ ] SubRPC Gist 1 - https://gist.githubusercontent.com/chevdor/a8b381911c28f6de02dde62ed1a17dec/raw/6992b0a2924f80f691e4844c1731564f0e2a62ec/data.json
      rpc endpoints: 2
      last update: None
- [X] Polkadot Network Directory - https://paritytech.github.io/polkadot_network_directory/registry.json
      rpc endpoints: 31
      last update: None
- [X] OnFinality - https://raw.githubusercontent.com/chevdor/subrpc/master/registry/dwellir.json
      rpc endpoints: 1
      last update: None
- [X] SubRPC Registry - https://raw.githubusercontent.com/chevdor/subrpc/master/registry/subrpc.json
      rpc endpoints: 3
      last update: None
- [X] Parity - https://raw.githubusercontent.com/chevdor/subrpc/master/registry/parity.json
      rpc endpoints: 13
      last update: None
- [X] Moonbeam Fundation - https://raw.githubusercontent.com/chevdor/subrpc/master/registry/moonbeam.json
      rpc endpoints: 3
      last update: None
- [ ] SubRPC Gist 2 - https://gist.githubusercontent.com/chevdor/a8b381911c28f6de02dde62ed1a17dec/raw/6992b0a2924f80f691e4844c1731564f0e2a62ec/data2.json
      rpc endpoints: 0
      last update: None

You may now tap in the list of known endpoints:

# `ep` is an alias for `endpoints`
subrpc ep ls

it will show the list of all known endpoints. The list is already quite long, you can test on your machine :slight_smile:

In general, you will only be interested in one specific chain at a time though, this is what the subrpc endpoint get command does:

subrpc ep get polkadot

will output:

wss://polkadot.api.onfinality.io:443/public-ws
wss://rpc.polkadot.io:443
wss://polkadot-rpc.dwellir.com:443

Those are only http or web socket urls, but do they work ? subrpc endpoint ping will tell you:

RUST_LOG=debug subrpc ep ping

and I am planning some improvements on that front…

Call to infrastructure providers

If you make public (or private) endpoints available to the community, you could of course start with a PR and submit a file looking like this example.

It would be however even better if you host your own registry on your own server and share the link to your registry as a PR to the subrpc repository.

Even even better if you make those registraies also available in IPFS :slight_smile:

Disclaimer: WIP & Early version

I decided to share this tool in its early state as I think it can be useful already adn trigger ideas and discussions. It is however definitely rough on the edges :slight_smile: The data, commands and args will more than likely change over time.

Feedback, PRs and especially new data are welcome in the repository.

3 Likes

If you end up juggling between several (para)chains in PolkadotJS, I published an update for subrpc that helps save quick some clicks and time (:technologist: terminal required):

  • install subrpc: cargo install subrpc and follow the quick start to have some data locally
  • source this function (it assumes you have fzf already installed… which you really should have anyway :))
  • you can then do this which opens the chain of your choice from your terminal without having to figure out whether your (para)chain is under Polkadot/Kusama or another of the category and without having to scroll all the way back up to “switch”