Polkadot in Numbers - Annual Polkadot Ecosystem Report 2023 [by Parity Data]

[pavla]

Polkadot 2023 Report

We’re thrilled to announce that our Data Team at Parity Technologies has just created an interactive Annual Polkadot Ecosystem Data Report 2023. This report covers various aspects of Polkadot and is the first of its kind, marking a significant milestone in our journey to harness the power of data for the advancement of our dynamic ecosystem.

Our belief is steadfast: data is the bedrock of progress. By diving deep into the data of the Polkadot ecosystem, we aim to unveil its true potential and showcase its value to the world. Our mission revolves around deciphering the vast array of on-chain and off-chain data for the Polkadot ecosystem, turning them into insightful gems for our community.

Our adventure started with “DotLake,” a data warehouse we initially established for Parity’s in-house data requirements. Dotlake hosts the complete history of over 70+ Polkadot & Kusama chains and is continually growing. Now, we’ve expanded our horizon, aiming to accelerate the sharing of our discoveries and methodologies with the broader community.

This report is one of the ways of sharing these insights, looking back at the whole year behind us. The charts you see in the report are a part of a much larger collection we aim to share with the community, and are a product of diligent analysis and iteration over the past year.

It combines charts and metrics with succinct wiki-style descriptions for each covered area, aiming to educate the Polkadot-newbie readers about key aspects of the network and the ecosystem.

For newcomers, we recommend starting with section 2, “What is Polkadot.” For those with a more technical background, dive straight into the first graphs in section 3, “Shared Security of the Network.” All charts depict a full 2023 monthly or YTD view, with a cutoff date of December 1, 2023, unless stated otherwise.

We’re eager to hear your thoughts on this report and any data or metrics you’d like us to include in future editions. Please feel free to share your feedback and suggestions with us.

Please note that this report is optimized for being viewed on larger screens (laptop, desktop pc) and in order to view the interactive charts, your browser needs to have JavaScript enabled on this page. We’re providing a PDF version, that you can access here, just in case. We hope you enjoy reading this report as much as we enjoyed preparing it.

Happy Holidays
Data Team, Parity Technologies

[Karim]

I’m joining Pavla to share just how happy I am seeing this project finally come to fruition. The team has worked tirelessly over the past few weeks to get you these insights, it’s great to see these efforts materialize.

Nobody really came to us with a request of “Please make a yearly report” but we felt this would be just the right thing to do, next to our many other ongoing projects. We wanted to see it happen, so we made it happen. The Parity Data Team has come a very long way, from the time where we were running hand-crafted workloads on n8n to now having a one of a kind cost-effective data lake / data warehouse to answer any Polkadot related question, be it on-chain or off-chain and creating websites with interactive visualizations to boot.

The Parity Data Team started out as a BI team with a focus on on-chain data, but graduated to a fully fledged data research unit within Parity & the Polkadot ecosystem, providing data, insights and guidance to ecosystem teams within and outside of Parity to a flurry of initiatives.

For Polkadot 2.0, we won’t be playing catch-up to enable accurate and proper representation of the Polkadot ecosystem or any of its members but we will be right at the forefront of the evolution, helping and guiding decisions supported by quality data and robust infrastructure.

Now today, we give you the past, in the form of interactive charts, highlights and summary of 2023. Next year, we want to give you the future. As a preview of what I mean:

• Many more data applications and demos of what is possible with our current data, with a focus on insights and applications centred around agile coretime, coreplay, Polkadot 2.0. Here, we are really big fans of Lastic’s pricing simulator and think this is the best way forward to leverage data and make it actionable, beyond just reports and SQL.
• Robust and reliable access to raw DotLake data:
  • Direct access (without requiring an account), stay tuned for this one as we have a really cool technical solution in place
  • Access to DotLake through Dune (Q1 2024) including our definitions, workbooks and more. Here we’ll follow multiple routes, one through direct data integration (leveraging DotLake) followed by a longer term official on-boarding of Polkadot data in the scope of a Data bounty.

Data is only ever a part of the equation, since it’s mostly a lagging indicator telling us what happened. Numbers can guide you, but instincts will give you the edge. The future belongs to the builders and we’re very happy to be part of it and contribute in our own way.

We wish you a great end of the year and a great start in the new one.

PS:

• Special for @Birdo : Next report will have dark-mode, it wasn’t possible in this short time-span (gotta read on some more front end stuff)
• Not all the charts and metrics we have on our side made it into this report, if you’re missing one, just ping us and we’ll make sure it is included in the next one
• Stay tuned for the updated December 2023 numbers, in January 2024
• If you notice any discrepancies or mistakes, you can write Pavla or myself directly on Twitter or on our email.
• You can find a Twitter/X thread here as well.

[dgencitizen]

Wooop wooop super exciting!

Thank you for sharing and putting this together @Pavla ; @Karim and the rest of the team. You rock!

[Birdo]

Couldn’t read the report - it was far too blinding.
dark mode by default next time please

Congrats all - it looks awesome

[sourabhniyogi]

@pavla @karim This is an awesome report, the best report I have seen in Polkadot Data this year.

Our goal is to integrate Dune with Polkadot to make it easy for everyone to compile reports like these based on substrate-etl and with your help, any unique data in DotLake as well. The choice for a Polkadot-Dune integration can be solved by the decentralized OpenGov system, which we have submitted as Ref #366. Without DotLake, the tradeoff is:

Colorful Notion’s vision for the future is that EVERYONE can build ANY dashboard for the ENTIRE Polkadot/Kusama ecosystem in Dune using substrate-etl (and DotLake, if you so choose). If #366 passes, everyone can build a dashboard like the one I just built today here:

We would appreciate access to DotLake and your wishlist as to what should be onboarded onto Dune. Having built substrate-etl + Polkaholic.io over the last 2 years, our wish is to cover as many relay chains and parachains as we can, extending to Polkadot 2.0’s CoreTime/CorePlay/CoreJam/DA future.

Would you be able to share this data soon so instead of requiring the above tradeoff we can onboard DotLake’s via #366 instead?

[pavla]

Thank you, @sourabhniyogi, @Birdo & @dgencitizen !

Our report’s focus on DotLake, an internal data platform developed initially by and for Parity and now increasingly being extended to the community, is factual - as highlighted in the report’s Introduction and architectural diagram. No data point or metric in the report was derived from any source other than Dotlake, which ingests data directly from chains.

Because we care for presentation of insights also, we chose echarts for embedding interactive graphs, while Metabase remains our internal dashboarding tool.

Please note that, even if we wanted to use substrate-etl for this purpose, your recent removal of EVM & XCM datasets and the announcement to reduce data retention and parachain coverage, following your approved proposal, led us to exclude substrate-etl as an option for a data source from our annual report due to reliability concerns.

That being said, thank you for your continued invaluable contributions and steadfast commitment to the challenges. The Parity Data Team was established with a vision to foster a dynamic data-driven ecosystem and to democratize not just data, but also insights, which this report is all about. By addressing the unique challenges of blockchain data, offering innovative solutions, and empowering others to gain deeper insights, we have maintained our competitive edge in the ecosystem. Your perspective on making data access even more straightforward and attracting a broader community engagement is highly appreciated.

With this report, we’ve achieved a significant milestone in making Polkadot analytics and metrics more accessible — a long-standing goal is still to democratize data access, with thousands of people already having viewed the report (and counting!). Rest assured, we are committed to fulfilling this mission! We welcome any feedback on our work, especially suggestions that can enhance the impact of our outcomes. We’re excited to continue tackling these challenges Together.

A happy New Year to you all!

[Karim]

Here’s a small thing we whipped together very quickly, DotLake metrics on Dune!

Here’s the Polkadot monthly transactions graph (Substrate + EVM), as shown in our yearly report, but on Dune:

Check it out:

https://dune.com/androidcat/dotlake-polkadot-monthly-transactions

Tell us which metrics you’d like, and we can add them without any issues. This is a quick POC where we can make metrics available, before loading the complete dataset for the community.

Meanwhile, thanks everyone for the tremendous support and amazing reception the report has received. Hoping to many more to come, on Dune, and every other platform to ensure appropriate and correct Polkadot representation.

PS: I’m not an artist so I’ll wait for others to make even prettier ones.

[Karim]

Small update since this caught some steam, we’ve made all the DotLake on-chain metrics that we used in the report public:

While we explore the 2 integrations (Data Sync, Dune-led data integration), don’t hesitate to tell us what metrics we can export for you so you can create amazing visualizations.

Can you make a prettier report than the one we did? Give it a shot and please share it

[alexdimes]

Congrats @pavla , @Karim and the rest of the amazing Data team!

We definitely need more of ecosystem based analytics. Will reach out with more suggestions!

DOTLake - spot on.

Happy holidays

[BEdasc]

When you’re short on time, it’s always useful to have an annual summary showing how Polkadot is evolving and working. Many thanks and Happy NY