Polkadot-API 2024 2nd update

Hello there :wave:

Continuing the tradition started in my previous posts (Post 1, Post 2), here is the latest quarterly update for the Polkadot-API!

We delivered!

During this quarter, we initiated a Forum Thread to share technical updates from the project, enhancing the transparency of our development process. For more details, you can refer to the thread. Here’s a brief summary of these last months:

polkadot-api@1.0.0

We just released the first stable version of PAPI! After months of stabilizing and improving the library, we believe it is ready to seal it and mark it as stable!

The maintenance and improvement of polkadot-api will be our #1 priority as it has been during the last months.

Compatibility Checks

Over the past few months, we have made significant updates to the Polkadot-API compatibility feature, following a step-by-step approach. These changes have brought it to a stable and reliable state for the long term. Previously, the checksums for each type in the metadata were fragile, lacking granularity and being overly strict. Now, we offer a detailed compatibility check that enables developers to understand how interactions might behave if the network changes. You can find more information here.

Codegen and CLI

Our beloved CLI tool, papi, and the code generation component have undergone significant rework. One major improvement is that you no longer need to install @polkadot-api/descriptors before starting to work with PAPI. Now, the only dependency required is polkadot-api.

Additionally, we have introduced the ability to generate descriptors and types directly from a compiled runtime in wasm.

@polkadot-api/merkleize-metadata

We released @polkadot-api/merkleize-metadata, a library implementing RFC#0078 entirely in TypeScript! By leveraging our scale-ts and @polkadot-api/substrate-bindings libraries for the bulk of the SCALE encoding work, we were able to focus on the core implementation. This library is feature-complete, and significant ecosystem players like Talisman have already seamlessly integrated it!

Bundle sizes

As you know, bundle sizes are a top priority for us during development. There has been room for improvement, especially for multichain dApps. We will continue to work on optimizing bundle sizes to ensure that dApp developers can achieve better loading times and performance for their applications.

Tx Improvements

Event Payload

Thanks to feedback from our early adopters, we successfully constructed and stabilized the event payload received when a transaction is signed and broadcasted. You can read detailed explanations of each event in the documentation.

Error handling

After some back and forth, we’ve introduced a named and strongly typed error to inform consumers why a transaction is invalid (or it became invalid after broadcasting). I.e. smaller nonce, mortality expired, etc. This is now clearly explained in the documentation.

Besides that, we also made easier for consumers to access Module Errors, i.e. errors that happen when executing the transaction and are defined by pallets itself. Thanks to Kheops for the report!

Documentation Improvements

Our documentation, now hosted at papi.how, has undergone significant updates, with many new entries added to address issues raised by our users. Feel free to explore and check it out!

A Busy Quarter

Since our last post and the OpenGov Proposal in mid-April 2024, many significant events have occurred alongside regular Polkadot-API maintenance. Let’s dive into them.

PBA (Polkadot Blockchain Academy)

@voliva and I attended the latest session of the PBA in Singapore during May and June 2024. It was an incredible experience where we expanded our knowledge in various areas of blockchain, Polkadot, and FRAME, and identified key pain points in core and dApp development. We worked really hard and we both graduated with distincion! We received NFTs proving that we attended and graduated: Josep and Victor

Polkadot Decoded 2024 in Brussels

We are extremely grateful to the Decoded staff for inviting us to speak at the event. My colleague @carlosala and I presented on building a successful dApp using modern Polkadot technologies, such as PAPI, the new JSON-RPC API, smoldot, etc; demonstrating how a dApp built with PAPI can seamlessly navigate a breaking runtime upgrade. We are very proud of the outcome and look forward to continuing to share our knowledge at future Polkadot events!

In case you missed the talk, it is already available in Youtube! Polkadot-API Decoded Talk

Team growth

We are excited to announce that Hannah Redler will be joining our team as the fourth member! Coming from the Ethereum ecosystem, we hope she finds the Polkadot community welcoming. We believe she will be a perfect fit for our team, and you’ll start seeing her around the community soon!

Looking forward :fast_forward:

polkadot-api improvements

ink! support

The next major enhancement we envision for PAPI is the integration of first-class ink! support. By understanding contract metadata and displaying strongly typed arguments and outputs to the consumer, we believe we will significantly enhance the developer experience for dApp developers working with smart contracts.

PAPI SDKs

Polkadot has been designed, developed, and enhanced with a focus on scalability, lower transaction fees, and the core development experience, but the end-user has often been overlooked. Libraries that would facilitate dApp developers’ use of Polkadot technology to deliver real value to end users are missing, resulting in a poor user experience. We aim to develop SDKs for the primary pallets across Polkadot/Kusama and their system chains, with a focus on making interactions with Polkadot as straightforward and seamless as using a traditional web2 application.

When creating polkadot-api, we strictly focused on building a chain-agnostic library with no business logic. While low-level endpoints are useful for advanced implementers, we need higher-level APIs that understand the pallet’s business logic. The main challenge we’ve identified in Polkadot is not the UI components but the highly low-level nature of the blockchain’s APIs. End users should not need to worry about storage queries, runtime calls, or other low-level implementation details of Polkadot, yet they currently do.

While developing the SDKs for the main use cases of the relay and system chains, we also aim to support the creation of new SDKs tailored for parachains. These chains may have unique business logic for each pallet, or they may need entirely new SDKs designed specifically for their own pallets.

We see this initiative as a natural extension of the work we’ve done (and will continue to do), bringing significant benefits to the entire Polkadot developer community.

Polkadot Actions

The initial use case we envision for our SDKs is Polkadot Actions. Drawing inspiration from Solana Actions and Blinks, we believe that creating a common language for dApps will greatly benefit the community. Simplifying routine blockchain interactions is essential and urgent for the ecosystem’s success and wider adoption. However, this must be done thoughtfully, with careful abstractions that prevent unnecessary complexity from reaching the end user, while also avoiding an overly simplified interface.

We plan to develop the first proof of concept at the WebZero hackathon, taking place during the Web3 Summit in Berlin from August 19-21. Our entire team will focus on building a robust working example, which we will refine over the coming months until it reaches a stable state. We aim to follow the same successful timelines that we implemented with polkadot-api.

Transparency

Since February 2024, the development of Polkadot-API has been exclusively funded by the Polkadot Treasury through OpenGov proposals.

While our proposals have covered the majority of our planned expenses, we did encounter a few additional costs that were not initially accounted for. These included going to Thailand to attend Polkadot sub0, where we performed a small talk in the summit, traveling to Brussels for the presentation we prepared for the Decoded in Brussels, and onboarding a new team member to prepare for the next phase of development. Given the importance of these activities, we decided to cover these costs ourselves.

Throughout our past proposals, we have consistently withdrawn only the exact amount requested in Euros, responsibly staking any excess tokens resulting from price fluctuations. This approach provided a buffer for future proposals. Specifically, we retained an excess of 3.313 DOT from the first proposal and 793 DOT from the second proposal.

However, with the option to submit new proposals in stablecoins like USDT or USDC, this practice is no longer necessary. The reduced volatility between EUR and USD minimizes the risk, making it something we can confidently assume moving forward.

Therefore, the next following weeks we will be transferring all of these excess tokens back to the Treasury. We will un-stake them and perform the transfer after the bonding period.

Independently from that, we will open some “tip” referenda for each of the additional expenses we’ve been having, so the community can decide if this should be retroactively funded.

Upcoming OpenGov proposal

In the coming days, we will be publishing our third OpenGov proposal. This proposal covers the development and maintenance of polkadot-api and the advancements outlined in the previous sections for six months, from August 2024 to January 2025. Stay tuned and lend us your support to help us continue delivering important and impactful code! Your support has been, and will remain, crucial to our success. Let’s keep developing and improving the ecosystem together! :rocket:

11 Likes

These detailed updates are great, and by all accounts, so is PAPI. Great work, team!

3 Likes