In the PAPI Q2023 update, I detailed the dual focus of the small but dedicated team I’ve been leading:
-
Development of @polkadot-api Libraries: We introduced these as an alternative to the existing PJS libraries. Our aim is to offer a top-level library, primarily designed to assist DApp developers. This light-client-first approach is intended to streamline the creation of DApps.
-
Transformation of the
@substrate/connectExtension: We’ve embarked on what I like to call the “metamorphosis” of this extension, a key aspect I will elaborate on in this post.
Additionally, I shared the decision to spin off from Parity in order to expedite the development of Polkadot-API (PAPI). This strategic move aims to establish a clearer distinction between the two initiatives mentioned above. By doing so, my external team can focus more intently on PAPI, propelling us towards our ambitious goals: releasing the first beta version by the end of Q1 2024 and the first stable version by Q2 of the same year.
In response to inquiries, I realize there’s a need for a more detailed explanation of the “metamorphosis” of the @substrate/connect extension. In this blog post, I aim to demystify this transformation and elucidate why it’s a pivotal step towards enriching the ecosystem with superior dApps and an enhanced user experience.
The Status Quo
The API currently utilized by DApps through the PJS extension represents the prevailing standard or “status quo”. Essentially, this is the primary API that PJS natively interfaces with. As a result, any other extension or tool seeking integration with a PJS DApp must adhere to this API.
However, it’s important to understand the designed role of the PJS extension. It primarily functions as a “signer,” enabling users to authorize transactions. This design choice implies that the PJS extension does not directly communicate with the blockchain it’s creating transactions for.
Consequently, the burden falls on the interacting DApps to supply the PJS extension with certain vital information, available only on the blockchain. A critical piece of this information is the latest metadata.
Regular users of PJS-based DApps might be familiar with this scenario: the DApp alerts you that your extension’s metadata is outdated, prompting an update. The risks associated with signing transactions using incorrect or compromised metadata are significant. If a DApp persuades a user to update their extension with an erroneous metadata version, the user might unknowingly authorize a transaction different from what they intended.
In the current landscape, users must trust not only the integrity of their PJS extension but also the reliability of various DApps and the means by which they obtain the current metadata.
It’s noteworthy, though, that there is an ongoing effort led by Zondax and Alzymologist teams. They are proposing a new sign-extension that would validate the metadata that was used by the signing device, which would provide a metadata digest value, ensuring that the metadata used in transaction creation matches with the one on chain.
Despite this, the status quo presents numerous challenges for both users and DApp developers. Each DApp is tasked with managing all the crosscutting concerns involved in transaction creation – from validation, dry-run execution, to displaying transaction fees, view and edit transaction details like tips, mortality, and so forth.
What if…?
Imagine a scenario where DApps are built using a library that first ensures the presence of a light-client in the running environment. This environment then communicates with a signer, receiving all necessary on-chain data directly from the light client. Such a setup could manage all the intricate aspects of transaction creation, offering a significantly more secure and user-friendly experience.
This isn’t just a hypothetical situation; it’s how DApps developed with PAPI will operate.
Envision a browser extension that provides DApps not just access to a light-client (akin to what the @substrate/connect extension does now) but also a signer API. This API would be designed to guarantee that all on-chain data is sourced straight from the light client.
Implications for Polkadot DApp Users
For you, the DApp user, the changes would be profound:
- No More Metadata Updates: The irritating prompts to update your extension’s metadata would be a thing of the past. Let’s face it, the average user shouldn’t need to worry about metadata or its maintenance.
- Seamless Transaction Management: The extension would handle all transaction-related tasks, including validation, dry runs to preclude error-prone transactions, and fee estimations.
- Advanced User Options: For those who want more control, features like editing the tip or setting the mortality of a transaction would still be available.
Less trust – you’d mainly need to ensure that your extension is secure and that you properly review the transaction before signing it. There would be no need for an API allowing DApps to update metadata. Why? Because the extension could reliably fetch this data directly from the blockchain via the light-client.
Implications for DApp Developers
For developers, the benefits are equally significant:
- Simplified Development Process: The need to manage the complexities of transaction creation would be eliminated. Developers could focus on the unique logic specific to their DApp.
- Flexibility in Integration: While handling transaction-related concerns wouldn’t be necessary, developers still have the option to incorporate such features into their DApps. This allows users to cross-verify information using the extension.
How Can We Get There?
The path to this envisioned future is not singular; there are multiple avenues we could explore. One particularly promising approach involves providing the ecosystem with a new “template extension.” This extension would be designed as a replacement for the PJS extension, but with a crucial difference: it would be built to leverage headless libraries such as @substrate/light-client-extension-helpers. This strategy would enable other contributors in the ecosystem to create (or transition) their extensions simply by implementing their unique user interfaces on top of this robust foundation.
In my view, the most efficient and direct route to achieve this is through the incremental enhancement of the @substrate/connect extension. As highlighted in the previous Polkadot-API update, many of the essential components required for this transformation are already in place. This existing foundation not only accelerates development but also ensures a smoother transition for the ecosystem.
Could PJS DApps Leverage the New Extension?
Absolutely, supporting PJS DApps with this new extension is not only feasible but also advisable. The extension could (and ideally should) offer DApps access to both APIs: the new, more efficient API promoted by Polkadot-API (based on the concept of a Polkadot Provider API) and the existing, albeit more complex, API currently used by PJS DApps. Let me delve deeper to explain how this integration can be realized:
The traditional approach of PolkadotJS extensions involves making themselves known to DApps by “injecting” themselves into a global-scope object (window.injectedWeb3) in the browser. However, this method is not without its challenges. It’s prone to race conditions and complicates the detection process for DApps trying to identify which PJS extensions a user has installed. Moreover, there’s a risk of extensions inadvertently overwriting each other within this global object.
A more efficient and reliable method would be to adopt an event-based protocol. This approach, similar to what is outlined in EIP-6963, involves both the DApp and the extension emitting and listening for events. This protocol facilitates a smoother discovery process for various Polkadot extensions.
In practice, this means that the “legacy” PJS API could continue to be injected via window.injectedWeb3, while the new API becomes discoverable through the proposed event-based protocol. Such a dual approach not only enhances compatibility with existing DApps but also paves the way for smoother adoption and transition to the new system.
Progress and Remaining Tasks
Significant progress has been made towards our goal, but several important tasks still lie ahead:
-
UI Development for Account Management: This involves creating a user-friendly interface for account creation and account derivation.
-
Transaction Review and Signing UI: Developing a seamless and intuitive UI flow for users to review and approve transactions.
-
PJS Extension Account Import UI: Creating a specific interface for importing accounts from the PJS extension.
-
Polkadot Vault Integration: This task entails integrating the extension with the Polkadot Vault.
-
Other third-Party Tool Integrations (Optional but Beneficial): Although not mandatory, integrating with popular third-party tools like Ledger, Wallet Connect, Kampela, etc., would be a huge win.
-
Event-Based System for API Discoverability: Implementing the event-based system discussed earlier is crucial for making the new API easily discoverable by DApps.
-
Implementation of PJS Injected API: This is essential for ensuring the extension’s compatibility and availability for current PJS applications.
Is This Extension Crucial for PAPI’s Success?
While this extension isn’t strictly essential for PAPI’s success, its absence would be felt. We’ve developed the @polkadot-api/legacy-provider to bridge this gap, emulating the new API by integrating smoldot with injected extensions. However, this workaround doesn’t quite match the superior user experience that a “native” API, fully leveraging the extension’s capabilities, would offer.
In essence, the presence of this extension would significantly enhance the functionality and appeal of DApps developed with Polkadot-API. Yet, it’s not a make-or-break factor for PAPI’s overall success.
It’s important to note that the same API envisioned for this new extension could also be adapted for environments that don’t support extensions, such as the DApps section of Nova Wallet.
Final Remarks
The aim of this blog post is to present my vision for the evolution of the @substrate/connect extension to the Polkadot community. I invite everyone to contribute, critique, and refine these ideas.
Regardless of the direction Parity chooses – whether to pursue these ideas, explore alternative paths, or even put this development on hold – I want to express my unwavering respect and support for their decision. My ultimate goal is to maintain a symbiotic relationship with Parity, collaboratively providing the ecosystem with a robust alternative to PJS.