The need for an improved `Signer` API

1

Introduction

In a previous post, I discussed the PJS extension’s role as primarily a signer, lacking direct blockchain access, and proposed the concept of an extension that marries a secure light-client connection with a comprehensive signer API. This union, I argued, would significantly benefit both dApp developers and users by streamlining interactions and enhancing security.

Yet, one critical aspect went unmentioned: the pressing need for a simple, standardized signer API properly decoupled from PJS. The rationale is straightforward—offline signers, which arguably should represent the majority due to their inherent security advantages, require a robust and simple API. Such signers should feature, at a minimum, an API for updating metadata across supported chains and another for enabling user prompts to sign transactions. These transactions, decoded with the specific chain’s metadata, would be presented to users in a humanly readable format. Notably, even if users were misled into updating metadata with a corrupted version, the introduction of a new sign-extension promises to invalidate any transaction whose metadata hash fails to align with its on-chain counterpart.

The ideal scenario, therefore, involves building the API I envisioned in my earlier discussion atop a universally accepted standard for offline signers. This contrasts sharply with the current landscape, dominated by the PJS extension’s bespoke approach—a situation that is far from optimal and calls for urgent reevaluation.

This blog post aims to outline what constitutes an “ideal” offline signer, contrast it with the current PJS model, and chart a course towards adopting a more resilient and standardized API.

Part 1: A Better Signer

For clarity, let’s define the context of the proposed signer interface to include:

  • The chain it interacts with
  • The key pair
  • The signing scheme (Ed25519, Sr25519, or Ecdsa)

One potential approach to access this signer interface (with its context) could involve a keyring interface, featuring a method like .getSigner(publicKey: Uint8Array, chainId: Uint8Array). In most cases, the chainId would correspond to the genesis hash, though we will not delve into that complexity here. This keyring interface could also facilitate adding new chains, among other functionalities.

Focusing on the Signer interface itself, a better API might resemble the following TypeScript definition:

export interface Signer {
  getSystemVersion: () => Uint8Array;
  getSupportedMetadataVersions: () => Array<number>;
  updateMetadata: (newMetadata: Uint8Array) => Promise<void>;
  signTransaction: (data: Uint8Array) => Promise<Uint8Array>;
  signData: (data: Uint8Array | string) => Promise<Uint8Array>;
}

  • getSystemVersion: This method serves to verify the currency of the metadata, which could become obsolete with the introduction of merkleized metadata, potentially leading to its deprecation.

  • getSupportedMetadataVersions: Enables querying which metadata versions the signer supports.

  • updateMetadata: Allows for the suggestion of updated metadata versions. Again, once the merkleized metadata is a reality, this could be obsoleted/removed, as it could make sense to send the metadata every time that we want to sign a transaction.

  • signTransaction: Triggers a user prompt with a human-readable description of the data being signed. The argument is scale-encoded, comprising Tuple<CallData, Extra, AdditionalSigned>:

  • signData: Initiates a prompt for the user to sign an arbitrary payload.

With advancements in Polkadot-SDK metadata, it’s now feasible to properly decode and present this data to users via a generic viewer in a human-readable format. This eliminates the need to frequently update chain-specific definitions, streamlining the process for a more user-friendly experience.

Part 2: Comparing the “Better” Signer with the Current PJS Signer

The SignerPayloadJSON Interface

Exploring the proposed straightforward and streamlined signer API reveals stark contrasts with the current API expected by PJS for transaction signing. This existing API suffers from several critical shortcomings: a lack of generality, insufficient documentation, and an overly tight coupling with PJS’s specific implementation quirks. To illustrate, consider an actual payload generated by a PJS application:

{
  "specVersion": "0x000f5d98",
  "transactionVersion": "0x0000000e",
  "address": "5H3wTChvMxtd4z31MEdAHdmHgaqNoUAvepJ473kPgQnARPUB",
  "blockHash": "0x3e6bde5824c9d38e84934df1dcf198a3a884aa9a151cf303aad8e1d1915cfeab",
  "blockNumber": "0x006211c4",
  "era": "0x4400",
  "genesisHash": "0x67f9723393ef76214df0118c34bbbd3dbebc8ed46a10973a8c969d48fe7598c9",
  "method": "0x32092000d0c8a7fb3071347144aff6fe64f484bf72e69582009c7638bae95f2be8d88f4ba10f",
  "nonce": "0x00000011",
  "signedExtensions": [
    "CheckNonZeroSender",
    "CheckSpecVersion",
    "CheckTxVersion",
    "CheckGenesis",
    "CheckMortality",
    "CheckNonce",
    "CheckWeight",
    "ChargeAssetTxPayment"
  ],
  "tip": "0x00000000000000000000000000000000",
  "version": 4,
  "assetId": {
    "parents": 0,
    "interior": {
      "x2": [{"palletInstance": 50}, {"generalIndex": 8}]
    }
  }
}

Issues with this approach include:

  1. Signed-Extensions Redundancy: The API requires consumers to specify signed-extensions that the signer should autonomously identify from the metadata, adding unnecessary complexity.

  2. Inconsistent Encoding: Surprisingly, not all hexadecimal fields adhere to SCALE-encoding consistently. This inconsistency hampers correct data interpretation and challenges developers. For instance, trying to pass SCALE-encoded values to any field other than the method results in the extension not interpreting those field correctly.

  3. Decoding Discrepancies: The API accepts data decoded in a way that’s non-standard and subject to arbitrary implementation details of PJS.

The Urgency of Abandoning Custom Type Definitions

The true depth of the issue extends beyond this convoluted API. The PJS extension’s approach to using metadata only for decoding the method field necessitates the creation of custom definitions for each chain. This method introduces several problems:

  • Delayed Adoption and Innovation: Custom definitions create a bottleneck, hindering the adoption of new chains or features until they are integrated into the PJS repository.

  • Increased Maintenance: Developers must continuously update these custom definitions to align with on-chain developments, a redundant and cumbersome process given the dynamic nature of blockchain metadata.

  • Compromised User Experience: Users are often the last link in this chain, facing delays in accessing new features or chains due to the time required to update and integrate custom definitions.

  • Security Risks: The lag between on-chain updates and their reflection in the PJS repository through custom definitions can expose users to vulnerabilities, undermining the security and integrity of their interactions.

“Half Blind-Signing” Shouldn’t Be a Thing

Another significant concern is the phenomenon of “half blind-signing.” This occurs when outdated metadata leads to key transaction details being displayed in unreadable hexadecimal, while other, less critical information is presented in a readable format. This approach is misleading, giving users a false sense of security about their understanding and control over the transactions they sign.

If an extension cannot fully utilize metadata to make transaction details comprehensible to users, it arguably should not attempt to partially decode or interpret this data. Presenting any part of the transaction as understandable under these circumstances is deceptive and potentially harmful.

Part 3: A Path Forward

The critical question now is whether we can evolve the current PJS extension into the envisioned “ideal” model without disrupting the existing dApps within the ecosystem. The answer is a resounding yes, and I argue that this transition is not only feasible but also imperative.

1) Maintaining SignerPayloadJSON Interface While Evolving Internally

First and foremost, ensuring backward compatibility is crucial. The PJS dApps currently in operation must continue to function seamlessly, which means the signer.signPayload method, utilizing the SignerPayloadJSON interface, should remain operational. However, this does not preclude us from enhancing how the PJS signer processes its inputs. By leveraging the type definitions found within the blockchain metadata—rather than outdated custom type definitions—we can maintain the existing API surface while significantly improving the underlying mechanics.

2) Introducing the signTransaction Method

Next, we propose adding a new signTransaction method, as outlined earlier. This method would not only adhere to the improved standards but also potentially streamline the internal workings of the existing signPayload method. By converting its inputs into the standardized Tuple<CallData, Extra, AdditionalSigned> format as defined in the metadata, we can achieve greater consistency and efficiency.

The good news is that the necessary tools and frameworks for this transformation have already been developed by the Polkadot-API team. We are fully prepared and eager to assist in making this transition a reality.

Conclusion

The path forward involves a delicate balance between preserving the functionality that current dApps rely on and introducing more robust, flexible, and secure methods of interaction. By carefully implementing these changes, we can significantly upgrade the user and developer experience without sacrificing the integrity of existing applications.

8 Likes