The Ledger App Debate: united we stand, divided we fall

We recently posted a Unified Ledger app proposal which gained both remarkable support and significant pushback within our ecosystem sparking a hot debate on multiple platforms and channels. It was quite surprising to uncover a fair bit of opposition and allegations based not on constructive criticism but rather efforts to block our initiative altogether. We believe that a common Ledger app is in the best interest of the entire ecosystem as illustrated by the widespread support we gained from both parachains and users. We would like to reiterate our unwavering commitment to an open, civil, and transparent discussion by inviting Parity, Zondax, Ledger, and all parachains to voice their concerns and questions regarding the proposal.

In this post, we would like to cover our motivation, solution, security, and implementation, and finally address two main concerns we faced so far.

Motivation

There are three key reasons why we believe there should be a universal Ledger app for all parachains:

  1. The Ledger team is overwhelmed with work and explicitly asks to curb the number of apps they need to support from the ecosystem, as well as limit the frequency of the app updates, making them no more than once per quarter.

  2. Unique derivation path for each app leads to various problems such as the inability to participate in a crowdloan with a Ledger, UX complications related to derivation path selection, and the need to upgrade all applications every time a new parachain is added.

  3. The Ledger app works as a full-scale decoder, where pallet numbers and their respective methods are hard-coded. This means some transactions become unavailable after the runtime upgrade and thus users lose access to their funds for extended periods of time while they wait for Ledger support to respond.

Proposed solution

The gist of our solution is to abandon call data decoding on the device (similar to EVM-compatible apps).

The main rationale for making a distinct Ledger app for each parachain is the need to decode the call data of every extrinsic.

There are three major differences between Substrate parachains and standard smart contract chains:

  1. Large transactions that are difficult to validate, even on a large screen.

  2. Frequent runtime upgrades.

  3. Transactions that are difficult/impossible for the average user to understand, such as XCM or ORML (excellent developments but designed to be as general as possible for various projects to use).

Our approach solves all three of the above problems. For now, the solution works only for asset and balance-related pallets, but it supports all types of balances and effectively covers the majority of on-chain transactions in Polkadot. Also, we’re not just adding metadata for each parachain; we’re transforming the transaction into text readable by humans for each parachain.

Security of the proposed solution

One of the key concerns often voiced during discussion is the security of our approach. Most of our opponents refer to insecure blind signing despite it being an industry standard for EVM. Furthermore, as said above, the most common transactions (asset transfers) will be subject to clear sign and the scope of clear signed transactions will be expanding as new app developments kick in. Additionally, we’ve been in constant communication with the Ledger team to ensure our solution complies with their security standards.

Let’s summarize why we believe the proposed solution is actually secure enough by any reasonable standard :

  1. As mentioned earlier, we modeled the application based on EVM, and our security complies with widespread EVM standards.

  2. The application’s architecture allows to add the processing of new commonly used transaction types, such as governance or staking in the future.

  3. While our solution might not be perfect out of the box,.we have a clear vision of how to make the app more secure and user-friendly (more on this later).

The motivation for this application was not born out of a desire to obtain a piece of the treasury but rather out of the necessity of having this application for our parachain and other parachains in our ecosystem. This is evident from the numerous comments and feedback we received while pursuing the common app goal.

It is very important to re-iterate that the exact implementation of the proposed app will work identically to the way Ledger apps work in Ethereum:

  1. Most common transactions such as Balance (and other asset operations) will be supported in the clear sign mode. See this spreadsheet for the breakdown of the different asset modules we’re planning to support.

  2. Blind sign mode is something that the user has to turn on manually on the device before he will have the ability to use it.

  3. Display the hash of the call data both in the extension/mobile application and on the Ledger device.

Communication with the Ledger Team

  1. Problem with multiple derivation paths: The proposed app will automatically select the needed derivation path between Polkadot and Kusama relay chains, depending on where the supported parachain is.

  2. Users should not blindly sign by default: The app will work the same way as EVM-based apps: users will have to manually turn on the hash signing option on the device themselves.

  3. Frequent app updates are time-consuming and not optimal, how can we avoid that? This is what we as an ecosystem need to sort out, from our end, we proposed to make updates no more frequent than once per quarter.

  4. If we manage app updates once per quarter, can we guarantee a clear sign when the app is not updated? No, we can’t, so those parachains who have implemented some breaking changes won’t work in the clear sign mode but still will have a blind sign functionality available. Yet these updates are quite seldom in our experience and we will monitor the main modules/pallets for the updates.

  5. It’s up to the ecosystem to decide on how to move forward with the app and what other common transactions to support for the clear sign mode: Indeed, this is the reason we’ve created a telegram group with multiple parachain representatives and have been vocal about our propositions during parachain round table calls and in discussions with Parity.

Addressing objections and concerns

We’ve received numerous polkassembly comments and posts related to the proposed solution and would like to address two major ones here.

  1. Security concern: Probably the most significant one we’ve received, which can be formulated like this:

Q: How do you verify that what you are signing is actually what the hot wallet shows you? This is impossible and therefore totally insecure.

A: The concerns associated with signing hashes are usually explained using the example of simple transactions, but let’s look at the example of an XCM transaction, which is the main feature of Polkadot parachains.

  • The size of the transaction - it is almost impossible to validate it on the Ledger device, moreover the device itself frequently freezes when presented with a transaction of such size
  • Arguments and methods that are incomprehensible to the user
  • Recipient address in public key format => You need to use third-party applications for validation
  • The transaction may appear in the encoded format of the recipient’s chain => you need to use third-party applications for validation

To check such a transaction user must use 3rd party instruments to validate it, raising all of the same security concerns as with the hash signing.

We see two possible approaches to increase security:

  • The first approach is to change the SignedPayload struct to create a human-readable string from extrinsic, so the users will sign a string, not a byte array. This requires major changes to be made in the substrate framework and this process can take a lot of time.

  • The second approach is to add a new SignedExtension, and double sign the transaction (byte array and human-readable text), it will add more data to extrinsic and will increase transaction validation times, yet it can be implemented without any modifications to the substrate framework. Any parachain can support the app by adding this SignedExtension. SignedExtension can support default trx. signing for backward compatibility.

We would love to hear community and expert thoughts on the above and Invite Parity, Parachains, and the Zondax team to further discussion. We’ve also reached out to Parity to organize a technical call on the issue.

  1. Zondax claims: Here is a quote:

Q: “We are shocked that the development plan seems to bluntly claim features and activities that already exist and have been available for a long time”

A: We want to build a universal Ledger app that will support multiple parachains. We ask you to carefully re-read our proposal to clearly understand the exact activities for which we’re requesting the funding. To be specific, funding is required to enable clear signing for all available balance pallets and other developments that require a lot of effort.

There are several issues with Zondax’s approach as we see it:

  1. Zondax APPs currently support only 4 main chains: Polkadot, Kusama, Statemine, and Statemint.

  2. In order to have Ledger support, Zondax requires each parachain to sign up for their services with no transparency about costs.

  3. Zondax’s approach is not universal, doesn’t serve ecosystem interests, and doesn’t provide a universal parachain app in a timely manner. We crave to speed up things a bit for the ultimate ecosystem and user benefit, yet the discussion around our solution has been going on for over 6 months now with no meaningful progress made so far. We believe this is not the way things should ultimately work in the ecosystem.

  4. There are concerns regarding the cost of the app and its support to the Polkadot treasury. Spending $700k of treasury funds just for maintaining the support of Zondax’s solution for 4 main chains and no parachain support without meaningful new development work seems unreasonably expensive to us. Additionally, the entire community would benefit if there was more disclosure about the proposed audit costs and the companies that will be doing the audit. There are known audit company names on the Ledger website and we as an ecosystem should reach out to them publicly.

We have never spoken out publicly against Zondax’s solution, and it is very strange for us to see so much negativity coming toward ours recently. We believe that Zondax’s solution for Polkadot and Kusama should remain in place at least until a better one is devised.

We operate on the premise that grants and open source are in place so that if any team can create a better solution, they can compete for funding.

Finally, we have always stated and continue to say that our solution is just the first step towards the right Ledger application, and as we as a community continue to figure out how to improve it, we fully expect and welcome other community members to come forward with initiatives similar to ours.

13 Likes

thank you for such a thorough write up.

I will attempt to keep my response concise:

  • I agree that a ledger app which supports all Substrate based chains is valuable, even if those transactions are “blind” to the user. We have seen that this does unlock tons of activity throughout other ecosystems which have taken a similar approach. Ethereum is a great example.
  • We should be focusing less on supporting clear signing of all messages (since this has shown to be tough), and instead focus on the set of messages which are most frequently used by cold wallets, which I believe is the main scenario for most ledger users.
    • Balance transfer and assigning a proxy seem like the primary focuses.
    • Beyond that, with the use of proxies, more day to day activities can be completed with things like Polkadot Vault, various browser extensions, or other hot wallet choices. These have much easier methods for updating metadata and performing clear signing.
  • Historical context is that the current byte encoding of transactions is to minimize the data footprint on Substrate based chains. Each transaction requires only 2 bytes to target a specific call (and then any extra data needed for the call parameters).
    • These two bytes are usually: (1) pallet byte index, (2) call index within the pallet.
    • This is already inherently limiting, for example, there is only at most 256x256 possible calls, assuming all pallet calls are optimally packed. This is plenty, but I am certain that at some point someone will ask for more than 256 calls in one pallet, or try to add more than 256 pallets to a runtime. When that day comes, I don’t know what solution we will come up with.
    • Passing raw strings to and from the runtime would be exactly opposite of the goal to reduce transaction byte size. Your suggestions about hacking the Signed Extensions would seem to work, but I do not think this is something we should encourage in the ecosystem.
  • I am in 100% agreement with all 4 problem points that you call out with Zondax’s solution.

I recently posted on twitter:

We should only fund teams who will implement the Ledger app using XCM.

In fact, I mentioned the same ideas in a previous thread about the Ledger app.

I think anyone who will take on building the Ledger app for the Polkadot ecosystem should read, and provide feedback / comments / criticisms / reality checks to this thread: XCM as a Standard for Reading And Interacting with Parachains

It seems that XCM is actually PERFECT for a universal ledger app for the entire Polkadot ecosystem and beyond. XCM describes a language to convey intention. In the context of token transfers, it can perfectly express transfering any kind of token to any user, no matter if their balance is represented on a parachain, contract, or any consensus system in between the two.

It should be that a Ledger app can present to me an XCM message, which I sign, and that will always do the same operation no matter the underlying runtime… because that is exactly what XCM was designed to do:

Using the Runtime Metadata was an “old way” of thinking because it was the only option at the time of Polkadot / Kusama launch. But now that XCM is getting more and more mature, it should be the place we go to in order to keep our app ecosystem working seamlessly.

XCM has historically updated only 1 time per year, and will likely not update more than every 6 months.

So given this information, why should we financially support any Ledger app which does not choose XCM as the base layer?

10 Likes

I agree, but this Polkassembly comment from the Head of the Developer Ecosystem at Ledger suggests that it’s not a realistic possibility. The whole post is worth a read, but this is the part I’m referring to specifically:

  • While the application may qualify for developer mode (subject to confirmation), its consideration for public release is unlikely.
1 Like

Because parachain teams are blocked from onboarding vast amounts of users right now. I think having XCM-based Ledger app would be great from a technical perspective and if Parity or other parties are willing to build it, great, all for it.

But the reality for parachain teams is that we are years behind EVM chains. The app developed by Equilibrium is solving an acute need and should thus be funded in addition to the Zondax and possibly other apps.

We are the builders in that ecosystem giving utility to Polkadot and Kusama. We are the ones in the trenches with users and we are the ones that need a solution to this problem.

6 Likes

Quite a hot topic right now - thanks for such a thorough write-up!

The way I see it is:

  • The current situation wrt Ledger & Zondax is simply unsustainable - the ongoing costs are exorbitant and there is absolutely no chance that the majority of parachains would be able to get their own Ledger app. As a result, anyone who sends DOT from their Ledger to any other parachain is basically immediately rekt.

  • It is concerning to see the comments from Ledger wrt this app never likely being considered for a full Ledger app - which heavily reduces the value of the Equilibrium solution. That being said, I do think this could be useful at the very least as an option for users who get their DOT “stuck” by the above; even if it won’t resolve the ongoing need for a retail friendly Ledger app that works for all.

I love the idea of a universal app based on XCM that would require far fewer updates moving forward - but sounds like this could be some way off if not a pipe dream. In the meantime the Equilibrium solution at least helps us to take steps forward, providing some way of users retrieving lost funds.

That being said, we also cannot afford to take any steps back wrt Ledger in our ecosystem because it’s already a major pain point for onboarding liquidity - therefore I will be voting for both the Zondax and Equilibrium proposals, as well as any future proposal to build an XCM version :eyes:

I hope that in the following 12 months we can indeed come to a solution that allows us to step away from the current untenable position.

8 Likes

Hi, Shawn!

I didn’t quite understand your proposition, can you please elaborate several things:

  1. Usually XCM transactions are way to long/complicated, how should we display those on the ledger device?
  2. As stated in the post, current XCM transfers use public key not the SS58 address, how should users validate their addresses?
  3. How should an XCM message which invokes call on the target chain look like?
  4. How do we account for non-trivial extrinsics without resorting to byte arrays? For example, we have DEX methods which take several parameters (price, quantity, side, e.t.c.)

It would be beneficial if you provided your vision more clearly with a simple example.

1 Like
  1. I do think some XCM calls are a bit more complicated than the average transaction, but basically all XCM are just a set of instructions. In the case of a balanace transfer on the same chain, I do think it is as simple as a single instruction TransferAsset. If there are multiple instructions, I would display those instructions in plain text, one by one, like a batch call.

  2. SS58 is just a specific display format for the public key. The raw bytes are the same. However the ledger app is converting bytes to the SS58 string should work just the same on a public key. (Either I am misunderstanding something niche, or it is quite concerning you don’t know this)

  3. At the moment, it would be a single call to pallet-xcm > execute with the inputs being the XCM instructions to be executed, and the max weight, which could be approximated or calculated like max gas for an ETH call. As long as this call index doesn’t change and XCM isn’t updated, then the ledger app would not need to upgrade. We can probably guarantee this will not happen more than 2x per year, more often just once per year.

  4. As I mentioned in my post, the first and most important thing to handle is transfers and proxies. This is a contained, and relatively simple set of instructions the ledger can certainly support. More complex stuff like DEX methods, the ledger is shitty for, for all the reasons mentioned about metadata, limited encoding capabilities, etc… Instead users should be easily guided to make a proxy for their account, and use a hot(ter) wallet for these operations. Browser extensions and Web UIs have no problem with this kind of thing.

2 Likes

@shawntabrizi - the solution to a generic app that can richly display every transaction is quite simple and elegant and was discussed at length in the past with @bkchr, Alexander Slesarew and others from the Wallet ecosystem.

Embedding a merkletree of the metadata into the signature payload means that a signer can simply be provided with the metadata proofs, sign the signature and the chain will reject it if the metadata root doesn’t match:

2 Likes

I had not seen that, and does sound like it would solve some of the problems at play here.

So:

  1. Latest metadata would be downloaded on the fly, and sent to the ledger device.
  2. Ledger device would hash the metadata, as well as use it.
  3. User can then sign the message, and submit it to the chain, with the metadata hash included in the signed payload.
  4. The chain can reject the payload if the metadata hash signed does not match what it on chain.

The flow makes sense to me. Questions would be:

  • Can we rely on always being able to download the latest metadata? I guess we already assume we are always online?
  • Can the ledger device handle this large payload, and hashing it?
  • Is using the metadata better than using XCM?

XCM in my mind would still unify all the experiences across chains, contracts, and anything in-between. For example, transferring a contract token and a native token would look the same, just with a different asset id. Chains using different pallets, for example balances pallet, assets pallet, multi-balances pallet, etc… would all have the same XCM instructions. A browser extension, which is usually the starting point for a ledger transaction, would be able to interface with all chains automatically if they supported XCM, whereas extensions would need to individually program how to handle operations for each parachain, depending on where they have customized pallets.

The problem is that Metadata only expresses the contents on the chain, not necessarily functionality and intention. XCM relays intention of the user, and allows the XCVM to interpret that into the appropriate on-chain operations.

@juan.leni could you comment on whether this is feasible? Ideally the specific metadata needed to sign the current transaction is passed along on every signature. Does Ledger’s API allow for such metadata to be passed along needed to construct the signature?

@shawntabrizi I agree, XCM will solve a lot of this. HOWVER the more we rely on XCM the more important it is that the signers (ledger, wallet or parity vault) actually understand XCM. Increased in complexity of XCM will mean more frequent updates to it. So we will end up replicating the infrastucture we have for metadata, would you agree?

XCM is about doing general interactions that any consensus engine could understand.

Metadata is about understanding specifically the composition of a chain.

There is a reason that chains use XCM to interact with one another, and not Metadata.

I am not arguing that we should completely halt all work on metadata based approaches, or that metadata does not play a part in the perfect UX, but what I am saying, back to the my original post, is that any proposal which is asking for hundreds of thousands in funding should have XCM based integration on the roadmap. It should be clear the value of XCM to the final experience that we want for users and application developers.

I understand the difference between chain metadata and XCM. My argument was: to support cross chain DeFi we will need a much larger set of XCM messages (like deposit DOT into an AMM, borrow USDT, or swap USDT for DOT with a maximum price of Z, deposit DOT into liquid staking and get LDOT). As the complexity of XCM grows it becomes more important for any signer to understand XCM well and present it in a human readable way so that the user signing it can know what they sign.

Hence all the work we do for metadata we will also have to do for XCM format. The challenge is how do you know what xcm message format users creates a message in? This technically depends on the receiving chain but there is no way for the wallet to know what XCM messages the receiving chain supports so getting XCM message format trustlessly will be rather hard.

@shawntabrizi do you agree with this?

1 Like

We can compare Pallet transactions to XCM transactions:

Pallet transactions:

  • Meaning determined by pallet and runtime authors.
  • Meaning can change with every upgrade of the chain; no limitation in frequency.
  • Unversioned.

XCM transactions:

  • Meaning determined through RFC process and community input.
  • Meaning changes only when a new XCM version is arrived at (historically once per year).
  • Versioned.

Constructing XCM transactions therefore seems like it is much more suited to offline devices which need to determine/display human-readable semantics. The signing device would still need to be updated when a new version of XCM is released with that new XCM documentation, but this is extremely infrequent and certainly no more often than once per quarter. XCM is designed to be highly composable and flexible - substantially more so than pallet calls. Complex de-fi operations involving multiple steps such as withdrawal, borrowing and exchanges are in fact better suited to a programmable environment like XCM than they are to regular Pallet calls and there are already some planned features which will make this stuff easy to express in XCM.

The Pallet metadata approach would still be needed for chain features which XCM cannot express. The Merklised metadata approach which @lucasvo pointed out above is the way to go; the chain would both include and sign transactions with the current metadata hash. Hot UIs can then append the relevant metadata together with proofs that they correspond to the given hash in the transaction payload given to the signing device. This new format of transactions could even co-exist with the older format, allowing the metadata signing to be optional (though offline devices would obviously refuse to sign if not included). This ensures that it is possible to make transactions which are desired to hang around over runtime-upgrade boundaries.

4 Likes

I am hearing two conflicting things from people working on Polkadot on the idea of using XCM as not just cross chain messaging but a generic user interface:

  • XCM is a better way to interact with parachains and we should push people to just use it
  • XCM won’t change much (1/year) and because of that will never be exhaustive

In the short term XCM could lead to faster adoption of simple token transfers but could actually hurt the ecosystem in the long run because it’s incomplete.

We have to be careful we don’t make XCM the first choice of integration for things like exchanges and wallets only to realize that because we’ve now locked them out of actually interacting on chain. What if most custodians decide to integrate XCM to enable parachain support but are now unable to offer governance support because XCM doesn’t support it. We risk that anyone that went the “easy route” of just using XCM won’t add regular governance functions because of the overhead of supporting both extrinsics and XCM.

2 Likes

XCM will never be exhaustive

XCM is unlikely to be able to cover all functionality offered by all chains but is eventually likely to cover most common functionality of chains. Governance is a good example and likely should be covered by XCM in the not-too-distant future.

I fear that attempting to convince all services to support the interpretation of all chains’ Pallet transactions is unrealistic. There would be a huge amount of duplicated work. Metadata is fine generalising the signing process over many chains, but it doesn’t help much with creating UIs for things like chain-agnostic defi and governance.

1 Like

Yes, I agree. Hence it’s important to not look at XCM as the savior for parachain integrations for anything that might remotely need to do more than sending and receiving tokens - which some people have suggested. Quite practically this means for any custodian XCM isn’t a good candidate because it doesn’t support governance features as of today because we do want custodians to support governance transactions.

Sorry for taking this thread off topic!

While I’m generally against pushing anything as being a “saviour”, XCM is designed to abstract all sorts of operations over chains. It already does a lot more than “sending and receiving tokens” and will evolve over time to cover a lot of use-cases. Eventually its coverage will likely become complete enough that the vast majority of transactions from the vast majority of users can be expressed in it. Of course if you want your custodian to support niche or custom operations on a specific chain then you’ll need them to support Pallet transactions. But I expect a lot of the time chains won’t have these operations or users won’t have any need of them. And custodians might be reluctant to support arbitrary transactions which can alter assets held in some arbitrary way - XCM is a lot safer in this regard.

2 Likes

Let’s recap what we’ve stated so far. Here are the issues we see with the integration of the ledger and other cold wallets:

  1. Frequent runtime updates
  2. The need to have an application for each parachain
  3. Inability to verify large transactions on the small screen of the Ledger device
  4. Inability to validate transactions without third-party applications using generic solutions like XCM or ORML. There are too many specific arguments that ordinary users are unfamiliar with: the average user could care less about what is XCM v1, v2, or v3 and what are their subtleties and differences.
  5. It is a daunting task for an average user to validate which asset has an ID of 100500 or how his public key is related to his displayed address. Users crave simple, human-readable instructions and interfaces, not technically challenging things they don’t understand.

To address those issues, different approaches related to ledger APP design have been discussed so far. Below we consider all of them and show why we think the approach we propose is optimal.

Metadata. Integration through metadata is highly specialized. In practice, only hot wallets + UI can use this. The downside is all five issues mentioned above.

XCM. XCM is an excellent solution for machine integration, enabling integration between parachains, exchanges, trading bots, arbitrage, etc. Once the program is written, it does not need to be updated for at least a year until a new version of XCM is released, but issues 3, 4, and 5 persist.

Additionally, at the moment, XCM only supports transfers, while other types of transactions require execution, which requires metadata parsing and brings us back to issue number one.

SignedPayload struct. The node can generate a human-readable string from the binary representation of the transaction and offer the user to sign that specific string. We attach a high-level schematic description (picture) of how it would work for the client.

This solution can be customized to make the parameters more understandable for humans (e.g., showing the ticker/symbol instead of the asset ID). Upgrading the runtime would simply involve generating a new string if the arguments have changed.

The signature verification would fail if the client signed an old version. Trust should be placed only in what the clients see when signing the transaction.

This solution addresses issues 1, 2, 4, 5, and partially issue number 3, as for the most common “large” transactions, a shorter and human-readable text can be devised.

@gavofyork curios to hear your thoughts on this

4 Likes

We had this week some good discussions around the topic. We also directly spoke with Zodax and other people of the ecosystem. We will go into the direction of merkelized metadata. There will be a new ledger app that will stream the metadata as proofs to the device to stream decode the extrinsic. For other hardware devices it shouldn’t be such a big problem as they already support the entire metadata, but they will let faster as we don’t need to transfer the entire metadata any more.

7 Likes

I think this GameBoy solution (links courtesy of @rich) is futureproof and is highly resilient because you can’t trust existing and future supply chains:

https://www.gamewallet.gg/

I believe many extrinsic signing UIs across all of Web3 can be mapped into GameBoy interfaces and can be done in an entirely open source way, with L1s/L2s+Substrate parathreads collaborating for a super decentralized resilient development path. Instead of the dependence on Ledger, we depend on an ancient technology stack:

(a) unlike Ledgers now and in the future, provably WITHOUT backdoor possibilities because people from the past could not have known about the future
(b) unlike Ledgers now, has a programmable low cost ($100) display that can display L1/L2 complexity with just a little bit of metadata

A proof of concept of Web3 GameBoy (or some such ancient device) signing a Substrate extrinsic or EVM Transaction would involve getting sufficient Substrate runtime metadata + EVM address ABI to be rendered on the ancient device. I think a good UI would prevent a lot of stupid things from happening, like getting the decimals right and mapping activity into the users USD/EUR/… native currency.

Can all of you remaining Ledger fans attack this GameBoy style of solution?

1 Like