Developer Experience must be our #1 Priority

Governance
1

tl;dr - This post looks into major pain points that development teams experience in Polkadot, advocates for making great developer experience an ecosystem-wide effort, and proposes a strategic mindset and a few tactics to go forward

Like many ecosystems, Polkadot is feeling the effects of the bear market. Some teams are turning their attention towards Ethereum to gain exposure to its capital and users. Some teams run out of funds and head for the exit. Some teams lose access to or face significant issues obtaining Treasury funding that they previously relied on and reconsider their business strategy. Many more teams are not yet there but are already facing related symptomatic issues. These dynamics affect morale and the optimistic outlook that was previously felt much stronger.

Everyone feels this evolutionary pressure.

At the same time, Polkadot is almost universally on the leading edge of crypto research. The Polkadot 1.0 roadmap is delivered, with Parachains offering a forward-looking scalable solution to anyone wishing to establish a sovereign and reliable base in Web3. The Polkadot 2.0 roadmap is starting to shine through with concept drafts like coretime, coreplay, and corecast. Polkadot will expand its scope to support smart contracts and additional chain/rollup types as first-class consumers of Polkadot blockspace.

How do we reconcile those two perspectives?

Many great projects in history failed despite having superior technology because the market turned against them. We must not let this happen! The success of Polkadot is crucial to giving humanity a positive outlook on Web3 and democratizing the Web. While a world without Polkadot is possible, I want to see a world where the vision and passion of the people in Polkadot will have a positive impact that is recognized in history books.

In recent days, many people in the ecosystem discussed these current issues, the possible root causes, and how to move forward. In this article, I want to summarize the discussions I have observed and share my perspective. We will begin by looking at the current symptoms. Then I will put forward a few definitions that look like they could find consensus and propose a few strategic pillars for advancing the Polkadot ecosystem. We will close with a few specific suggestions for initiatives.

Pain Points

Let’s discuss some commonly mentioned pain points:

  • Parachain development is rough
  • Parachain teams feeling unsupported and unheard by Parity
  • Uncertainty around Treasury funding
  • Perception of Parity’s focus on enterprise vs. Web3 values.
  • Polkadot’s lack of representation in the thought leadership space.
  • VCs fud Polkadot

Parachain development is rough. While 1.0 is delivered, bootstrapping a chain is still incurring considerable effort and costs (running collator networks/initiatives, testnets, DevOps, QA, customer support, integration support, and Substrate maintenance) with a time to market of 2-4 years. Projects like Tanssi are removing obstacles (by supplying better templates, integration support, running collators, etc.) and reducing the time to market, and core contracts might reduce the development effort in the future, but we still have a long way to go.

Parachain teams feeling unsupported and unheard by Parity. This is not universal and teams give different accounts of this. Parity is hosting several public and semi-private formats for teams to gather around certain themes and advance them together. I for example am invited as a Mangata representative to a monthly DeFi call for DeFi teams in the space. Earlier this year there was also a survey specifically targeted at developers in the ecosystem. There is also 1-on-1 consulting with teams that have secured a parachain slot on Polkadot. I believe this issue is felt more strongly by teams that are a bit further out from the “center” of Polkadot’s attention. E.g. as a member of a team that has been building since 2020 but has not procured a Polkadot slot yet, my last official 1-on-1 with Parity was in 2020. Another common pain point mentioned in this context is that Parity strives to maintain neutrality, which results in them avoiding mentioning specific projects. The Polkadot website for example has not yet managed to explain to a prospective user what you can actually do in the Polkadot ecosystem. No mention of specific projects or products. In the year 2023. 7 years after the whitepaper.

Uncertainty around Treasury funding. Projects that have built their business case reliant on Treasury funding feel uncertain, as OpenGov is still fresh and has not established basic rails and guidelines and what is expected to get funded and what not. I have started a more in-depth discussion on the issue here: Towards a Treasury Budget

Perception of Parity’s focus on enterprise vs. Web3 values. A contentious discussion. Many cypherpunks and OGs feel that focus on Enterprise adoption has little or negative value, while others feel the complete opposite. There is a perception that this focus on Enterprise adoption is counter to Web3 values. Parity employees have rejected the notion that Parity is overly focused on Enterprise adoption.

Polkadot’s lack of perception in the thought leadership space. Some people are observing that Ethereum and Cosmos enjoy much better communication than Polkadot with those. To me it looks like this is because Polkadot has coherent tech leadership, which resulted in some amount of bubble-ification. Polkadot was a thought leader for many years focused on delivering the 1.0 roadmap and had little active communication with the remaining crypto space. Cosmos meanwhile which is much more federated has built a mental model that interacts more closely with Ethereum. I would argue that this problem has been observed already since the beginning of the year and Protocol Berg this year saw Polkadot OGs like Gavin and Rob engage with thought leaders from Ethereum and Cosmos.

VCs fud Polkadot. Some parachain project leaders report that VCs are giving them a hard time for being Polkadot projects. This indicates that VCs get nervous about their ROI and don’t see enough measurable success. I believe this has to do with narratives to a great extent, but there are still systematic causes that we need to attack.

Let’s Fix the Root Cause: Developer Experience

The root cause of the issues described above is one thing: developer experience (DX). The way developers experience the ecosystem.

Launching a parachain is hard, expensive, and takes at least 4 people and 2 years (except GM chain). And somewhere along the way, many teams realize that they probably didn’t need to build a chain to build an app to begin with.

There is definitely a market for parachains, appchains, rollups, etc. We need them to extend the ecosystem with additional processing capabilities and features. An analogy is how we extend computers with new components like CPU → GPU → TPU, HDD → SSD to gain new, better properties. Parachains also offer better sovereignty.

But at the same time, it is clear that many apps don’t need to be a chain and will revert to being smart contracts, go chainless, or become a secret, third thing eventually. Operating under these conditions is tough. Crypto is volatile and working on the bleeding edge makes it even more uncertain.

I want to offer an alternative perspective and then explain why we should work towards it…

Imagine a world where Polkadot has become a rich marketplace of experimentation and innovation. Prospective developers find a lot of helpful resources and documentation. They have the tooling to

  • (fast) try out building in virtual environments that show immediate results,
  • (powerful) access all of Polkadot’s primitives and parachain features,
  • (easy) have access to unified APIs that capture many use cases, and are able to deploy a new parachain within minutes, and
  • (fun) there are tons of great people who share a vision and positive spirit that makes you want to stay forever.

Such a place would give a great impression to developers and would provide a strong foundation for organic growth. There would be lots of experiments inspiring each other and an influx of great minds.

But why should we focus on DX and not on onboarding more users or a big enterprise push?

Defining Polkadot Success

When is the ecosystem successful? You can pick many KPIs and they will all correlate on a large enough time horizon. Price of DOT, blockspace sold, on-chain volume, TVL, number of users, etc…

I argue that the best measure of success is the number of new and diverse apps being deployed. A big number of new apps means we did a lot of things right in the past in terms of acquiring new developers:

  • they became aware of and interested in(!) the ecosystem and started to build
  • they had access to resources like documentation, developer relationship advocates, other teams
  • they actually were able to build far enough to eventually deploy

A large number of new apps being deployed means there is a great chance that some of them will be successful and that we are creating an environment that is inducing experimentation and innovation.

It also ties in and correlates with many other KPIs. If we have many apps:

  • Polkadot sells more blockspace
  • It will attract new users of the apps
  • There will be speculative volume around those projects, attracting more TVL
  • etc…

Considering everything that has been said so far, there is one conclusion:

Positive Developer Experience must be the #1 priority of our ecosystem.

The 3-2-1 Rule of Developer Satisfaction

Memorize the 3-2-1 rule of developer satisfaction. There are:

  • 3 tiers of Polkadot technology
  • 2 types of developers we want to speak to
  • 1 common goal: PMF

3 tiers of Polkadot technology: Polkadot consists of 3 tiers of technology:

  1. The Polkadot relay chain: The thing that produces blockspace.
  2. The parachains and other future blockspace consumers
  3. Apps

Parity and the Polkadot fellowship are building (1). It’s a complex machine and nobody should be forced to hear about it unless they want to. Parity’s original job was to promote Polkadot to parachain builders (2). This was sufficient for many years. But our ecosystem has grown and now that parachains are here, we as an ecosystem need to focus on delivering actual apps (3) to end users. So the requirements on ecosystem marketing are splitting and now targeting…

2 types of developers we want to speak to: There are chain developers and app developers. For a long time, we as an ecosystem conflated the two and assumed that everyone wanting to build an app in Polkadot would want to build a chain. But that is not right. As a dev, you should always only put as little effort as necessary into the environment. Your core focus should always be the pure app experience. Thus we now arrive at the these two types of developers that we want to make happy.

This implies a perspective shift: Instead of just chain developers, we need to think about the whole value chain and how app developers that build apps for the eventual end user are catered to. We do this by focusing on…

1 common goal: Product-Market Fit: Polkadot has not yet reached product-market fit. PMF means that the product has found a sufficient market. It’s unclear if Polkadot is already there. Who is the market? Since we define ecosystem success as many devs building many apps, the market is app developers. Devs still have a hard time building apps, so we can safely say that we don’t yet have a proper product-market fit.

The basic mindset to converge toward product-market fit is: Talk, Learn, Improve. Talk with as many devs as possible. Learn from the conversations. Improve the process to make the product work better for them. Our goal is to make the developer experience as good as possible by talking with developers and learning how we can do better and implementing it.

Specific Suggestions for Improvement

There are 4 specific improvements that were mentioned by ecosystem actors in the previous days. I am adding my own, fifth suggestion at the end:

  1. Ask Parity to prioritize customer success with dedicated key accounts for as many prospective developer teams as possible
  2. Increase Polkadot’s visibility in events and discussions. Establish more thought leadership across ecosystem boundaries.
  3. Embrace cross-collaboration and cross-seeding of ideas by inviting members from other ecosystems to our events.
  4. Highlight existing Polkadot projects much more
  5. Make Developer Experience a priority that Polkadot Governance demonstratively puts on the agenda and funds activities for

We All are Polkadot

I know that it is easy to point fingers at some parts of the ecosystem and say: “They should do better.” But the truth is: We all are Polkadot. Polkadot’s biggest asset might not be its tech, but the people that are here. We all share the vision of a Web3 that puts ownership of the web in people’s hands and allows everyone to shape it. Polkadot is rich in people who carry that vision and make it a reality. It is upon all of us to help other devs join the ecosystem and create an environment that kicks ass in terms of developer experience.

Share this post as thread on Twitter: https://twitter.com/alice_und_bob/status/1703833748121215296

37 Likes
2

I don’t want to appear overly negative by saying that, but I think that when it comes to specifically the client (as it’s my domain of predilection), Substrate will never offer an easy experience.

Substrate is over-complicated and full of technical debt, and very little effort is spent fixing that. I sometimes casually follow client-side pull requests, and in my impression most of them actually add more complexity or technical debt.
Even in the unlikely event where extremely radical refactoring measures were taken today, it would IMO take 2 years or more to go down to sane levels.

The only way out to me is to create alternatives to the Substrate client. That, however, would require precisely specifying how a “runtime host” should behave, which historically many in the Parity engineering have refused to consider as they believe that the runtime and client should be entangled.

8 Likes
4

It’s exactly the developer experience in the Polkadot ecosystem that makes me struggle.

The value proposition of Polakdot I was believing was Pallet development, to create more powerful dApps, that you can create with smart contracts. However, this dev experience ends in the need to find a home for your pallet, which puts you in front of the parachain. Is this still the value proposition for devs?

I will claim that making parachains easier, and cheaper to spin up and maintain will, even more, fragment and hurt the ecosystem into irrelevancy, and therefore not improve the developer experience.

What if the opposite is the case? And we should consolidate.

Let’s throw a thought into the discussion.

Could Consolidating parachains into verticals be a solution, trying to unify teams, and building efficient parachains in terms of operation, transactions, users, and products, by accumulating them into a powerful vertical parachain, that creates revenue and can sustain.

This would allow us to create a strong funnel for a vertical-specific developer experience and create solutions that could probably better compete in the broader market.

Utopic, maybe? For sure many challenges, but active consolidation would preserve our biggest good, which is the builders and creators of those projects. The market will consolidate sooner or later, and the way it does it burns the people.

More than happy to start elaborating and solving the challenges.

1 Like
5

This is exactly what needs to happen, but there also needs to be a back to basics approach to how we conceive “verticals”.

If we presume that KSM/DOT upside is limited, all of the (huge) upside is going to be in packaging up “community collectives” who can refine what is effectively a free resource (coretime) into a more valuable one.

This is basically the approach for Kabocha - it has always been a relatively blank sheet of paper for just this purpose.:.

When you say “verticals” what do you see/mean/think?

6

With verticals I mean DeFi, Identity, Gaming, Privacy, EVM … some might make more sense to consolidate than others.

8

When it comes to Substrate developer experience specifically, @kianenigma originally wrote up this proposal and I’ve commented with my 2c on how starting a node and tinkering with Substrate should look.

More broadly, it’s clear that tools and SDKs don’t need to just work well, but feel good to use. In retrospect this is one of the largest pieces of developer adoption. For this we need to be very thoughtful about APIs.

6 Likes
9

I remember extremely similar discussions back in 2018, and yet we’re still at the exact same place.

In general, I strongly disagree with vision 1, but the reasons don’t matter here.

I just wish it was possible to actually write alternatives to Substrate. You know, the whole competition moving things forward thing.
But it’s not possible, because the way things work is held hostage by Parity employees refusing to contribute to the specification and in general people not giving a shit about this specification (which is paradoxical for an ecosystem where anyone is supposed to be able to review how their software works), plus IMO questionable decisions like giving up on determinism when it comes to PVF verification.

6 Likes
10

I don’t think that it is only a technical issue, or debt, on the SDK level, at least not from my perspective.

I think it is a sizing problem, where we lose.

We offer only the elephant-size project (own parachain) and the rest is to be cared for by the parachains, where they usually only offer the mouse-size, which is smart contracts (on EVM parachain) or no developer experience at all.

If you want to leverage the value proposition of pallets, which is in my opinion the strongest thing in this ecosystem, then you are always bound to the elephant-size.

What if we could create a home for a mid-size project that uses one pallet and a UI Integration, with or even without its own token economy?

1 Like
11

what this leads into is a substrate template for a specific type of parachain - neither for-profit or system, where the core technology stays the same, but the assets accruing and the culture surrounding makes each instantiation unique that is far more tightly aligned to the core economic drivers of the internal economy - accruing value to KSM/DOT in the process.

this is the solution to the elephant and the mouse.

the game will be played between relay, coretime and collectives.

12

In general, I strongly disagree with vision 1, but the reasons don’t matter here.

I do too, which is why I proposed this alternative:

1 Like
13

It’s not really on the API itself that I disagree, but on the whole approach of trying to propose an API.

Whatever API you design, it’s going to be useless if it’s not stable. API stability is one of if not the main prerequisite to proposing a good dev experience.
The API of the smoldot JavaScript package, for example, has had three minor changes in the last two years.

Designing a large-scale API on paper, like this issue is trying to do, is very hard. It is very likely that you forget about some detail, and thus the chances that you get it right on the first try are extremely slim. This is even more true in an ecosystem like blockchains that continuously evolves. Most likely, an API like that will require many iterations and will need to evolve over time. This is not good.

Vision 2 is actually a good API IMO, but only if there’s one universal client (rather than multiple different binaries depending on the consensus, as suggested in the issue). It would be a good API because the idea of starting a node and it “just works” would be stable. Users would just have to plug the client and runtime together, simply making sure that they are compatible through some version system, and not worry about this “system” ever breaking.
A consequence in order for this to work is that they would also need clear indications of what they can modify or not in the runtime code in order to stay compatible.
If you have multiple binaries depending on the consensus algorithm, then you introduce potential clusterfucks such as the situation where you would like to change the consensus algorithm of an existing chain.

1 Like
14

Glad to see people acknowledge developer experience as a major pain point in the ecosystem, this needs to be our north star and be really prioritized, like really stop the world kind of prioritization. There’s too much narrative around blockspace for example, do we need more atm when it’s wasted and available for “free”? instead let’s all talk about ways we can make entry to the ecosystem as simple as possible at all levels, opportunities for improvement are everywhere, not only in the client, deployments, libraries, SDKs, XCM(A LOT of room for improvement here with DX), etc. and it’s not only up to Parity, also the Fellowship, Prachains or the community via OpenGov and financing the right projects.

3 Likes
15

There have been very good reasons for this and it is also not correct to frame this as a decision when in fact we listened to feedback and have not decided this way.

Honestly, you claim you don’t want to sound too negative, but there are multiple views on the same thing, you have one, others have a different one - all have tradeoffs. Which do matter more - depends a lot on where you are coming from. There is no right or wrong.

The above was rightly posted in a public forum, instead of getting secretly decided at Parity behind closed doors - precisely to get insights into different perspectives. We are already moving in the direction you envision us to be, we are not there yet, but not being perfect from one point of view, does by no means mean it is not good and more importantly does not mean things can’t improve. I can clearly see that they do.

16

There’s never a right or wrong and everything has a trade-off, yes. But to me the trade-off is overwhelmingly in favor of very precisely defining what PVF succeeds or not. If it was just me, this would even justify getting rid of wasmtime and switching to wasmi or to RISC-V.

Posting on the forum to gather feedback doesn’t automatically make the decision correct. As far as I can tell the discussion in the forum post just died out. The concern about for example Kagome becoming by definition a dead project if this goes forward simply got shrugged off.

The consequence of not having a precise definition of whether a PVF succeeds is that the Polkadot network is owned by the Parity engineers.
What happens for example if tomorrow a vulnerability is found in wasmtime, but updating the wasmtime version breaks a parachain? Is the answer that if the parachain is important they’ll delay the update or fork, and if the parachain isn’t important they’ll go ahead anyway? Isn’t that against the entire principle of blockchains?

17

@eskimor @tomaka separate thread pls, sers

2 Likes
18

thanks for this, a good recap of the recent discussion :slight_smile:
I’m in agreeance with all of the specific suggestions for improvement.

Re:
#1 - we do have dedicated people to work with Polkadot parachain teams (as you mentioned above) and infrastructure players, but we could expand the scope to also discuss with teams like Mangata or Vara that are building cool products that aren’t specifically on Polkadot.

#2 - couldn’t agree more. IMO it’s one of the biggest weaknesses of our eco vs the others. There is a project underway to help boost ecosystem players as speakers and thought leaders.

#3 - yes yes yes yes we need to do this more. WebZero and Kodadot I believe have been doing this at their events. A web3 Summit rebirth would be nice.

#4 - There has recently been a much stronger focus on marketing ecosystem teams, and we are still thinking up new and creative ways to do this. The website is sorely in need of an update to promote eco teams - there is a project underway to address this also.

#5 - This is harder for me to speak to, but there are efforts to re-do the docs and the devrel team has a fresh plan for Q4. There are some new initiatives being rolled out like the Polkadot Heros https://www.polkadot.network/blog/introducing-the-polkadot-developer-heroes-program

Overall, personally speaking I’d like to see stronger narratives and positioning for Polkadot in regards to marketing. We need a stronger narrative for the next evolution of Polkadot to resonate with developers and users - I’m hoping that the brand refresh will help drive some of these key messages.

There are many other changes I’d like to see, maybe for another time :innocent:

4 Likes
19

I disagree that this is off-topic.

This thread will be full of people saying “this or that should be improved”. Yes, everyone agrees that things should be improved. Saying that things should be improved is the easy part. Do you want the improvements to actually happen or not? Because saying it alone isn’t going to make it happen.

If you want Parity to actually make the improvements happen, well their track record is they have failed to provide a good developer experience since basically forever.
If you want the community to be able to improve things, then you need to put systems in place so that technical aspects of Polkadot no longer rely entirely on Parity’s shoulders, otherwise the community is simply handcuffed.

And these systems are: an accurate and up to date specification, and no delegating to a reference implementation to dictate which blocks are valid or not.

8 Likes
20

I agree with all the points shared by @alice_und_bob, and I’d like to add one more point that’s equally crucial.

The Polkadot ecosystem is undeniably complex, featuring numerous parachains offering services to the ecosystem. For instance, you have Zeitgeist offering prediction markets and Oak Network providing automation. If dApp developers on Moonbeam wish to build applications that leverage the services provided by these parachains, the prescribed path is through XCM.

Here’s the crux of my argument:

** Building on top of Polkadot parachains can be quite challenging. - This statement is a generalization and doesn’t always hold true. However, I want to zoom in on what is perhaps the flagship Polkadot technology: XCM.

If a dApp developer wants to utilize Oak’s automation, they need to understand the weight of the XCM message, the weight of the Transact , the weight-to-fee ratio, and also how to construct the XCM message itself. Terms like BuyExecution , WithdrawAsset , and HoldingRegister are terminologies that dApp developers shouldn’t necessarily have to grapple with. The usability of XCM is filled with so much complexity that it’s essentially broken. We’ve been creating tutorials around XCM for over 1.5 years, and they are, by far, the most challenging ones to create and follow.

Now, let’s compare this to building on top of other cross-chain technologies like Axelar/Wormhole, where their SDK is exceptionally user-friendly and encourages developers to build on the technology while abstracting away as much complexity as possible. With XCM, it often feels like we intentionally want things to be extremely intricate, viewing complexity as a feature rather than a problem.

As the Polkadot ecosystem matures, XCM is poised to be the primary driving force behind seamless cross-chain interactions. We often hear about BridgeHub and AssetHub being the key drivers for assets and bridging to other ecosystems. Furthermore, at Sub0, we gained some insight into how Snowfork was going to function (more or less). All three of these projects will rely on XCM to thrive. Therefore, it’s crucial that XCM’s usability becomes more accessible and welcoming; otherwise, it’s likely to face challenges.

A Suggested Solution

One straightforward example is to provide a way for the source chain to request the destination chain to calculate the necessary weight and fee for an XCM message. This could be achieved through a runtime API call, making the use of XCM significantly more user-friendly. Then, individual teams could develop SDKs that abstract away as much of the complexity as possible.

Wdyt?

7 Likes
21

Thx for sharing your thoughts on this! I think it takes someone with your insight and expertise to highlight those issues.

At what level would such an improvement have to be implemented? Is that Cumulus?

Is there a way to better wrap all that complexity away? Who would have the best lever to get it done?

2 Likes
22

I totally agree with what you said in your post.

It is now much harder than ever for small developer teams to build especially common good tools and even harder to get funding from the whale-dominated treasury.

Some may say, that denial of further funding from the treasury kills projects that are there only to drain the treasury.

I disagree with that strongly. There are teams, that passionately build for the ecosystem and they are still there despite the fact that they got their funding rejected. They in my opinion deserve the spotlight because what is a better example of dedication to the ecosystem than that?

3 Likes