Polkadot Native Storage

Hello everyone! I am Kyle from Eiger. I’d like to present to you an idea to build Filecoin like native storage on Polkadot and Kusama.

This is mainly inspired by the vision Gavin presented at Decoded 23 about building the Ubiquitous Supercomputer. When thinking about the app-centricity design and the Space and Cores model it will be critical for us to have native storage and not depend on other networks.

We would like to propose that we build a system parachain to serve this functionality where anyone in the ecosystem through XCM would be able to store and retrieve data. The fees would be paid in native DOT. This basically extends the Polkadot value offering to include storage, similar to Filecoin, and brings us closer to the supercomputer vision.

We believe its very important that this work is done as a public good with DOT used as the fee token, otherwise any dependence on other networks or other tokens would undermine the goals.

We have been working on this since Decoded and already got a grant from the Web3 Foundation to research how to do this which we just completed.

https://github.com/dtuzi/Grants-Program/blob/4d5e7ff1cc90ab0416604dc705bd02c29ba136e6/applications/Eiger_Storage_on_Polkadot_1.md

Here is a 48 page :sweat_smile: document of our completed research and implementation plan.

https://github.com/eigerco/polkadot-native-storage/blob/72efd1e18e4d0ce2c3712571c7669301831f2328/doc/report/polkadot-native-storage-v1.0.0.pdf

It’s not going to be easy but having native storage is going to be awesome. We would greatly appreciate your feedback regarding this idea as its critical we have alignment and build something useful for the community.

Oh yeah one more thing: What should it be called? We are thinking about PolkaDisk but if you have any cool suggestions please write them below.

13 Likes
3 Likes

Super excited about this initiative - been waiting on something like this for a long time, ever since the Polkadot on Filecoin explorations years ago.

It’s crucial we go on this development as we are seeing something similar happening on ethereum too via ETH Storage, and simultaneously Filecoin itself is gaining ground by adding its own VM to their solution.

Sounds very interesting. It would also be very interesting to see how this would compare with e.g. Crust.

As for a name suggestion: DotFile(s)/PolkaSpace. :upside_down_face:

2 Likes

Here are the main differences compared to Crust:

  • Trusted Execution Environments are not used
  • We want the fee token to be native DOT throughout (no CRU)
  • Crust uses IPFS for actual storage, the parachain bridges to and offloads the storage to IPFS; on the other hand, this would be native storage provided by Polkadot storage providers.
  • While we plan to employ IPFS technology (e.g., CIDs, IPLD, etc), storage providers are completely orthogonal to IPFS per se.

Regarding the names

DotFile(s)/PolkaSpace

very nice :+1:

2 Likes

Related: Storage Hub

Yes, your team is also working on a storage solution. From the posted design doc, I see you have spent a good deal of time thinking about the problem. We wish you the best of luck, and are very interested to see what you come up with!

Hi! Thank you for your kind words! We’ve reviewed your proposal and found it very interesting and thorough. Your support means a lot to us, and we look forward to sharing our progress with the community. Best of luck with your own endeavors!

1 Like

Hello everyone, we have been busy working on this, so here’s a quick update from us.
We are continuing to research and plan towards a complete implementation of PolkaDisk. This includes:

  • fleshing out of the original milestones sketched in our grant submission. This work will feed into future proposals …with clarity, completeness, and correctness.
  • we have identified more Rust resources, which will accelerate the implementation of the collator node and storage subsystem.
  • we are progressively defining the API for both the collator and storage.
  • we are working on an aggressive implementation plan to get this into your hands as soon as possible. :grinning:
2 Likes

Hi everyone!
Since our last update we have been working on and improved our proposal, check out the latest draft on Polkassembly:

Please review it and leave your thoughts, your feedback would be very valuable.

Wow, this sounds like a great proposal. I like it and especially that it could leverage existing nodes.

A slight concern would be to start a habit of adding to many features into the “polkadot core”.
There are plenty of databases and datastructures and decentralized storage solutions.

Starting with bittorrent and other early solutions all the way to the latest generation like IPFS and similar ones, it seems hard to tell which storage solutions might stick around long term, so having this implemented as a parachain for others to use feels a lot more modular and future proof than adding ot to the core imho.

2 Likes

Thank you for sharing your thoughts! We certainly aim to make this solution future proof – i.e., resilient. :grin:

I definitely think the idea of leveraging existing nodes, so you don’t have to wait for users to e.g. adopt and run your parachain makes it work from day1, which is beneficial.

On the other hand, if everyone did it this way, we wouldn’t even need parachains and probably end up with a single chain with every feature under the sun integrated into existing nodes.

Putting additional functionality into the core takes consensus away from everyone to opt-in or opt-out of a feature they might or might not need in their context.

To me, the core polkadot value proposition of parachains or let’s say apps, is that over time things can evolve more freely and you use and run nodes for features you need. You could almost claim that this helps with scalability and a lot of additional concerns and is one or maybe the most important reason why Gavin Wood set out to create Polkadot after all the learnings in Ethereum.

I am curious to hear how you think about this or what are your thoughts or reasoning context that guides the proposal you made?

In our proposal, our main goal is to align with Gavin’s vision of building a “supercomputer”, where each parachain tackles a specific problem. Currently, there’s no parachain in Polkadot focused on native storage using the native currency (DOT). Introducing a native storage parachain gets us closer to realizing this vision, emphasizing the importance of each parachain having its own boundaries and responsibilities.

4 Likes