Polkadot Native Storage Updates

I saw that you have a ticket for S3 compatibility, that would be nice: S3-compatible API · Issue #692 · eigerco/polka-storage · GitHub
I am using filebase.com currently to archive things, since it integrates with the AWS cli through S3 compat. But I would rather use something Polkadot powered :grin:

Another thing that I think is very useful is mounting it into the file system. I tried Sia coin (storage as well) a few times over the years and was always disappointed at how badly it integrates with Linux or MacOs. They do not offer a simple CLI where you can do sia mount --config-whatever.... --mount-path /mnt/archive. It could be very nice to have this, since it makes it usable for non-tech users as well.
As example, there is the ProtonDrive MacOs UI app which makes it a no-brainer to mount a remote drive.

1 Like

2025-02-11 Update

We’re working on placing multiple pieces in a sector, support was there but there’s more to it than just “adding them”:

We fixed our uploads:

Added support for 512MiB & 1GiB sectors:

And work continues on P2P:

Hey Oliver!

We’re glad you took a look! Yes, S3-compat is up for research & implementation but not the priority right now.

About mounting, that might be trickier due to the way sectors are kept around, research will be needed to decide whether it would be feasible :thinking:

1 Like

2025-02-17 Update

We’re working on finishing up the storage provider crate benchmarks:

We’ve done some work on the P2P side:

We’re working on testing proof GPU support:

The storage provider now takes the deal’s start block into account:

Improved our CAR crate:

Made proofs more efficient:

Finally, we addressed one of the existing test network shortcomings:

2 Likes

2025-02-26 Update

We’ve fixed a bug in our proof pipeline:

As we focus on fetching, we’ve been improving the mater crate:

We’re started work on replication:

And we’re also working on deal parameters:

Behind the scenes, we’re currently running some experiments to gather data on the cost of running a Storage Provider.

2025-03-06 Update

Some small improvements on storagext:

We finished our improvements to polka-fetch:

We’re pushing for 1GiB sector sizes:

The work on deal parameters keeps going:

We’ve implemented DealId-based content retrieval:

We’re working on improving Delia with the aim of adding automatically replicated deals:

Similarly, we have written an example on how a CLI-based deal replication flow could work:

2025-03-12 Update

The new Delia is under review:

We’re finishing the server side:

We’ve done a small QoL improvement:

Some more work on deal parameters:

And implemented Deal ID based retrieval:

2025-03-28 Update

On Delia, we did several improvements:

output

On our main project:

2025-16-04 Update

On Delia, we did several improvements:

On our polka-storage:

1 Like

2025-04-23 Update

We’ve been hard at work, simplifying our end user experience!

You can take a glimpse at the upload experience here:
output

We’ve also drastically reduced the complexity in fetching files, simply:

  • Keep the receipt you got at the end of your deal
  • Upload it to Delia and you’ll have your file back in no time!

output1

On our parachain’s end, we’ve been mostly finishing up benchmarks:

1 Like

2025-04-30 Update

We’ve been focused on Delia:

1 Like