# Polkadot Native Storage Updates **URL:** \ **Category:** Tech Talk\ **Created:** [March 26, 2024, 1:40am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021 "2024-03-26T01:40:36Z")\ **Posts on this page:** 20\ **Page:** 3
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [November 22, 2024, 10:31am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/41 "2024-11-22T10:31:37Z")
@somedude, lets use the discussions here [GitHub - eigerco/polka-storage: The Polka Storage Parachain Project](https://github.com/eigerco/polka-storage) ---
**Author:** ![somedude](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/somedude/32/7263_2.png) [@somedude](https://forum.polkadot.network/u/somedude)\ **Post date:** [November 22, 2024, 11:31am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/42 "2024-11-22T11:31:33Z")
Great. Just fix that 404 [1] in README.md and you’ll be golden! The Web site link in repo metadata is also 404 (URL of the same Web site which seems to be the place for docs). I don’t know what’s supposed to be on the site, but I hope you’ll have a getting started walk-through, so that those who aren’t experts can at least put and get some data without too much pain 🙂 [1] ```auto Check out the book at https://eigerco.github.io/polka-storage/getting-started/index.html. ``` ---
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [November 22, 2024, 11:58am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/43 "2024-11-22T11:58:25Z")
Our apologies! While we fix it, you can check the book here: > **[Introduction - The Polka Storage Book](https://eigerco.github.io/polka-storage-book/)** > > The Polka Storage project aims to bring Filecoin-like storage to Polkadot! ---
**Author:** ![somedude](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/somedude/32/7263_2.png) [@somedude](https://forum.polkadot.network/u/somedude)\ **Post date:** [November 24, 2024, 5:48am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/44 "2024-11-24T05:48:08Z")
For anyone looking to try, this is _almost_ ready, but if you’re not a Rust developer-level troubleshooter it’s probably better to wait another 1-2 weeks. Some sharing for those looking to try right now (Nov 24, 2024): - Errors exposed to the user are not informative (WIP already, but not done yet) - see the example below - The docs are still being improved and, in the current iteration, may be confusing (in short, installation/setup workflow is confusing, but also the docs are out of sync with the repo, which made me waste 1-2 hours until I discovered that). This will be improved as well - Binaries/containers are out of sync with the docs as well (e.g. the faucet command) - The repo defaults to `develop` but the docs refer to releases and some PRs with fixes don’t have useful recent commits, while latest release (and therefore, containers) has a 3 week old release). Not unexpected, but it may be hours until you figure all this out. The good news is it’s looking promising and while I spent hours trying to get past the error below (and couldn’t) that is the **last** step in the workflow (the last should be “read” (`GET`) from storage, but that step is not yet in the docs). Looks like an interesting and useful project. I look forward to next release when the docs and containers get in sync and this bug hopefully fixed. Wish list: - More info about max size of blobs that can be `PUT` - `GET` steps - (Maybe this is a stupid question): I wonder if there will be a way to differentiate public vs. private access - (May be already documented, didn’t see it): recommended approach to encrypt objects, in the case public access be the default (I can’t tell if it will or won’t, since `GET` step isn’t in the docs right now) - (May be common knowledge for chain storage): it’s not very clear - although it didn’t impact my evaluation - if it’s OK to submit and sign a proposal before`start_block` as long as it’s not published before it. I wouldn’t expect that `PUT` can be performed before `start_block`, but maybe other steps can although, even in that case, the question is how much before `start_block` (i.e. N-14400, N-256, etc.) ### Example 1 - This confusing error in the last step (`publish-deal`) doesn’t tell you anything. Maybe your balance isn’t enough, maybe your JSON is populated with wrong data, maybe it’s something else… Go figure.. This is being fixed, but not fixed yet as far as I can tell. The docs don’t say how much balance is required (there’s a per-block cost formula, but I can’t be sure if there are other costs and in any case the error seems WASM-related and not balance related ```raw Error: RpcClient(Call(ErrorObject { code: InternalError, message: "proposal has not been accepted", data: None })) ``` - Server-side log doesn’t tell me much either, and currently there are no open issues related to this. ```raw 2024-11-24 01:56:06.020 INFO tokio-runtime-worker runtime::market: [Parachain] on_finalize: no deals to process in block: 168 2024-11-24 01:56:06.020 DEBUG tokio-runtime-worker runtime::system: [Parachain] [168] 2 extrinsics, length: 3670 (normal 0%, op: 0%, mandatory 0%) / normal weight:Weight(ref_time: 0, proof_size: 0) (0%) op weight Weight(ref_time: 0, proof_size: 0) (0%) / mandatory weight Weight(ref_time: 10532249000, proof_size: 77889) (0%) 2024-11-24 01:56:06.021 INFO tokio-runtime-worker sc_basic_authorship::basic_authorship: [Parachain] 🎁 Prepared block for proposing at 168 (2 ms) [hash: 0x12621b07d07bb160888e9e729154479000e946168062ee2eff792157a9d7bcee; parent_hash: 0xcc6d…bda4; extrinsics (2): [0x5a3a…8d1c, 0x662d…a456] 2024-11-24 01:56:06.022 INFO tokio-runtime-worker aura::cumulus: [Parachain] 🔖 Pre-sealed block for proposal at 168. Hash now 0xe6510c8b77d52dea373b714603e2ef6947a13e7d9ba5a3db47906b8d406a740f, previously 0x12621b07d07bb160888e9e729154479000e946168062ee2eff792157a9d7bcee. 2024-11-24 01:56:06.022 INFO tokio-runtime-worker substrate: [Parachain] 🏆 Imported #168 (0xcc6d…bda4 → 0xe651…740f) 2024-11-24 01:56:06.023 INFO tokio-runtime-worker aura::cumulus: [Parachain] PoV size { header: 0.21875kb, extrinsics: 3.5849609375kb, storage_proof: 8.9638671875kb } 2024-11-24 01:56:06.023 INFO tokio-runtime-worker aura::cumulus: [Parachain] Compressed PoV size: 9.857421875kb 2024-11-24 01:56:06.023 DEBUG tokio-runtime-worker parachain::collation-generation: [Relaychain] candidate is generated candidate_hash=0x3f66be629f2094c229722a38888c5239cbf1fda880835da4557524e8039fe65d pov_hash=0x81c5baeced98def98965cc0d1fc164c0de82bb2a3abcce14023cff9ab72416e9 relay_parent=0xc83f1e6c2f8e391c95c4e3ad97dea52841c28649fe1d6c75673470423d413ede para_id=1000 traceID=84274839485452377306051434561466552889 2024-11-24 01:56:06.024 DEBUG tokio-runtime-worker parachain::collator-protocol: [Relaychain] Received session info session_index=19 2024-11-24 01:56:06.024 DEBUG tokio-runtime-worker parachain::collator-protocol: [Relaychain] Accepted collation, connecting to validators. para_id=1000 candidate_relay_parent=0xc83f…3ede relay_parent_mode=Enabled { max_candidate_depth: 3, allowed_ancestry_len: 2 } candidate_hash=0x3f66be629f2094c229722a38888c5239cbf1fda880835da4557524e8039fe65d pov_hash=0x81c5baeced98def98965cc0d1fc164c0de82bb2a3abcce14023cff9ab72416e9 core=CoreIndex(0) current_validators=[Public(d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d (5GrwvaEF...))] traceID=84274839485452377306051434561466552889 2024-11-24 01:56:06.024 DEBUG tokio-runtime-worker parachain::collator-protocol: [Relaychain] Advertising collation. relay_parent=0xc83f1e6c2f8e391c95c4e3ad97dea52841c28649fe1d6c75673470423d413ede candidate_hash=0x3f66be629f2094c229722a38888c5239cbf1fda880835da4557524e8039fe65d peer_id=12D3KooWQCkBm1BYtkHpocxCwMgR8yjitEeHGx8spzcDLGt2gkBm traceID=84274839485452377306051434561466552889 2024-11-24 01:56:06.024 DEBUG tokio-runtime-worker parachain::validator-discovery: [Relaychain] New ConnectToValidators request peer_set=Collation requested=1 failed_to_resolve=0 2024-11-24 01:56:06.024 DEBUG tokio-runtime-worker parachain::validator-discovery: [Relaychain] New ConnectToValidators resolved request peer_set=Collation num_peers=1 removed=0 2024-11-24 01:56:06.025 INFO tokio-runtime-worker sc_basic_authorship::basic_authorship: [Parachain] 🙌 Starting consensus session on top of parent 0xe6510c8b77d52dea373b714603e2ef6947a13e7d9ba5a3db47906b8d406a740f (#168) 2024-11-24 01:56:06.025 ERROR tokio-runtime-worker runtime: [Parachain] panicked at /home/user/.cargo/git/checkouts/polkadot-sdk-cff69157b985ed76/dba2dd5/substrate/frame/aura/src/lib.rs:135:21: Slot must increase 2024-11-24 01:56:06.025 ERROR tokio-runtime-worker aura::cumulus: [Parachain] err=Error { inner: Proposing Caused by: 0: Error at calling runtime api: Execution failed: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed WASM backtrace: error while executing at wasm backtrace: 0: 0x43a5e6 - polka_storage_runtime.wasm!rust_begin_unwind 1: 0x3fa02 - polka_storage_runtime.wasm!core::panicking::panic_fmt::h58961fa2eb0f3a16 2: 0x3874b8 - polka_storage_runtime.wasm! as frame_support::traits::hooks::OnInitialize<<<::Block as sp_runtime::traits::HeaderProvider>::HeaderT as sp_runtime::traits::Header>::Number>>::on_initialize::h79eda421d9430a77 3: 0x12b384 - polka_storage_runtime.wasm!<(TupleElement0,TupleElement1,TupleElement2,TupleElement3,TupleElement4,TupleElement5,TupleElement6,TupleElement7,TupleElement8,TupleElement9,TupleElement10,TupleElement11,TupleElement12,TupleElement13,TupleElement14,TupleElement15,TupleElement16,TupleElement17,TupleElement18,TupleElement19,TupleElement20) as frame_support::traits::hooks::OnInitialize>::on_initialize::h6d131c8a66e07cd7 4: 0x2edafa - polka_storage_runtime.wasm!frame_executive::Executive::initialize_block::h7cd8d6f9864c02a6 5: 0x22360e - polka_storage_runtime.wasm!Core_initialize_block 1: Execution failed: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed WASM backtrace: error while executing at wasm backtrace: 0: 0x43a5e6 - polka_storage_runtime.wasm!rust_begin_unwind 1: 0x3fa02 - polka_storage_runtime.wasm!core::panicking::panic_fmt::h58961fa2eb0f3a16 2: 0x3874b8 - polka_storage_runtime.wasm! as frame_support::traits::hooks::OnInitialize<<<::Block as sp_runtime::traits::HeaderProvider>::HeaderT as sp_runtime::traits::Header>::Number>>::on_initialize::h79eda421d9430a77 3: 0x12b384 - polka_storage_runtime.wasm!<(TupleElement0,TupleElement1,TupleElement2,TupleElement3,TupleElement4,TupleElement5,TupleElement6,TupleElement7,TupleElement8,TupleElement9,TupleElement10,TupleElement11,TupleElement12,TupleElement13,TupleElement14,TupleElement15,TupleElement16,TupleElement17,TupleElement18,TupleElement19,TupleElement20) as frame_support::traits::hooks::OnInitialize>::on_initialize::h6d131c8a66e07cd7 4: 0x2edafa - polka_storage_runtime.wasm!frame_executive::Executive::initialize_block::h7cd8d6f9864c02a6 5: 0x22360e - polka_storage_runtime.wasm!Core_initialize_block Stack backtrace: 0: anyhow::error::::new 1: as cumulus_client_consensus_proposer::ProposerInterface>::propose::{{closure}} 2: cumulus_client_consensus_aura::collators::lookahead::run_with_export::{{closure}} 3: as core::future::future::Future>::poll 4: as core::future::future::Future>::poll 5: as core::future::future::Future>::poll 6: as core::future::future::Future>::poll 7: tokio::runtime::task::core::Core::poll 8: tokio::runtime::task::harness::Harness::poll 9: tokio::runtime::scheduler::multi_thread::worker::Context::run_task 10: tokio::runtime::scheduler::multi_thread::worker::Context::run 11: tokio::runtime::context::runtime::enter_runtime 12: tokio::runtime::scheduler::multi_thread::worker::run 13: as core::future::future::Future>::poll 14: tokio::runtime::task::core::Core::poll 15: tokio::runtime::task::harness::Harness::poll 16: tokio::runtime::blocking::pool::Inner::run 17: std::sys::backtrace::__rust_begin_short_backtrace 18: core::ops::function::FnOnce::call_once{{vtable.shim}} 19: std::sys::pal::unix::thread::Thread::new::thread_start 20: start_thread at ./nptl/pthread_create.c:442:8 21: __GI___ clone3 at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 } 2024-11-24 01:56:06.749 INFO tokio-runtime-worker substrate: [Relaychain] 💤 Idle (2 peers), best: #196 (0xc83f…3ede), finalized #193 (0xe4cd…2da7), ⬇ 1.1kiB/s ⬆ 2.6kiB/s 2024-11-24 01:56:06.804 INFO tokio-runtime-worker substrate: [Parachain] 💤 Idle (0 peers), best: #168 (0xe651…740f), finalized #163 (0x755b…760e), ⬇ 72 B/s ⬆ 40 B/s 2024-11-24 01:56:10.015 DEBUG tokio-runtime-worker parachain::validator-discovery: [Relaychain] New ConnectToValidators request peer_set=Collation requested=1 failed_to_resolve=0 2024-11-24 01:56:10.015 DEBUG tokio-runtime-worker parachain::validator-discovery: [Relaychain] New ConnectToValidators resolved request peer_set=Collation num_peers=1 removed=0 ``` ---
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [November 25, 2024, 7:53pm UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/45 "2024-11-25T19:53:33Z")
Hey @somedude! Thank you for your through feedback! We’ve rebuilt the binaries and reviewed the docs to ensure they’re in-sync (like adding the faucet). We also added more information to the book which should answer your questions — in short - we don’t support `GET` yet — we’re working on adding retrieval to the system; - the maximum `PUT` size is 2048 bytes — we’re also working on it, - all accesses are public and as such we recommend you encrypt your files (this can be done using [7zip](https://www.7-zip.org/7z.html), [age](https://github.com/FiloSottile/age), or [VeraCrypt](https://www.veracrypt.fr/code/VeraCrypt/), we don’t endorse any in specific), - you can and should submit your deals early so that the storage provider has time to process them (implementation details here will vary as work progresses) Finally, we’d like to ask you to elaborate what was not clear in the install/setup workflow so that we can address that as soon as possible! In the meantime, we invite you to take a look again and once more, thank you for your feedback! ---
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [November 26, 2024, 3:35pm UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/46 "2024-11-26T15:35:24Z")
**2024-11-26 Update** We’re currently working on: - Finishing our spring cleaning — i.e. reducing tech debt - PoSt proofs - Gregor — the deal aggregator We’re starting work on: - Indexer research - Fetching research ---
**Author:** ![somedude](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/somedude/32/7263_2.png) [@somedude](https://forum.polkadot.network/u/somedude)\ **Post date:** [November 27, 2024, 5:51am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/47 "2024-11-27T05:51:31Z")
Thanks for taking care of these things and the info is very helpful. Regarding the maximum size I wondered if something like multipart upload was possible using individual 2KiB chunks, but it seems that needs to be done on the client (and that’s fine) for those who need it. Then maybe use the first chunk to store a map of other chunks, and the rest to store object content. Or maybe put a pointer to the next part at the end of each part that has the next part… I happened to need just 1KiB, so I’m good for now. Those who need more would probably prefer that one common and tested method of chunking be implemented by the project so that they don’t have to build their own. > [@jmg-duarte](#): > > Finally, we’d like to ask you to elaborate what was not clear in the install/setup workflow so that we can address that as soon as possible! The workflow was a bit confusing in terms of the sequence of steps required to get everything deployed. IMO there should be 1 section for deploy/install with 2 subsections (docker, build from source or download + run from CLI), 1 section to getting faucet funds and making sure all the required accounts are there and working, and 1 section to getting things done (regardless of how you run, from Docker or CLI). I suppose it’s not necessary for people used to working with parachains, but as the first time user I struggled a bit even though I often use a Substrate chain (w/o parachains). I assume the bug I was encountering was fixed as well? As that was the last step and everything else appears to be working, that’s the only fix I strictly need. > [@jmg-duarte](#): > > you can and should submit your deals early so that the storage provider has time to process them (implementation details here will vary as work progresses) Do you mean this “in general” or related to the bug I encountered? Regarding the bug I encountered, I would assume that `publish-deal` should be able to start _after_ the start block (as I’ve committed to paying anyway) and that your comment was generic - in which case I wonder if the docs could define “early” (that would be useful so that we ca add current\_height+X where X is the recommended “advance notification”. Thanks for making these improvements! ---
**Author:** ![th7nder](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/th7nder/32/4924_2.png) [@th7nder](https://forum.polkadot.network/u/th7nder)\ **Post date:** [November 27, 2024, 9:19am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/48 "2024-11-27T09:19:15Z")
> [@somedude](#): > > Regarding the maximum size I wondered if something like multipart upload was possible using individual 2KiB chunks, but it seems that needs to be done on the client (and that’s fine) for those who need it. Then maybe use the first chunk to store a map of other chunks, and the rest to store object content. Or maybe put a pointer to the next part at the end of each part that has the next part… > > I happened to need just 1KiB, so I’m good for now. Those who need more would probably prefer that one common and tested method of chunking be implemented by the project so that they don’t have to build their own. That’s a great point and chunking is the solution if the allowed size is X and you need to store the file \> X. We didn’t plan providing a chunking solution, as the small sector size is experimental - for small file sizes, proving time is fast, so testing time is fast. We are going to introduce bigger file size limits, probably the ‘unit of storage’ will be 32GiB/64 GiB on-chain. Files in our chain are stored in sectors (unit of storage), so a Storage Provider gathers deals from the clients and packs them into a sector. Currently the Sector Size is 2KiB, hence the limitation. When we move to bigger sector sizes, there will be another problem to handle - not chunking but batching. Storage Providers will prefer to store bigger data deals, so if you would like to store small pieces of data (1KiB in 32GiB) you may not find a Storage Provider that is willing to accept a deal. That’s why we’re building a deal aggregation component - ‘Gregor’, it’ll gather small pieces of data from the clients and propose it in a bigger batch. I hope I understood your concerns correctly and explained our reasoning, if not, looking for follow-up questions and we’re here! ---
**Author:** ![somedude](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/somedude/32/7263_2.png) [@somedude](https://forum.polkadot.network/u/somedude)\ **Post date:** [November 27, 2024, 9:38am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/49 "2024-11-27T09:38:06Z")
Thanks for the update, yes, that’s the right way. I like Gregor - “Gregor The Aggregator” 😃 There are “traditional” (lol) Web3 players that heavily favor big blobs, so I think there’s space for these tiny ones (if you need just a few) and medium-sized ones (say up to 32MB, enough for many apps). Above that is good and fine, but also overlaps with those “bulk” storage options and may be much pricier per MB than bulk storage. Still, it may be desirable for folks who need a few GB and prefer to use 1 provider/back-end for many small & medium plus some large, than code and test two integrations and pay to multiple providers (complexity, cost, inconvenience). ---
**Author:** ![aidan46](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/aidan46/32/7117_2.png) [@aidan46](https://forum.polkadot.network/u/aidan46)\ **Post date:** [November 27, 2024, 11:10am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/50 "2024-11-27T11:10:53Z")
Currently, `publish_deals` should always be done before the `start_block`, any deal published after the `start_block` will be rejected by the chain. There is no recommended advance notice, as long as the deals are submitted before the `start_block` they will be accepted. However, if it’s done too late, then there won’t be enough time to do `pre-commit` and prove the sectors, we’ll figure out appropriate values and provide them as a recommendation (we haven’t estimated them yet) - thank you for noticing that! Regarding the documentation, we will review them and improve on the clarity and flow of the documentation. Thank you for your extensive feedback and helping us improve 🙏. ---
**Author:** ![somedude](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/somedude/32/7263_2.png) [@somedude](https://forum.polkadot.network/u/somedude)\ **Post date:** [November 27, 2024, 11:56am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/51 "2024-11-27T11:56:46Z")
> [@aidan46](#): > > Currently, `publish_deals` should always be done before the `start_block`, any deal published after the `start_block` will be rejected by the chain. Whoa - that’s interesting! Didn’t see that coming. I’ve tried both before and after, IIRC, but I’ll avoid “after” from now on. It’d be good to have these points highlighted in the docs. Hit a small landmine with binaries this time (Ubuntu 22.04 LTS has an older `glibc`) so I’m building binaries instead. [Sys reqs](https://eigerco.github.io/polka-storage-book/getting-started/index.html#system-requirements) are technically correct (e.g. if you run from the source or use Docker, but not if you use the prebuilt binaries) which is an example of why I suggested to structure the docs in distinct stages. Not a big problem now that I understand this area better - just FYI. Another concrete example is that [build from source](https://eigerco.github.io/polka-storage-book/getting-started/building/index.html) doesn’t mention (so this is an exercise for the reader) [all](https://eigerco.github.io/polka-storage-book/getting-started/local-testnet/index.html) binaries required to run - some (external) binaries need to be downloaded even if you build from the source. Makes sense once you compare the pages, but not until then 🙂 Thanks for the info! ```sh │ Command │ bash -c polka-storage-node build-spec --disable-default-bootnode > /tmp/zombie-dc56376f7f0702cc4… │ └─────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘ Error: Error: Command failed with exit code 1: bash -c polka-storage-node build-spec --disable-default-bootnode > /tmp/zombie-dc56376f7f0702cc4e627013b6c9d277_-3343252-7vTh1SAP4wOs/cfg/undefined-plain.json polka-storage-node: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.38' not found (required by polka-storage-node) polka-storage-node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.39' not found (required by polka-storage-node) polka-storage-node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by polka-storage-node) ``` ---
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [December 3, 2024, 7:18pm UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/52 "2024-12-03T19:18:34Z")
**2024-12-03 Update** We’ve worked on improving the docs. We’re working on: - Researching randomness methods - Researching the indexer node - Researching fetching methods - Implementing PoSt in the pipeline - Researching for the Gregor POC ---
**Author:** ![driemworks](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/driemworks/32/9746_2.png) [@driemworks](https://forum.polkadot.network/u/driemworks)\ **Post date:** [December 3, 2024, 7:33pm UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/53 "2024-12-03T19:33:05Z")
Regarding research on randomness methods, this might be of interest: [Ideal Labs | Updates & Next Steps - #3 by driemworks](https://forum.polkadot.network/t/ideal-labs-updates-next-steps/11046/3) We’re working towards introducing publicly verifiable randomness-as-a-service to Polkadot. ---
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [December 10, 2024, 3:10pm UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/54 "2024-12-10T15:10:57Z")
2024-12-10 Update 👨‍🔬 We’re currently researching solutions for: - 🎲 Randomness — [docs: randomness research rfc by jmg-duarte · Pull Request #626 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/626) - 📦 Retrieval/Fetching — [docs: retrieval research rfc by cernicc · Pull Request #623 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/623) - 🗂 Deal Indexing - [https://github.com/eigerco/polka-storage/pull/620](https://github.com/eigerco/polka-storage/pull/620) - [docs: deal scraper RFC by aidan46 · Pull Request #631 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/631) - [https://github.com/eigerco/polka-storage/pull/629](https://github.com/eigerco/polka-storage/pull/629) We’re pushing forward our proving implementation, we’ve merged [ongoing PoSt submissions](https://github.com/eigerco/polka-storage/pull/617) and we’re working on fixing related bugs and supporting proving multiple partitions. We’re still working on the Gregor POC. Finally, we’ve open sourced our Delia POC 🚀 — [GitHub - eigerco/delia: Web application for proposing and publishing deals with providers on the Polka Storage System Parachain.](https://github.com/eigerco/delia) ---
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [December 17, 2024, 9:34pm UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/55 "2024-12-17T21:34:35Z")
**2024-12-17 Update** 🎄 We have merged: - 🎲 [AuthorVRF randomness](https://github.com/eigerco/polka-storage/pull/633) - 📦 [Retrieval RFC](https://github.com/eigerco/polka-storage/pull/623) - 🔍 [PeerId → Multiadress Resolver RFC](https://github.com/eigerco/polka-storage/pull/629) Ongoing: - 🚧 Initial work on PeerId resolving — [feat: Add peer-resolver lib by aidan46 · Pull Request #640 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/640) - 🧑‍🔬 Initial work on multiple sector support — [fix(storage-provider): multiple sectors, multiple cache dirs by th7nder · Pull Request #642 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/642) - 📁 Gregor POC ---
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [December 24, 2024, 4:08pm UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/56 "2024-12-24T16:08:08Z")
**2024-12-24 Update** We’ve solved an handful of issues: - [fix(storagext): allow extrinsics to be submitted in parallel by th7nder · Pull Request #643 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/643) - [fix: configure async backing properly by th7nder · Pull Request #645 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/645) - [fix: upgrade to polkadot2412 and use fork-aware txpool by th7nder · Pull Request #648 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/648) - [fix: use subxt wait-for-success by th7nder · Pull Request #649 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/649) - [fix(storagext): losing transactions on reorg by th7nder · Pull Request #651 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/651) We’ve merged the randomness weights: - [feat(pallet-randomness): add benchmarks by jmg-duarte · Pull Request #656 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/656) And progress continues on retrieval and Gregor. 🎄 Happy Holidays! ---
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [January 8, 2025, 6:17pm UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/57 "2025-01-08T18:17:12Z")
**2025-01-08 Update** Progress continues on benchmarks: - [feat(pallet-proofs): add benchmarks by jmg-duarte · Pull Request #657 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/657) And on the proving pipeline: - [feat: verify multiple partitions in pallets by th7nder · Pull Request #658 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/658) - [feat: generate multiple proofs by th7nder · Pull Request #661 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/661) - [feat: throttle parallel prove commits by th7nder · Pull Request #662 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/662) As well as on extending our CAR library: - [feat: implement file blockstore by cernicc · Pull Request #660 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/660) And some work into the P2P aspects of the Storage Provider: - [feat: Add P2P functionality by aidan46 · Pull Request #666 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/666) ---
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [January 14, 2025, 4:34pm UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/58 "2025-01-14T16:34:52Z")
**2025-01-14 Update** We’ve added configuration file support: - [feat(polka-storage-provider-server): implement config file by jmg-duarte · Pull Request #674 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/674) We’re still working on retrieval and P2P ---
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [January 29, 2025, 10:19am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/59 "2025-01-29T10:19:09Z")
**2025-01-29 Update** On the P2P front, we’ve been working to implement peer discovery: - [feat(storage-provider-server): add p2p bootstrap and registration by aidan46 · Pull Request #666 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/666) - [docs: Update docs to include p2p Peer ID and keys by aidan46 · Pull Request #686 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/686) - [feat: Add re-registration on expiry for p2p node by aidan46 · Pull Request #685 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/685) Retrieval is more than just moving the bytes over the network, we’ve been working on improving mater to support retrieval: - [feat: setup retrieval server and client by cernicc · Pull Request #681 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/681) - [feat: mater car reader read\_block\_metadata by cernicc · Pull Request #706 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/706) On the benchmarks front, we found opportunities to improve the codebase and even a bug! - [feat: extract DummyProofsVerification to primitives by jmg-duarte · Pull Request #694 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/694) - [fix(primitives): reduce cid feature set by jmg-duarte · Pull Request #700 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/700) - [fix(primitives): benchmark compilation issues by jmg-duarte · Pull Request #701 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/701) - [ci: run benchmarks by jmg-duarte · Pull Request #702 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/702) - [feat(pallet-storage-provider): extract ProveCommitSector by jmg-duarte · Pull Request #704 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/704) - [feat(pallet-market): implement benchmarks by jmg-duarte · Pull Request #705 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/705) Finally, we’ve been working on getting our E2E tests working with proofs: - [refactor: PoSt e2e test by th7nder · Pull Request #699 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/699) - [fix(storagext): update .scale & api accordingly by jmg-duarte · Pull Request #707 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/707) ---
**Author:** ![jmg-duarte](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.polkadot.network/jmg-duarte/32/7112_2.png) [@jmg-duarte](https://forum.polkadot.network/u/jmg-duarte)\ **Post date:** [February 5, 2025, 11:36am UTC](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021/60 "2025-02-05T11:36:32Z")
**2025-02-05 Update** We’ve had some issues with Nix: - [fix: update polkadot to 2412 and subxt-cli to 0.38. by th7nder · Pull Request #711 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/711) - [feat: resolve rococo-runtime error by th7nder · Pull Request #719 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/719) - [fix: polkadot nix build fails due to clang std=c++17 on MacOS by th7nder · Pull Request #730 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/730) And with incomplete reads: - [fix: zero padding reader incomplete reads by th7nder · Pull Request #710 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/710) We’ve added the deal status RPC to the server: - [feat(storage-provider): add status RPC by jmg-duarte · Pull Request #712 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/712) We’ve added support for 8MiB sectors: - [https://github.com/eigerco/polka-storage/pull/708](https://github.com/eigerco/polka-storage/pull/708) And we’re working on 512MiB and 1GiB: - [feat: support 512 & 1GiB with huge refactors by th7nder · Pull Request #717 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/717) We’ve also improved our tests: - [feat(maat): add caching for porep/post params by th7nder · Pull Request #721 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/721) - [fix(storage-provider): query sectory expiration by jmg-duarte · Pull Request #722 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/722) Fixed a bug where the server would spam pre-commit messages: - [fix(storage-provider): wait for deadline end on empty partition by jmg-duarte · Pull Request #731 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/731) And we keep pushing the P2P front: - [rfc: Add P2P info exchange by aidan46 · Pull Request #713 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/713) - [feat: Move bootstrap node into collator by aidan46 · Pull Request #727 · eigerco/polka-storage · GitHub](https://github.com/eigerco/polka-storage/pull/727) [Previous page](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021.md?page=2) [Next page](https://forum.polkadot.network/t/polkadot-native-storage-updates/7021.md?page=4)