Publish Substrate to crates.io

@bkchr @joao-paulo some updates? Are there any tracking issues that we can monitor? Otherwise I may need to write a bot to ping for updates every two weeks.

JP does not work for Parity anymore… so sadly he cannot push this forward (see his GH page).
I dont think it is currently assigned to anyone. The situation is that all Substrate crates get released from master every week i think. For example sp-core is now at version 20.

Thanks for the update. Can someone else get assigned? This is a top priority task.

I will take on to oversee this process together with @joyce. Someone is about to join Parity and that will then work on this task.

4 Likes

I think the plan is to change that so that we sync it with our actual releases instead of doing it periodically. Do you know if anybody is on that @bkchr?

1 Like

We kicked this off last week. The person working on it is currently creating a write up on how to tackle it.

Will post more updates when I have more information.

4 Likes

bumppp

1 Like

Crates for Substrate as well as Polkadot and Cumulus are now live. The plan is to publish new crate versions every time we have a Polkadot node release.

Currently we’re just bumping the major version of each crate on each release but we are working on having semver aware versioning at some point in the future.

10 Likes

Any updates about this regards the new polkadot-sdk repo? Is the new publish pipeline ready? What’s the release frequency? What’s the latest release? Release notes? Is it ready for downstream projects to use?

1 Like

Re-echoing @xlc 's comments. Is the case that the crates.io versions will match the GitHub version? Wen Polkadot-v1.1.0?

Yes, every release of the Parity Polkadot node is done together with a release of all crates to crates.io.

Every 2 weeks.

If you mean the latest crates version, we don’t have any right tooling yet. I think @OliverTY was using something.

We are working on a proper system for release notes.

Yes you can use it.

Already on crates.io since the release.

All the crates seem to have their version number chosen at random.

Is it possible to just have v1.1.0 on all of them. Literally No way to use them because of the different versions

This doesn’t work. Because you can overwrite old crate versions. So, we have to start with the old crate versions and bump them as we go.

There could probably be introduced some kind of tooling that updates all of your crates to the latest version on crates.io.

If that’s the case, then maybe set all of them to the highest major version. So that this way, versions move in lockstep. I can tell you that different major versions will cause incompatibilities across crates.

In the future we don’t want to bump only the major versions. Bump only major, minor or patch depending on the changes. Then these versions will also not go in lockstep.

Doubt it’ll be useful in this case

Not sure why it should be less useful than with every crate having the same version. One of the major advantages being that you can update node side and runtime side for example separately. Meaning you can adapt for example faster to node side changes and only update the runtime every X weeks.

As I said, we could provide some tooling to update your crates to the latest version. For example we could use diener to do this.

A tool to update to latest version will be helpful.
It will be great if we can somehow specify what packages to not update so we can easily update the client without update the runtime.

3 Likes

Yea it was only a one-off script for the migration. Basically it pulled each crate version from index.crates.io.
I think this tool could be able to list all the latest versions GitHub - paritytech/parity-publish: A tool to manage publishing Parity's crates in one go, but not sure how far it is.

This idea of updating the node but not the runtime would only work when no crates that are shared between runtime and node are bumped, or?
Maybe it is easier to have two cargo workspaces; one for the runtime and one for the node such that the dependency resolutions dont interfere with each other.

Be able to decouple runtime and the binary into separate workspaces will be the best. But I am not sure if it is possible to do it now. Some examples will be great.