This time the problem wasn’t that the migration failed, but more that the migration took too long. This is also not the first time this has happened and will probably also not be the last time. As I had said around the “famous” How to recover a parachain talks, we need to “test test test and test”. We already have chopsticks from @xlc and that is a very good tool that we should leverage more. We also have try-runtime
and both are probably capable of showing you the PoV size of the migration, which would have shown the issue here.
However, what I would like to see, instead of coming retroactively when the shit hit the fan, people asking before applying. Maybe we should create some repo or discussions here in the forum when someone wants to apply an upgrade. People could then ask others for help, other people out of the ecosystem who may already have more knowledge when it comes to runtime upgrades. Just go out and ask for help I would also be very very happy if we could do this outside of parity I would also be willing to look into this as well from time to time. Maybe we could create this repo under the fellowship or somewhere else? We just need someone who would be willing to make this happening. We could also write down learnings from these migrations to help others in the future.
I also started to write down instructions on how to rescue a chain, instead of just writing them down in a chat. My idea is that people can use this to learn. Here are the two documents I currently have: