Eventual consistency issues when publishing many packages at once

[Jordan-Nelson]

Problem

TL;DR - Pub says that a package version does not exist for up to ~30 seconds after it has been published.

I am one of several maintainers of a large monorepo. There are a few dozen packages in this repo, many of which depend on each other and often get published at the same time. When we publish packages we create a dependency graph and start publishing all the leaf nodes and then work up each branch until all the packages are published. This is automated. This ensures package_b is not published before package_a if package_b depends on package_a. This is important when package_b depends on the latest version of package_a and both are being published at the same time.

This use to work great. More recently we have seen issues where after package_a version x.y.z is published, publishing package_b fails with Because package_b depends on package_a ^X.Y.Z which doesn't match any versions, version solving failed.

If we wait around 10 seconds sometimes the publish command works on the second try. Sometimes it takes as long as 30 seconds to recognize that package_a ^X.Y.Z does exist.

This makes it very hard to automate our publishing process. We currently use a script to publish, which is kicked of by a human. In the future we would like to fully automate this. This issue makes this error prone as the best solution that I can think of is to wait ~1 minute after each publish command.

Expected behavior

Once a package is published, pub should not say it does not exist

Actual behavior

Once a package is published, pub sometimes says it does not exist

[jonasfj]

TL;DR - Pub says that a package version does not exist for up to ~30 seconds after it has been published.

Where does it say that? I think we recently made it 2 minutes, so I'd like to update all messages to be more correct.

---

This use to work great. More recently we have seen issues where after package_a version x.y.z is published, publishing package_b fails with Because package_b depends on package_a ^X.Y.Z which doesn't match any versions, version solving failed.

Yeah, we recently changed Cache-Control on pub.dev for API responses, to allow a CDN to cache the response for up to 2 minutes.

Honestly, we tried going for 10 minutes, but that caused a lot of push back from people manually publishing dependent packages.

Caching gives us faster responses and better reliability.

I think that in the future we should consider being even more aggressive about caching.
We have talked about pub.dev hosting end-points that lists date-time for packages updated in the past day, week, month. And then having pub get simply fetch said file, and purge its cache, instead of listing versions for every package that wants to use in the solve.
This is arguably further out in the future, and we haven't even work out how such a thing should work. But it's the kind of thing that could enable pub get to run very FAST! 🚀

In practice, it rarely matters to package consumers if a new version of a package comes out in 30s, 2 minutes or 30 minutes.
But it does matter to some advanced mono-repository package authors! 🙈

Long term, as in: some day in a far fetched future, maybe we could have a feature where package authors could stage new versions on pub.dev; test and update staged versions, perhaps even release multiple staged versions together. This would be cool, and it would be the solution to caching issues like this.

At the moment, I suspect that the benefit of faster responses and increased reliability benefits a lot of people. Where as the mono-repository publishing issues, only affects a few very advanced package authors.

Workaround with dart pub publish --skip-validation

I think that for advanced users it might be better to simply use --skip-validation, when publishing.
This is a relatively recently added feature. One should be careful when using it (obviously).

But the validations we do are not fantastic, we mostly try to resolve dependencies, run dart analyze and a few other tricks.

I don't know if dart pub publish --dry-run would allow you to run the validation, I'm guessing those might fail because the resolution will fail.

When we publish packages we create a dependency graph and start publishing all the leaf nodes and then work up each branch until all the packages are published. This is automated. This ensures package_b is not published before package_a if package_b depends on package_a. This is important when package_b depends on the latest version of package_a and both are being published at the same time.

But it's probably better that you spend your energy testing your code, possibly even testing with dart pub downgrade (to the extend possible), rather than you spending time writing logic to publish packages in order of dependency, just to satisfy the publishing validations we have in dart pub publish.

I think that it would be cool if we could make dart pub publish validations run dart pub downgrade and then analyze the code for errors. But for something like that to work in a mono-repository where you depend on package versions you're publishing along side the package would be complicated. It's more likely that you're better of developing your own testing procedures to do this. I would suspect that many different mono-repositories have different workflows for things like this.

---

Closing this bug because:

• Caching offers benefits (speed and reliability)
• Negative effects mostly affects a few very advanced package authors.
• We have a workaround --skip-validation.
• While not ideal: It might be preferable to accept that it won't be.

[jonasfj]

For reference, see: #4489 for staged publishing discussions.

Feel free to upvote #4489 if you think it's important.

[jonasfj]

Also filed dart-lang/site-www#5803 to update documentation to cover --skip-validation.