Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publishing multiple packages from a mono_repo (atomically?) #4131

Open
dcharkes opened this issue Jan 31, 2024 · 2 comments
Open

Publishing multiple packages from a mono_repo (atomically?) #4131

dcharkes opened this issue Jan 31, 2024 · 2 comments
Labels
type-enhancement A request for a change that isn't a bug

Comments

@dcharkes
Copy link
Contributor

When working with mono repos, it is common for multiple packages to depend on each other.

it would be splendid if we could somehow support publishing multiple interdependent packages.

A couple of things we'd need:

  1. mono_repo/pkgs/my_package$ dart pub publish --dry-run --not-yet-published=../my_package2 --not-yet-published=../my_package3
    The version in ../my_package2/pubspec.yaml can now also be used in package resolution.

  2. mono_repo$ dart pub publish_all --path=pkgs/my_package1 --path=pkgs/my_package2
    This first runs validation on all packages with --not-yet-published arguments. And then when it's happy it publishes all of them atomically.

If we don't want to support step 2. Then we can build step 2 into https://pub.dev/packages/mono_repo. dart pub global run mono_repo:publish_all. It would use step 1, and then non-atomically do a dart pub publish --force in each package. This would lead to packages temporarily not having all their dependencies existing until this step is done.

(To make CI work properly, we could also add --not-yet-published to pub get. This is not necessary though, as the same can be achieved with dependency overrides. But it might make sense to build for API symmetry.)

cc @jonasfj @sigurdm @mosuem

@sigurdm sigurdm added the type-enhancement A request for a change that isn't a bug label Feb 1, 2024
@sigurdm
Copy link
Contributor

sigurdm commented Feb 1, 2024

I think that if we want to do this, it would be as part of "staged publishing" where you would then be able to release all packages in one go.

Also I think the problem is rather small:

  • For sets of packages without cyclic dependencies (that are (almost?) never really needed) it should be possible to publish in an order where resolution will go through.
  • Users of the packages will never get not-yet-resolving versions

@dcharkes
Copy link
Contributor Author

dcharkes commented Feb 1, 2024

Correct, I'm not so interested in cyclic dependencies. (Also, really with native assets we should not have cyclic dependencies! But who knows, if we'd support it I might start using cyclic dependencies myself and split stuff up in smaller packages...)

it should be possible to publish in an order where resolution will go through.

This requires multiple commits:

  • Merge a commit on my_package1 with a stable version in it's pubspec in the mono repo.
  • Publish my_package1.
  • Merge a commit on my_package2 which updates the dependency on my_package1 (from a path dependency to a published dependency) and updates the version of my_package2.
  • Publish my_package2.
  • Merge a commit on my_package3 ...

Having multiple commits and multiple publish steps takes time to baby-sit if you want the CI to run to sanity check that what you're doing is correct.

@mosuem suggested writing a a github bot in dart-lang/mono_repo that does these steps for you on a mono_repo.

  • The bot needs to be able to make branches and a PR.
  • Bots can already merge with auto-submit labels.
  • The bot needs to be able to push tags (or even make releases in the GitHub CI) to trigger the auto publishing on pub.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-enhancement A request for a change that isn't a bug
Projects
None yet
Development

No branches or pull requests

2 participants