Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(POC) Transitive dependency overriding #2981

Closed
wants to merge 2 commits into from
Closed

Conversation

PgBiel
Copy link
Contributor

@PgBiel PgBiel commented Apr 13, 2024

Proof of concept for #2899. Basically implements most of the hard parts; this PR should be easy to adjust based on the result of the discussion there.

Current design

Adds a new [patch] section to gleam.toml, which works exactly like [dependencies], except that any package in the root project's [patch] will override any transitive dependencies to that package. For example, consider this gleam.toml in the test1 project:

[dependencies]
gleam_stdlib = ">= 0.34.0 and < 2.0.0"
test2 = { path = "../test2" }

[dev-dependencies]
gleeunit = ">= 1.0.0 and < 2.0.0"

[patch]
test2 = { path = "../test3" }
simplifile = "1.6.0"
filepath = { path = "../filepath" }

If we assume that test3 depends on simplifile 1.7.0, what will happen here is that:

  1. test1 will depend on test3 instead of test2 (for now assuming the package at test3 is also named test2).
  2. test2 will depend on simplifile 1.6.0 instead of 1.7.0, even if incompatible (the error is silenced).
  3. simplifile 1.6.0 (depended on by test2) will now depend on filepath at ../filepath instead of the Hex version it usually depends on (1.0.0).

This [patch] section is added to the manifest.toml to ensure the manifest updates when the [patch] changes.

Current implementation

  1. On compiler-core, most changes occur in dependency. There, we have a check for version incompatibilities; it is simply skipped when the package is patched by the root config. Additionally, when working with pubgrub's traits for dependency resolving, we ensure we filter packages' versions according to the root config patch, and also update the fetched packages' dependencies accordingly so pubgrub doesn't get confused. We also, of course, add the patch section to PackageConfig, and also require specifying a patch to access a config's dependencies.
  2. On compiler-cli, most changes are in dependencies. Here, we had to refactor providing local and Git packages into a PackageProvider struct containing provide_*_package as methods instead of top-level functions. This was necessary as the PackageFetcher now stores the PackageProvider in a RefCell, such that, if a Hex package depends on a package which was patched to point to a local or Git package instead of a Hex package, the PackageProvider is invoked to provide that package instead of fetching from Hex (or get it from cache if the package was already provided before - hence the need for RefCell, so we can update the cache, which is also later reused to add those local packages to the manifest). Additionally, in the provide_*_package methods, we update each package's dependencies according to the root config's patch during traversal of dependencies. Finally, in the appropriate location within compiler-cli, we also ensure the manifest is updated when the patch section changes.

TODO

I will wait for further discussion before progressing on this PR, so here's more or less what still needs to be done.

  • Decide on the final design (via discussion at the relevant issue)
  • Add support for overriding a package with another package, even if it has a different name (could be separate PR)
  • Add unit tests
  • Update CHANGELOG

- Ensures it is updated when config changes
@lpil
Copy link
Member

lpil commented May 20, 2024

Closing due to inactivity. Please feel free to reopen! Thank you

@lpil lpil closed this May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants