Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incremental sysimage build doesn't support packages with same name but different uuid #768

Open
krynju opened this issue Jan 25, 2023 · 3 comments · May be fixed by #807
Open

Incremental sysimage build doesn't support packages with same name but different uuid #768

krynju opened this issue Jan 25, 2023 · 3 comments · May be fixed by #807

Comments

@krynju
Copy link

krynju commented Jan 25, 2023

This one is similiar to #767, but setup and place where it errors are different.

Setup

Simple two projects that have a top level dependency PackageA, but with different uuids

using PackageCompiler

using Pkg

Pkg.activate("project1") # has Package1 with uuid=1234
create_sysimage(["Package1"]; sysimage_path="sysimage5.so")

Pkg.activate("project2") # has Package1 with uuid=5678
create_sysimage(["Package1"]; sysimage_path="sysimage6.so", base_sysimage="sysimage5.so")

Error

julia> create_sysimage(["Package1"]; sysimage_path="sysimage6.so", base_sysimage="sysimage5.so")
⠙ [00m:07s] PackageCompiler: compiling incremental system imageLoadError
⡆ [00m:08s] PackageCompiler: compiling incremental system imageimporting Package1 into Main conflicts with an existing identifier
in expression starting at /tmp/jl_bOn2oN:7
✖ [00m:08s] PackageCompiler: compiling incremental system image
ERROR: failed process: Process(`/home/krynju/julia-1.7.3/bin/julia --color=yes --startup-file=no --cpu-target=native -O3 --sysimage=sysimage5.so --project=/home/krynju/sameuuidtest/project2 --output-o=/tmp/jl_sKRDQC.o /tmp/jl_bOn2oN`, ProcessExited(1)) [1]

Error appears due to this code:

PackageCompiler.jl/src/PackageCompiler.jl

Lines 377 to 382 in 05f6854

# Make packages available in Main. It is unclear if this is the right thing to do.
for pkg in packages
print(julia_code_buffer, """
import $pkg
""")
end

Solution

I worked around this by commenting out this part of code and everything looks to be functioning just right.

The error didn't appear if the package was the same uuid, but a different version, which is probably something weird to handle as well

A build flag to turn off this import behavior would probably be good enough as I'm not sure what the import to Main is dedicated for
@KristofferC
Copy link
Sponsor Member

A build flag to turn off this import behavior would probably be good enough as I'm not sure what the import to Main is dedicated for

It is there to be able to get a reference to the module (e.g. using .Plots) without having to have a Project file available to lookup the UUID when doing using Plots.

@krynju krynju mentioned this issue Mar 16, 2023
Open
@staticfloat
Copy link
Sponsor Member

Here's a simple way to reproduce this:

julia> versioninfo()
Julia Version 1.9.0-rc1
Commit 3b2e0d8fbc1 (2023-03-07 07:51 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 32 × AMD Ryzen 9 3950X 16-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, znver2)
  Threads: 1 on 32 virtual cores
Environment:
  JULIA_PKG_PRECOMPILE_AUTO = 1
  LD_LIBRARY_PATH = 
  JULIA_PKG_SERVER = https://pkg.julialang.org

$ cat broken/Project.toml 
[deps]
Accessors = "7d9f7c33-5ae7-4f3b-8dc6-eff91059b697"
IntervalSets = "8197267c-284f-5f27-9208-e0e47529a953"

$ cat broken/broken.jl 
using PackageCompiler, Pkg

Pkg.activate(@__DIR__)
create_sysimage(["Accessors", "IntervalSets"]; sysimage_path=joinpath(@__DIR__,"broken.so"))

$ julia +1.9 --project broken/broken.jl 
  Activating project at `~/.julia/dev/PackageCompiler/broken`
Precompiling environment...
  4 dependencies successfully precompiled in 1 seconds. 12 already precompiled.
⠸ [00m:08s] PackageCompiler: compiling incremental system imageinvalid redefinition of constant IntervalSetsExt
Stacktrace:
 [1] top-level scope
   @ none:1
⣠ [00m:08s] PackageCompiler: compiling incremental system imageModule, e::Any)
   @ Core ./boot.jl:370
 [3] top-level scope
   @ /tmp/jl_w3cHYriBwz:61
 [4] eval(m::Module, e::Any)
   @ Core ./boot.jl:370
 [5] top-level scope
   @ /tmp/jl_w3cHYriBwz:56
in expression starting at /tmp/jl_w3cHYriBwz:56
✖ [00m:08s] PackageCompiler: compiling incremental system image

@KristofferC
Copy link
Sponsor Member

There are two different issues being discussed here:

  1. The OP which is caused by us trying to make packages available in Main. If two packages have the same name but different UUID this will not work. This will only happen when you are making an incremental sysimage with a package name on top of another sysimage with a different package of the same name.
  2. The one described by @staticfloat. This occurs when we are replaying precompile statements we need to have a reference to the module. If there are multiple modules with the same name, this used to error because we tried import all of those into the same module. This is a dup of Precompile stage doesn't support packages with the same names and different uuids #767 and has been fixed by Some fixes to precompile "replay" #805

@krynju krynju linked a pull request May 10, 2023 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add import_to_main flag for Main module package name conflicts resolution krynju/PackageCompiler.jl
3 participants
@staticfloat @KristofferC @krynju