Fix bug in create_sysimg_from_object_file for Xcode 15 CLT #935
Conversation
src/PackageCompiler.jl
Outdated
| if Sys.isapple() | ||
| cltools_version_cmd = `pkgutil --pkg-info=com.apple.pkg.CLTools_Executables` | ||
| cltools_version = match(r"version: (.*)\n", readchomp(cltools_version_cmd))[1] | ||
| if startswith(cltools_version, "15") |
There was a problem hiding this comment.
I am wondering if we should make this >= 15 ?
I presume this issue will still be around in xcode 16 and we don't want to go back to the flags for <=14?
There was a problem hiding this comment.
I'm mildly torn on this. As-is, it forces us to revisit this fix for xcode 16 and investigate whether the issue has been solved or if our mildly hacky solution is still needed. Ultimately, though, that comes at the expense of (most likely) breaking development for anyone who upgrades to Xcode 16. I see no reason to assume that this will not continue to be an issue in xcode 16+, so making it ≥ 15 seems like the best course of action. Thanks!
src/PackageCompiler.jl
Outdated
| o_file_flags = Sys.isapple() ? `-Wl,-all_load $object_files` : `-Wl,--whole-archive $object_files -Wl,--no-whole-archive` | ||
| if Sys.isapple() | ||
| cltools_version_cmd = `pkgutil --pkg-info=com.apple.pkg.CLTools_Executables` | ||
| cltools_version = match(r"version: (.*)\n", readchomp(cltools_version_cmd))[1] |
There was a problem hiding this comment.
This should handle the case that pkgutil is unable to locate com.apple.pkg.CLTools_Executables
There was a problem hiding this comment.
Good call. Do you have any thoughts on what version we should assume under this failure case? I don't have a good feel for whether it's more likely that a system has been upgraded to the latest version or not, but given how Apple ties Xcode version to macOS version (and professional dev envs are eventually going to force OS updates) it may be safest to default to 15+ (i.e. to apply the flags).
There was a problem hiding this comment.
Unfortunately it looks like there's a syntax ambiguity so that a lot of linkers interpret this flag as -ld_classic i.e. link against libd_classic.so, which will obviously fail.
I'm not sure how far back we have to go before that becomes an issue. It also sounds like we might have to worry about using a different compiler not from XCode CLT.
Could we execute the compiler binary and ask it for its version directly?
There was a problem hiding this comment.
While I have a general understanding of the problem you're describing, this is admittedly over my head. Maybe something like the first answer on this SO post could work?
There was a problem hiding this comment.
My biggest concern is that this patch assumes that the user is using the XCode CLT to compile on macOS, which is not necessarily true
By "execute the compiler binary directly", I mean a combination of either:
- Modifying get_compiler_cmd to inspect the path to the binary and see that it's provided by Xcode, and/or
- Execute
run_compiler(`` --version``)or similar to determine whether we're running the Xcode-provided compiler and which version.
There was a problem hiding this comment.
I've tinkered with this a bit and am still having a bit of trouble.
I've tried getting the path to the compiler with
base_dir = dirname(sysimg_source_path)
compiler_source_path = joinpath(base_dir, "compiler", "compiler.jl")
Printing compiler_source_path results in /Users/jaycook/.julia/juliaup/julia-1.8.5+0.aarch64.apple.darwin14/share/julia/base/compiler/compiler.jl. I was hoping more for something like /Library/Developer/CommandLineTools/usr/bin, where I could check for a string like "xcode" or "CommandLineTools".
run_compiler()'s underlying call to run() does not appear to return the result of the command that's run. For example, print(run_compiler(`` --version``)) results in Process(``gcc --version``, ProcessExited(0)).
If you're so inclined, do you care to toss up a commit for the implementation that you have in mind?
| if startswith(cltools_version, "15") | ||
| try | ||
| cltools_version_cmd = `pkgutil --pkg-info=com.apple.pkg.CLTools_Executables` | ||
| cltools_version = match(r"version: (.*)\n", readchomp(cltools_version_cmd))[1] |
There was a problem hiding this comment.
I think you can check for nothing here, rather than try/catch.
There was a problem hiding this comment.
Can you suggest a good way to test this? If I change the --pkg-info to force a failure I get the following error, but I imagine that the "real" failure mode here is simply returning nothing. Just not sure how to go about getting my system into a state that allows me to test this before shipping it.
Fix for issue #738, which conditionally adds the
-Wl,-ld_classicflags too_file_flagsincreate_sysimg_from_object_filebased on Xcode CLT version.All credit to @PhilReinhold for this code; I'm simply opening this PR based on his latest correspondence on the issue.
Tested locally on gcc 15.0.0 and Julia 1.8.5. @hbe72 confirmed in the issue correspondence that it's working on gcc 15.0.0 and Julia 1.10
Re: CI failures, I'm seeing other recent PRs (1, 2) that are also failing the nightly build tests and buildkite. Please let me know if you see anything in this change that should address these failures.