Cannot find Quartz library on MacOS Sonoma

[takacsmark]

After upgrading to MacOS Sonoma 14.0 I encountered the following error running stable diffusion depth map script that uses Vispy under the hood.

*** Error loading script: depthmap.py
    Traceback (most recent call last):
      File "/Users/mark/dev/chi/stable-diffusion-webui/modules/scripts.py", line 382, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "/Users/mark/dev/chi/stable-diffusion-webui/modules/script_loading.py", line 10, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/Users/mark/dev/chi/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/depthmap.py", line 8, in <module>
        from src import common_ui
      File "/Users/mark/dev/chi/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/src/common_ui.py", line 6, in <module>
        from src import backbone, video_mode
      File "/Users/mark/dev/chi/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/src/video_mode.py", line 8, in <module>
        from src import core
      File "/Users/mark/dev/chi/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/src/core.py", line 27, in <module>
        from inpaint.mesh import write_mesh, read_mesh, output_3d_photo
      File "/Users/mark/dev/chi/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/inpaint/mesh.py", line 12, in <module>
        from vispy import scene, io
      File "/Users/mark/dev/chi/stable-diffusion-webui/venv/lib/python3.10/site-packages/vispy/__init__.py", line 27, in <module>
        from .util import config, set_log_level, keys, sys_info  # noqa
      File "/Users/mark/dev/chi/stable-diffusion-webui/venv/lib/python3.10/site-packages/vispy/util/__init__.py", line 14, in <module>
        from . import fonts       # noqa
      File "/Users/mark/dev/chi/stable-diffusion-webui/venv/lib/python3.10/site-packages/vispy/util/fonts/__init__.py", line 13, in <module>
        from ._triage import _load_glyph, list_fonts  # noqa, analysis:ignore
      File "/Users/mark/dev/chi/stable-diffusion-webui/venv/lib/python3.10/site-packages/vispy/util/fonts/_triage.py", line 14, in <module>
        from ._quartz import _load_glyph, _list_fonts
      File "/Users/mark/dev/chi/stable-diffusion-webui/venv/lib/python3.10/site-packages/vispy/util/fonts/_quartz.py", line 12, in <module>
        from ...ext.cocoapy import cf, ct, quartz, CFRange, CFSTR, CGGlyph, UniChar, \
      File "/Users/mark/dev/chi/stable-diffusion-webui/venv/lib/python3.10/site-packages/vispy/ext/cocoapy.py", line 1288, in <module>
        quartz = cdll.LoadLibrary(util.find_library('quartz'))
      File "/opt/homebrew/Cellar/python@3.10/3.10.13_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ctypes/__init__.py", line 452, in LoadLibrary
        return self._dlltype(name)
      File "/opt/homebrew/Cellar/python@3.10/3.10.13_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ctypes/__init__.py", line 374, in __init__
        self._handle = _dlopen(self._name, mode)
    OSError: dlopen(Quartz.framework/Quartz, 0x0006): tried: 'Quartz.framework/Quartz' (no such file), '/System/Volumes/Preboot/Cryptexes/OSQuartz.framework/Quartz' (no such file), '/opt/homebrew/lib/Quartz.framework/Quartz' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/Quartz.framework/Quartz' (no such file), '/usr/lib/Quartz.framework/Quartz' (no such file, not in dyld cache), 'Quartz.framework/Quartz' (no such file)

I'm using Vispy 0.14.1.

The file vispy/ext/cocoapy.py on line 1288 has this code:
quartz = cdll.LoadLibrary(util.find_library('quartz'))
Changing quartz to Quartz resolves the issue.

[djhoese]

Very weird. What is the return value of util.find_library("quartz") and of util.find_library("Quartz")? I quick google doesn't reveal anything about a name change in the Sonoma changes. And you can see that it is trying to load with dlopen something called Quartz.framework/Quartz so it is finding something.

I don't have a mac, but I've asked some coworkers who do if they know anything about this. If anyone else reading this can help investigate that would be appreciated.

[takacsmark]

@djhoese these are the return values:

❯ python --version
Python 3.10.13
❯ python
Python 3.10.13 (main, Aug 24 2023, 12:59:26) [Clang 15.0.0 (clang-1500.0.40.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes.util
>>> print(ctypes.util.find_library('quartz'))
None
>>> print(ctypes.util.find_library('Quartz'))
/System/Library/Frameworks/Quartz.framework/Quartz

[djhoese]

@takacsmark Can you make a pull request with your fix? We'll see how CI feels about this. Maybe this has been a suggestion for a long time and we've just been getting away with it and it was finally removed in Sonoma.

[takacsmark]

@djhoese looked into it and see that there was already a patch at the beginning of the file for Big Sur based on a Stackoverfow solution. I patched that patch to be in line with the Stackoverflow logic. Tested for my usecase, ok.

Created pull request 2549

[psobolewskiPhD]

Do you have a small snippet to reproduce this?
I'm on Ventura 13.6.2 and I have:

Python 3.11.4 | packaged by conda-forge | (main, Jun 10 2023, 18:08:41) [Clang 15.0.7 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes.util
>>> print(ctypes.util.find_library('quartz'))
None
>>> print(ctypes.util.find_library('Quartz'))
/System/Library/Frameworks/Quartz.framework/Quartz

So apparently this isn't Sonoma specific?

Edit:
Also, looking at that monkey patch, in my case:

>>> import OpenGL.GL
>>>

So it appears that the except branch of that logic won't even trigger...

[djhoese]

Ok so maybe this is an edge case of an edge case. Thanks for checking @psobolewskiPhD.

[takacsmark]

My understanding is that the monkey patch was created for Big Sur based on the PR and the Stackoverflow question and should trigger also on Ventura. Can you check that?

Just to recap the issue; the monkey patch maps in the current prod release:

quartz --> Quartz.framework/Quartz, thus the code will try cdll.LoadLibrary("Quartz.framework/Quartz") and throws the error on Sonoma:

OSError: dlopen(Quartz.framework/Quartz, 0x0006): tried: 'Quartz.framework/Quartz' (no such file), '/System/Volumes/Preboot/Cryptexes/OSQuartz.framework/Quartz' (no such file), '/opt/homebrew/lib/Quartz.framework/Quartz' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/Quartz.framework/Quartz' (no such file), '/usr/lib/Quartz.framework/Quartz' (no such file, not in dyld cache), 'Quartz.framework/Quartz' (no such file)

So my understanding is this:

• quartz = cdll.LoadLibrary(util.find_library('quartz')) used to work before Big Sur and is left as is for backwards compatibility
• I have no way to test if the monkey patch was actually triggering after Big Sur, but either cdll.LoadLibrary(util.find_library('quartz')) or cdll.LoadLibrary("Quartz.framework/Quartz") worked
• Both cdll.LoadLibrary(util.find_library('quartz')) and cdll.LoadLibrary("Quartz.framework/Quartz") fail on Sonoma, so updated the monkey patch to reflect the Stackoverflow solution exactly.

[djhoese]

Kind of related: Should we drop support for Big Sur and older?

https://en.wikipedia.org/wiki/MacOS_version_history#Releases

[psobolewskiPhD]

That's the thing, the monkey patch is gated by a Try:

vispy/vispy/ext/cocoapy.py

Lines 23 to 38 in 5b4731c

	try:
	import OpenGL.GL # noqa
	except ImportError:
	# print('Drat, patching for Big Sur')
	orig_util_find_library = util.find_library
	def new_util_find_library(name):
	res = orig_util_find_library(name)
	if res:
	return res
	lut = {
	'objc': 'libobjc.dylib',
	'quartz': 'Quartz.framework/Quartz'
	}
	return lut.get(name, name+'.framework/'+name)
	util.find_library = new_util_find_library

For me Ventura 13.6.2 arm64, import OpenGL.GL doesn't throw an exception.
Python 3.11.4, 3.10.12 -- both from conda-forge.

I'll test some more this evening.

[djhoese]

Ok it is all starting to come back to me. @takacsmark what version of python are you using and where did it come from? I think conda-forge included patches for this issue in the Python that they distribute for a long time, but it has since been released upstream by Python. I'm having trouble tracking down what patch versions of Python 3.9+ include the fix for it, but any modern-ish version should be good.

My guess is that you @takacsmark don't have pyopengl installed and that must be (or so it seems) a requirement for this cocoapy code. If you remove the try/except do you get a dlopen issue or do you get a module/package not found error?

[psobolewskiPhD]

Is there a better way to test this? a snippet I could run?
Because on my Ventura Macs, I get:

>>> import OpenGL.GL
>>> 
>>> from vispy.ext.cocoapy import quartz
>>> quartz
<CDLL 'None', handle fffffffffffffffe at 0x1053f3050>

So that seems not correct? But I can run napari and e.g. turntable_box example just fine.
Edit: I do have pyopengl
Let me try a vispy only env: mamba create -n vispy python=3.10 vispy
Boom:

╰─ python                                                             (vispy) ─╯
Python 3.10.13 | packaged by conda-forge | (main, Oct 26 2023, 18:09:17) [Clang 16.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import OpenGL.GL
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'OpenGL'

and

>>> from vispy.ext.cocoapy import quartz
>>> quartz
<CDLL 'Quartz.framework/Quartz', handle 3ad9bf508 at 0x10ebd9c60>

So that makes sense.
Which comes back to needing a proper test snippet that uses quartz

[takacsmark]

I'm using Python 3.10.13 in a venv created by stable diffusion webui using venv, not conda. pyopengl is not installed, indeed.

[djhoese]

I don't even remember what cocoapy is used for. If you install pyopengl, without your fix, does your use case start working?

[takacsmark]

After installing pyopengl the monkey patch does not fire and the error is not thrown, my stable diffusion use case works as expected.

When I look at the libraries in REPL I get this:

❯ python
Python 3.10.13 (main, Aug 24 2023, 12:59:26) [Clang 15.0.0 (clang-1500.0.40.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import cdll, util
>>> print(util.find_library('objc'))
/usr/lib/libobjc.dylib
>>> print(util.find_library('CoreFoundation'))
/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation
>>> print(util.find_library('AppKit'))
/System/Library/Frameworks/AppKit.framework/AppKit
>>> print(util.find_library('quartz'))
None
>>> cdll.LoadLibrary(util.find_library('quartz'))
<CDLL 'None', handle fffffffffffffffe at 0x103187280>
>>> print(util.find_library('CoreText'))
/System/Library/Frameworks/CoreText.framework/CoreText
>>> print(util.find_library('Foundation'))
/System/Library/Frameworks/Foundation.framework/Foundation

The find_library return value for quartz is None, it's loaded as <CDLL 'None', handle fffffffffffffffe at 0x103187280> it does no throw an error. Looks strange, though.

[djhoese]

So...do we spend a ton of time investigating what the CDLL "None" really means or do we just say that this is a missing dependency and close this issue and the PR...or is there a way this can be documented better? I really don't understand what the cocoapy stuff is trying to do.

[psobolewskiPhD]

I do wonder if the line

vispy/vispy/ext/cocoapy.py

Line 1288 in 5b4731c

quartz = cdll.LoadLibrary(util.find_library('quartz'))

should be changed to cdll.LoadLibrary(util.find_library('quartz') or util.find_library('Quartz'))
On my Ventura it returns:

>>> cdll.LoadLibrary(util.find_library('quartz') or util.find_library('Quartz'))
<CDLL '/System/Library/Frameworks/Quartz.framework/Quartz', handle 3aa411628 at 0x102b97910>

which seems correct versus the current:

>>> cdll.LoadLibrary(util.find_library('quartz'))
<CDLL 'None', handle fffffffffffffffe at 0x102b80410> 

But without a something that specifically tests this it's hard to say?

[takacsmark]

Your solution cdll.LoadLibrary(util.find_library('quartz') or util.find_library('Quartz')) works on Sonoma, too:

>>> cdll.LoadLibrary(util.find_library('quartz') or util.find_library('Quartz'))
<CDLL '/System/Library/Frameworks/Quartz.framework/Quartz', handle 3cd212088 at 0x102c8f370>

If PyOpenGL is not available though, then the monkey patch will run and it has an override for quartz. So the patch will fire, return `Quartz.framework/Quartz' and throw the error in LoadLibrary. To handle this we can keep the fix that I created in the pull request. My fix changes the logic of the patch, it's in line with the info on stackoverflow that seems legit, but testing on multiple versions of MacOs would be the safe path.

[hekuli]

Also experiencing this issue and looking forward to a fix. Using an M1 Max.