Failed to build - possible BLAS issue

[ktattan]

Hello, I'm trying to pip install scs on macOS Moneterey, M1 chip via the installation guide, but I keep getting the error below.

 blas_mkl_info:
        NOT AVAILABLE
      blis_info:
        NOT AVAILABLE
      openblas_info:
          libraries = ['openblas', 'openblas']
          library_dirs = ['/opt/arm64-builds/lib']
          language = c
          define_macros = [('HAVE_CBLAS', None)]
          runtime_library_dirs = ['/opt/arm64-builds/lib']
      blas_opt_info:
          libraries = ['openblas', 'openblas']
          library_dirs = ['/opt/arm64-builds/lib']
          language = c
          define_macros = [('HAVE_CBLAS', None)]
          runtime_library_dirs = ['/opt/arm64-builds/lib']
      lapack_mkl_info:
        NOT AVAILABLE
      openblas_lapack_info:
          libraries = ['openblas', 'openblas']
          library_dirs = ['/opt/arm64-builds/lib']
          language = c
          define_macros = [('HAVE_CBLAS', None)]
          runtime_library_dirs = ['/opt/arm64-builds/lib']
      lapack_opt_info:
          libraries = ['openblas', 'openblas']
          library_dirs = ['/opt/arm64-builds/lib']
          language = c
          define_macros = [('HAVE_CBLAS', None)]
          runtime_library_dirs = ['/opt/arm64-builds/lib']
      {'libraries': ['openblas', 'openblas'], 'library_dirs': ['/opt/homebrew/opt/openblas/lib'], 'language': 'c', 'define_macros': [('HAVE_CBLAS', None)], 'runtime_library_dirs': ['/opt/homebrew/opt/openblas/lib']}
      {'libraries': ['openblas', 'openblas'], 'library_dirs': ['/opt/homebrew/opt/openblas/lib'], 'language': 'c', 'define_macros': [('HAVE_CBLAS', None)], 'runtime_library_dirs': ['/opt/homebrew/opt/openblas/lib']}
      error: Command "clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include -DPYTHON -DCTRLC=1 -DDLONG=1 -DUSE_LAPACK -DHAVE_CBLAS -DHAVE_CBLAS -Iscs/include -Iscs/linsys -Iscs/linsys/cpu/direct/ -Iscs/linsys/external/amd -Iscs/linsys/external/dqlql -I/private/var/folders/69/zykcchfx29s95t3v93270g140000gn/T/pip-build-env-adc48o08/overlay/lib/python3.9/site-packages/numpy/core/include -I/Users/killian/.pyenv/versions/3.9.9/include/python3.9 -c scs/linsys/scs_matrix.c -o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/scs_matrix.o -O3" failed with exit status 1

I have BLAS, OPENBLAS and LAPACK all installed via brew. I'm not quite sure from the error if this is a OPENBLAS issue or Command Line Tools. In either case, I've installed and updated the latest versions of both. Is there something else I can do here to fix this? Thanks

[bodono]

Is that the only error output? Because I don't actually see any error other than failed with exit status 1 which doesn't tell me anything.

[ktattan]

There's a much longer error, so long it's even truncated on the terminal, I just copied the last part which I thought had most information. Here's a small section of the rest of the error

      /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include/signal.h:106:31: warning: pointer is missing a nullability type specifier (_Nonnull, _Nullable, or _Null_unspecified) [-Wnullability-completeness]
      int     sigvec(int, struct sigvec *, struct sigvec *);
                                        ^
      /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include/signal.h:106:31: note: insert '_Nullable' if the pointer may be null
      int     sigvec(int, struct sigvec *, struct sigvec *);
                                        ^
                                         _Nullable
      /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include/signal.h:106:31: note: insert '_Nonnull' if the pointer should never be null
      int     sigvec(int, struct sigvec *, struct sigvec *);
                                        ^
                                         _Nonnull
      /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include/signal.h:106:48: warning: pointer is missing a nullability type specifier (_Nonnull, _Nullable, or _Null_unspecified) [-Wnullability-completeness]
      int     sigvec(int, struct sigvec *, struct sigvec *);
                                                         ^
      /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include/signal.h:106:48: note: insert '_Nullable' if the pointer may be null
      int     sigvec(int, struct sigvec *, struct sigvec *);
                                                         ^
                                                          _Nullable
      /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include/signal.h:106:48: note: insert '_Nonnull' if the pointer should never be null
      int     sigvec(int, struct sigvec *, struct sigvec *);
                                                         ^
                                                          _Nonnull

[bodono]

Those are all warnings, is there an error anywhere?

[ktattan]

I grepped the entire output and this is the only place where I could find error

...
failed with exit status 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for scs
Failed to build scs
ERROR: Could not build wheels for scs, which is required to install pyproject.toml-based projects

I don't think it's very informative.

[bodono]

Hmmm, can you post the entire stack trace as a gist: https://gist.github.com/

[ktattan]

Here it is: https://gist.github.com/ktattan/6788decb2ab01d4f82d9971a5e728bba

[bodono]

Is that everything? I don't see a clear error anywhere, and all those warnings are coming from /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include/stdlib.h not from SCS.

If that is everything can you try adding the --verbose flag when running pip install?

[ktattan]

Yep that's everything I got, but I tried it with --verbose as you suggested: https://gist.github.com/ktattan/bc679b796a2812ce512fa386ed4d7c30
Thanks again for your help.

[bodono]

I still can't see anything there, other than warnings that are not coming from SCS. Try installing from source, instructions available here.

[ktattan]

Thanks - install from the source produces a slightly different error

clang: error: no such file or directory: 'build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/scs_matrix.o'
clang: error: no such file or directory: 'build/temp.macosx-12.2-arm64-cpython-39/scs/src/util.o'
error: Command "clang -bundle -undefined dynamic_lookup -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/killian/.pyenv/versions/3.9.9/lib -L/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/killian/.pyenv/versions/3.9.9/lib -L/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/lib build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/cpu/direct/private.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/csparse.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/SuiteSparse_config.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_1.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_2.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_aat.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_control.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_defaults.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_dump.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_global.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_info.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_order.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_post_tree.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_postorder.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_preprocess.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/amd/amd_valid.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/external/qdldl/qdldl.o build/temp.macosx-12.2-arm64-cpython-39/scs/linsys/scs_matrix.o build/temp.macosx-12.2-arm64-cpython-39/scs/src/aa.o build/temp.macosx-12.2-arm64-cpython-39/scs/src/cones.o build/temp.macosx-12.2-arm64-cpython-39/scs/src/ctrlc.o build/temp.macosx-12.2-arm64-cpython-39/scs/src/linalg.o build/temp.macosx-12.2-arm64-cpython-39/scs/src/normalize.o build/temp.macosx-12.2-arm64-cpython-39/scs/src/rw.o build/temp.macosx-12.2-arm64-cpython-39/scs/src/scs.o build/temp.macosx-12.2-arm64-cpython-39/scs/src/scs_version.o build/temp.macosx-12.2-arm64-cpython-39/scs/src/util.o build/temp.macosx-12.2-arm64-cpython-39/src/scspy.o -o build/lib.macosx-12.2-arm64-cpython-39/_scs_direct.cpython-39-darwin.so -Wl,-framework -Wl,Accelerate -Wl,-framework -Wl,Accelerate" failed with exit status 1

[bodono]

Can you post the entire stack trace as another gist?

[ktattan]

Sure, its here: https://gist.github.com/ktattan/8d49c0d69bf1853a42bda495fe542b8f

[bodono]

I don't see the clang: error: no such file or directory in there?

[ktattan]

Sorry I copied the wrong error, this one should have clang error in it now: https://gist.github.com/ktattan/8d49c0d69bf1853a42bda495fe542b8f

[bodono]

Hmmm, I still don't see the clang error, is it the right one? If it is, which line is the clang error on?

[ktattan]

I dont get it, I am seeing it close to the very end (I think around line 56,000) - you need to view the raw file: https://gist.githubusercontent.com/ktattan/8d49c0d69bf1853a42bda495fe542b8f/raw/276fd3cc126b4084c7101892349ee5dd9a25e9d2/build_source_error.txt

[bodono]

I don't understand what is happening, I haven't seen that issue before. I would make sure to clear away any partially compiled artefacts and try again. You can also just try compiling the plain C version using make to ensure that runs successfully.

[imh]

I'm seeing the same issue now (M2 chip on Ventura). @ktattan, did you ever find a solution?

(edit a few months later: I'm also getting this issue on an M1 machine I have. ARM may be the common denominator?)