BLD: fix use of non-default interpreter, improve f2py handling #20612
Conversation
rgommers
added
enhancement
github-actions
bot
added
scipy.stats
scipy.special
scipy.io
scipy.sparse.linalg
scipy.linalg
scipy.sparse
scipy.optimize
scipy.interpolate
scipy.integrate
scipy._lib
Build issues
rgommers
removed
scipy.stats
scipy.special
scipy.io
scipy.sparse.linalg
scipy.linalg
scipy.sparse
scipy.optimize
scipy.interpolate
scipy.integrate
scipy._lib
Cython
rgommers
force-pushed
the
fix-nondefault-interpreter
branch
from
3a33dab
to
ca35f3f
Compare
This guards against issues with trying to use the `python3` interpreter for anything that actually requires the Python interpreter we are building for. In such a build config with multiple interpreters, we need to be careful not to pick up the wrong one when invoking `custom_target` or `run_command` in meson.build files.
This removes all remaining invocations of `py3` in meson.build files. Not invoking the Python interpreter we're building for at all helps with cross-compilation, since that interpreter typically isn't able to run without the use of an emulator like QEMU.
This private namespace was already deprecated in the `__getattr__` in `interpolate/__init__.py`, however since the f2py-generated extension module `dfitpack` had to be moved to `_dfitpack`, this .py file has to be added instead. Do so with the exact same pattern as `fitpack.py` and co.
The `meson` executable isn't guaranteed to be installed for the same Python interpreter as we're running the tests for (this may for example happen in distro setups when we're testing a non-default Python). Explicitly passing the Python interpreter we're testing for to Meson via a native file should be a robust solution.
rgommers
force-pushed
the
fix-nondefault-interpreter
branch
from
ca35f3f
to
3c37112
Compare
rgommers
requested review from
larsoner,
andyfaff,
person142,
steppi,
perimosocordiae,
ev-br and
ilayn
as code owners
|
Okay, this is happy now! This is a fairly significant build change and fixes some nontrivial issues, so I'd like to get it merged soon so it has a few weeks in |
|
@thalassemia maybe one for you to review? |
scipy/stats/meson.build
Outdated
| py3.extension_module('_mvn', | ||
| [mvn_module, 'mvndst.f'], | ||
| #[mvn_module, 'mvndst.f'], |
There was a problem hiding this comment.
Good catch. And thanks for the review!
|
|
||
| - name: Build wheel and install | ||
| run: | | ||
| python3.11 -m build -wnx -Csetup-args=-Dblas=blas-atlas -Csetup-args=-Dlapack=lapack-atlas -Ccompile-args=-j2 |
There was a problem hiding this comment.
Are we using build -x because pyproject.toml requires numpy>=2.0.0rc1?
There was a problem hiding this comment.
Yes indeed. -n and -x almost always go together, because usually you're skipping build isolation to target an environment that is somehow different from what you'd get from build isolation.
Addresses review comments on PR 20612. [skip ci]
|
The only change post-review-approval is a tweak to code comments, so I'll hit the green button here. Thanks again @thalassemia |
|
I got the following errors on scipy/meson.build:156:15: ERROR: Command `/Users/czgdp1807/mambaforge/envs/scipy-dev-clang-15/bin/f2py -v` failed with status 1.Program f2py found: YES (/Users/czgdp1807/mambaforge/envs/scipy-dev-clang-15/bin/f2py)
Running command: /Users/czgdp1807/mambaforge/envs/scipy-dev-clang-15/bin/f2py -v
--- stdout ---
--- stderr ---
Traceback (most recent call last):
File "/Users/czgdp1807/mambaforge/envs/scipy-dev-clang-15/bin/f2py", line 5, in <module>
from numpy.f2py.f2py2e import main
File "/Users/czgdp1807/mambaforge/envs/scipy-dev-clang-15/lib/python3.11/site-packages/numpy/f2py/__init__.py", line 14, in <module>
import subprocess
File "/Users/czgdp1807/mambaforge/envs/scipy-dev-clang-15/lib/python3.11/subprocess.py", line 49, in <module>
import signal
File "/Users/czgdp1807/Quansight/scipy/scipy/signal/__init__.py", line 307, in <module>
from . import _sigtools, windows
File "/Users/czgdp1807/Quansight/scipy/scipy/signal/windows/__init__.py", line 42, in <module>
from ._windows import *
File "/Users/czgdp1807/Quansight/scipy/scipy/signal/windows/_windows.py", line 7, in <module>
from scipy import linalg, special, fft as sp_fft
ModuleNotFoundError: No module named 'scipy'After reverting this PR locally the above errors are removed. |
|
I'm seeing the same error as @czgdp1807 when building SciPy on |
|
The traceback indicates that from within the stdlib's |
This PR does the following:
python3.10(default, getspython3alias) andpython3.11in Ubuntu 22.04f2pyto a generator, making the code inmeson.buildfiles much more concise.pyffiles are consistently named, matching their import name (modulo an underscore). This is necessary to support thegeneratorusage.interpolate.dfitpack, which were still missing.test_extending.pytest setup so it works when themesonthat is found isn't in the active environment but the defaultpython3one.custom_target(..., command: [py3, ...]). This will improve the cross-compilation experience, since QEMU or similar is no longer needed to run the cross interpreter.f2pyrather thanpython -m f2py.f2pygenerates code that does not depend on the platform or Python version it's installed for (with one single exception for long-double on some platforms - when the code hits_selected_real_kind_func- that we should not be affected by). This allows runningf2pyas a regular code generator, just likecython.Closes gh-20535.