Deprecated NumPy syntax : needs new windows binary?

[tieneupin]

I am running ImageD11 1.9.8 on a Windows 10 OS, and encountered the following issue with the columnfile() module/function:

C:\Anaconda3\envs\imaged11\lib\site-packages\ImageD11\columnfile.py in __init__(self, filename, new)
    152         self.nrows = 0
    153         if not new:
--> 154             self.readfile(filename)
    155 
    156     def get_bigarray(self):

C:\Anaconda3\envs\imaged11\lib\site-packages\ImageD11\columnfile.py in readfile(self, filename)
    311                 nrows = len(raw)-i-1 # skip the last row
    312                 last = len(raw)-1
--> 313             cols = [ np.empty( nrows , np.float ) for _ in range(len(row0))]
    314             fillcols( raw[i:last], cols )
    315             self.__data=cols

C:\Anaconda3\envs\imaged11\lib\site-packages\ImageD11\columnfile.py in <listcomp>(.0)
    311                 nrows = len(raw)-i-1 # skip the last row
    312                 last = len(raw)-1
--> 313             cols = [ np.empty( nrows , np.float ) for _ in range(len(row0))]
    314             fillcols( raw[i:last], cols )
    315             self.__data=cols

C:\Anaconda3\envs\imaged11\lib\site-packages\numpy\__init__.py in __getattr__(attr)
    303 
    304         if attr in __former_attrs__:
--> 305             raise AttributeError(__former_attrs__[attr])
    306 
    307         # Importing Tester requires importing all of UnitTest which is not a

AttributeError: module 'numpy' has no attribute 'float'.
`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

If I have understood the error message correctly, ImageD11 is written using deprecated NumPy syntax. Would it be possible to consider updating the syntax to make it compatible with newer versions of NumPy (I am running version 1.24.4)? This should involve replacing np.float, np.int and other such terms with the up-to-date variant (np.float_, np.int_, etc.).

[jonwright]

Sure. This was fixed in git a little while ago.

Is the underlying problem that you need a binary release to be uploaded on pypi so that you can install on windows?

[tieneupin]

Hi Jon,

Thanks for the quick response! I am still learning the jargon for programming, but yes, think a new Windows binary might be needed. For context, I installed this version of ImageD11 (1.9.8) via pip about 1 month ago, as I'm not yet proficient enough to update my packages directly with git. The version of the scripts associated with the pip installation would need to be updated, in that case.

[jonwright]

If you are in a hurry, you can install the C compiler (https://wiki.python.org/moin/WindowsCompilers) and then try running the pip install again (there is a v2.0.0 source release on pypi now). Something like:

python -m pip install ImageD11 --no-build-isolation --upgrade --no-deps

The --no-build-isolation means it should compile against the version you have.

The --upgrade means to upgrade ImageD11

The --no-deps means to please leave the rest of your environment alone.

Otherwise let me know which python version you are using and I can see if I can make a wheel for it here.

[tieneupin]

Thanks for your quick response, Jon! I've followed the pip installation instructions you've given, and can confirm that ImageD11 v2.0.0 was successfully installed.

With regards to Python version, I am using Python 3.8.18. I have previously tried installing ImageD11 1.9.8 in environments running on newer versions of Python (3.9 - 3.11), but the pip installation failed due to issues with building a wheel. Python 3.8 is the only one in which the pip installation ran successfully.

[jonwright]

New wheels should be on the way soon, thanks to @t20100 and #191

[t20100]

Once all dependencies are available for python3.12, to enable tests for Python3.12 when releasing wheels, remove *cp312-* here:

ImageD11/.github/workflows/release.yml

Lines 56 to 58 in 24c372f

	# Skip tests for 32bits and emulated architectures, arm64 macos and on Windows
	# Skip cp312 tests for now: not all dependencies are available.
	CIBW_TEST_SKIP: "*cp312-* *-*linux_i686 *-*linux_{aarch64,ppc64le,s390x} *-macosx_arm64 *-macosx_universal2:arm64 *-win32 *-win_amd64"

[tieneupin]

That's great news, thanks for working on this matter! I'd be happy to test the new wheels for Windows once they're released.

[jonwright]

They should be up on pip as version 2.0.1 now. Please let me know if you have a problem to install them - otherwise I am hoping this is fixed now?
Thanks!

[tieneupin]

Hi Jon, thanks for notifying me about the version 2.0.1 being available now!

I've given it a test on my Windows 10 desktop, but it has unfortunately failed to install thus far. I'll copy out the steps I've taken:

1. Created a fresh Python environment to install ImageD11 into:

conda create -n imaged11test python=3.12

# PACKAGES INCLUDED UPON INSTALLATION
  bzip2              pkgs/main/win-64::bzip2-1.0.8-he774522_0
  ca-certificates    pkgs/main/win-64::ca-certificates-2023.08.22-haa95532_0
  expat              pkgs/main/win-64::expat-2.5.0-hd77b12b_0
  libffi             pkgs/main/win-64::libffi-3.4.4-hd77b12b_0
  openssl            pkgs/main/win-64::openssl-3.0.12-h2bbff1b_0
  pip                pkgs/main/win-64::pip-23.3.1-py312haa95532_0
  python             pkgs/main/win-64::python-3.12.0-h1d929f7_0
  setuptools         pkgs/main/win-64::setuptools-68.0.0-py312haa95532_0
  sqlite             pkgs/main/win-64::sqlite-3.41.2-h2bbff1b_0
  tk                 pkgs/main/win-64::tk-8.6.12-h2bbff1b_0
  tzdata             pkgs/main/noarch::tzdata-2023c-h04d1e81_0
  vc                 pkgs/main/win-64::vc-14.2-h21ff451_1
  vs2015_runtime     pkgs/main/win-64::vs2015_runtime-14.27.29016-h5e58377_2
  wheel              pkgs/main/win-64::wheel-0.41.2-py312haa95532_0
  xz                 pkgs/main/win-64::xz-5.4.5-h8cc25b3_0
  zlib               pkgs/main/win-64::zlib-1.2.13-h8cc25b3_0

2. Ran pip install in this fresh environment, which returned the following error message:

pip install imaged11==2.0.1

# COMMAND LINE OUTPUT
Collecting imaged11==2.0.1
  Downloading ImageD11-2.0.1-cp312-cp312-win_amd64.whl.metadata (4.4 kB)
Collecting six (from imaged11==2.0.1)
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting numpy (from imaged11==2.0.1)
  Downloading numpy-1.26.2-cp312-cp312-win_amd64.whl.metadata (61 kB)
     ---------------------------------------- 61.2/61.2 kB 653.5 kB/s eta 0:00:00
Requirement already satisfied: setuptools in c:\anaconda3\envs\imaged11test\lib\site-packages (from imaged11==2.0.1) (68.0.0)
Collecting xfab>=0.0.4 (from imaged11==2.0.1)
  Downloading xfab-0.0.6-py2.py3-none-any.whl (68 kB)
     ---------------------------------------- 68.6/68.6 kB 3.6 MB/s eta 0:00:00
Collecting matplotlib (from imaged11==2.0.1)
  Downloading matplotlib-3.8.2-cp312-cp312-win_amd64.whl.metadata (5.9 kB)
Collecting pyopengltk (from imaged11==2.0.1)
  Downloading pyopengltk-0.0.4-py2.py3-none-any.whl (11 kB)
Collecting scipy (from imaged11==2.0.1)
  Downloading scipy-1.11.4-cp312-cp312-win_amd64.whl.metadata (60 kB)
     ---------------------------------------- 60.4/60.4 kB 3.3 MB/s eta 0:00:00
Collecting h5py (from imaged11==2.0.1)
  Downloading h5py-3.10.0-cp312-cp312-win_amd64.whl.metadata (2.5 kB)
Collecting pyyaml (from imaged11==2.0.1)
  Downloading PyYAML-6.0.1-cp312-cp312-win_amd64.whl.metadata (2.1 kB)
Collecting pytest (from imaged11==2.0.1)
  Downloading pytest-7.4.3-py3-none-any.whl.metadata (7.9 kB)
Collecting bslz4-to-sparse (from imaged11==2.0.1)
  Downloading bslz4_to_sparse-0.0.15-cp312-cp312-win_amd64.whl.metadata (533 bytes)
Collecting fast-histogram (from imaged11==2.0.1)
  Downloading fast_histogram-0.12-cp38-abi3-win_amd64.whl.metadata (6.8 kB)
Collecting scikit-image (from imaged11==2.0.1)
  Downloading scikit_image-0.22.0-cp312-cp312-win_amd64.whl.metadata (13 kB)
Collecting tqdm (from imaged11==2.0.1)
  Downloading tqdm-4.66.1-py3-none-any.whl.metadata (57 kB)
     ---------------------------------------- 57.6/57.6 kB 3.0 MB/s eta 0:00:00
Collecting hdf5plugin (from imaged11==2.0.1)
  Downloading hdf5plugin-4.3.0-py3-none-win_amd64.whl.metadata (2.8 kB)
Collecting numba (from imaged11==2.0.1)
  Downloading numba-0.58.1.tar.gz (2.6 MB)
     ---------------------------------------- 2.6/2.6 MB 11.2 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\Users\Tien Eu Pin\AppData\Local\Temp\pip-install-435pb5lb\numba_01d2166f73a347b197adf64e47acb803\setup.py", line 51, in <module>
          _guard_py_ver()
        File "C:\Users\Tien Eu Pin\AppData\Local\Temp\pip-install-435pb5lb\numba_01d2166f73a347b197adf64e47acb803\setup.py", line 48, in _guard_py_ver
          raise RuntimeError(msg.format(cur_py, min_py, max_py))
      RuntimeError: Cannot install on Python version 3.12.0; only versions >=3.8,<3.12 are supported.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

One other thing to note would be that I already have Visual Studio C++ Build Tools installed separately on my computer. Input on what has happened above would be much appreciated. Are there other arguments that I should have included but didn't?

[jonwright]

Sorry for the mess. The problem is the dependency on numba that is not ready for 3.12 yet (numba/numba#9197). ImageD11 should install on 3.8 up to 3.11, and on 3.12 as soon as numba release their next version. You can install it now with a --no-deps flag but then you need to install dependencies manually.

[jonwright]

They are aiming at 11-Dec https://github.com/numba/numba/milestone/73, so it seems reasonable to just wait for that to arrive.

[tieneupin]

Thanks for the quick reply! I've downgraded to Python 3.11, and can confirm that the installation has progressed successfully through to completion. Thanks for resolving this installation issue!

[jonwright]

Note to self: we still need a new windows binary build for numba 3.12 to go to pypi... almost there now.