Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: scipy.ndimage.value_indices returns empty dict for intc/uintc dtype on Windows #19423

Closed
kmaehashi opened this issue Oct 22, 2023 · 1 comment · Fixed by #20644
Closed

BUG: scipy.ndimage.value_indices returns empty dict for intc/uintc dtype on Windows #19423

kmaehashi opened this issue Oct 22, 2023 · 1 comment · Fixed by #20644
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.ndimage
Milestone
1.13.1

Comments

@kmaehashi
Copy link

Describe your issue.

scipy.ndimage.value_indices returns an empty dict on Windows, if the input is intc/uintc dtype.

Discovered in cupy/cupy#7947.

Reproducing Code Example

import numpy as xp
import scipy as xps


image = (xp.random.rand(10,12) * 4).astype(xp.int32)
val_idx = xps.ndimage.value_indices(image)
print(val_idx.keys())  # dict_keys([0, 1, 2, 3])

image = (xp.random.rand(10,12) * 4).astype(xp.intc)
val_idx = xps.ndimage.value_indices(image)
print(val_idx.keys())  # dict_keys([])   <- UNEXPECTED

Error message

n/a

SciPy/NumPy/Python version and system information

1.11.3 1.26.1 sys.version_info(major=3, minor=12, micro=0, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /c/opt/64/include
    lib directory: /c/opt/64/lib
    name: openblas
    openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
      NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= SKYLAKEX MAX_THREADS=2
    pc file directory: c:/opt/64/lib/pkgconfig
    version: 0.3.21.dev
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /c/opt/64/include
    lib directory: /c/opt/64/lib
    name: openblas
    openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
      NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= SKYLAKEX MAX_THREADS=2
    pc file directory: c:/opt/64/lib/pkgconfig
    version: 0.3.21.dev
  pybind11:
    detection method: config-tool
    include directory: unknown
    name: pybind11
    version: 2.11.0
Compilers:
  c:
    commands: cc
    linker: ld.bfd
    name: gcc
    version: 10.3.0
  c++:
    commands: c++
    linker: ld.bfd
    name: gcc
    version: 10.3.0
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 0.29.36
  fortran:
    commands: gfortran
    linker: ld.bfd
    name: gcc
    version: 10.3.0
  pythran:
    include directory: C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-r6tmzm88\overlay\Lib\site-packages/pythran
    version: 0.14.0
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: windows
  cross-compiled: false
  host:
    cpu: x86_64
    endian: little
    family: x86_64
    system: windows
Python Information:
  path: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-q67zaj2m\cp312-win_amd64\build\venv\Scripts\python.exe
  version: '3.12'
@kmaehashi kmaehashi added the defect A clear bug or issue that prevents SciPy from being installed or used as expected label Oct 22, 2023
@kmaehashi kmaehashi mentioned this issue Oct 23, 2023
Merged
@neilflood
Copy link
Contributor

Thanks @kmaehashi

I am not sure if this is a problem with value_indices or not. The value_indices C code contains a few switch statements like this

        switch(arrType) {
        case NPY_INT8:   CASE_VALUEINDICES_SET_MINMAX(npy_int8); break;
        case NPY_UINT8:  CASE_VALUEINDICES_SET_MINMAX(npy_uint8); break;
        case NPY_INT16:  CASE_VALUEINDICES_SET_MINMAX(npy_int16); break;
        case NPY_UINT16: CASE_VALUEINDICES_SET_MINMAX(npy_uint16); break;
        case NPY_INT32:  CASE_VALUEINDICES_SET_MINMAX(npy_int32); break;
        case NPY_UINT32: CASE_VALUEINDICES_SET_MINMAX(npy_uint32); break;
        case NPY_INT64:  CASE_VALUEINDICES_SET_MINMAX(npy_int64); break;
        case NPY_UINT64: CASE_VALUEINDICES_SET_MINMAX(npy_uint64); break;
        }

where arrType comes from PyArray_TYPE(array). So, at first it would seem that this does not deal with intc/uintc types. However, reading the numpy source code, it appears that these are just aliases for the explicitly sized types. In npy_common.h, there are a series of #defines such as

#elif NPY_BITSOF_INT == 32
#ifndef NPY_INT32
#define NPY_INT32 NPY_INT
#define NPY_UINT32 NPY_UINT

with many cases for different machine type sizes.

At the Python level, on Linux at least, numpy.uintc seems to be identical to numpy.uint32, e.g.

>>> import numpy
>>> a = numpy.zeros(5, dtype=numpy.uintc)
>>> a.dtype
dtype('uint32')

So, I am not sure what would be happening in the cupy code (on Windows) that allows these to be distinct.

I am a bit puzzled by this. I am not sure whether the problem is in value_indices, numpy, or cupy. Do you have any thoughts?

@tylerjereddy tylerjereddy added this to the 1.14.0 milestone May 5, 2024
tylerjereddy added a commit to tylerjereddy/scipy that referenced this issue May 5, 2024
@tylerjereddy
BUG: value_indices unfixed types
266330b 
* Fixes scipygh-19423

* Add a few more `case` statements to account for
the (i.e., Windows) data types that don't have a fixed
width, and add a regression test.

[skip circle] [skip cirrus]
@tylerjereddy tylerjereddy mentioned this issue May 5, 2024
Merged
tylerjereddy added a commit to tylerjereddy/scipy that referenced this issue May 5, 2024
@tylerjereddy
BUG: value_indices unfixed types
40fc88b 
* Fixes scipygh-19423

* Add a few more `case` statements to account for
the (i.e., Windows) data types that don't have a fixed
width, and add a regression test.

[skip circle] [skip cirrus]
tylerjereddy added a commit to tylerjereddy/scipy that referenced this issue May 5, 2024
@tylerjereddy
BUG: value_indices unfixed types
e1a4c36 
* Fixes scipygh-19423

* Add a few more `case` statements to account for
the (i.e., Windows) data types that don't have a fixed
width, and add a regression test.

[skip circle] [skip cirrus]
@lucascolley lucascolley modified the milestones: 1.14.0, 1.13.1 May 15, 2024
tylerjereddy added a commit to tylerjereddy/scipy that referenced this issue May 15, 2024
@tylerjereddy
BUG: value_indices unfixed types
5c770e8 
* Fixes scipygh-19423

* Add a few more `case` statements to account for
the (i.e., Windows) data types that don't have a fixed
width, and add a regression test.

[skip circle] [skip cirrus]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.ndimage
Projects
None yet
Milestone
1.13.1
Development

Successfully merging a pull request may close this issue.

BUG: ndimage.value_indices: deal with unfixed types tylerjereddy/scipy
5 participants
@kmaehashi @tylerjereddy @neilflood @lucascolley @j-bowhay