BLD: test wheels and sdist

[neutrinoceros]

PR Summary

In #4905 it became apparent that our wheels are not tested, and while they're not broken now (as far as I know), it seems worth it to make building safer (albeit probably much longer). Let's see how long it even takes.

PR Checklist

• New features are documented, with docstrings and narrative docs
• Adds a test for any bugs fixed. Adds tests for new features.

[neutrinoceros]

I don't know exactly what I'm doing wrong but

• I cannot seem to make pytest's configuration (which lives in pyproject.toml) portable, so pytest crashes on collection because it doesn't know to ignore nose-only files
• I also couldn't make sdist testing work

I'm running out of time for today, but I'll try to come back soon. Any help is welcome.

[yut23]

I was able to reproduce the linux segfaults locally (inside the cibuildwheel containers), and the backtraces point to static initialization of std::locale objects. Maybe something is up with the locale settings in the containers?

They initially showed up for std::ctype<wchar_t>::do_scan_not() and std::ctype<wchar_t>::do_is() in _octree_raytracing.*.so, but I commented out the unused includes for <iostream>, <string>, and <fstream> in yt/utilities/lib/_octree_raytracing.hpp, which moved the problem to std::__cxx11::numpunct<char>::~numpunct() in ewah_bool_wrap.*.so.

Full backtraces:

_octree_raytracing.cpython-39-x86_64-linux-gnu.so

Program received signal SIGSEGV, Segmentation fault.
#0  0x00007facd39364fc in std::ctype<wchar_t>::do_is(wchar_t const*, wchar_t const*, unsigned short*) const ()                                                 
   from /tmp/tmp.urqU0Z4nXZ/venv/lib/python3.9/site-packages/yt/utilities/lib/_octree_raytracing.cpython-39-x86_64-linux-gnu.so                                
#1  0x00007facd392ce1a in void std::__facet_shims::__numpunct_fill_cache<char>(std::integral_constant<bool, false>, std::locale::facet const*, std::__numpunct_cache<char>*) () from /tmp/tmp.urqU0Z4nXZ/venv/lib/python3.9/site-packages/yt/utilities/lib/_octree_raytracing.cpython-39-x86_64-linux-gnu.so
#2  0x00007facd394aa67 in std::locale::facet::_M_sso_shim(std::locale::id const*) const ()                                                                     
   from /tmp/tmp.urqU0Z4nXZ/venv/lib/python3.9/site-packages/yt/utilities/lib/_octree_raytracing.cpython-39-x86_64-linux-gnu.so                                
#3  0x00007facd39230c6 in std::locale::_Impl::_M_install_facet(std::locale::id const*, std::locale::facet const*) ()                                           
   from /tmp/tmp.urqU0Z4nXZ/venv/lib/python3.9/site-packages/yt/utilities/lib/_octree_raytracing.cpython-39-x86_64-linux-gnu.so                                
#4  0x00007facd3923db1 in std::locale::_Impl::_Impl(unsigned long) ()
   from /tmp/tmp.urqU0Z4nXZ/venv/lib/python3.9/site-packages/yt/utilities/lib/_octree_raytracing.cpython-39-x86_64-linux-gnu.so                                
#5  0x00007facd39246c5 in std::locale::_S_initialize_once() ()
   from /tmp/tmp.urqU0Z4nXZ/venv/lib/python3.9/site-packages/yt/utilities/lib/_octree_raytracing.cpython-39-x86_64-linux-gnu.so                                
#6  0x00007facff07220b in __pthread_once_slow (once_control=0x7facd39fcdf8 <std::locale::_S_once>,                                                             
    init_routine=0x7facd39246b0 <std::locale::_S_initialize_once()>) at pthread_once.c:117                                                                     
#7  0x00007facd3924711 in std::locale::_S_initialize() ()
   from /tmp/tmp.urqU0Z4nXZ/venv/lib/python3.9/site-packages/yt/utilities/lib/_octree_raytracing.cpython-39-x86_64-linux-gnu.so                                
#8  0x00007facd3924773 in std::locale::locale() ()
   from /tmp/tmp.urqU0Z4nXZ/venv/lib/python3.9/site-packages/yt/utilities/lib/_octree_raytracing.cpython-39-x86_64-linux-gnu.so                                
#9  0x00007facd39218b3 in std::ios_base::Init::Init() ()
   from /tmp/tmp.urqU0Z4nXZ/venv/lib/python3.9/site-packages/yt/utilities/lib/_octree_raytracing.cpython-39-x86_64-linux-gnu.so                                
#10 0x00007facd3902610 in __static_initialization_and_destruction_0 (__priority=65535, __initialize_p=1) at yt/utilities/lib/_octree_raytracing.cpp:31059      
#11 _GLOBAL__sub_I__octree_raytracing.cpp(void) () at yt/utilities/lib/_octree_raytracing.cpp:31059                                                            
#12 0x00007facff2979c3 in call_init (env=0x17b8cd0, argv=0x7fff34d96098, argc=3, l=<optimized out>) at dl-init.c:82                                            
#13 _dl_init (main_map=main_map@entry=0x27f1430, argc=3, argv=0x7fff34d96098, env=0x17b8cd0) at dl-init.c:131                                                  
#14 0x00007facff29c59e in dl_open_worker (a=a@entry=0x7fff34d891b8) at dl-open.c:559                                                                           
#15 0x00007facff2977d4 in _dl_catch_error (objname=objname@entry=0x7fff34d891a8, errstring=errstring@entry=0x7fff34d891b0,                                     
    mallocedp=mallocedp@entry=0x7fff34d891a0, operate=operate@entry=0x7facff29c160 <dl_open_worker>, args=args@entry=0x7fff34d891b8) at dl-error.c:177         
#16 0x00007facff29bb8b in _dl_open (                                           
    file=0x7facd8dac550 "/tmp/tmp.urqU0Z4nXZ/venv/lib/python3.9/site-packages/yt/utilities/lib/_octree_raytracing.cpython-39-x86_64-linux-gnu.so",             
    mode=-2147483646, caller_dlopen=<optimized out>, nsid=-2, argc=3, argv=0x7fff34d96098, env=0x17b8cd0) at dl-open.c:649
...

ewah_bool_wrap.cpython-39-x86_64-linux-gnu.so

Program received signal SIGSEGV, Segmentation fault.
#0  0x00007fcd06471fb6 in _int_free () from /lib64/libc.so.6
#1  0x00007fccd390d2a1 in std::__cxx11::numpunct<char>::~numpunct() ()
   from /tmp/tmp.DPJcC1Zg2e/venv/lib/python3.9/site-packages/ewah_bool_utils/ewah_bool_wrap.cpython-39-x86_64-linux-gnu.so                                     
#2  0x00007fccd38b841f in std::locale::facet::_M_sso_shim(std::locale::id const*) const [clone .cold] ()                                                       
   from /tmp/tmp.DPJcC1Zg2e/venv/lib/python3.9/site-packages/ewah_bool_utils/ewah_bool_wrap.cpython-39-x86_64-linux-gnu.so                                     
#3  0x00007fccd3909246 in std::locale::_Impl::_M_install_facet(std::locale::id const*, std::locale::facet const*) ()                                           
   from /tmp/tmp.DPJcC1Zg2e/venv/lib/python3.9/site-packages/ewah_bool_utils/ewah_bool_wrap.cpython-39-x86_64-linux-gnu.so                                     
#4  0x00007fccd3909f31 in std::locale::_Impl::_Impl(unsigned long) ()
   from /tmp/tmp.DPJcC1Zg2e/venv/lib/python3.9/site-packages/ewah_bool_utils/ewah_bool_wrap.cpython-39-x86_64-linux-gnu.so                                     
#5  0x00007fccd390a845 in std::locale::_S_initialize_once() ()
   from /tmp/tmp.DPJcC1Zg2e/venv/lib/python3.9/site-packages/ewah_bool_utils/ewah_bool_wrap.cpython-39-x86_64-linux-gnu.so                                     
#6  0x00007fcd06ece20b in __pthread_once_slow () from /lib64/libpthread.so.0
#7  0x00007fccd390a891 in std::locale::_S_initialize() ()
   from /tmp/tmp.DPJcC1Zg2e/venv/lib/python3.9/site-packages/ewah_bool_utils/ewah_bool_wrap.cpython-39-x86_64-linux-gnu.so                                     
#8  0x00007fccd390a8f3 in std::locale::locale() ()
   from /tmp/tmp.DPJcC1Zg2e/venv/lib/python3.9/site-packages/ewah_bool_utils/ewah_bool_wrap.cpython-39-x86_64-linux-gnu.so                                     
#9  0x00007fccd3907a33 in std::ios_base::Init::Init() ()
   from /tmp/tmp.DPJcC1Zg2e/venv/lib/python3.9/site-packages/ewah_bool_utils/ewah_bool_wrap.cpython-39-x86_64-linux-gnu.so                                     
#10 0x00007fccd38b96f0 in __static_initialization_and_destruction_0 (__priority=65535, __initialize_p=1) at ewah_bool_utils/ewah_bool_wrap.cpp:63387           
#11 _GLOBAL__sub_I_ewah_bool_wrap.cpp(void) () at ewah_bool_utils/ewah_bool_wrap.cpp:63387                                                                     
#12 0x00007fcd070f39c3 in _dl_init_internal () from /lib64/ld-linux-x86-64.so.2
...

[neutrinoceros]

Thanks @yut23. I also dropped the unused includes you referred to here just in case, but I actually didn't realized there was a segfault at al... Maybe it's something that should get reported to cibuildwheel ? In any case, I think we need to figure out how to properly run pytest from outside the source repo first, so we have at least some evidence that the segfault is specifically a problem with the ubuntu container. Currently, sdist and macOS wheels are blocked by the same problem, so, progress !

[neutrinoceros]

Made some progress regarding configuration, but I'm still seeing collection errors where files named *_nose.py are still not ignored. I really don't get it.

[yut23]

It looks like --ignore just doesn't work with --pyargs: pytest-dev/pytest#3287

[yut23]

Alright, I've figured out why: pytest makes the --ignore and --ignore-glob paths absolute before checking against the test file. This prepends the current working directory (which is /tmp/tmp.<random>/test_cwd inside the container), so they will never match the installed files under /tmp/tmp.<random>/venv.

Relevant pytest code: https://github.com/pytest-dev/pytest/blob/cbf6bd9dd28d8464f7eebe32e51b398e2a5e3039/src/_pytest/main.py#L407-L416

[neutrinoceros]

Thank you so much for investigating this. Indeed, I never thought it could be a problem with pytest itself. Do you think this should be fixed upstream or is pytest working as intended ?

[neutrinoceros]

superseded by #4913