[flang][driver] support symbol visibility flag

[h-vetinari]

While working on upgrading the SciPyb-builds in conda-forge to use flang 18, I'm running into the fact that flang got a lot stricter about unknown flags (which previously were just ignored, presumably).

Some of this probably needs to be fixed on the meson side (which generates the actual compiler invocations from the build files), but I noticed one case where there's both arguably missing functionality, and the warnings don't make sense, namely -fvisibility=, which clang documents as:

Set the default symbol visibility for all global definitions. must be ‘default’, ‘hidden’, ‘internal’ or ‘protected’.

SciPy specifies hidden visibility for some targets, and when this gets hit by flang (on windows), the result at first is

[962/1475] Compiling Fortran object scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj
FAILED: scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj 
"flang-new" "-Iscipy\linalg\_interpolative.cp311-win_amd64.pyd.p" "-Iscipy\linalg" "-I..\scipy\linalg" "-I..\..\_h_env\Lib\site-packages\numpy\_core\include" "-I..\..\_h_env\Lib\site-packages\numpy\f2py\src" "-Iscipy\lib_fortranobject.a.p" "-I%PREFIX%\Include" "-fvisibility=hidden" "-DNDEBUG" "-D_FILE_OFFSET_BITS=64" "-O3" "-D_CRT_SECURE_NO_WARNINGS" "-D_MT" "-D_DLL" "--target=x86_64-pc-windows-msvc" "-DMS_WIN64=" "-module-dir" "scipy\linalg\_interpolative.cp311-win_amd64.pyd.p" -o scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj "-c" ../scipy/linalg/src/id_dist/src/dfft.f
flang-new: error: unknown argument '-fvisibility=hidden'; did you mean '-Xclang -fvisibility=hidden'?

Using the suggested -Xclang -fvisibility=hidden, I then get

[962/1475] Compiling Fortran object scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj
FAILED: scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj 
"flang-new" "-Iscipy\linalg\_interpolative.cp311-win_amd64.pyd.p" "-Iscipy\linalg" "-I..\scipy\linalg" "-I..\..\_h_env\Lib\site-packages\numpy\_core\include" "-I..\..\_h_env\Lib\site-packages\numpy\f2py\src" "-Iscipy\lib_fortranobject.a.p" "-I%PREFIX%\Include" "-Xclang" "-fvisibility=hidden" "-DNDEBUG" "-D_FILE_OFFSET_BITS=64" "-O3" "-D_CRT_SECURE_NO_WARNINGS" "-D_MT" "-D_DLL" "--target=x86_64-pc-windows-msvc" "-DMS_WIN64=" "-module-dir" "scipy\linalg\_interpolative.cp311-win_amd64.pyd.p" -o scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj "-c" ../scipy/linalg/src/id_dist/src/dfft.f
flang-new: error: unknown argument '-Xclang'; did you mean '-Xflang'?

Finally, using -Xflang -fvisibility=hidden, the result is

[962/1475] Compiling Fortran object scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj
FAILED: scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj 
"flang-new" "-Iscipy\linalg\_interpolative.cp311-win_amd64.pyd.p" "-Iscipy\linalg" "-I..\scipy\linalg" "-I..\..\_h_env\Lib\site-packages\numpy\_core\include" "-I..\..\_h_env\Lib\site-packages\numpy\f2py\src" "-Iscipy\lib_fortranobject.a.p" "-I%PREFIX%\Include" "-Xflang" "-fvisibility=hidden" "-DNDEBUG" "-D_FILE_OFFSET_BITS=64" "-O3" "-D_CRT_SECURE_NO_WARNINGS" "-D_MT" "-D_DLL" "--target=x86_64-pc-windows-msvc" "-DMS_WIN64=" "-module-dir" "scipy\linalg\_interpolative.cp311-win_amd64.pyd.p" -o scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj "-c" ../scipy/linalg/src/id_dist/src/dfft.f
error: unknown argument: '-fvisibility=hidden'

At the very least the warnings are misleading. But setting symbol visibility should probably be supported in general?

[llvmbot]

@llvm/issue-subscribers-flang-driver

Author: None (h-vetinari)

While working on upgrading the SciPyb-builds in conda-forge to use flang 18, I'm running into the fact that flang got a lot stricter about unknown flags (which previously were just ignored, presumably).

Some of this probably needs to be fixed on the meson side (which generates the actual compiler invocations from the build files), but I noticed one case where there's both arguably missing functionality, and the warnings don't make sense, namely -fvisibility=, which clang documents as:
> Set the default symbol visibility for all global definitions. <arg> must be ‘default’, ‘hidden’, ‘internal’ or ‘protected’.

SciPy specifies hidden visibility for some targets, and when this gets hit by flang (on windows), the result at first is

[962/1475] Compiling Fortran object scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj
FAILED: scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj 
"flang-new" "-Iscipy\linalg\_interpolative.cp311-win_amd64.pyd.p" "-Iscipy\linalg" "-I..\scipy\linalg" "-I..\..\_h_env\Lib\site-packages\numpy\_core\include" "-I..\..\_h_env\Lib\site-packages\numpy\f2py\src" "-Iscipy\lib_fortranobject.a.p" "-I%PREFIX%\Include" "-fvisibility=hidden" "-DNDEBUG" "-D_FILE_OFFSET_BITS=64" "-O3" "-D_CRT_SECURE_NO_WARNINGS" "-D_MT" "-D_DLL" "--target=x86_64-pc-windows-msvc" "-DMS_WIN64=" "-module-dir" "scipy\linalg\_interpolative.cp311-win_amd64.pyd.p" -o scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj "-c" ../scipy/linalg/src/id_dist/src/dfft.f
flang-new: error: unknown argument '-fvisibility=hidden'; did you mean '-Xclang -fvisibility=hidden'?

Using the suggested -Xclang -fvisibility=hidden, I then get

[962/1475] Compiling Fortran object scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj
FAILED: scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj 
"flang-new" "-Iscipy\linalg\_interpolative.cp311-win_amd64.pyd.p" "-Iscipy\linalg" "-I..\scipy\linalg" "-I..\..\_h_env\Lib\site-packages\numpy\_core\include" "-I..\..\_h_env\Lib\site-packages\numpy\f2py\src" "-Iscipy\lib_fortranobject.a.p" "-I%PREFIX%\Include" "-Xclang" "-fvisibility=hidden" "-DNDEBUG" "-D_FILE_OFFSET_BITS=64" "-O3" "-D_CRT_SECURE_NO_WARNINGS" "-D_MT" "-D_DLL" "--target=x86_64-pc-windows-msvc" "-DMS_WIN64=" "-module-dir" "scipy\linalg\_interpolative.cp311-win_amd64.pyd.p" -o scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj "-c" ../scipy/linalg/src/id_dist/src/dfft.f
flang-new: error: unknown argument '-Xclang'; did you mean '-Xflang'?

Finally, using -Xflang -fvisibility=hidden, the result is

[962/1475] Compiling Fortran object scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj
FAILED: scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj 
"flang-new" "-Iscipy\linalg\_interpolative.cp311-win_amd64.pyd.p" "-Iscipy\linalg" "-I..\scipy\linalg" "-I..\..\_h_env\Lib\site-packages\numpy\_core\include" "-I..\..\_h_env\Lib\site-packages\numpy\f2py\src" "-Iscipy\lib_fortranobject.a.p" "-I%PREFIX%\Include" "-Xflang" "-fvisibility=hidden" "-DNDEBUG" "-D_FILE_OFFSET_BITS=64" "-O3" "-D_CRT_SECURE_NO_WARNINGS" "-D_MT" "-D_DLL" "--target=x86_64-pc-windows-msvc" "-DMS_WIN64=" "-module-dir" "scipy\linalg\_interpolative.cp311-win_amd64.pyd.p" -o scipy/linalg/_interpolative.cp311-win_amd64.pyd.p/src_id_dist_src_dfft.f.obj "-c" ../scipy/linalg/src/id_dist/src/dfft.f
error: unknown argument: '-fvisibility=hidden'

At the very least the warnings are misleading. But setting symbol visibility should probably be supported in general?