Using AdaptiveCpp on Windows

Using AdaptiveCpp on Windows

It is possible to make the OpenMP and CUDA backend work on Windows, but currently requires going through some manual steps. Also NOTE: this requires a Clang Plugin and Clang CUDA, which both are not officially supported on Windows. Thus this is considered highly experimental. We are providing compatible LLVM 11.1 and Boost 1.75 binaries to ease the process. The binaries require the VC++ 2019 redistributable to be installed (which you most likely already have).

Dependencies (non-exhaustive)

• Ninja
• LLVM (patched with D69322, included in LLVM 12+, prebuilt 11.1.0 binary)
• Boost (built for Clang toolchain, tested with 1.70 and 1.75, prebuilt 1.75 binary)
• Python 3

AdaptiveCpp depends on Boost and mostly uses Clang for compilation. Thus these are our main dependencies.

Getting the LLVM with plugin support

LLVM currently does not officially support Clang plugins on Windows. As AdaptiveCpp requires the use of a Clang plugin for compiling for CUDA, LLVM has to be built manually:

• Get the LLVM source (e.g. git clone https://github.com/llvm/llvm-project)
• Apply the required patch
• Open Visual Studio Command (usually want to use x64 Native Tools)
  • if using Ninja, execute set CC=cl and set CXX=cl to use VS compiler
• Configure LLVM with CMake with enabled RTTI and plugin support -DLLVM_EXPORT_SYMBOLS_FOR_PLUGINS=ON -DLLVM_ENABLE_RTTI=ON
  • See LLVM docs for available options.
• Build and install LLVM using ninja and ninja install

A full install might look like this:

git clone https://github.com/llvm/llvm-project llvm
cd llvm
@rem if not using LLVM 12+ execute the next line:
@rem git am --3way --ignore-space-change --keep-cr "D69322.diff"
@rem check whether there were merge conflicts.. and if so, solve them
mkdir build && cd build
set CC=cl
set CXX=cl
cmake ../llvm -G Ninja -DCMAKE_INSTALL_PREFIX=<INSTALL_PREFIX> -DLLVM_ENABLE_PROJECTS="clang;lld" -DLLVM_EXPORT_SYMBOLS_FOR_PLUGINS=ON -DLLVM_ENABLE_RTTI=ON -DLLVM_TARGETS_TO_BUILD=X86 -DCMAKE_BUILD_TYPE=RelWithDebInfo
ninja
ninja install
@rem the clang lib required for the AdaptiveCpp plugin is not installed, do it manually
copy RelWithDebInfo\lib\clang.lib <INSTALL_PREFIX>/lib

You might want to add your <INSTALL PREFIX> to your PATH variable. Otherwise, all steps below require you to actively specify the full path to clang.

Building the OpenMP runtime fails for LLVM 11 and probably 12, if using MSVC, but seems to be fixed on main in this commit. Building the runtime separately, using Clang might work, otherwise, it also is possible to just copy libomp.dll and libomp.lib from a binary distribution of LLVM to <INSTALL_PREFIX>/bin and <INSTALL_PREFIX>/lib folders respectively.

Getting Boost built with Clang

Get a recent boost source archive, extract and open a console in the extracted folder. Then execute:

bootstrap.bat
.\b2 toolset=clang-win address-model=64 variant=release --build-type=complete stage

For faster building, you might want to add --with-fiber --with-context --with-test.

Building AdaptiveCpp

Now that we have all dependencies installed, we can go about building AdaptiveCpp, just as one would for any other platform.

Change over to your AdaptiveCpp clone. Note, AdaptiveCpp is currently not ready to work with multi-config build tools like the Visual Studio generators, thus ninja is the recommended build tool.

mkdir build && cd build
# Configure with CMake, standard mechanisms for issues apply
cmake .. -G Ninja -DCMAKE_C_COMPILER=clang.exe -DCMAKE_CXX_COMPILER=clang++.exe -DBOOST_ROOT=<BOOST_DIR> -DLLVM_DIR=<INSTALL_PREFIX>/lib/cmake/llvm -DCMAKE_BUILD_TYPE=RelWithDebInfo
ninja
ninja install # might want to add a -DCMAKE_INSTALL_PREFIX=<INSTALL_PREFIX> to the CMake command above to install locally..

As Windows has no idea about -rpath, to use AdaptiveCpp in a project, you either have to copy hipSYCL-rt.dll next to your executable or ensure that <INSTALL_PREFIX>/bin is in your PATH.

Pitfalls

python.exe not found or Microsoft Store always opens when trying to build with AdaptiveCpp

Ensure your python installation is activated (in case of virtual environments or Anaconda), your python.exe is in your system path.

If the MS Store always opens, when executing python, remove the dummy python.exes as explained here.

Linker error with something like iterator level

Make sure all your libraries are built with the same build type (debug vs. release).

Something doesn't work

This is super likely to happen and something that worked today might stop working tomorrow, as there's little commitment to supporting LLVM Plugins and Clang CUDA on Windows. So everything is experimental!

Something is not found or there is a parsing error

Windows is special when it comes to paths, so check, whether you have any whitespaces in a path and if so, try adding quotes around the path. Also, avoid using backslashes \ in paths and instead just replace them with forward-slashes /.

Known issues

Itanium and MS-ABI mismatch

CUDA uses the Itanium ABI on Windows (regardless of Clang / NVCC). As host code still uses MS-ABI there, mismatches regarding struct/class layout are possible. One such example is the empty bases issue, which was fixed for sycl::accessor in #623. There isn't much we can do about this but hope that MSVC changes it's ABI sooner than later or Clang adds a warning for these cases.

MS-STL versions incompatible with Clang CUDA

Due to a version check introduced in the MS-STL, Clang CUDA cannot be used with a current version of the MS-STL. Multiple workarounds are possible: use a version without that change (probably broken since 16.10), specify -D_ALLOW_COMPILER_AND_STL_VERSION_MISMATCH (unverified) or apply the proposed patch.