Possibly mistaken usage of the cupyx.jit.thrust.device execution policy in the JIT context?

[leofang]

When discussing the Thrust JIT support with the CCCL team, a question was raised regarding the usage of the jit.thrust.device policy in the test suite, ex:

cupy/tests/cupyx_tests/jit_tests/test_thrust.py

Lines 954 to 959 in be5d7f6

	jit.thrust.sort_by_key(
	jit.thrust.device,
	x_array.begin(),
	x_array.end(),
	y_array.begin(),
	)

According to what I learned, when using Thrust algorithms in a kernel the expectation should be to always use jit.thrust.seq instead. Before CUDA 12, using the device policy would actually silently launch additional kernels using the "old" CUDA dynamic parallelism, so-called CDP1 in the CUDA Programming Guide (although why it worked without us passing -rdc explicitly to the compiler remains to be seen). Worst of all, this is not something that a debugger or profiler can warn. Starting CUDA 12, it'd fall back to the seq policy and at some point it will become a compile-time error.

@jrhemstad @gevtushenko please correct me if the above summary is inaccurate 🙂 Thanks!

[kmaehashi]

cc/ @asi1024

[leofang]

Sorry, apparently I only typed the note without leaving the actual question... the question is: did you see issues when using the seq policy during development, thus deciding to switch to device in some, but not all, of the tests?

[jrhemstad]

See NVIDIA/cccl#1534 for a more thorough explanation of the background and Thrust's plans for the future.