Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-threaded build sometimes fails to build Python client #49

Open
etremel opened this issue Aug 4, 2023 · 0 comments
Open

Multi-threaded build sometimes fails to build Python client #49

etremel opened this issue Aug 4, 2023 · 0 comments
Labels
bug Something isn't working

Comments

@etremel
Copy link
Contributor

etremel commented Aug 4, 2023

Sometimes when I attempt to do an incremental rebuild of Cascade after making some changes, the Python client (which I did not change) will fail to build with an error message like this:

[build] * Building wheel from sdist
[build] * Creating virtualenv isolated environment...
[build] * Installing packages in isolated environment... (setuptools >= 40.8.0, wheel)
[build]   Determining projects to restore...
[build]   Restored /home/edward/research/cascade/src/service/cs/CascadeClient.csproj (in 124 ms).
[build] error: [Errno 2] No such file or directory: 'derecho.cascade-1.0rc0/derecho/cascade/udl.py'
[build] 
[build] ERROR Backend subproccess exited when trying to invoke build_sdist
[build] gmake[2]: *** [src/service/python/CMakeFiles/member_client_py.dir/build.make:107: src/service/python/derecho/cascade/member_client.cpython-310-x86_64-linux-gnu.so] Error 1
[build] gmake[2]: *** Deleting file 'src/service/python/derecho/cascade/member_client.cpython-310-x86_64-linux-gnu.so'
[build] gmake[1]: *** [CMakeFiles/Makefile2:559: src/service/python/CMakeFiles/member_client_py.dir/all] Error 2

Apparently the Python package builder failed to find the file udl.py when it needed it. This file is located in the Python client's source directory (in src/service/python/derecho/cascade/udl.py), and the CMakeLists.txt in the python folder specifies that it should be copied to the binary (build) directory before attempting to build the Python client:

add_custom_command(TARGET member_client_py PRE_LINK
COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_CURRENT_SOURCE_DIR}/derecho ${CMAKE_CURRENT_BINARY_DIR}/derecho)

But for some reason this copy_directory command is not executed before attempting to build the member_client_py target, so the directory (and udl.py) is not found.

I believe this error happens due to a race condition in multi-threaded make/CMake, because I can reliably make it go away (and build successfully) by trying again with a single-threaded build, i.e. make -j1.

@etremel etremel added the bug Something isn't working label Aug 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant