You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
2024-03-28T08:00:31,792 [DEBUG] W-9000-embeddings_1.1 org.pytorch.serve.wlm.WorkerLifeCycle - launcherAvailable cmdline: [python, -m, torch.backends.xeon.run_cpu, --use_logical_core, --enable_jemalloc, --no_python, hostname]
2024-03-28T08:00:31,821 [WARN ] W-9000-embeddings_1.1 org.pytorch.serve.wlm.WorkerLifeCycle - torch.backends.xeon.run_cpu is not available. Proceeding without worker core pinning. For better performance, please make sure torch.backends.xeon.run_cpu is available.
launcherAvailable cmdline is running outside the virtual env of torchserve like the below command which fails
python -m torch.backends.xeon.run_cpu --use_logical_core --no_python hostname
/usr/bin/python: Error while finding module specification for 'torch.backends.xeon.run_cpu' (ModuleNotFoundError: No module named 'torch')
Instead using the virtual env where the torchserve is running works
[rocky@hostname ~]$ source /opt/venv-torchserve/bin/activate
(venv-torchserve) [rocky@hostname ~]$ python -m torch.backends.xeon.run_cpu --use_logical_core --no_python hostname
2024-03-28 08:04:22,935 - __main__ - INFO - Use TCMalloc memory allocator
2024-03-28 08:04:22,936 - __main__ - INFO - OMP_NUM_THREADS=4
2024-03-28 08:04:22,936 - __main__ - INFO - Using Intel OpenMP
2024-03-28 08:04:22,936 - __main__ - INFO - KMP_BLOCKTIME=1
2024-03-28 08:04:22,936 - __main__ - INFO - LD_PRELOAD=/opt/venv-torchserve/lib/libiomp5.so:/usr/lib64/libtcmalloc.so
2024-03-28 08:04:22,936 - __main__ - INFO - numactl -C 0-3 -m 0 hostname
Installation instructions
Torchserve 0.10.0 installed in a virtual env with pip
馃悰 Describe the bug
cpu launcher doesn't work
Error logs
launcherAvailable cmdline is running outside the virtual env of torchserve like the below command which fails
Instead using the virtual env where the torchserve is running works
Installation instructions
Torchserve 0.10.0 installed in a virtual env with pip
Model Packaging
not applicable
config.properties
Versions
TorchServe 0.10.0
Python 3.9
Repro instructions
Start a torchserve from a virtual env with cpu_launcher_enable true
Possible Solution
i think https://github.com/pytorch/serve/blob/v0.10.0/frontend/server/src/main/java/org/pytorch/serve/wlm/WorkerLifeCycle.java#L69 should be updated to use python environment used by the running torchserve
The text was updated successfully, but these errors were encountered: