Why GPU isn't used?

[ysyyork]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Ubuntu22.04

Mobile device if the issue happens on mobile device

No response

Browser and version if the issue happens on browser

No response

Programming Language and version

python

MediaPipe version

No response

Bazel version

No response

Solution

hand tracking gpu

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

Want to leverage Nvidia GPU but couldn't

Describe the expected behaviour

Should leverage Nvidia GPU

Standalone code/steps you may have used to try to get what you need

I followed the tutorial here (https://developers.google.com/mediapipe/framework/getting_started/gpu_support#opengl_es_setup_on_linux_desktop) for setting up OpenGL support on my ubuntu laptop where a RTX 3060 is installed. 

specifically, I used `nvidia/opengl:1.0-glvnd-devel-ubuntu22.04` as base docker image, then I run the docker container like below:

docker run -idt --gpus all -e __GLX_VENDOR_LIBRARY_NAME=nvidia -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY --network host --name ubuntu22.04-opengl-test4  nvidia/opengl:1.0-glvnd-devel-ubuntu22.04 bash

In this container, I followed the above link and installed mesa dependencies. I also checked with `glxinfo | grep -i opengl` command and I can see the output contains NVIDIA. 

Then I build with this command
`bazel build --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu`

It was successfully built

Then I tested it with `GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_gpu  --calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_desktop_live_gpu.pbtxt  --input_video_path=mediapipe/examples/desktop/object_detection/test_video.mp4`

and I used `glances` to monitor GPU usage. it only shows very high CPU usage and 0 GPU usage.

Other info / Complete Logs

No response

[ysyyork]

Sorry wanna add some picture and realize the template didn't enable me to do that. So I add more info following the description above here.

this is the usage

However in the log for running hand_tracking_gpu, I do see the tflite mode is delegate to GPU. But the reality is there is no GPU usage.

Also, in the log, it shows mesa as openGL vendor. Not nvidia. Based on this issue, it seems I have to get it to Nvidia to make it work? but Idk how.

[ysyyork]

Within the same docker container, I also tried using the pip installed mediapipe and leverage task. I followed the tutorial code listed here and made it one script. I modified
base_options = python.BaseOptions(model_asset_path='gesture_recognizer.task')
to
base_options = python.BaseOptions(model_asset_path='gesture_recognizer.task', delegate=mp.tasks.BaseOptions.Delegate.GPU)

It can run but again GPU usage is 0. Log shows it's already delegated to GPU

[ysyyork]

ok, i figured by myself. it's related to the x11 server I passed into my docker. I used a VNC server and passed a virtual VNC x11 session in and by default it would only use Mesa. After I remove this x11 pass in or I directly run this on host system, my openGL changed to nvidia.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No