Problems with egl rendering

[ambitious-octopus]

I am working on a mujoco simulation inside a docker container (using nvidia-docker for GPU access). My simulation use egl for hardware rendering and on my local machine works perfectly. I would like to run my simulation on a server with 3 NVIDIA A40.
When I run this script in the same docker image but on the server

import os
os.environ["MUJOCO_GL"]="egl"
os.environ["MUJOCO_EGL_DEVICE_ID"] = "1"
from dm_control import mujoco

I get this error:

ImportError: Cannot initialize a headless EGL display.

Server:
Nvidia Driver Version: 525.85.12 CUDA Version: 12.0

Local Machine:
Nvidia Driver Version: 525.116.04 CUDA Version: 12.0

If I use osmesa (os.environ["MUJOCO_GL"]="egl") everything works correctly. But I would like to use egl.
I tried to install libnvidia-gl (apt install libnvidia-gl-525) as in #214 and #365. but I get the same error.

I should also add that I do not use NVIDIA Multi-Instance GPU.
Am I doing something wrong here?
Thank you in advance for the support!

[saran-t]

This almost definitely means that your Docker container doesn't have permission to access the GPU hardware specified.

Have you tried setting MUJOCO_EGL_DEVICE_ID to 0 ?

[ambitious-octopus]

I suspected that. It works with MUJOCO_EGL_DEVICE_ID=0.
I would also add that nvidia-smi within the container shows all the gpu's.
And I'm still not using the GPU! What am I missing?