Add queue per GPU to ensure sequential inference requests

[yondonfu]

No description provided.

[ad-astra-video]

I like this! Most easily accessible GPUs likely have a 1 job at a time limit. Possibly a 1 job queue per gpu? Could this tie into the worker callback saying what step is completed so the O could know its say 75% done with the current job? The O could then allocate to the GPU closest to being done I think.

[rickstaa]

Tracked internally in https://linear.app/livepeer-ai-spe/issue/LIV-318/.