Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simple multi-gpu configuration #1938

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Conversation

borg323
Copy link
Member

@borg323 borg323 commented Jan 1, 2024

This adds a --gpus=x (0<=x<=8) to specify the number of gpus to use, which is ignored if --backend-opts is used. Calling meson build with -Dgpus_opt=false disables this option, and this is done for the blas and android appveyor builds.

Using --gpus=1 (default) is equivalent to --backend-opts=gpu=0, which is mainly affecting the onednn backend, making the gpu first choice, so I added a fallback to cpu there, if the gpu is not available.

The old behavior is obtained by using --gpus=0, In practice this only makes sense with the onednn backend.

For a higher value of gpus, a backend options line of the form (backend=X,gpu=0),(backend=X,gpu=1) and so on is constructed, with X being either the backend specified with --backend (unless it is multiplexing, demux or roundrobin) or the default backend. The multiplexing backend is used, unless demux or roundrobin is specified with --backend.

@borg323 borg323 marked this pull request as draft January 15, 2024 11:03
@borg323
Copy link
Member Author

borg323 commented Jan 15, 2024

I'm having second thoughts about the (ab)use of backend options, made it draft for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant