mistral does not support NVIDIA V100 (compute_cap <= 800)

[thesues]

Describe the bug
it does not support some old hardware.

Can it just convert bfloat16 to float16 before loading model. just like vllm is doing?

[EricLBuehler]

Hi @thesues!

This is a problem with the way we compile our CUDA kernels. I'll try to see if we can modify the compilation process (likely with conditional compilation) to avoid this, but it will probably be difficult.