Cuda 12.9 support

[MLConvexAI]

Feature request

I have Windows with NVidia RTX-5090. This is the new Blackwell chip which requires Cuda 12.9 to work. Unsloth requires Triton, Torch and Bitsandbytes. By compiling from the source code, I can get Triton and Torch to work. Should Bitsandbytes also support Cuda 12.9 or when could this be available?

Motivation

Ask for Nvidia RTX-5090 support.

Your contribution

Just a question

[matthewdouglas]

Hi,

I'll try to clarify a little bit here.

In order to use bitsandbytes, you do not necessarily need a CUDA Toolkit installed. If you do have it installed, it does not have to match the same as the latest CUDA runtime version that your driver indicates it supports. I am assuming you indicate the request for CUDA 12.9 because this is what nvidia-smi shows you. This is not the same thing as the CUDA Toolkit, and you can also run applications compiled for earlier versions than 12.9.

Blackwell (e.g. RTX 5090) support was added in the CUDA 12.8 toolkit, and is present in the CUDA 12.8 builds of PyTorch 2.7.0. We ship CUDA 12.8 binaries with Blackwell support in the latest bitsandbytes releases.

You should install torch==2.7.0+cu128:

pip install torch --index-url https://download.pytorch.org/whl/cu128

Then, when you install bitsandbytes, it should load the libbitsandbytes_cu128.dll, which supports the RTX 5090.

Hope this helps!

[MLConvexAI]

Thanks. Somehow I do not get the combination torch, bitsandbytes and triton to work together. If torch==2.7.0+cu128: is supported, would you happen to know what is the triton version that fits in to this combination?

[matthewdouglas]

Triton isn't officially supported on Windows. With that said, triton==3.3.0 is what is specified for the PyTorch Linux wheels.

[MLConvexAI]

Small correction here: I am using windows wsl ubuntu (pure windows is not working at all). I have tried with triton==3.3.0. Thanks for these comments. I need to figure out something else.

[matthewdouglas]

Closing due to inactivity. Please bump or open a new issue if needed. Thanks!