Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/scripts/build-cuda.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ elif [ "${build_arch}" = "aarch64" ]; then
# CUDA 12.8+: Add sm100/sm120
[[ "${cuda_version}" == 12.8.* || "${cuda_version}" == 12.9.* ]] && build_capability="75;80;90;100;120"
else
# By default, target Maxwell through Hopper.
build_capability="50;60;70;75;80;86;89;90"
# By default, target Pascal through Hopper.
build_capability="60;70;75;80;86;89;90"

# CUDA 12.8+: Add sm100 and sm120; remove < sm70 to align with PyTorch 2.8+cu128 minimum
[[ "${cuda_version}" == 12.8.* || "${cuda_version}" == 12.9.* ]] && build_capability="70;75;80;86;89;90;100;120"
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ bitsandbytes has the following minimum requirements for all platforms:
<tr>
<td></td>
<td>🟩 NVIDIA GPU <br><code>cuda</code></td>
<td>SM50+ minimum<br>SM75+ recommended</td>
<td>SM60+ minimum<br>SM75+ recommended</td>
<td>✅</td>
<td>✅</td>
<td>✅</td>
Expand All @@ -87,7 +87,7 @@ bitsandbytes has the following minimum requirements for all platforms:
</td>
<td>✅</td>
<td>✅</td>
<td>🚧</td>
<td>〰️</td>
</tr>
<tr>
<td></td>
Expand Down Expand Up @@ -127,7 +127,7 @@ bitsandbytes has the following minimum requirements for all platforms:
<tr>
<td></td>
<td>🟩 NVIDIA GPU <br><code>cuda</code></td>
<td>SM50+ minimum<br>SM75+ recommended</td>
<td>SM60+ minimum<br>SM75+ recommended</td>
<td>✅</td>
<td>✅</td>
<td>✅</td>
Expand Down
26 changes: 14 additions & 12 deletions docs/source/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,17 +16,19 @@ Welcome to the installation guide for the `bitsandbytes` library! This document

## CUDA[[cuda]]

`bitsandbytes` is currently supported on NVIDIA GPUs with [Compute Capability](https://developer.nvidia.com/cuda-gpus) 5.0+.
The library can be built using CUDA Toolkit versions as old as **11.6** on Windows and **11.4** on Linux.
`bitsandbytes` is currently supported on NVIDIA GPUs with [Compute Capability](https://developer.nvidia.com/cuda-gpus) 6.0+.
The library can be built using CUDA Toolkit versions as old as **11.8**.

| **Feature** | **CC Required** | **Example Hardware Requirement** |
|---------------------------------|-----------------|---------------------------------------------|
| LLM.int8() | 7.5+ | Turing (RTX 20 series, T4) or newer GPUs |
| 8-bit optimizers/quantization | 5.0+ | Maxwell (GTX 900 series, TITAN X, M40) or newer GPUs |
| NF4/FP4 quantization | 5.0+ | Maxwell (GTX 900 series, TITAN X, M40) or newer GPUs |
| LLM.int8() | 7.5+ | Turing (RTX 20 series, T4) or newer GPUs |
| 8-bit optimizers/quantization | 6.0+ | Pascal (GTX 10X0 series, P100) or newer GPUs|
| NF4/FP4 quantization | 6.0+ | Pascal (GTX 10X0 series, P100) or newer GPUs|

> [!WARNING]
> Support for Maxwell GPUs is deprecated and will be removed in a future release. For the best results, a Turing generation device or newer is recommended.
> Support for Maxwell GPUs is deprecated and will be removed in a future release.
> Maxwell support is not included in PyPI distributions from `v0.48.0` on and must be built from source.
> For the best results, a Turing generation device or newer is recommended.

### Installation via PyPI[[cuda-pip]]

Expand All @@ -36,12 +38,12 @@ The currently distributed `bitsandbytes` packages are built with the following c

| **OS** | **CUDA Toolkit** | **Host Compiler** | **Targets**
|--------------------|------------------|----------------------|--------------
| **Linux x86-64** | 11.8 - 12.6 | GCC 11.2 | sm50, sm60, sm75, sm80, sm86, sm89, sm90
| **Linux x86-64** | 12.8 | GCC 11.2 | sm75, sm80, sm86, sm89, sm90, sm100, sm120
| **Linux x86-64** | 11.8 - 12.6 | GCC 11.2 | sm60, sm70, sm75, sm80, sm86, sm89, sm90
| **Linux x86-64** | 12.8 - 12.9 | GCC 11.2 | sm70, sm75, sm80, sm86, sm89, sm90, sm100, sm120
| **Linux aarch64** | 11.8 - 12.6 | GCC 11.2 | sm75, sm80, sm90
| **Linux aarch64** | 12.8 | GCC 11.2 | sm75, sm80, sm90, sm100
| **Linux aarch64** | 12.8 - 12.9 | GCC 11.2 | sm75, sm80, sm90, sm100, sm120
| **Windows x86-64** | 11.8 - 12.6 | MSVC 19.43+ (VS2022) | sm50, sm60, sm75, sm80, sm86, sm89, sm90
| **Windows x86-64** | 12.8 | MSVC 19.43+ (VS2022) | sm75, sm80, sm86, sm89, sm90, sm100, sm120
| **Windows x86-64** | 12.8 - 12.9 | MSVC 19.43+ (VS2022) | sm70, sm75, sm80, sm86, sm89, sm90, sm100, sm120

Use `pip` or `uv` to install:

Expand All @@ -67,7 +69,7 @@ For example, to install a compiler and CMake on Ubuntu:
apt-get install -y build-essential cmake
```

You should also install CUDA Toolkit by following the [NVIDIA CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html) guide. The current minimum supported CUDA Toolkit version that we test with is **11.8**.
You should also install CUDA Toolkit by following the [NVIDIA CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html) guide. The current minimum supported CUDA Toolkit version that we support is **11.8**.

```bash
git clone https://github.com/bitsandbytes-foundation/bitsandbytes.git && cd bitsandbytes/
Expand All @@ -84,7 +86,7 @@ pip install -e . # `-e` for "editable" install, when developing BNB (otherwise

Compilation from source on Windows systems require Visual Studio with C++ support as well as an installation of the CUDA Toolkit.

To compile from source, you need CMake >= **3.22.1** and Python >= **3.9** installed. You should also install CUDA Toolkit by following the [CUDA Installation Guide for Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html) guide from NVIDIA. The current minimum supported CUDA Toolkit version that we test with is **11.8**.
To compile from source, you need CMake >= **3.22.1** and Python >= **3.9** installed. You should also install CUDA Toolkit by following the [CUDA Installation Guide for Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html) guide from NVIDIA. The current minimum supported CUDA Toolkit version that we support is **11.8**.

```bash
git clone https://github.com/bitsandbytes-foundation/bitsandbytes.git && cd bitsandbytes/
Expand Down
Loading