[Doc] update Contributing page's testing section

[davidxia]

• make it apparent the set up instructions are for Nvidia CUDA
• the pip install command requires the PyTorch index
• install the python3-dev Debian package that's required for building mamba-ssm wheel from source

The output of python collect_env.py

$ python vllm/collect_env.py INFO 05-16 18:10:30 [__init__.py:248] Automatically detected platform cuda. Collecting environment information... PyTorch version: 2.7.0+cu128 Is debug build: False CUDA used to build PyTorch: 12.8 ROCM used to build PyTorch: N/A OS: Ubuntu 24.04.2 LTS (x86_64) GCC version: (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0 Clang version: Could not collect CMake version: version 3.28.3 Libc version: glibc-2.39 Python version: 3.12.3 (main, Feb 4 2025, 14:48:35) [GCC 13.3.0] (64-bit runtime) Python platform: Linux-6.11.0-1014-gcp-x86_64-with-glibc2.39 Is CUDA available: True CUDA runtime version: 12.9.41 CUDA_MODULE_LOADING set to: LAZY GPU models and configuration: GPU 0: NVIDIA A100-SXM4-40GB Nvidia driver version: 575.51.03 cuDNN version: Could not collect HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True CPU: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 46 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 12 On-line CPU(s) list: 0-11 Vendor ID: GenuineIntel Model name: Intel(R) Xeon(R) CPU @ 2.20GHz CPU family: 6 Model: 85 Thread(s) per core: 2 Core(s) per socket: 6 Socket(s): 1 Stepping: 7 BogoMIPS: 4400.43 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat avx512_vnni md_clear arch_capabilities Hypervisor vendor: KVM Virtualization type: full L1d cache: 192 KiB (6 instances) L1i cache: 192 KiB (6 instances) L2 cache: 6 MiB (6 instances) L3 cache: 38.5 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-11 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown Vulnerability Reg file data sampling: Not affected Vulnerability Retbleed: Mitigation; Enhanced IBRS Vulnerability Spec rstack overflow: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence; BHI SW loop, KVM SW loop Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown Versions of relevant libraries: [pip3] mypy-extensions==1.0.0 [pip3] numpy==1.26.4 [pip3] nvidia-cublas-cu12==12.8.3.14 [pip3] nvidia-cuda-cupti-cu12==12.8.57 [pip3] nvidia-cuda-nvrtc-cu12==12.8.61 [pip3] nvidia-cuda-runtime-cu12==12.8.57 [pip3] nvidia-cudnn-cu12==9.7.1.26 [pip3] nvidia-cufft-cu12==11.3.3.41 [pip3] nvidia-cufile-cu12==1.13.0.11 [pip3] nvidia-curand-cu12==10.3.9.55 [pip3] nvidia-cusolver-cu12==11.7.2.55 [pip3] nvidia-cusparse-cu12==12.5.7.53 [pip3] nvidia-cusparselt-cu12==0.6.3 [pip3] nvidia-nccl-cu12==2.26.2 [pip3] nvidia-nvjitlink-cu12==12.8.61 [pip3] nvidia-nvtx-cu12==12.8.55 [pip3] pyzmq==26.4.0 [pip3] sentence-transformers==3.2.1 [pip3] torch==2.7.0+cu128 [pip3] torchaudio==2.7.0+cu128 [pip3] torchvision==0.22.0+cu128 [pip3] transformers==4.51.3 [pip3] transformers-stream-generator==0.0.5 [pip3] triton==3.3.0 [pip3] tritonclient==2.51.0 [pip3] vector-quantize-pytorch==1.21.2 [conda] Could not collect ROCM Version: Could not collect Neuron SDK Version: N/A vLLM Version: 0.8.5.dev708+g451da4bcb (git sha: 451da4bcb) vLLM Build Flags: CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled GPU Topology:	GPU0	CPU Affinity	NUMA Affinity	GPU NUMA ID GPU0 X	0-11	0	N/A Legend: X = Self SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI) NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU) PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge) PIX = Connection traversing at most a single PCIe bridge NV# = Connection traversing a bonded set of # NVLinks LD_LIBRARY_PATH=/usr/local/cuda-12.9/lib64:/usr/local/cuda-12.9/lib64 NCCL_CUMEM_ENABLE=0 PYTORCH_NVML_BASED_CUDA_CHECK=1 TORCHINDUCTOR_COMPILE_THREADS=1 CUDA_MODULE_LOADING=LAZY 

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[davidxia]

without the --torch-backend=auto I got this error

ERROR: Ignored the following versions that require a different python version: 1.10.0 Requires-Python <3.12,>=3.8; 1.10.0rc1 Requires-Python <3.12,>=3.8; 1.10.0rc2 Requires-Python <3.12,>=3.8; 1.10.1 Requires-Python <3.12,>=3.8; 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11; 1.6.2 Requires-Python >=3.7,<3.10; 1.6.3 Requires-Python >=3.7,<3.10; 1.7.0 Requires-Python >=3.7,<3.10; 1.7.1 Requires-Python >=3.7,<3.10; 1.7.2 Requires-Python >=3.7,<3.11; 1.7.3 Requires-Python >=3.7,<3.11; 1.8.0 Requires-Python >=3.8,<3.11; 1.8.0rc1 Requires-Python >=3.8,<3.11; 1.8.0rc2 Requires-Python >=3.8,<3.11; 1.8.0rc3 Requires-Python >=3.8,<3.11; 1.8.0rc4 Requires-Python >=3.8,<3.11; 1.8.1 Requires-Python >=3.8,<3.11; 1.9.0 Requires-Python >=3.8,<3.12; 1.9.0rc1 Requires-Python >=3.8,<3.12; 1.9.0rc2 Requires-Python >=3.8,<3.12; 1.9.0rc3 Requires-Python >=3.8,<3.12; 1.9.1 Requires-Python >=3.8,<3.12 ERROR: Could not find a version that satisfies the requirement torch==2.7.0+cu128 (from versions: 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0, 2.7.0) ERROR: No matching distribution found for torch==2.7.0+cu128 

[davidxia]

without this package, I got this error when building mamba from source. This is with uv pip install, but the error with pip install is similar.

full error

$ uv pip install --no-build-isolation "git+https://github.com/state-spaces/mamba@v2.2.4" Resolved 42 packages in 190ms × Failed to build `mamba-ssm @ git+https://github.com/state-spaces/mamba@95d8aba8a8c75aedcaa6143713b11e745e7cd0d9` ├─▶ The build backend returned an error ╰─▶ Call to `setuptools.build_meta.build_wheel` failed (exit status: 1) [stdout] torch.__version__ = 2.7.0+cu126 running bdist_wheel Guessing wheel URL: https://github.com/state-spaces/mamba/releases/download/v2.2.4/mamba_ssm-2.2.4+cu12torch2.7cxx11abiTRUE-cp312-cp312-linux_x86_64.whl Precompiled wheel not found. Building from source... running build running build_py copying mamba_ssm/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm copying mamba_ssm/distributed/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/distributed copying mamba_ssm/distributed/distributed_utils.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/distributed copying mamba_ssm/distributed/tensor_parallel.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/distributed copying mamba_ssm/models/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/models copying mamba_ssm/models/mixer_seq_simple.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/models copying mamba_ssm/models/config_mamba.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/models copying mamba_ssm/utils/generation.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/utils copying mamba_ssm/utils/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/utils copying mamba_ssm/utils/hf.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/utils copying mamba_ssm/utils/torch.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/utils copying mamba_ssm/ops/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops copying mamba_ssm/ops/selective_scan_interface.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops copying mamba_ssm/modules/mha.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules copying mamba_ssm/modules/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules copying mamba_ssm/modules/mamba2_simple.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules copying mamba_ssm/modules/mamba_simple.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules copying mamba_ssm/modules/ssd_minimal.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules copying mamba_ssm/modules/mamba2.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules copying mamba_ssm/modules/block.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules copying mamba_ssm/modules/mlp.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules copying mamba_ssm/ops/triton/selective_state_update.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton copying mamba_ssm/ops/triton/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton copying mamba_ssm/ops/triton/ssd_chunk_scan.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton copying mamba_ssm/ops/triton/softplus.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton copying mamba_ssm/ops/triton/layer_norm.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton copying mamba_ssm/ops/triton/ssd_combined.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton copying mamba_ssm/ops/triton/layernorm_gated.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton copying mamba_ssm/ops/triton/ssd_chunk_state.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton copying mamba_ssm/ops/triton/ssd_bmm.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton copying mamba_ssm/ops/triton/k_activations.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton copying mamba_ssm/ops/triton/ssd_state_passing.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton running egg_info writing mamba_ssm.egg-info/PKG-INFO writing dependency_links to mamba_ssm.egg-info/dependency_links.txt writing requirements to mamba_ssm.egg-info/requires.txt writing top-level names to mamba_ssm.egg-info/top_level.txt reading manifest file 'mamba_ssm.egg-info/SOURCES.txt' adding license file 'LICENSE' adding license file 'AUTHORS' writing manifest file 'mamba_ssm.egg-info/SOURCES.txt' running build_ext building 'selective_scan_cuda' extension [1/1] c++ -MMD -MF /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/csrc/selective_scan/selective_scan.o.d -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/csrc/selective_scan -I/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include -I/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/usr/local/cuda-12.9/include -I/home/dxia/src/github.com/vllm-project/vllm/.venv/include -I/usr/include/python3.12 -c -c /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/csrc/selective_scan/selective_scan.cpp -o /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/csrc/selective_scan/selective_scan.o -O3 -std=c++17 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1016"' -DTORCH_EXTENSION_NAME=selective_scan_cuda -D_GLIBCXX_USE_CXX11_ABI=1 FAILED: /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/csrc/selective_scan/selective_scan.o c++ -MMD -MF /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/csrc/selective_scan/selective_scan.o.d -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/csrc/selective_scan -I/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include -I/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/usr/local/cuda-12.9/include -I/home/dxia/src/github.com/vllm-project/vllm/.venv/include -I/usr/include/python3.12 -c -c /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/csrc/selective_scan/selective_scan.cpp -o /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/csrc/selective_scan/selective_scan.o -O3 -std=c++17 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1016"' -DTORCH_EXTENSION_NAME=selective_scan_cuda -D_GLIBCXX_USE_CXX11_ABI=1 In file included from /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include/torch/csrc/Device.h:4, from /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include/torch/csrc/api/include/torch/python.h:8, from /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/csrc/selective_scan/selective_scan.cpp:7: /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include/torch/csrc/python_headers.h:12:10: fatal error: Python.h: No such file or directory 12 | #include <Python.h> | ^~~~~~~~~~ compilation terminated. ninja: build stopped: subcommand failed. [stderr] /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated !! ******************************************************************************** Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0). By 2026-Feb-18, you need to update your project and remove deprecated calls or your builds will no longer be supported. See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! corresp(dist, value, root_dir) /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsWarning: `install_requires` overwritten in `pyproject.toml` (dependencies) corresp(dist, value, root_dir) /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py:61: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: BSD License See https://packaging.python.org/en/latest/guides/writing-pypro /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: BSD License See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! self._finalize_license_expression() /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py:480: UserWarning: The detected CUDA version (12.9) has a minor version mismatch with the version that was used to compile PyTorch (12.6). Most likely this shouldn't be a problem.  warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))  /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py:490: UserWarning: There are no x86_64-linux-gnu-g++ version bounds defined for CUDA version 12.9  warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')  Emitting ninja build file /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/build.ninja...  Compiling objects...  Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)  Traceback (most recent call last):  File "<string>", line 317, in run  File "/usr/lib/python3.12/urllib/request.py", line 240, in urlretrieve  with contextlib.closing(urlopen(url, data)) as fp:  ^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen  return opener.open(url, data, timeout)  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3.12/urllib/request.py", line 521, in open  response = meth(req, response)  ^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3.12/urllib/request.py", line 630, in http_response  response = self.parent.error(  ^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3.12/urllib/request.py", line 559, in error  return self._call_chain(*args)  ^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain  result = func(*args)  ^^^^^^^^^^^  File "/usr/lib/python3.12/urllib/request.py", line 639, in http_error_default  raise HTTPError(req.full_url, code, msg, hdrs, fp)  urllib.error.HTTPError: HTTP Error 404: Not Found   During handling of the above exception, another exception occurred:   Traceback (most recent call last):  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2506, in _run_ninja_build  subprocess.run(  File "/usr/lib/python3.12/subprocess.py", line 571, in run  raise CalledProcessError(retcode, process.args,  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.   The above exception was the direct cause of the following exception:   Traceback (most recent call last):  File "<string>", line 11, in <module>  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 432, in build_wheel  return _build(['bdist_wheel'])  ^^^^^^^^^^^^^^^^^^^^^^^  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 423, in _build  return self._build_with_temp_dir(  ^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir  self.run_setup()  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 317, in run_setup  exec(code, locals())  File "<string>", line 336, in <module>  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/__init__.py", line 117, in setup  return distutils.core.setup(**attrs)  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 186, in setup  return run_commands(dist)  ^^^^^^^^^^^^^^^^^^  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 202, in run_commands  dist.run_commands()  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands  self.run_command(cmd)  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/dist.py", line 1104, in run_command  super().run_command(command)  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command  cmd_obj.run()  File "<string>", line 334, in run  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/command/bdist_wheel.py", line 370, in run  self.run_command("build")  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command  self.distribution.run_command(command)  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/dist.py", line 1104, in run_command  super().run_command(command)  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command  cmd_obj.run()  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/build.py", line 135, in run  self.run_command(cmd_name)  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command  self.distribution.run_command(command)  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/dist.py", line 1104, in run_command  super().run_command(command)  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command  cmd_obj.run()  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 99, in run  _build_ext.run(self)  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 368, in run  self.build_extensions()  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 1010, in build_extensions  build_ext.build_extensions(self)  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 484, in build_extensions  self._build_extensions_serial()  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 510, in _build_extensions_serial  self.build_extension(ext)  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 264, in build_extension  _build_ext.build_extension(self, ext)  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 565, in build_extension  objects = self.compiler.compile(  ^^^^^^^^^^^^^^^^^^^^^^  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 815, in unix_wrap_ninja_compile  _write_ninja_file_and_compile_objects(  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2159, in _write_ninja_file_and_compile_objects  _run_ninja_build(  File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2522, in _run_ninja_build  raise RuntimeError(message) from e  RuntimeError: Error compiling objects for extension   hint: This usually indicates a problem with the package or the build environment.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @davidxia.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @davidxia.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[davidxia]

@hmellor I rebased the conflicts away if you get a chance to review. Thanks! 🙏

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @davidxia.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[davidxia]

@hmellor lmk how the PR looks. Thanks!

[DarkLight1337]

FYI there is no need to workaround mamba-ssm since #21421

[hmellor]

@davidxia sorry about the delay on this one. If you reverted the change about mamba-ssm I'd be happy to accept the other parts of this PR

[davidxia]

@hmellor Thanks, removed that part.

[hmellor]

A couple of comments, but other than that this looks good to me