Add support for Python 3.14

[albertvillanova]

Add support for Python 3.14, which was first released yesterday:

• First release: 2025-10-07

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[albertvillanova]

Currently, there are some issues with torchvision dependency: https://github.com/huggingface/trl/actions/runs/18335378071/job/52218404203

  × No solution found when resolving dependencies:
  ╰─▶ Because vllm==0.10.2 depends on torchvision==0.23.0 and
      torchvision==0.23.0 has no wheels with a matching Python ABI tag (e.g.,
      `cp314`), we can conclude that vllm==0.10.2 cannot be used.
      And because trl[dev]==0.24.0.dev0 depends on vllm==0.10.2, we can
      conclude that trl[dev]==0.24.0.dev0 cannot be used.
      And because only trl[dev]==0.24.0.dev0 is available and you require
      trl[dev], we can conclude that your requirements are unsatisfiable.

      hint: You require CPython 3.14 (`cp314`), but we only found wheels for
      `torchvision` (v0.23.0) with the following Python ABI tags: `cp39`,
      `cp310`, `cp311`, `cp312`, `cp313`, `cp313t`

Check upstream issue on pytorch:

• Python 3.14 support for PyTorch pytorch/pytorch#156856

[albertvillanova]

Related PR:

• Exclude vllm dependencies from dev extra #4229

[qgallouedec]

Ok so we need to wait until python 3.14 is supported then

[albertvillanova]

The torchvision and vllm error reported above (#4225 (comment)) was fixed after the merge of (#4225 (comment)):
#4229

However, there is still another issue with torch.compile that prevents the support for Python 3.14: https://github.com/huggingface/trl/actions/runs/19697104308/job/56424394269?pr=4225

 tests/experimental/test_bco_trainer.py:27: in <module>
    from ..testing_utils import TrlTestCase, require_no_wandb, require_peft, require_sklearn
tests/testing_utils.py:83: in <module>
    not is_bitsandbytes_multi_backend_available() and not torch_device == "cuda",
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/testing_utils.py:75: in is_bitsandbytes_multi_backend_available
    import bitsandbytes as bnb
.venv/lib/python3.14/site-packages/bitsandbytes/__init__.py:19: in <module>
    from .backends.default import ops as default_ops
.venv/lib/python3.14/site-packages/bitsandbytes/backends/default/ops.py:324: in <module>
    @torch.compile
     ^^^^^^^^^^^^^
.venv/lib/python3.14/site-packages/torch/__init__.py:2590: in compile
    raise RuntimeError("torch.compile is not supported on Python 3.14+")
E   RuntimeError: torch.compile is not supported on Python 3.14+