Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
07e3831
[CB] Changes for long generation (#45530)
remi-or Apr 23, 2026
706acf5
Allow for registered experts from kernels hub (#45577)
winglian Apr 23, 2026
bd69ed2
[docs] multi-turn tool calling (#45554)
stevhliu Apr 23, 2026
8e64e53
[AMD CI] Fix expectations for Gemma3n (#45602)
Abdennacer-Badaoui Apr 23, 2026
0323898
fix transformers + torchao nvfp4 serialization (#45573)
vkuzo Apr 23, 2026
533c4e1
SonicMoe (#45433)
IlyasMoutawwakil Apr 23, 2026
1e071b2
Processing Utils: continue when content is a string (#45605)
RyanMullins Apr 23, 2026
57f9936
qa: bumped mlinter and allow local override (#45585)
tarekziade Apr 23, 2026
91904ac
Fix configuration reading and error handling for kernels (#45610)
hmellor Apr 23, 2026
5cf7951
fix: compute auxiliary losses when denoising is disabled in D-FINE (#…
Abineshabee Apr 23, 2026
16f3dde
Remove unnecessary generate warnings (#45619)
Cyrilvallez Apr 24, 2026
f0a5a1c
generate: drop stale num_return_sequences warning on continuous batch…
joaquinhuigomez Apr 24, 2026
a66638d
Skip failing offloading tests (#45624)
Cyrilvallez Apr 24, 2026
f0f456b
chore(qa): split pipeline and add type checking (#45432)
tarekziade Apr 24, 2026
23ca437
Allow more artifacts to be download in CI (#45629)
ydshieh Apr 24, 2026
622b8e9
chore: bump doc-builder SHA for main doc build workflow (#45631)
rtrompier Apr 24, 2026
678e871
CircleCI with torch 2.11 (#45633)
ydshieh Apr 24, 2026
c472755
Raise clear error for `problem_type="single_label_classification"` wi…
gaurav0107 Apr 24, 2026
47a512b
Fix xdist collisions for captured_info artifacts and preserve CI debu…
stationeros Apr 25, 2026
ded2b74
Add `supports_gradient_checkpointing` to `NemotronHPreTrainedModel` (…
sergiopaniego Apr 27, 2026
4f85f85
Fix whisper return language (#42227)
FredHaa Apr 27, 2026
d94ced8
Merge branch 'main' into main
stationeros Apr 27, 2026
9cb1a72
Merge PR #45645: Fix xdist captured_info collisions
evalstate Apr 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/build_documentation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on:

jobs:
build:
uses: huggingface/doc-builder/.github/workflows/build_main_documentation.yml@90b4ee2c10b81b5c1a6367c4e6fc9e2fb510a7e3 # main
uses: huggingface/doc-builder/.github/workflows/build_main_documentation.yml@2430c1ec91d04667414e2fa31ecfc36c153ea391 # main
with:
commit_sha: ${{ github.sha }}
package: transformers
Expand All @@ -23,7 +23,7 @@ jobs:
hf_token: ${{ secrets.HF_DOC_BUILD_PUSH }}

build_other_lang:
uses: huggingface/doc-builder/.github/workflows/build_main_documentation.yml@90b4ee2c10b81b5c1a6367c4e6fc9e2fb510a7e3 # main
uses: huggingface/doc-builder/.github/workflows/build_main_documentation.yml@2430c1ec91d04667414e2fa31ecfc36c153ea391 # main
with:
commit_sha: ${{ github.sha }}
package: transformers
Expand Down
14 changes: 9 additions & 5 deletions .github/workflows/check_failed_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ jobs:
n_runners: ${{ steps.set-matrix.outputs.n_runners }}
process: ${{ steps.set-matrix.outputs.process }}
steps:
- uses: actions/download-artifact@v4
- uses: actions/download-artifact@v8
continue-on-error: true
with:
name: ci_results_${{ inputs.job }}
Expand Down Expand Up @@ -127,12 +127,14 @@ jobs:
image: ${{ inputs.docker }}
options: --gpus all --shm-size "16gb" --ipc host -v /mnt/cache/.cache/huggingface:/mnt/cache/
steps:
- uses: actions/download-artifact@v4
- uses: actions/download-artifact@v8
with:
name: ci_results_${{ inputs.job }}
path: /transformers/ci_results_${{ inputs.job }}

- uses: actions/download-artifact@v4
- uses: actions/download-artifact@v8
env:
ACTIONS_ARTIFACT_MAX_ARTIFACT_COUNT: 2000
with:
pattern: setup_values*
path: setup_values
Expand Down Expand Up @@ -255,12 +257,14 @@ jobs:
image: ${{ inputs.docker }}
options: --gpus all --shm-size "16gb" --ipc host -v /mnt/cache/.cache/huggingface:/mnt/cache/
steps:
- uses: actions/download-artifact@v4
- uses: actions/download-artifact@v8
with:
name: ci_results_${{ inputs.job }}
path: /transformers/ci_results_${{ inputs.job }}

- uses: actions/download-artifact@v4
- uses: actions/download-artifact@v8
env:
ACTIONS_ARTIFACT_MAX_ARTIFACT_COUNT: 2000
with:
pattern: new_failures_with_bad_commit_${{ inputs.job }}*
path: /transformers/new_failures_with_bad_commit_${{ inputs.job }}
Expand Down
13 changes: 12 additions & 1 deletion .github/workflows/model_jobs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,18 @@ jobs:
env:
report_name_prefix: ${{ inputs.report_name_prefix }}
run: |
cat "/transformers/reports/${machine_type}_${report_name_prefix}_${matrix_folders}_test_reports/captured_info.txt"
shopt -s nullglob
captured_info_files=("/transformers/reports/${machine_type}_${report_name_prefix}_${matrix_folders}_test_reports"/captured_info*.txt)

if [ ${#captured_info_files[@]} -eq 0 ]; then
echo "No captured information files found."
exit 0
fi

for captured_info_file in "${captured_info_files[@]}"; do
echo "===== ${captured_info_file##*/} ====="
cat "$captured_info_file"
done

- name: Copy test_outputs.txt
if: ${{ always() }}
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/self-scheduled.yml
Original file line number Diff line number Diff line change
Expand Up @@ -601,7 +601,9 @@ jobs:
- name: Create output directory
run: mkdir warnings_in_ci

- uses: actions/download-artifact@v4
- uses: actions/download-artifact@v8
env:
ACTIONS_ARTIFACT_MAX_ARTIFACT_COUNT: 2000
with:
path: warnings_in_ci

Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/slack-report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,9 @@ jobs:
# Security: checkout to the `main` branch for untrusted triggers (issue_comment, pull_request_target), otherwise use the specified ref
ref: ${{ (github.event_name == 'issue_comment' || github.event_name == 'pull_request_target') && 'main' || (inputs.commit_sha || github.sha) }}

- uses: actions/download-artifact@v4
- uses: actions/download-artifact@v8
env:
ACTIONS_ARTIFACT_MAX_ARTIFACT_COUNT: 2000

- name: Prepare some setup values
run: |
Expand Down
2 changes: 1 addition & 1 deletion docker/consistency.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ ARG REF=main
RUN apt-get update && apt-get install -y time git g++ pkg-config make git-lfs
ENV UV_PYTHON=/usr/local/bin/python
RUN pip install uv && uv pip install --no-cache-dir -U pip setuptools GitPython
RUN uv pip install --no-cache-dir --upgrade 'torch<=2.10.0' 'torchaudio' 'torchvision' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir --upgrade 'torch<=2.11.0' 'torchaudio' 'torchvision' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir pypi-kenlm
RUN uv pip install --no-cache-dir "git+https://github.com/huggingface/transformers.git@${REF}#egg=transformers[quality,testing,torch-speech,vision]"
RUN git lfs install
Expand Down
2 changes: 1 addition & 1 deletion docker/custom-tokenizers.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ RUN make install -j 10

WORKDIR /

RUN uv pip install --no-cache --upgrade 'torch<=2.10.0' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache --upgrade 'torch<=2.11.0' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir --no-deps accelerate --extra-index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir "git+https://github.com/huggingface/transformers.git@${REF}#egg=transformers[ja,testing,sentencepiece,spacy,rjieba]" unidic unidic-lite
# spacy is not used so not tested. Causes to failures. TODO fix later
Expand Down
2 changes: 1 addition & 1 deletion docker/examples-torch.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ USER root
RUN apt-get update && apt-get install -y --no-install-recommends libsndfile1-dev espeak-ng time git g++ cmake pkg-config openssh-client git-lfs ffmpeg curl
ENV UV_PYTHON=/usr/local/bin/python
RUN pip --no-cache-dir install uv && uv pip install --no-cache-dir -U pip setuptools
RUN uv pip install --no-cache-dir 'torch<=2.10.0' 'torchaudio' 'torchvision' 'torchcodec<=0.10.0' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir 'torch<=2.11.0' 'torchaudio' 'torchvision' 'torchcodec<=0.11.0' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-deps timm accelerate --extra-index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir librosa "git+https://github.com/huggingface/transformers.git@${REF}#egg=transformers[sklearn,sentencepiece,vision,testing]" seqeval albumentations jiwer

Expand Down
2 changes: 1 addition & 1 deletion docker/exotic-models.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ USER root
RUN apt-get update && apt-get install -y libsndfile1-dev espeak-ng time git libgl1 g++ tesseract-ocr git-lfs curl
ENV UV_PYTHON=/usr/local/bin/python
RUN pip --no-cache-dir install uv && uv pip install --no-cache-dir -U pip setuptools
RUN uv pip install --no-cache-dir 'torch<=2.10.0' 'torchaudio' 'torchvision' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir 'torch<=2.11.0' 'torchaudio' 'torchvision' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir --no-deps timm accelerate
RUN uv pip install -U --no-cache-dir pytesseract python-Levenshtein opencv-python nltk
# RUN uv pip install --no-cache-dir natten==0.15.1+torch210cpu -f https://shi-labs.com/natten/wheels
Expand Down
2 changes: 1 addition & 1 deletion docker/pipeline-torch.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ USER root
RUN apt-get update && apt-get install -y --no-install-recommends libsndfile1-dev espeak-ng time git pkg-config openssh-client git ffmpeg curl
ENV UV_PYTHON=/usr/local/bin/python
RUN pip --no-cache-dir install uv && uv pip install --no-cache-dir -U pip setuptools
RUN uv pip install --no-cache-dir 'torch<=2.10.0' 'torchaudio' 'torchvision' 'torchcodec<=0.10.0' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir 'torch<=2.11.0' 'torchaudio' 'torchvision' 'torchcodec<=0.11.0' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-deps timm accelerate --extra-index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir librosa "git+https://github.com/huggingface/transformers.git@${REF}#egg=transformers[sklearn,sentencepiece,vision,testing]"

Expand Down
2 changes: 1 addition & 1 deletion docker/torch-light.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ USER root
RUN apt-get update && apt-get install -y --no-install-recommends libsndfile1-dev espeak-ng time git g++ cmake pkg-config openssh-client git-lfs ffmpeg curl
ENV UV_PYTHON=/usr/local/bin/python
RUN pip --no-cache-dir install uv && uv pip install --no-cache-dir -U pip setuptools
RUN uv pip install --no-cache-dir 'torch<=2.10.0' 'torchaudio' 'torchvision' 'torchcodec<=0.10.0' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir 'torch<=2.11.0' 'torchaudio' 'torchvision' 'torchcodec<=0.11.0' --index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-deps timm accelerate --extra-index-url https://download.pytorch.org/whl/cpu
RUN uv pip install --no-cache-dir librosa "git+https://github.com/huggingface/transformers.git@${REF}#egg=transformers[sklearn,sentencepiece,vision,testing,tiktoken,num2words,video]"

Expand Down
16 changes: 8 additions & 8 deletions docs/source/en/modeling_rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,22 +13,22 @@ specific language governing permissions and limitations under the License.

# Model structure rules

Transformers enforces a set of static rules on every `modeling_*.py`, `modular_*.py`, and `configuration_*.py` file. The [mlinter](https://github.com/huggingface/transformers-mlinter) tool checks them as part of `make typing` and errors out if violations are found.
Transformers enforces a set of static rules on every `modeling_*.py`, `modular_*.py`, and `configuration_*.py` file. The [mlinter](https://github.com/huggingface/transformers-mlinter) package provides the checker engine, and the repository keeps its active rule set in `utils/rules.toml`. That local TOML lets us enable, disable, or tweak rules quickly without waiting for a new `transformers-mlinter` release.

These are the expected model conventions for adding or changing modeling code. They keep the codebase consistent and ensure compatibility with features like pipeline parallelism, device maps, and weight tying.

## Running the checker

`make typing` runs `mlinter` alongside the `ty` type checker. Run `mlinter` on its own with the following commands.
`make typing` runs `mlinter` alongside the `ty` type checker through the repo wrapper, so it picks up `utils/rules.toml`. Run the same wrapper directly with the following commands.

```bash
mlinter # check all modeling files
mlinter --changed-only # check only files changed vs origin/main
mlinter --list-rules # list all rules and their enabled status
mlinter --rule TRF001 # show built-in docs for a specific rule
python utils/check_modeling_structure.py # check all modeling files
python utils/check_modeling_structure.py --changed-only # check only files changed vs origin/main
python utils/check_modeling_structure.py --list-rules # list all rules and their enabled status
python utils/check_modeling_structure.py --rule TRF001 # show built-in docs for a specific rule
```

The `--changed-only` flag is the fastest option during development. It only checks the files you've modified relative to the main branch.
The `--changed-only` flag is the fastest option during development. It only checks the files you've modified relative to the main branch. If you invoke `mlinter` directly instead of the wrapper, pass `--rules-toml utils/rules.toml` so local overrides are applied.

## Fixing a violation

Expand All @@ -52,7 +52,7 @@ Use the rule ID to look up the fix in the [rules reference](#rules-reference). T

## Rules reference

Each rule below lists what it enforces and a diff showing the fix. Run `mlinter --rule TRF001` to see the built-in docs for any rule.
Each rule below lists what it enforces and a diff showing the fix. Run `python utils/check_modeling_structure.py --rule TRF001` to see the built-in docs for any rule with the repo's current rule set.

<!-- BEGIN RULES REFERENCE -->

Expand Down
Loading