Skip to content

[docs] model testing#45152

Open
stevhliu wants to merge 8 commits intohuggingface:mainfrom
stevhliu:testing
Open

[docs] model testing#45152
stevhliu wants to merge 8 commits intohuggingface:mainfrom
stevhliu:testing

Conversation

@stevhliu
Copy link
Copy Markdown
Member

@stevhliu stevhliu commented Mar 31, 2026

refactors the testing docs to be more contributor-facing organized around writing model tests instead of being a collection of pytest usage examples and CI maintenance. also updates the pr checks doc so contributors are better prepared to pass the checks on first try

testing.md

  • adds CausalLMModelTest and VLMModelTest paths (should use these more often)
  • explains using ModelTester and ModelTest for non-causal and vlm models
  • explains what mixins provides
  • explains using real pretrained weights versus tiny models
  • flags for controlling what to test

pr_checks.md

  • clear make command flow
  • describe what make check-repo does
  • mention slow tests
  • condense # Copied from section

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@stevhliu stevhliu marked this pull request as ready for review April 3, 2026 18:23
@stevhliu stevhliu requested review from tarekziade and vasqu April 3, 2026 18:24
This was referenced Apr 6, 2026
@vasqu
Copy link
Copy Markdown
Contributor

vasqu commented Apr 9, 2026

@ydshieh @tarekziade could you do a first pass? Would take a look then, quite a lot to catch up on 😬

Copy link
Copy Markdown
Collaborator

@tarekziade tarekziade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, thanks a lot! I took a first pass focusing on the CI aspects.

A couple of meta remarks:

It might be better to avoid referring to CircleCI directly in the docs, since where/how we run CI is evolving. Maybe we can use a more generic term like “Hugging Face CI”?
Similarly, the detailed description of CI jobs could become outdated fairly quickly given the changes we’re planning. It might be more useful to emphasize the Makefile targets instead, as we’re aiming to make them the source of truth for both CI and local development.

Comment thread docs/source/en/pr_checks.md Outdated
Comment thread docs/source/en/pr_checks.md Outdated
Comment thread docs/source/en/pr_checks.md Outdated
Comment thread docs/source/en/pr_checks.md Outdated
Comment thread docs/source/en/pr_checks.md Outdated
Comment thread docs/source/en/testing.md Outdated
Comment thread docs/source/en/pr_checks.md Outdated
Copy link
Copy Markdown
Collaborator

@tarekziade tarekziade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM just one nit about mlinter

Copy link
Copy Markdown
Contributor

@vasqu vasqu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial comments from my side, sorry a bit jumbled thoughts here and there. Hope it's clear enough otherwise happy to clarify

Comment thread docs/source/en/pr_checks.md Outdated
Comment thread docs/source/en/pr_checks.md Outdated
Comment thread docs/source/en/pr_checks.md Outdated
```bash
pip install -e ".[dev]"
```
## Code quality
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have a small intro that gives an intro what these subsections are, i.e. referencing the explicit names of our repo checks

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we are missing a TOC or similar with some short quick intros what they actually refer to

  • What you should do as contributor / how can you fix the CI
  • Internals
    • Each check
    • Utilities for each of those, for exampel make style for if only the code quality is failing

Comment thread docs/source/en/testing.md
Comment thread docs/source/en/testing.md
Comment thread docs/source/en/testing.md
Comment thread docs/source/en/testing.md
Comment thread docs/source/en/testing.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants