Conversation
|
[For maintainers] Suggested jobs to run (before merge) run-slow: vit |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
This comment contains models: ["models/vit"] |
CI Results |
|
This comment contains models: ["models/vit"] |
CI ResultsModel CI Report❌ Failed tests |
|
This comment contains models: ["models/vit"] |
CI ResultsModel CI Report❌ Failed tests
|
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.event.issue.number }}-${{ startsWith(github.event.comment.body, 'run-slow') || startsWith(github.event.comment.body, 'run slow') || startsWith(github.event.comment.body, 'run_slow') }} | ||
| cancel-in-progress: true | ||
| pull_request: |
There was a problem hiding this comment.
ig these diffs are only for testing and will be reverted? Going straight to notificaition_service.py
| fp.write(f"job name: {matrix_name}\n") | ||
| fp.write(f'job link: {matrix_job_results_extra[matrix_name]["captured_info"]["single"]["link"]}\n\n') | ||
| fp.write(f'captured_info: {matrix_job_results_extra[matrix_name]["captured_info"]["single"]["captured_info"]}') | ||
|
|
There was a problem hiding this comment.
iiuc, we are creating an easy-to-analyze table with "expected vs reality" output, and adding it in hub-dataset when the test fails
Imo it will still require a lot of human intervention to check if those "failures" are real or just flaky, failing on main, etc. Can we log it only if actual_result_main_branch != actual_result_pr_branch, that will make the list of failing tests shorter and help us find models that are actually broken
Also we might never get an actual_result_pr_branch if the test fails with a ValueError/AttributeError. I think this is another case to be logged and reported
3 participants
What does this PR do?
Fixes # (issue)
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.