Skip to content

Conversation

@ndido98
Copy link
Contributor

@ndido98 ndido98 commented Oct 21, 2025

@ mention of reviewers

@ihsaan-ullah

A brief description of the purpose of the changes contained in this PR.

This PR ensures that BestModeStrategy, used for publishing results on the leaderboard by forcing the best result, only picks up submissions that have finished (and may have a score attached) and are not soft deleted.

Issues this PR resolves

Fixes #2044
Fixes #1284 (failed submissions are now discarded, blocking n/a scores associated with failed submissions).
Fixes #1840
Fixes #2057

A checklist for hand testing

  • Create a leaderboard with "Force best" strategy
  • Create a submission, it should pop up on the leaderboard, as it's the only one
  • Soft delete it, it should disappear from the leaderboard but still count towards the daily/per-person limit
  • Reupload the same submission (or upload a submission with a worse score), this should now be present on the leaderboard

Checklist

  • Code review by me
  • Hand tested by me
  • I'm proud of my work
  • Code review by reviewer

@Didayolo
Copy link
Member

Didayolo commented Nov 18, 2025

Thank you very much for your contribution. This is a very important bug that we need to fix as soon as possible.

May fixes also #1840.

@Didayolo Didayolo self-assigned this Nov 18, 2025
@Didayolo Didayolo changed the base branch from develop to fix-bestmodestrategy November 18, 2025 14:46
@Didayolo
Copy link
Member

Merged into codalab/codabench to trigger the CI tests.

@Didayolo Didayolo merged commit 71a50d3 into codalab:fix-bestmodestrategy Nov 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants