Skip to content

fix: deepscaler-24k test reduce to 10 steps to safely finish in 4 hr#1280

Merged
terrykong merged 2 commits intomainfrom
tk/deepscaler-not-enough-time
Oct 7, 2025
Merged

fix: deepscaler-24k test reduce to 10 steps to safely finish in 4 hr#1280
terrykong merged 2 commits intomainfrom
tk/deepscaler-not-enough-time

Conversation

@terrykong
Copy link
Copy Markdown
Collaborator

@terrykong terrykong commented Oct 6, 2025

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Issues

List issues that this PR closes (syntax):

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

Summary by CodeRabbit

  • Tests
    • Updated the LLM GRPO Deepscaler 1.5b-24K test suite to reduce steps per run and total steps (15 → 10), decreasing execution time and resource usage.
    • Test iteration counts adjust automatically based on the new values.
    • Improves CI throughput and feedback speed with no changes to user-facing behavior or features.

Signed-off-by: Terry Kong <terryk@nvidia.com>
@terrykong terrykong requested a review from chtruong814 October 6, 2025 16:07
@terrykong terrykong requested a review from a team as a code owner October 6, 2025 16:07
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Oct 6, 2025

📝 Walkthrough

Walkthrough

Adjusted test parameters in a single LLM test script by reducing STEPS_PER_RUN and MAX_STEPS from 15 to 10; derived NUM_RUNS will change accordingly. No other logic or public interfaces were modified.

Changes

Cohort / File(s) Summary
Test suite parameter tweaks
tests/test_suites/llm/grpo-deepscaler-1.5b-24K.sh
Reduced STEPS_PER_RUN: 15→10 and MAX_STEPS: 15→10; NUM_RUNS implicitly affected; no other logic changes.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

Suggested labels

CI:L0

Suggested reviewers

  • chtruong814
  • guyueh1

Pre-merge checks and finishing touches

✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title clearly and concisely summarizes the main change by indicating that the deepscaler-24k test steps are reduced to 10 to ensure completion within four hours, directly matching the modifications to STEPS_PER_RUN and MAX_STEPS in the changeset.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
Test Results For Major Changes ✅ Passed The PR only tweaks step counts in a test script, which is a minor configuration adjustment rather than a major feature or refactor, so the absence of documented test results is acceptable under the custom check criteria.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch tk/deepscaler-not-enough-time

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7aabb81 and 1f4f886.

📒 Files selected for processing (1)
  • tests/test_suites/llm/grpo-deepscaler-1.5b-24K.sh (1 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
**/*.sh

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

**/*.sh: Follow the Google Shell Style Guide for all shell scripts
Use uv run to execute Python scripts in shell/driver scripts instead of activating virtualenvs and calling python directly
Add the NVIDIA copyright header (with current year) at the top of all shell scripts, excluding tests/ and test-only scripts

Files:

  • tests/test_suites/llm/grpo-deepscaler-1.5b-24K.sh
tests/test_suites/llm/*.sh

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

LLM driver script filenames must mirror the YAML base name and follow the same pattern with .sh extension

Files:

  • tests/test_suites/llm/grpo-deepscaler-1.5b-24K.sh
tests/test_suites/**

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Place driver shell scripts and common.env under tests/test_suites// and list nightly tests in tests/test_suites/nightly.txt

Files:

  • tests/test_suites/llm/grpo-deepscaler-1.5b-24K.sh
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Lint check
  • GitHub Check: Post automodel integration comment / Comment on PR
  • GitHub Check: Post submodule check comment / Comment on PR
🔇 Additional comments (1)
tests/test_suites/llm/grpo-deepscaler-1.5b-24K.sh (1)

7-8: Config tweak looks good

Lowering both STEPS_PER_RUN and MAX_STEPS to 10 keeps the NUM_RUNS math and downstream checkpoint/metric steps consistent while shortening the overall run as intended. Thanks for tightening the runtime window.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@terrykong terrykong added CI:docs Run doctest CI:L0 Run doctests and unit tests labels Oct 7, 2025
@terrykong terrykong enabled auto-merge (squash) October 7, 2025 00:40
@terrykong terrykong added r0.4.0 and removed CI:docs Run doctest labels Oct 7, 2025
@terrykong terrykong added CI:L0 Run doctests and unit tests and removed CI:L0 Run doctests and unit tests labels Oct 7, 2025
@terrykong terrykong added the CI:docs Run doctest label Oct 7, 2025
@terrykong terrykong removed the CI:L0 Run doctests and unit tests label Oct 7, 2025
@terrykong terrykong merged commit 806e285 into main Oct 7, 2025
38 of 39 checks passed
@terrykong terrykong deleted the tk/deepscaler-not-enough-time branch October 7, 2025 21:17
chtruong814 pushed a commit that referenced this pull request Oct 7, 2025
…1280)

Signed-off-by: Terry Kong <terryk@nvidia.com>
Signed-off-by: NeMo Bot <nemo-bot@nvidia.com>
PrinsYin pushed a commit to PrinsYin/RL that referenced this pull request Nov 30, 2025
yuanhangsu1986 pushed a commit to yuanhangsu1986/RL-Nemontron-Edge-Omni that referenced this pull request Feb 21, 2026
…VIDIA-NeMo#1280)

Signed-off-by: Terry Kong <terryk@nvidia.com>
Signed-off-by: yuanhangs <yuanhangs@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants