chore: Bump vllm to 0.11.2, torch to 2.9, transformers to 4.57.1

[yfw]

What does this PR do ?

Updates vllm to 0.11.2, torch to 2.9, transformers to 4.57.1. Also updates Automodel to use main branch.

Issues

List issues that this PR closes (syntax):

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

Summary by CodeRabbit

Release Notes

• Updates

  • Upgraded core dependencies: PyTorch (2.9.0), Transformers (4.57.1), and vLLM (0.11.2)
  • Added multi-node distributed training configuration support

• Improvements

  • Enhanced vLLM integration robustness and compatibility
  • Optimized FP8 weight handling for improved efficiency

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[github-actions]

❌ Submodule Fast-Forward Check Failed

Check based on commit: 82b6f95 (PR #1563 from yifu/vllm0112_bump)

❌ Submodules that need attention:

Automodel: ❌ Commits have DIVERGED from a common ancestor
TARGET (main branch): https://github.com/NVIDIA-NeMo/Automodel/commits/a2db048383cd54b3fafc928df4c30bf7bbf7c430/
CURRENT (PR #1563 from yifu/vllm0112_bump): https://github.com/NVIDIA-NeMo/Automodel/commits/f9fc82c055e1cc69a68ff0bc7614aabe507a43ea/

Please ensure all submodule commits are fast-forwards of the main branch before merging.

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: 39a9b03 (PR #1563 from yifu/vllm0112_bump)

✅ Submodules that are properly updated:

Automodel: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: fa2ccf4 (PR #1563 from yifu/vllm0112_bump)

✅ Submodules that are properly updated:

Automodel: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: baf37d6 (PR #1563 from yifu/vllm0112_bump)

✅ Submodules that are properly updated:

Automodel: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: b671719 (PR #1563 from yifu/vllm0112_bump)

✅ Submodules that are properly updated:

Automodel: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR updates core dependencies (PyTorch 2.9.0, transformers 4.57.1, vLLM 0.11.2), refactors vLLM worker initialization to use dynamic file lookup instead of hardcoded paths, introduces in-place FP8 weight post-processing, and migrates test configurations from single-node to two-node setups.

Changes

Cohort / File(s)	Summary
Dependency Updates pyproject.toml, tools/build-custom-vllm.sh	Bumped PyTorch from 2.8.0 to 2.9.0, transformers from ≥4.55.4 to ≥4.57.1, and vLLM from 0.11.0 to 0.11.2; updated CUDA wheel index from cu128 to cu129.
Submodule Updates .gitmodules, 3rdparty/Automodel-workspace/Automodel	Changed Automodel submodule branch to yifu/bump-torch-and-hf and updated commit pointer.
Configuration & Test Updates examples/configs/recipes/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.yaml, tests/test_suites/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.sh, tests/test_suites/nightly.txt	Updated checkpoint/log paths from 1n8g to 2n8g, set cluster.num_nodes: 2, changed NUM_NODES from 1 to 2, and switched nightly test reference from 1n8g to 2n8g configuration.
FP8 Weight Processing nemo_rl/models/generation/fp8.py	Added maybe_post_process_fp8_weight_block() function for in-place FP8 weight and scale re-quantization; adjusted process_weights_after_loading() to call the new function without extra parameters.
vLLM Worker Initialization nemo_rl/models/generation/vllm/vllm_worker.py	Replaced hardcoded vLLM import paths with dynamic file lookup via importlib.find_spec(); introduced _get_vllm_file() helper and two new patching functions (_patch_vllm_init_workers_ray() and _patch_vllm_vit_flash_attn_backend()) for robust version-agnostic patching.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• vllm_worker.py: Carefully verify dynamic file lookup logic handles vLLM versions robustly and that patched behaviors (Ray executor env variables, attention backend override) remain correct.
• fp8.py: Ensure in-place weight/scale updates preserve gradient flow and model training behavior; confirm compatibility with DeepGemm E8M0 quantization.
• Dependency compatibility: Cross-check PyTorch 2.9.0, transformers 4.57.1, and vLLM 0.11.2 compatibility across different compute environments and CUDA versions (cu129).
• Multi-node test configurations: Validate that 2-node setup produces expected distributed behavior and resource allocation.

Possibly related PRs

• fix: update the custom vllm instructions #1116: Updates custom vLLM build tooling and dependency instructions similar to this PR's torch/vLLM version bumps.
• chore: major version bump (torch 2.8, vllm 0.11, ray 2.49) & SP fixes #1334: Modifies the same vLLM worker integration and related dependency/test configurations.

Suggested reviewers

• terrykong
• parthchadha
• guyueh1

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.
Test Results For Major Changes	⚠️ Warning	PR introduces major dependency bumps (torch 2.9.0, transformers 4.57.1, vllm 0.11.2) and code changes to FP8 quantization and vLLM workers, but PR description contains no test results, performance benchmarks, or compatibility verification. Review comments flag incompatibility issues with this dependency combination.	Document comprehensive test results in PR description including: integration test results with updated dependencies, training convergence verification, 2-node test configuration results, FP8 quantization validation, and confirmation that torch 2.9.0 + transformers 4.57.1 combination has been tested.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main changes: dependency version bumps for vllm, torch, and transformers, which are the primary modifications across multiple files.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch yifu/vllm0112_bump

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 4

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 25ff3f6 and b671719.

⛔ Files ignored due to path filters (1)

• uv.lock is excluded by !**/*.lock

📒 Files selected for processing (9)

• .gitmodules (1 hunks)
• 3rdparty/Automodel-workspace/Automodel (1 hunks)
• examples/configs/recipes/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.yaml (2 hunks)
• nemo_rl/models/generation/fp8.py (2 hunks)
• nemo_rl/models/generation/vllm/vllm_worker.py (2 hunks)
• pyproject.toml (5 hunks)
• tests/test_suites/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.sh (1 hunks)
• tests/test_suites/nightly.txt (2 hunks)
• tools/build-custom-vllm.sh (1 hunks)

🧰 Additional context used

📓 Path-based instructions (9)

**/*.sh

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

**/*.sh: Use uv run instead of python to execute scripts
Follow the Google Shell Style Guide for shell scripts

Files:

• tools/build-custom-vllm.sh
• tests/test_suites/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.sh

!(**/tests/**|**/test_*.py|**/test_*.sh)

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Add the NVIDIA copyright header to all Python files and shell scripts (excluding tests). The header should include the current year

Files:

• tools/build-custom-vllm.sh
• pyproject.toml
• .gitmodules
• examples/configs/recipes/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.yaml
• tests/test_suites/nightly.txt
• tests/test_suites/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.sh
• nemo_rl/models/generation/vllm/vllm_worker.py
• nemo_rl/models/generation/fp8.py
• 3rdparty/Automodel-workspace/Automodel

**/*.{py,sh}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

The NVIDIA copyright header should appear at the top of all Python files and shell scripts (excluding tests)

Files:

• tools/build-custom-vllm.sh
• tests/test_suites/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.sh
• nemo_rl/models/generation/vllm/vllm_worker.py
• nemo_rl/models/generation/fp8.py

examples/configs/recipes/**/*.yaml

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

When adding support for a new model, create a recipe YAML under examples/configs/recipes/ in the appropriate domain subdirectory (llm, vlm, etc.)

Files:

• examples/configs/recipes/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.yaml

examples/configs/recipes/llm/*.yaml

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Recipe YAML files should follow the naming pattern: --ng-[-modifiers][-long][.vN].yaml for LLM recipes

Files:

• examples/configs/recipes/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.yaml

tests/test_suites/nightly.txt

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

When adding a nightly test for a new model, append the driver script path (relative to tests/test_suites/) to tests/test_suites/nightly.txt

Files:

• tests/test_suites/nightly.txt

tests/test_suites/**/*.sh

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

tests/test_suites/**/*.sh: When adding support for a new model, create a corresponding driver shell script under tests/test_suites/ in the matching domain
Driver shell scripts should match the YAML base name with .sh extension and invoke training entrypoint with uv run

Files:

• tests/test_suites/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.sh

**/*.py

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

**/*.py: Conform code to Python 3.12+
Indent code with 4 spaces. Do not use tabs
Use snake_case for file names
Use PascalCase for class names
Use snake_case for function and method names
Use snake_case for local variables
Prefix variable names that start with a number with 'k' (e.g., k_99th_percentile)
Use upper snake_case with 'G' prefix for global variables (e.g., G_MY_GLOBAL)
Use upper snake_case for constants
Avoid shadowing variables declared in an outer scope
Initialize all externally visible members of a class in the constructor
Prefer docstrings over comments for interfaces that may be used outside a file
Reserve comments for code within a function or interfaces that are local to a file
If a piece of code is commented out, include a comment describing its usage and why it's commented out. Remove debug comments before merging
Use Google style docstrings for classes and functions in Python, which can be parsed by Sphinx
Avoid using reflection when functionality can be easily achieved without reflection
When using try-except blocks, limit the except clause to the smallest set of specific errors possible
When using try-except blocks for duck-typing, keep the body of the try as small as possible and use the else block for logic
YAML is the single source of truth for configuration defaults. Do not set non-None defaults in code for configuration values
For required configuration attributes, access config directly and expect presence (e.g., policy_cfg['precision']) without hidden defaults
Use typing.NotRequired to mark optional attributes in TypedDict for configuration
When adding a new config key to a TypedDict subclass, document the key's purpose, valid values/types, and recommended default, and reflect the default in exemplar YAMLs under examples/configs/*.yaml
Follow the Google Python Style Guide for Python code

Files:

• nemo_rl/models/generation/vllm/vllm_worker.py
• nemo_rl/models/generation/fp8.py

nemo_rl/**/*.py

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

For any source file under nemo_rl/*.py that defines a class or function decorated with @ray.remote, add a coverage pragma (# pragma: no cover) because these run in separate Ray processes

Files:

• nemo_rl/models/generation/vllm/vllm_worker.py
• nemo_rl/models/generation/fp8.py

🧠 Learnings (6)

📚 Learning: 2025-11-24T17:24:41.976Z

Learnt from: CR
Repo: NVIDIA-NeMo/RL PR: 0
File: CODING_GUIDELINES.md:0-0
Timestamp: 2025-11-24T17:24:41.976Z
Learning: Applies to examples/configs/recipes/llm/*.yaml : Recipe YAML files should follow the naming pattern: <algo>-<model>-<nodes>n<gpus>g-<strategy-and-params>[-modifiers][-long][.vN].yaml for LLM recipes

Applied to files:

• examples/configs/recipes/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.yaml

📚 Learning: 2025-11-24T17:24:41.976Z

Learnt from: CR
Repo: NVIDIA-NeMo/RL PR: 0
File: CODING_GUIDELINES.md:0-0
Timestamp: 2025-11-24T17:24:41.976Z
Learning: Applies to examples/configs/recipes/vlm/*.yaml : Recipe YAML files should follow the naming pattern: vlm_<algo>-<model>-<nodes>n<gpus>g-<strategy>[-modifiers][.vN].yaml for VLM recipes

Applied to files:

• examples/configs/recipes/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.yaml

📚 Learning: 2025-09-18T13:26:43.307Z

Learnt from: zpqiu
Repo: NVIDIA-NeMo/RL PR: 1006
File: examples/configs/recipes/llm/distillation-qwen3-32b-to-8b-base-2n8g-fsdp2tp2.v1.yaml:19-26
Timestamp: 2025-09-18T13:26:43.307Z
Learning: In on-policy distillation workflows, validation can use downstream task performance (like math problem solving) as RL-like reward metrics rather than traditional distillation metrics like KL divergence. In this case, "val_reward" with "higher_is_better: true" is the correct checkpoint monitoring configuration.

Applied to files:

• examples/configs/recipes/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.yaml

📚 Learning: 2025-11-24T17:24:41.976Z

Learnt from: CR
Repo: NVIDIA-NeMo/RL PR: 0
File: CODING_GUIDELINES.md:0-0
Timestamp: 2025-11-24T17:24:41.976Z
Learning: Applies to tests/test_suites/nightly.txt : When adding a nightly test for a new model, append the driver script path (relative to tests/test_suites/) to tests/test_suites/nightly.txt

Applied to files:

• tests/test_suites/nightly.txt

📚 Learning: 2025-10-12T14:46:57.171Z

Learnt from: zpqiu
Repo: NVIDIA-NeMo/RL PR: 1324
File: tests/test_suites/llm/distillation-qwen3-32b-to-1.7b-base-1n8g-megatron-tp2pp2cp2-pack.sh:6-11
Timestamp: 2025-10-12T14:46:57.171Z
Learning: Test scripts in tests/test_suites/llm/ follow a standard configuration pattern that includes NUM_NODES, STEPS_PER_RUN, MAX_STEPS, NUM_RUNS (calculated as `$(( (MAX_STEPS + STEPS_PER_RUN - 1) / STEPS_PER_RUN ))`), and NUM_MINUTES. These variables are part of the test infrastructure's standard interface and should not be flagged as unused even if not directly referenced within the individual script, as they are consumed by external launch tooling or common.env.

Applied to files:

• tests/test_suites/nightly.txt
• tests/test_suites/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.sh

📚 Learning: 2025-11-06T22:30:22.860Z

Learnt from: ZhiyuLi-Nvidia
Repo: NVIDIA-NeMo/RL PR: 1477
File: nemo_rl/models/generation/vllm/vllm_backend.py:163-168
Timestamp: 2025-11-06T22:30:22.860Z
Learning: For Ray actor methods in the vLLM generation worker code (vllm_backend.py, vllm_worker.py, vllm_worker_async.py), error handling should use print/traceback + return False pattern rather than raising exceptions, following the Ray RPC practice where exceptions may not propagate well across process boundaries.

Applied to files:

• nemo_rl/models/generation/vllm/vllm_worker.py

🪛 Ruff (0.14.6)

nemo_rl/models/generation/vllm/vllm_worker.py

177-181: Avoid specifying long messages outside the exception class

(TRY003)

---

187-192: Avoid specifying long messages outside the exception class

(TRY003)

---

221-221: zip() without an explicit strict= parameter

Add explicit value for parameter strict=

(B905)

🪛 Shellcheck (0.11.0)

tests/test_suites/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.sh

[warning] 6-6: NUM_NODES appears unused. Verify use (or export if used externally).

(SC2034)

🔇 Additional comments (12)

examples/configs/recipes/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.yaml (1)

9-9: LGTM—consistent 2-node config update with proper naming.

The changes correctly scale the configuration from 1-node to 2-node setup:

• Directory/checkpoint identifiers updated consistently (1n8g → 2n8g).
• cluster.num_nodes: 2 properly specified with gpus_per_node: 8 (line 59).
• Filename follows the required pattern per coding guidelines: <algo>-<model>-<nodes>n<gpus>g-<strategy-and-params>[-modifiers][-long>[.vN].yaml.
• Changes align with the broader PR context (test suite migrations and dependency updates).

Also applies to: 51-51, 56-56, 58-58

.gitmodules (1)

14-14: Clarify submodule branch intent—potential merge blocker.

Line 14 points to yifu/bump-torch-and-hf (a feature branch). Pinning submodules to feature branches creates build fragility: the branch can be deleted, have its history rewritten, or go stale, potentially breaking CI and releases.

Before merging, confirm:

1. Is this feature branch intended to be temporary, or should it remain in production?
2. If temporary, ensure the Automodel repo merges this branch to main (or another stable ref) before this PR merges.
3. If this branch must remain referenced, update the PR description to document the dependency and expected lifetime.

3rdparty/Automodel-workspace/Automodel (1)

1-1: Verify that the new submodule commit is compatible with the dependency bumps.

The submodule pointer has been updated to a new commit. Given that this PR includes significant dependency bumps (torch 2.9, transformers 4.57.1, vLLM 0.11.2), please verify that the new Automodel commit (910f4e0402ec3af0c3b8642639f0347732067630) is compatible with these updated versions and does not introduce any breaking changes or version conflicts.

Consider verifying:

• The new commit exists in the Automodel repository
• What changes are included in the new commit and whether they align with the dependency upgrades
• Whether any configuration or compatibility adjustments are needed in consuming code

tests/test_suites/nightly.txt (1)

19-19: LGTM!

The nightly test suite update correctly reflects the shift from 1n8g to 2n8g configuration, and the comment update properly describes the moonlight run section.

Also applies to: 42-42

tests/test_suites/llm/grpo-llama3.1-8b-instruct-2n8g-megatron-fp8-e2e.sh (1)

5-11: LGTM!

The NUM_NODES=2 update correctly aligns with the 2n8g configuration indicated in the filename. The ShellCheck warning about NUM_NODES being unused is a false positive—based on learnings, these variables are consumed by external launch tooling or common.env.

pyproject.toml (2)

60-60: LGTM!

The vllm version is consistently updated to 0.11.2 across all optional-dependencies sections (automodel, vllm, mcore).

Also applies to: 72-72, 95-95

---

106-108: LGTM!

The build dependency group correctly updates torch to 2.9.0, maintaining consistency with the main dependencies section.

tools/build-custom-vllm.sh (1)

69-69: LGTM!

The torch installation correctly updates to 2.9.0 with the cu129 wheel index, maintaining consistency with pyproject.toml. The xformers version (0.0.32.post1) on line 61 is appropriately updated to be compatible with torch 2.9.

nemo_rl/models/generation/fp8.py (2)

426-457: LGTM! In-place FP8 weight post-processing preserves weight_loader compatibility.

The implementation correctly:

1. Checks if DeepGemm should be used before processing
2. Uses .data.copy_() for in-place updates instead of creating new torch.nn.Parameter objects, preserving the weight_loader attribute needed for refit
3. Properly references the vLLM source for traceability

The lazy imports are appropriate for optional vLLM dependencies.

---

459-484: LGTM!

The integration of maybe_post_process_fp8_weight_block at the end of process_weights_after_loading is correct. The call order ensures layer.weight_scale is properly initialized before the DeepGemm-specific post-processing is applied.

nemo_rl/models/generation/vllm/vllm_worker.py (2)

19-19: LGTM!

The import of find_spec is appropriate for the new dynamic vLLM file discovery approach.

---

168-194: LGTM!

The helper function provides robust runtime discovery of vLLM files with clear error messages. The detailed error messages flagged by ruff (TRY003) are actually beneficial for debugging installation and version issues.

[github-actions]

⚠️ File Consistency Check

Check based on commit: 7899902 (PR #1563 from yifu/vllm0112_bump)

⚠️ DTensor Policy Worker Synchronization Warning

The file nemo_rl/models/policy/workers/dtensor_policy_worker_v2.py was modified in this PR, but nemo_rl/models/policy/workers/dtensor_policy_worker.py was not updated.

Why this matters:
These files contain related DTensor policy worker implementations that should be kept synchronized to ensure consistency across different versions.

Action required:

• Please review if the changes in nemo_rl/models/policy/workers/dtensor_policy_worker_v2.py should also be applied to nemo_rl/models/policy/workers/dtensor_policy_worker.py
• Update nemo_rl/models/policy/workers/dtensor_policy_worker.py if necessary to maintain consistency
• If the files are intentionally different, please add a comment in the PR explaining why

Files to check:

• Modified: nemo_rl/models/policy/workers/dtensor_policy_worker_v2.py
• Not modified: nemo_rl/models/policy/workers/dtensor_policy_worker.py

---

_{This check ensures that related file implementations remain synchronized across the codebase. If you believe this warning is incorrect or the files should intentionally differ, please add a comment explaining the reasoning.}

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: 7899902 (PR #1563 from yifu/vllm0112_bump)

✅ Submodules that are properly updated:

Automodel: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨