fix(pd): fix adamw for paddle backend#5308
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes Paddle backend optimizer selection so that optimizer/type: AdamW actually uses Paddle’s AdamW optimizer (instead of always constructing Adam), and updates a profiler label to be optimizer-agnostic.
Changes:
- Select
paddle.optimizer.Adamvspaddle.optimizer.AdamWbased onself.opt_type. - Rename the nvprof step label from “Adam update” to “Optimizer update”.
Comments suppressed due to low confidence (1)
deepmd/pd/train/training.py:669
- Newly enabling
AdamWfor the Paddle backend changes behavior, but there doesn’t appear to be any Paddle-side test that asserts the trainer actually constructspaddle.optimizer.AdamWwhenoptimizer/typeis set toAdamW. Adding a small unit/integration test (similar to existingsource/tests/pd/test_training.pycoverage) would prevent regressions back toAdamand confirm weight-decay behavior for this backend.
opt_cls = (
paddle.optimizer.Adam
if self.opt_type == "Adam"
else paddle.optimizer.AdamW
)
self.optimizer = opt_cls(
learning_rate=self.scheduler,
parameters=self.wrapper.parameters(),
beta1=float(self.opt_param["adam_beta1"]),
beta2=float(self.opt_param["adam_beta2"]),
weight_decay=float(self.opt_param["weight_decay"]),
)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
📝 WalkthroughWalkthroughThe change introduces dynamic selection of the optimizer class (Adam or AdamW) based on configuration in the training code, replacing a hard-coded Adam instantiation. A profiling label is also updated from "Adam update" to "Optimizer update" to reflect the generic optimizer usage. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Suggested reviewers
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@deepmd/pd/train/training.py`:
- Around line 658-663: Runtime now constructs paddle.optimizer.AdamW in
training.py via opt_cls when self.opt_type == "AdamW", but the optimizer
docs/schema still mark "AdamW" as doc_only_pt_supported; update the optimizer
schema to reflect Paddle support by removing "AdamW" from the
doc_only_pt_supported set (or adding a separate marker for paddle support), and
ensure the optimizer name "AdamW" appears in the supported backends list used by
the doc generation code (look for the doc_only_pt_supported symbol and the
optimizer support mapping in argcheck.py and adjust it so "AdamW" is not
PyTorch-only).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 682953ff-1a57-4e22-8140-71e3d03f9abc
📒 Files selected for processing (1)
deepmd/pd/train/training.py
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #5308 +/- ##
==========================================
- Coverage 82.28% 82.28% -0.01%
==========================================
Files 773 773
Lines 77330 77332 +2
Branches 3659 3659
==========================================
Hits 63631 63631
- Misses 12528 12529 +1
- Partials 1171 1172 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
ba7e289
This pull request updates the optimizer handling in the
deepmd/pd/train/training.pyfile, adding support for selecting betweenAdamandAdamWoptimizers based on configuration, and improves profiling clarity.Optimizer selection improvements:
paddle.optimizer.Adamandpaddle.optimizer.AdamWdepending on the value ofself.opt_type. This allows for more flexibility in choosing the optimizer.Profiling and logging enhancements:
"Adam update"to"Optimizer update"to accurately reflect the optimizer in use, improving the clarity of profiling output.Summary by CodeRabbit