Skip to content

[coati] add vf_coef argument for PPOTrainer#3318

Merged
ht-zhou merged 1 commit intohpcaitech:mainfrom
zhang-yi-chi:feature/vf_coef_for_ppo_trainer
Apr 11, 2023
Merged

[coati] add vf_coef argument for PPOTrainer#3318
ht-zhou merged 1 commit intohpcaitech:mainfrom
zhang-yi-chi:feature/vf_coef_for_ppo_trainer

Conversation

@zhang-yi-chi
Copy link
Copy Markdown
Contributor

@zhang-yi-chi zhang-yi-chi commented Mar 29, 2023 •
edited
Loading

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

#3317

📝 What does this PR do?

  1. Adding an argument vf_coef for value loss which is common in standard PPO implementation.
  2. Setting default multiplier of value loss to 0.5 as shown in the original paper.

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

@zhang-yi-chi zhang-yi-chi changed the title [coati]: add vf_coef argument for PPOTrainer [coati] add vf_coef argument for PPOTrainer Mar 29, 2023
ht-zhou
ht-zhou reviewed Apr 6, 2023
View reviewed changes
Comment thread applications/Chat/coati/models/loss.py
@ht-zhou ht-zhou self-assigned this Apr 6, 2023
@ht-zhou ht-zhou merged commit e6a132a into hpcaitech:main Apr 11, 2023
@zhang-yi-chi zhang-yi-chi mentioned this pull request Apr 11, 2023
Closed
jamesthesnake added a commit to jamesthesnake/ColossalAI that referenced this pull request Apr 11, 2023
@jamesthesnake
Merge pull request #18 from hpcaitech/main
aa1fedb 
[chat]: add vf_coef argument for PPOTrainer (hpcaitech#3318)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@ht-zhou ht-zhou ht-zhou approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@ht-zhou ht-zhou

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@zhang-yi-chi @ht-zhou