Skip to content

[ShardFormer]: Bert fine tune example with hybrid parallel plugin #4566

Closed
ppt0011 wants to merge 3 commits intohpcaitech:feature/shardformerfrom
ppt0011:feature/shardformer
Closed

[ShardFormer]: Bert fine tune example with hybrid parallel plugin #4566
ppt0011 wants to merge 3 commits intohpcaitech:feature/shardformerfrom
ppt0011:feature/shardformer

Conversation

@ppt0011
Copy link
Copy Markdown
Contributor

@ppt0011 ppt0011 commented Aug 31, 2023

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge
contributed #4567
e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

hybrid parallel plugin can now be used for bert fine tune example, except pp+zero1
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

@ppt0011 ppt0011 changed the title Feature/shardformer [ShardFormer]: Bert fine tune example with hybrid parallel plugin Aug 31, 2023
@ppt0011 ppt0011 closed this Sep 5, 2023
@ppt0011
Copy link
Copy Markdown
Contributor Author

ppt0011 commented Sep 5, 2023

a similar pr has been merged #4584

@Issues-translate-bot
Copy link
Copy Markdown

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


a similar pr has been merged #4584

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants