Skip to content

[Sharderformer] Support zbv in Sharderformer Policy#6150

Merged
ver217 merged 59 commits intohpcaitech:mainfrom
duanjunwen:feature/sharderformer_support_zbv
Jan 2, 2025
Merged

[Sharderformer] Support zbv in Sharderformer Policy#6150
ver217 merged 59 commits intohpcaitech:mainfrom
duanjunwen:feature/sharderformer_support_zbv

Conversation

@duanjunwen
Copy link
Copy Markdown
Contributor

@duanjunwen duanjunwen commented Nov 21, 2024

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs
  • I have installed pre-commit: pip install pre-commit && pre-commit install

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

@duanjunwen duanjunwen requested a review from a team as a code owner November 21, 2024 09:40
@duanjunwen duanjunwen requested a review from ver217 November 21, 2024 10:58
@duanjunwen duanjunwen force-pushed the feature/sharderformer_support_zbv branch from ba7fc35 to 8cb74e7 Compare December 10, 2024 07:09
@duanjunwen duanjunwen force-pushed the feature/sharderformer_support_zbv branch from 3fd2402 to 70b0ae1 Compare December 10, 2024 08:50
@duanjunwen duanjunwen force-pushed the feature/sharderformer_support_zbv branch from 44b5786 to 37b670e Compare December 10, 2024 11:26
ver217
ver217 reviewed Dec 18, 2024
View reviewed changes
Comment thread colossalai/shardformer/layer/qkv_fused_linear.py Outdated
Comment thread colossalai/shardformer/layer/qkv_fused_linear.py Outdated
Comment thread colossalai/shardformer/policies/gpt2.py Outdated
Comment thread docs/source/en/features/zerobubble_pipeline_parallelism.md
@duanjunwen duanjunwen requested a review from ver217 December 18, 2024 06:02
ver217
ver217 reviewed Dec 24, 2024
View reviewed changes
Comment thread colossalai/shardformer/layer/__init__.py
Comment thread colossalai/shardformer/layer/qkv_fused_linear.py Outdated
Comment thread colossalai/shardformer/layer/qkv_fused_linear.py Outdated
Comment thread docs/source/en/features/zerobubble_pipeline_parallelism.md Outdated
Comment thread docs/source/en/features/zerobubble_pipeline_parallelism.md
Comment thread requirements/requirements.txt Outdated
Comment thread tests/test_shardformer/test_layer/test_gpt2_qkv_fused_linear_1d.py Outdated
@duanjunwen duanjunwen requested a review from ver217 December 24, 2024 11:01
ver217
ver217 reviewed Dec 25, 2024
View reviewed changes
Comment thread colossalai/shardformer/layer/_operation.py
Comment thread docs/source/en/features/zerobubble_pipeline_parallelism.md Outdated
Comment thread tests/test_shardformer/test_layer/test_gpt2_qkv_fused_linear_1d.py Outdated
@duanjunwen duanjunwen requested a review from ver217 December 25, 2024 06:09
ver217
ver217 reviewed Dec 25, 2024
View reviewed changes
Comment thread docs/source/zh-Hans/features/zerobubble_pipeline_parallelism.md Outdated
@duanjunwen duanjunwen requested a review from ver217 December 25, 2024 09:18
@ver217 ver217 merged commit a9bedc7 into hpcaitech:main Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@ver217 ver217 ver217 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@duanjunwen @ver217