[Feature] Support eplb for fd #4599
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
kevincheng2
previously approved these changes
Oct 31, 2025
kevincheng2
approved these changes
Nov 3, 2025
yuanlehome
reviewed
Nov 3, 2025
| rank_expert_list, logical_to_physical_map, expert_count | ||
| ) | ||
| # TO BE FIXED | ||
| self.worker.get_model().update_state_dict(state_dicts) |
Collaborator
There was a problem hiding this comment.
注意，后续这里需要采用copy_覆盖原始权重的形式，而不是替换原Tensor对象
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Support EPLB.
为保证MoE部分不同专家之间的负载均衡，会将共享专家和高负载的细粒度专家在集群的不同GPU做多个复制，让GPU把更多的热数据（发给共享专家的）跑起来。
EPLB 通过复制高负载专家（Redundant Experts Strategy）并对专家分配进行启发式调整，确保不同 GPU 之间的负载均衡。这种方法解决了专家并行中因专家负载不均导致的计算资源浪费问题。分层负载平衡策略也可用于预填充阶段，具有较小的专家并行规模。