[Reverted][RL] Support Rollout Routing Replay by gongshaotian · Pull Request #5321 · PaddlePaddle/FastDeploy

gongshaotian · 2025-12-01T14:27:36Z

Reverted.
New PR: #5405

paddle-bot · 2025-12-01T17:41:49Z

Thanks for your contribution!

codecov-commenter · 2025-12-02T04:15:45Z

Codecov Report

❌ Patch coverage is 60.24590% with 97 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@b5a7abe). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...model_executor/layers/moe/routing_indices_cache.py	56.75%	59 Missing and 5 partials ⚠️
...el_executor/layers/moe/fused_moe_triton_backend.py	11.11%	6 Missing and 2 partials ⚠️
...del_executor/layers/moe/fused_moe_wint2_backend.py	0.00%	5 Missing ⚠️
fastdeploy/model_executor/layers/moe/moe.py	76.19%	4 Missing and 1 partial ⚠️
...l_executor/layers/moe/fused_moe_cutlass_backend.py	42.85%	4 Missing ⚠️
fastdeploy/engine/args_utils.py	70.00%	2 Missing and 1 partial ⚠️
...el_executor/layers/moe/fused_moe_marlin_backend.py	0.00%	3 Missing ⚠️
...odel_executor/layers/moe/fused_moe_backend_base.py	50.00%	2 Missing ⚠️
..._executor/layers/moe/fused_moe_deepgemm_backend.py	0.00%	2 Missing ⚠️
fastdeploy/worker/gpu_model_runner.py	94.11%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #5321   +/-   ##
==========================================
  Coverage           ?   59.05%           
==========================================
  Files              ?      326           
  Lines              ?    40464           
  Branches           ?     6131           
==========================================
  Hits               ?    23895           
  Misses             ?    14726           
  Partials           ?     1843

Flag	Coverage Δ
GPU	`59.05% <60.24%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

This PR introduces a Rollout Routing Replay (R3) feature for RL training tasks, enabling the recording of routing information during inference and its direct utilization in the training process to address consistency issues between training and inference in MOE models.

Key Changes

Adds RoutingReplayManager class for request-level routing table management within FastDeploy
Adds RoutingStore abstraction with local file system implementation (RDMA implementation is work-in-progress)
Integrates routing replay into MOE layers and all quantization backends via hook functions

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
fastdeploy/config.py	Adds `RoutingReplayConfig` class with configuration options for routing replay feature
fastdeploy/engine/args_utils.py	Adds CLI argument parsing and config creation for routing replay
fastdeploy/engine/engine.py	Passes routing replay config to worker service
fastdeploy/worker/worker_process.py	Adds routing replay config argument and initializes it in FDConfig
fastdeploy/worker/gpu_model_runner.py	Integrates routing replay manager with model runner, adds `is_chunk_step` tracking
fastdeploy/model_executor/forward_meta.py	Adds routing_replay_table field to ForwardMeta for passing routing table through model layers
fastdeploy/model_executor/layers/moe/routing_indices_cache.py	Core implementation: RoutingReplayManager, RoutingStore classes, and Triton kernel for saving routing data
fastdeploy/model_executor/layers/moe/moe.py	Adds forward_meta parameter and topk_ids_hookfunc to enable routing capture in MOE layers
fastdeploy/model_executor/layers/moe/*.py	Updates all MOE backend implementations to support routing replay hook function
fastdeploy/model_executor/models/glm4_moe.py	Passes forward_meta to MOE layer forward calls
fastdeploy/rl/rollout_config.py	Adds routing_replay_config parameter to rollout config

Comments suppressed due to low confidence (1)

fastdeploy/config.py:1491

Unnecessary 'pass' statement.

        pass

fastdeploy/config.py

fastdeploy/worker/worker_process.py

fastdeploy/worker/gpu_model_runner.py

fastdeploy/model_executor/layers/moe/routing_indices_cache.py

Copilot · 2025-12-04T08:45:16Z

fastdeploy/worker/worker_process.py

+        "--routing_replay_config",
+        type=json.loads,
+        default=None,
+        help="Configation of Rollout Routing Replay.",


Typo in help text: 'Configation' should be 'Configuration'.

Suggested change

help="Configation of Rollout Routing Replay.",

help="Configuration of Rollout Routing Replay.",

fastdeploy/model_executor/layers/moe/moe.py

fastdeploy/engine/args_utils.py

fastdeploy/model_executor/layers/moe/moe.py

fastdeploy/model_executor/layers/moe/routing_indices_cache.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

gongshaotian · 2025-12-04T12:56:44Z

fastdeploy/model_executor/layers/moe/moe.py


-    def forward_chunked_moe(self, x: paddle.Tensor, gate: nn.Layer, forward_meta: ForwardMeta):
+    def forward_chunked_moe(
+        self, x: paddle.Tensor, gate: nn.Layer, forward_meta: ForwardMeta, topk_ids_hookfunc: Callable = None


forward_meta 加个默认值

…nto dev_r3

…into dev_r3

…nto dev_r3

yuanlehome

LGTM

This reverts commit 96d2d48.

This reverts commit c45e064.

* [RL] Support Rollout Routing Replay * add routing indices cache * fix config bug and moe forward bug * R3 Support GLM * support eb4.5 * fix merge bug * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add routing replay ci * support glm topk * support orther top_k * fix ci bug * pre-commit * only support chatcmpl * Revert "Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402)" This reverts commit c45e064. * Fix XPU and NPU bug --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>

* [RL] Support Rollout Routing Replay * add routing indices cache * fix config bug and moe forward bug * R3 Support GLM * support eb4.5 * fix merge bug * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add routing replay ci * support glm topk * support orther top_k * fix ci bug * pre-commit * only support chatcmpl --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>

This reverts commit 96d2d48.

* [RL] Support Rollout Routing Replay * add routing indices cache * fix config bug and moe forward bug * R3 Support GLM * support eb4.5 * fix merge bug * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add routing replay ci * support glm topk * support orther top_k * fix ci bug * pre-commit * only support chatcmpl * Revert "Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402)" This reverts commit c45e064. * Fix XPU and NPU bug --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>

* [RL] Support Rollout Routing Replay * add routing indices cache * fix config bug and moe forward bug * R3 Support GLM * support eb4.5 * fix merge bug * Apply suggestion from @Copilot * Apply suggestion from @Copilot * Apply suggestion from @Copilot * Apply suggestion from @Copilot * add routing replay ci * support glm topk * support orther top_k * fix ci bug * pre-commit * only support chatcmpl * Revert "Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402)" This reverts commit c45e064. * Fix XPU and NPU bug --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com>

… tools (#5418) * feat(fmq): add ZMQ-based FMQ implementation and benchmark tools * move FMQ_CONFIG_JSON to envs * fix top_p_candidates (#5400) Co-authored-by: freeliuzc <lzc842650834@gmail.com> * [RL] Support Rollout Routing Replay (#5321) * [RL] Support Rollout Routing Replay * add routing indices cache * fix config bug and moe forward bug * R3 Support GLM * support eb4.5 * fix merge bug * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add routing replay ci * support glm topk * support orther top_k * fix ci bug * pre-commit * only support chatcmpl --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com> * [Bug fix] Fix the multi-input accuracy issue in the pooling model. (#5374) * fix multi-inputs * fix threshold * fix threshold * fix * [BugFix]remove _execute_empty_input (#5396) * Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402) This reverts commit 96d2d48. * [New][RL] Support Rollout Routing Replay (#5405) * [RL] Support Rollout Routing Replay * add routing indices cache * fix config bug and moe forward bug * R3 Support GLM * support eb4.5 * fix merge bug * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add routing replay ci * support glm topk * support orther top_k * fix ci bug * pre-commit * only support chatcmpl * Revert "Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402)" This reverts commit c45e064. * Fix XPU and NPU bug --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com> * bf16 deepseek (#5379) * fix deepseek (#5410) * Update tests/inter_communicator/test_fmq_factory.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update benchmarks/benchmark_fmq.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/inter_communicator/fmq.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: GoldPancake <56388518+Deleter-D@users.noreply.github.com> Co-authored-by: freeliuzc <lzc842650834@gmail.com> Co-authored-by: RAM <gstian5555@outlook.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Yuanle Liu <yuanlehome@163.com> Co-authored-by: lizexu123 <39205361+lizexu123@users.noreply.github.com> Co-authored-by: 周周周 <39978853+zhoutianzi666@users.noreply.github.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: bukejiyu <52310069+bukejiyu@users.noreply.github.com>

[RL] Support Rollout Routing Replay

67c4f78

gongshaotian marked this pull request as ready for review December 1, 2025 14:28

merge develop

7105af5

add routing indices cache

10f6e8b

gongshaotian added 4 commits December 2, 2025 21:20

fix config bug and moe forward bug

8d028a4

R3 Support GLM

8a46f7d

support eb4.5

0390058

merge develop

a015647

gongshaotian mentioned this pull request Dec 4, 2025

[Cherry-Pick][RL] Support Rollout Routing Replay #5166

Merged

8 tasks

fix merge bug

8df6ba4

yuanlehome requested a review from Copilot December 4, 2025 08:39

Copilot started reviewing on behalf of yuanlehome December 4, 2025 08:39 View session

Copilot finished reviewing on behalf of yuanlehome December 4, 2025 08:43

Copilot AI reviewed Dec 4, 2025

View reviewed changes

gongshaotian and others added 4 commits December 4, 2025 20:31

Apply suggestion from @Copilot

db5f169

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Apply suggestion from @Copilot

cc7c414

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Apply suggestion from @Copilot

d89c5fb

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Apply suggestion from @Copilot

62cc1f5

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

gongshaotian commented Dec 4, 2025

View reviewed changes

gongshaotian added 8 commits December 4, 2025 21:34

add routing replay ci

4c45b5f

Merge branch 'dev_r3' of https://github.com/gongshaotian/FastDeploy i…

125b704

…nto dev_r3

Merge branch 'develop' of https://github.com/PaddlePaddle/FastDeploy …

a107996

…into dev_r3

support glm topk

e37bbea

Merge branch 'dev_r3' of https://github.com/gongshaotian/FastDeploy i…

3a368de

…nto dev_r3

support orther top_k

957434f

fix ci bug

ac8c077

pre-commit

2762302

yuanlehome previously approved these changes Dec 5, 2025

View reviewed changes

yuanlehome and others added 2 commits December 5, 2025 13:53

Merge branch 'develop' into dev_r3

6122ecb

only support chatcmpl

11a68bb

lizexu123 dismissed yuanlehome’s stale review via 11a68bb December 5, 2025 06:51

yuanlehome approved these changes Dec 5, 2025

View reviewed changes

YuanRisheng added the skip-ci: coverage label Dec 5, 2025

Jiang-Jia-Jun merged commit 96d2d48 into PaddlePaddle:develop Dec 5, 2025
12 of 18 checks passed

Jiang-Jia-Jun added a commit that referenced this pull request Dec 5, 2025

Revert "[RL] Support Rollout Routing Replay (#5321)"

f4c69f3

This reverts commit 96d2d48.

Jiang-Jia-Jun mentioned this pull request Dec 5, 2025

Revert "[RL] Support Rollout Routing Replay" #5402

Merged

Jiang-Jia-Jun added a commit that referenced this pull request Dec 5, 2025

Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402)

c45e064

This reverts commit 96d2d48.

gongshaotian added a commit that referenced this pull request Dec 5, 2025

Revert "Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402)"

ad4317e

This reverts commit c45e064.

gongshaotian changed the title ~~[RL] Support Rollout Routing Replay~~ [Reverted][RL] Support Rollout Routing Replay Dec 5, 2025

This was referenced Dec 5, 2025

[Cherry-Pick][New][RL] Support Rollout Routing Replay (#5405) #5407

Closed

[Cherry-Pick][New][RL] Support Rollout Routing Replay (#5405) #5408

Merged

sunlei1024 pushed a commit that referenced this pull request Dec 6, 2025

Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402)

e70351c

This reverts commit 96d2d48.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Reverted][RL] Support Rollout Routing Replay#5321

[Reverted][RL] Support Rollout Routing Replay#5321
Jiang-Jia-Jun merged 22 commits intoPaddlePaddle:developfrom
gongshaotian:dev_r3

gongshaotian commented Dec 1, 2025 •

edited

Loading

Uh oh!

paddle-bot bot commented Dec 1, 2025

Uh oh!

codecov-commenter commented Dec 2, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gongshaotian Dec 4, 2025

Uh oh!

yuanlehome left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	help="Configation of Rollout Routing Replay.",
	help="Configuration of Rollout Routing Replay.",

Conversation

gongshaotian commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paddle-bot bot commented Dec 1, 2025

Uh oh!

codecov-commenter commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gongshaotian Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

yuanlehome left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gongshaotian commented Dec 1, 2025 •

edited

Loading

codecov-commenter commented Dec 2, 2025 •

edited

Loading