[Graph Optimization] Refactor default capture list #4630

gongshaotian · 2025-10-28T12:58:54Z

Motivation

When use CUDAGraph with MTP, the default-constructed capture list method ignores the max shape, which is computed as:
self.scheduler_config.max_num_seqs * (self.speculative_config.num_speculative_tokens + 1)

Origin PR: #4617

Modifications

Refactor default capture list
Refie debug log

Usage or Command

Pass

Accuracy Tests

Pass

Checklist

Add at least a tag in the PR title.
- Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
- You can add new tags based on the PR content, but the semantics must be clear.
Format your code, run pre-commit before commit.
Add unit tests. Please write the reason in this PR if no unit tests.
Provide accuracy results.
If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

* update enable chunked_prefill * update code * update code * update code

…ddle#3794) Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com>

* Update config.py * Update ep.py * Update fused_moe_backend_base.py * Update dynamic_weight_manager.py * Update worker_process.py * fix ci

* Update serving_chat.py * Update serving_completion.py * Update serving_completion.py

…ddlePaddle#3817)

…) (PaddlePaddle#3804) * 延迟 import Config * support chunked_prefill * support chunked_prefill

…ePaddle#3810) * speed up eb45 * update

* fix scheduler bug * fix

* add moe noaux_tc tatics in trition backend * fix * add dp config

* Update no_proxy environment variable in CI workflow * Install lsof and kill api_server processes Install lsof tool and kill processes using it.

…se (PaddlePaddle#3855) * Update no_proxy environment variable in CI workflow * Install lsof and kill api_server processes Install lsof tool and kill processes using it. * Update dependency versions for stable release * Update CI script to use stable dependencies

…le#3771) (PaddlePaddle#3835) * fix w4afp8 * 增加集中式配置 * codestyle * fix fa3 append attn

Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>

* Support for async processor added. * remove yappi code

* [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default * [Feature] Set scheduler v1 as default

* fix scheduler bug * fix * Update api_server.py

* add reasoning parser plugin * fix finish reason --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com>

* [DEBUG] Adapt validation for paddleformers==0.2 in release/2.2 * [CI] update paddleformers==0.2 in release/2.2

* disable scheduler v1 in guided decoding * disable scheduler v1 in guided decoding

…he_messager in cudagraph and PD (PaddlePaddle#4252) * fix the bug for prefilled_step_idx signal of cache_messager in cudagraph and PD * support dp

* [Executor]CUDAGraph support Speculate Decode * fix problem * solve problem * fix * fast compile * CUDAGraph + mtp support eb5(only target model) * Revert "fast compile" This reverts commit 3cfe837. * fix precommit * solve comment * fix comment about #pragram unroll --------- Co-authored-by: gongshaotian <gstain5555@outlook.com> Co-authored-by: gongshaotian <gstian5555@outlook.com>

…le#4392) * supports spec dynamic cfp8 * supports spec dynamic cfp8 --------- Co-authored-by: lzy <569782149@qq.com>

* Update common_engine.py * Update zmq_client.py * Update expert_service.py --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>

* support mtp chunk_prefill_v1 * fix mtp chunkprefill output * fix mtp chunkprefill output, fix unit test * fix save_output --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>

* [setup optimize]Support git submodule (PaddlePaddle#4033) * support git submodule * update setup * fix ci network * fix clone * revert clone linux * delete args * fix ci * update * fix --------- Co-authored-by: YuanRisheng <yuanrisheng@baidu.com>

* support speculate_limit_thinking_content_length_v2 * fix * fix import

* fix: image token output * fix: code style * fix: CompletionOutput.decode_type

… MTP + EP (PaddlePaddle#4430) * Fix MTP dummy run bug * Target Model and Draft Model using the same flag * aovid moe bug in cudagraph padding * In mtp replace use_cudagraph as step_use_cudagraph

* add einops * add einops

* support logprob in mtp * remove debug code * fix * feat: add draft_logprobs for Speculative Decode MTP * Revert "feat: add draft_logprobs for Speculative Decode MTP" This reverts commit d5a3c5c. * fix * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * fix some bugs * fix codestyle * fix bugs * fix bugs * fix bugs * fix bus * fix bugs * fix unitest --------- Co-authored-by: sunlei1024 <sunlei5788@gmail.com> Co-authored-by: sunlei18 <sunlei18@sunlei18deMacBook-Pro.local>

* support mm prefix cache close * add * fix * fix * fix --------- Co-authored-by: ltd0924 <luotingdan@baidu.com>

* [fix] fix requests & block metrics * [chore] rename variables

) * [CI] update ernie-4_5-vl baseline

* fix * fix * fix * add test case * add test case * add test case

…ddle#4566) * support mtp distribution equivalence verification * fix bugs * add unit test

paddle-bot · 2025-10-28T12:59:01Z

Thanks for your contribution!

Jiang-Jia-Jun and others added 30 commits August 31, 2025 21:31

Update FASTDEPLOY_VERSION to 2.2.0

1953c7c

fix ce build job (PaddlePaddle#3777)

2b0a745

fix ce compile task upload error (PaddlePaddle#3788)

0cdbc95

Fix chunked prefill (PaddlePaddle#3778)

a86b35a

* update enable chunked_prefill * update code * update code * update code

[Feature] Setting number of apiserver workers automatically (PaddlePa…

d1d063e

…ddle#3794) Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com>

[Feature] support model weight update in ep (PaddlePaddle#3802)

0f42771

* Update config.py * Update ep.py * Update fused_moe_backend_base.py * Update dynamic_weight_manager.py * Update worker_process.py * fix ci

[BugFix] fix max streaming tokens invalid (PaddlePaddle#3799)

cd09384

* Update serving_chat.py * Update serving_completion.py * Update serving_completion.py

[Executor] Fix bug of import paddle with RLHF (PaddlePaddle#3781) (Pa…

a6c8f17

…ddlePaddle#3817)

Update qwen_vl_processor.py (PaddlePaddle#3806)

5cda326

[BugFix] fix error of import paddle.base.core.Config (PaddlePaddle#3761…

1745101

…) (PaddlePaddle#3804) * 延迟 import Config * support chunked_prefill * support chunked_prefill

[v1loader]Reduce EB300B model loading time (PaddlePaddle#3700) (Paddl…

f975f7d

…ePaddle#3810) * speed up eb45 * update

[BugFix] fix scheduler (PaddlePaddle#3818)

37cb37b

* fix scheduler bug * fix

add reasoning parser plugin (PaddlePaddle#3820)

1968c65

Update installation method for paddlepaddle-xpu (PaddlePaddle#3834)

42402c8

【BugFix】add moe noaux_tc tatics in trition backend (PaddlePaddle#3821)

05b6591

* add moe noaux_tc tatics in trition backend * fix * add dp config

[XPU]Update XPU CI Case (PaddlePaddle#3844)

abcd214

* Update no_proxy environment variable in CI workflow * Install lsof and kill api_server processes Install lsof tool and kill processes using it.

【Fix bug] w4afp8 的nblock固定为256，并且fa3的append attn 增加mask参数 (PaddlePadd…

9213a58

…le#3771) (PaddlePaddle#3835) * fix w4afp8 * 增加集中式配置 * codestyle * fix fa3 append attn

[Bug Fix] Fix bug of multimodal inputs only text (PaddlePaddle#3850)

1432e33

fix port (PaddlePaddle#3865)

b56b015

Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>

Support for async processor added. (PaddlePaddle#3870)

8c0e7d6

* Support for async processor added. * remove yappi code

fix mem boom in ep (PaddlePaddle#3852)

fbf0e9d

[Bug fix] Fix prompt token ids dtype in v1 (PaddlePaddle#3861)

a0c0351

[bugfix] scheduler (PaddlePaddle#3871)

8550e19

* fix scheduler bug * fix * Update api_server.py

[bug] fix finish reason (PaddlePaddle#3858)

b8d0f1c

* add reasoning parser plugin * fix finish reason --------- Co-authored-by: Yuanle Liu <yuanlehome@163.com>

fix DP&&TP (PaddlePaddle#3872)

d40d3a5

[CI] update paddleformers==0.2 in release/2.2 (PaddlePaddle#3828)

afcde19

* [DEBUG] Adapt validation for paddleformers==0.2 in release/2.2 * [CI] update paddleformers==0.2 in release/2.2

[Fix] disable scheduler v1 in guided decoding (PaddlePaddle#3877)

8567ada

* disable scheduler v1 in guided decoding * disable scheduler v1 in guided decoding

paddleformers==0.1.4 (PaddlePaddle#3908)

e9f72df

zeroRains and others added 28 commits October 13, 2025 10:18

[Cherry-Pick][BugFix]fix the bug for prefilled_step_idx signal of cac…

07db281

…he_messager in cudagraph and PD (PaddlePaddle#4252) * fix the bug for prefilled_step_idx signal of cache_messager in cudagraph and PD * support dp

[CI] fix diff_error temporarily (PaddlePaddle#4391)

7b04a12

[MTP][Cfp8]supports spec dynamic cfp8 (PaddlePaddle#4290) (PaddlePadd…

bf12bee

…le#4392) * supports spec dynamic cfp8 * supports spec dynamic cfp8 --------- Co-authored-by: lzy <569782149@qq.com>

Update token_processor.py (PaddlePaddle#4395)

fa9a3ee

增加4合一视频选择 (PaddlePaddle#4372)

bc7193f

[BugFix] fix cache port and zmq close bugs (PaddlePaddle#4371)

dd425b8

* Update common_engine.py * Update zmq_client.py * Update expert_service.py --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>

[CI] Fix download instability issues (PaddlePaddle#4424)

55064b8

[MTP]support mtp chunk_prefill_v1 (PaddlePaddle#4365)

c349987

* support mtp chunk_prefill_v1 * fix mtp chunkprefill output * fix mtp chunkprefill output, fix unit test * fix save_output --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>

fix ep perf (PaddlePaddle#4381)

74ae214

support speculate_limit_thinking_content_length_v2 (PaddlePaddle#4428)

83f97d1

* support speculate_limit_thinking_content_length_v2 * fix * fix import

fix: image token output (PaddlePaddle#4399)

cfd93c0

* fix: image token output * fix: code style * fix: CompletionOutput.decode_type

[Graph Optimization][Speculative Decoding] Fix the bug of CUDAGraph +…

920df5b

… MTP + EP (PaddlePaddle#4430) * Fix MTP dummy run bug * Target Model and Draft Model using the same flag * aovid moe bug in cudagraph padding * In mtp replace use_cudagraph as step_use_cudagraph

[Fix] add einops for requirements (PaddlePaddle#4477)

beaec37

* add einops * add einops

[CI] update paddlepaddle-xpu==3.2.0 in 0908 (PaddlePaddle#4489)

9558912

[Fearture] Support mm model close prefix cache (PaddlePaddle#4502)

3cd9d30

* support mm prefix cache close * add * fix * fix * fix --------- Co-authored-by: ltd0924 <luotingdan@baidu.com>

[Cherry-pick] fix requests & block metrics (PaddlePaddle#4500)

ec499a0

* [fix] fix requests & block metrics * [chore] rename variables

[CI] update ernie-4_5-vl baseline (PaddlePaddle#4495) (PaddlePaddle#4505

31ac6c6

) * [CI] update ernie-4_5-vl baseline

fix image token output (PaddlePaddle#4487)

1531004

* fix * fix * fix * add test case * add test case * add test case

Support CUDAGraph Padding + MTP (PaddlePaddle#4546)

0718210

add hasher and ImagePosition

dd7fe27

add test case

fdce0d2

support static C8 (PaddlePaddle#4565)

0885371

support mtp draft model with ep (PaddlePaddle#4581)

1a21e6c

[Feature] support mtp distribution equivalence verification (PaddlePa…

2cf0b0b

…ddle#4566) * support mtp distribution equivalence verification * fix bugs * add unit test

refactore default capture list & refine code

6932aa5

gongshaotian closed this Oct 28, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Graph Optimization] Refactor default capture list #4630

[Graph Optimization] Refactor default capture list #4630

Uh oh!

gongshaotian commented Oct 28, 2025

Uh oh!

paddle-bot bot commented Oct 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

[Graph Optimization] Refactor default capture list #4630

[Graph Optimization] Refactor default capture list #4630

Uh oh!

Conversation

gongshaotian commented Oct 28, 2025

Motivation

Modifications

Usage or Command

Accuracy Tests

Checklist

Uh oh!

paddle-bot bot commented Oct 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants