[Cherry-Pick] add a new reasoning parser (#4571) #4664
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
add new reasoning_parser initial commit
add parser file content
add register
ernie_test_reasoning_parser
support <tool_call> token and add tool_parser
add and fix unit tests
modify reasoning_parser
modify reasoning parser and tool parser
modify unit tests
modify reasoning_parser and tool_parser
modify unit tests
fix tool_parser
modify the logic of reasoning_parser and tool_parser
add and modify unit tests
standardize code style
simplify reasoning_parser and tool_parser
modify unit test
Motivation
Add a new reasoning_parser
The model's output format with reasoning includes two types:
No tool invocation: xxxyyy
Tool invocation: xxx\n<tool_call>zzz</tool_cal>
x represents the reasoning content, y represents the response content, and z represents the tool invocation content.
Modifications
Add a new reasoning_parser ErnieTestReasoningParser
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.