Skip to content

fix: preserve tool-call context in tokenization #527

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
corbt merged 2 commits into from
Jan 21, 2026
Merged

fix: preserve tool-call context in tokenization #527

corbt merged 2 commits into from
Jan 21, 2026

Conversation

@corbt
Copy link
Contributor

@corbt corbt commented Jan 21, 2026 •
edited by vivekkalyan
Loading

Summary

  • Avoid dropping tool calls from assistant messages in context by only inserting sentinels for trainable assistant messages.
  • Preserve tool_calls in the chat template for trainable assistants and fail fast if a tool-call dict would be tokenized via content-only splicing.
  • Proposal: drop support for allow_training_without_logprobs (importance sampling needs logprobs; this mode complicates paths and enables subtle tool-call bugs).

Test plan

  • Not run (tokenization-only change)

fix: preserve tool-call context in tokenization
2d329a2 
Only splice trainable assistant spans and keep tool_calls
in the template; error if tool_calls would be dropped.
Fix type errors in tool-call tokenization
25ffbaf 
- Use .get() instead of direct [] access for tool_calls to handle
  message types that don't have this key
- Cast message to dict[str, Any] when appending to token_template_messages
@corbt corbt mentioned this pull request Jan 21, 2026
Open
@corbt corbt merged commit b220140 into main Jan 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bradhilton bradhilton bradhilton approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@corbt @bradhilton