[transformers] set return dict false for transformers v5 compatibility#1325
Merged
erictang000 merged 4 commits intoNovaSky-AI:mainfrom Mar 20, 2026
Merged
[transformers] set return dict false for transformers v5 compatibility#1325erictang000 merged 4 commits intoNovaSky-AI:mainfrom
erictang000 merged 4 commits intoNovaSky-AI:mainfrom
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
erictang000
commented
Mar 19, 2026
| # For a simple chat template, the fixed base approach is expected to behave the same | ||
| # as `apply_chat_template` | ||
| expected_token_ids = tokenizer_w_dummy_template.apply_chat_template(messages) | ||
| # as `apply_chat_template`. We compare decoded strings rather than raw token IDs |
Collaborator
Author
There was a problem hiding this comment.
this comparison between full chat template vs stripping base only happens in this test and not in the SkyRLGymGenerator
Member
There was a problem hiding this comment.
is it broken by v5? I wonder why it doesn't surface until now
Collaborator
Author
There was a problem hiding this comment.
yeah i asked claude it told me this was the relevant PR: huggingface/transformers#40936 and that in that PR
LlamaTokenizer was rewritten to inherit from TokenizersBackend (Rust) instead of the old Python SentencePiece backend
legacy was changed from defaulting to True to defaulting to False
The _get_prepend_scheme helper was added to select "first" vs "always" based on the legacy flag
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR makes 2 changes for transformers-v5 compatibility:
return_dict=Falsewhere needed for transformers-v5 compatibility - this can be merged prior to explicitly upgrading to transformers v5 in the pyproject.toml, since vllm still does not technically fully support it in the latest release. (This change should be backwards compatible with transformers v4.*, since the behavior prior to v5 was that the default value forreturn_dictwasNone, which was interpreted as False.)huggingface/transformers breaking PR
return_dict=falsePR for v5: huggingface/transformers#42567Validation
Checked all CPU tests are passing both with transformers < 5.0.0 and for transformers==5.3.0, and checked that
examples/train/gsm8k/run_gsm8kruns for both old/new transformers.