[transformers] Fixed decode output cls#807
Merged
tridao merged 1 commit intostate-spaces:mainfrom Nov 11, 2025
Merged
Conversation
Contributor
Author
|
cc: @tridao |
Contributor
Author
This was referenced Nov 10, 2025
vasqu
reviewed
Nov 11, 2025
| print(f"Prompt processing + decoding time: {(start.elapsed_time(end)):.0f}ms") | ||
| output_cls = GreedySearchDecoderOnlyOutput if top_k == 1 else SampleDecoderOnlyOutput | ||
| return output_cls(sequences=torch.cat(sequences, dim=1), scores=tuple(scores)) | ||
| return GenerateDecoderOnlyOutput(sequences=torch.cat(sequences, dim=1), scores=tuple(scores)) |
Contributor
There was a problem hiding this comment.
Just for reference, this is the respective PR that removed it huggingface/transformers#40998
It has been used as alias for a while now, maybe there should be a safety check to still allow this output class
Collaborator
|
Thanks! |
Esthercyz
pushed a commit
to Esthercyz/mamba_1
that referenced
this pull request
Jan 29, 2026
Knarf04
added a commit
to Knarf04/mamba
that referenced
this pull request
Feb 3, 2026
migdalskiy
pushed a commit
to migdalskiy/mamba
that referenced
this pull request
Mar 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The generation class
GreedySearchDecoderOnlyOutput,SampleDecoderOnlyOutputhave been deprecated from thetransformerslibrary and will be removed in the next version. I have updated it with the ouput class which is more generic.Since the transformers version is not pinned in
setup.py, this fix is crucial. The fix is backward compatible and hence won't break anything.For example, with the current release of transformers,
transformers==4.57.1, this code works:but as soon as we install transformers from main, even the import breaks
After the fix in the PR, this import issue is solved.