Skip to content

[transformers] Fixed decode output cls#807

Merged
tridao merged 1 commit intostate-spaces:mainfrom
romitjain:romit/fix-decode-generation
Nov 11, 2025
Merged

[transformers] Fixed decode output cls#807
tridao merged 1 commit intostate-spaces:mainfrom
romitjain:romit/fix-decode-generation

Conversation

@romitjain
Copy link
Copy Markdown
Contributor

@romitjain romitjain commented Oct 17, 2025

The generation class GreedySearchDecoderOnlyOutput, SampleDecoderOnlyOutput have been deprecated from the transformers library and will be removed in the next version. I have updated it with the ouput class which is more generic.

Since the transformers version is not pinned in setup.py, this fix is crucial. The fix is backward compatible and hence won't break anything.

For example, with the current release of transformers, transformers==4.57.1, this code works:

import mamba_ssm

selective_state_update = mamba_ssm.ops.triton.selective_state_update.selective_state_update
mamba_chunk_scan_combined = mamba_ssm.ops.triton.ssd_combined.mamba_chunk_scan_combined
mamba_split_conv1d_scan_combined = mamba_ssm.ops.triton.ssd_combined.mamba_split_conv1d_scan_combined

but as soon as we install transformers from main, even the import breaks

Traceback (most recent call last):
  File "/workspace/mamba/test.py", line 1, in <module>
    import mamba_ssm
  File "/workspace/mamba/mamba_ssm/__init__.py", line 6, in <module>
    from mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel
  File "/workspace/mamba/mamba_ssm/models/mixer_seq_simple.py", line 20, in <module>
    from mamba_ssm.utils.generation import GenerationMixin
  File "/workspace/mamba/mamba_ssm/utils/generation.py", line 14, in <module>
    from transformers.generation import GreedySearchDecoderOnlyOutput, SampleDecoderOnlyOutput, TextStreamer
ImportError: cannot import name 'GreedySearchDecoderOnlyOutput' from 'transformers.generation' (/workspace/transformers/src/transformers/generation/__init__.py). Did you mean: 'GenerateBeamDecoderOnlyOutput'?

After the fix in the PR, this import issue is solved.

@romitjain romitjain changed the title Updated decode output cls Fixed decode output cls Oct 17, 2025
@romitjain
Copy link
Copy Markdown
Contributor Author

cc: @tridao

@romitjain romitjain changed the title Fixed decode output cls [transformers] Fixed decode output cls Nov 10, 2025
@romitjain
Copy link
Copy Markdown
Contributor Author

Bumping up again @tridao @ko3n1g

print(f"Prompt processing + decoding time: {(start.elapsed_time(end)):.0f}ms")
output_cls = GreedySearchDecoderOnlyOutput if top_k == 1 else SampleDecoderOnlyOutput
return output_cls(sequences=torch.cat(sequences, dim=1), scores=tuple(scores))
return GenerateDecoderOnlyOutput(sequences=torch.cat(sequences, dim=1), scores=tuple(scores))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for reference, this is the respective PR that removed it huggingface/transformers#40998

It has been used as alias for a while now, maybe there should be a safety check to still allow this output class

@tridao
Copy link
Copy Markdown
Collaborator

tridao commented Nov 11, 2025

Thanks!

@tridao tridao merged commit 35e927b into state-spaces:main Nov 11, 2025
@romitjain romitjain deleted the romit/fix-decode-generation branch November 12, 2025 05:29
Esthercyz pushed a commit to Esthercyz/mamba_1 that referenced this pull request Jan 29, 2026
Knarf04 added a commit to Knarf04/mamba that referenced this pull request Feb 3, 2026
migdalskiy pushed a commit to migdalskiy/mamba that referenced this pull request Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants