Skip to content

[whisper] allow to pass text/audio specific kwargs#43863

Open
eustlb wants to merge 3 commits intomainfrom
whisper-proc-specific-kwargs
Open

[whisper] allow to pass text/audio specific kwargs#43863
eustlb wants to merge 3 commits intomainfrom
whisper-proc-specific-kwargs

Conversation

@eustlb
Copy link
Copy Markdown
Contributor

@eustlb eustlb commented Feb 9, 2026

What does this PR do?

This allows to do something like:

from transformers import AutoProcessor
from datasets import Audio, load_dataset

model_id = "openai/whisper-large-v3"
processor = AutoProcessor.from_pretrained(model_id)

dataset = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
dataset = dataset.cast_column("audio", Audio(processor.feature_extractor.sampling_rate))
sample = dataset[0]["audio"]

inputs = processor(
    audio=sample["array"],
    text="this is the transcription text",
    audio_kwargs={
        "max_length": 10000,
    },
    text_kwargs={
        "max_length": 5,
    },
    common_kwargs={
        "return_tensors": "pt",
    }
)
for k, v in inputs.items():
    print(f"{k}: {v.shape}")

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link
Copy Markdown
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okayyy not sure a test is necessary?

@eustlb
Copy link
Copy Markdown
Contributor Author

eustlb commented Feb 10, 2026

run-slow: whisper

@github-actions
Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: whisper

@github-actions
Copy link
Copy Markdown
Contributor

This comment contains run-slow, running the specified jobs:

models: ["models/whisper"]
quantizations: []

@github-actions
Copy link
Copy Markdown
Contributor

CI Results

Workflow Run ⚙️

Commit Info

Context Commit Description
RUN be16e231 merge commit
PR 4d187bfc branch commit
main b7b9d252 base commit

Model CI Report

1 new failed tests from this PR 😭

  • whisper:
    tests/models/whisper/test_processing_whisper.py::WhisperProcessorTest::test_tokenizer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants