Whisper models in Foundry Local SDK: v1 models return empty text, v2 models only transcribe first 30 seconds

[yuvalshmaryahu]

Hi,

I’m trying to use the Whisper-CPU models through the Foundry Local SDK, but I’m encountering some unexpected behavior.
There appear to be two model versions: v1 and v2.
For v1 models (e.g., base, small, medium), the API consistently returns empty text, even for audio files that produce correct transcriptions when using whisper-tiny.
For v2 models (e.g., tiny, large), the transcription is returned, but only for the first ~30 seconds of the audio. The rest of the audio is not transcribed.
Because of this, I’m currently unable to get a full transcription using the available models.

Could you please help clarify:
Whether this is a known issue with these models in the Foundry Local SDK?
If there is a recommended configuration or workaround to obtain full transcriptions?

Thank you!

[leestott]

Thanks @yuvalshmaryahu

Issue #517 confirmed! The SDK's AudioClient only transcribed the first ~30 seconds of the 4m12s audio file. The transcription cuts off at "The floor was old cracked" — exactly matching the reported behavior.

Completed: tested a 4 min audio .wav file

Validation results

Runtime	Audio length	Transcribed	Result
JavaScript (SDK AudioClient)	4m 12s	~30 sec	Truncated — cuts off at "The floor was old cracked"
Python (ONNX Runtime direct)	4m 12s	~30 sec	Truncated — identical cutoff point

Root cause: Whisper's encoder processes a fixed 30-second mel spectrogram window (3000 frames at the standard hop length). Neither the SDK's AudioClient nor the Python sample implements audio chunking — splitting longer audio into overlapping ~30 second segments, transcribing each, and concatenating the results. This is a limitation in the Foundry Local SDK / ONNX model pipeline, not a model quality issue.

Fix

I have provide a sample at https://github.com/microsoft-foundry/Foundry-Local-Lab

Solution implemented: All three language samples now split long audio into ≤30-second chunks, transcribe each chunk independently, and concatenate the results there is a sample of over 4mins available

Changes by language:

1. Python (foundry-local-whisper.py) — A transcribe() function splits audio using librosa into CHUNK_SAMPLES (30 × 16000 samples), runs _transcribe_chunk() on each segment via direct ONNX Runtime encoder/decoder inference, and joins the text with spaces.

2. JavaScript (foundry-local-whisper.mjs) — A splitWavIntoChunks() function parses the WAV header, splits the PCM data into 30-second segments, writes each to a temporary WAV file with a valid RIFF header, then calls audioClient.transcribe() on each chunk. Temporary files are cleaned up after processing.

3. C# (WhisperTranscription.cs) — A SplitWavIntoChunks() method performs the same WAV-header-aware split, writes chunk files to a temp directory, then calls audioClient.TranscribeAudioAsync() on each. The temp directory is cleaned up afterwards.

Supporting changes:

• Test audio: A longer sample (zava-full-project-walkthrough.wav, ~4 min) was added specifically for testing the chunking workaround.
• All three files include a comment linking back to https://github.com/microsoft/Foundry-Local/issues/517 for traceability.

Pattern used (all languages):

1. Load the WAV file
2. If duration ≤ 30 s → single-pass transcription (no change)
3. If duration > 30 s → split PCM data into 30-second chunks
4. Transcribe each chunk independently
5. Concatenate chunk texts with spaces
6. Clean up any temporary files

This is a client-side workaround — the underlying Whisper encoder limitation (30-second window) still exists in the model architecture and the Foundry Local SDK. The fix ensures the workshop samples produce complete transcriptions for audio of any length.