ECHO-261-route-english-transcription-through-litellm#171
Conversation
- Added ENABLE_ENGLISH_TRANSCRIPTION_WITH_LITELLM environment variable to control English transcription behavior. - Updated transcribe_conversation_chunk function to conditionally use RunPod transcription based on non en language and configuration. - Improved error handling for RunPod job status checks, including logging for unexpected responses and exceptions. - Return whisper prompt to the old prompts
WalkthroughA new environment variable flag is introduced to control routing of English transcription through LiteLLM. The transcription logic is updated to use LiteLLM for English when the flag is enabled, bypassing RunPod. Prompt templates for multiple languages are replaced with concise, informal introductions. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant TranscribeModule
participant LiteLLM
participant RunPod
User->>TranscribeModule: Request transcription (with language)
alt ENABLE_ENGLISH_TRANSCRIPTION_WITH_LITELLM is true and language is English
TranscribeModule->>LiteLLM: Transcribe audio (English)
LiteLLM-->>TranscribeModule: Return transcription
else
TranscribeModule->>RunPod: Transcribe audio (non-English or flag disabled)
RunPod-->>TranscribeModule: Return transcription
end
TranscribeModule-->>User: Return result
Assessment against linked issues
Assessment against linked issues: Out-of-scope changesNo out-of-scope changes detected. Possibly related PRs
Suggested reviewers
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (7)
echo/server/dembrane/config.py(1 hunks)echo/server/dembrane/transcribe.py(4 hunks)echo/server/prompt_templates/default_whisper_prompt.de.jinja(1 hunks)echo/server/prompt_templates/default_whisper_prompt.en.jinja(1 hunks)echo/server/prompt_templates/default_whisper_prompt.es.jinja(1 hunks)echo/server/prompt_templates/default_whisper_prompt.fr.jinja(1 hunks)echo/server/prompt_templates/default_whisper_prompt.nl.jinja(1 hunks)
🧰 Additional context used
🪛 Pylint (3.3.7)
echo/server/dembrane/transcribe.py
[convention] 205-205: Line too long (116/100)
(C0301)
[refactor] 205-303: Unnecessary "elif" after "return", remove the leading "el" from "elif"
(R1705)
[convention] 232-232: Trailing whitespace
(C0303)
[convention] 235-235: Trailing whitespace
(C0303)
[convention] 238-238: Line too long (121/100)
(C0301)
[convention] 245-245: Trailing whitespace
(C0303)
[convention] 250-250: Line too long (112/100)
(C0301)
[convention] 253-253: Line too long (106/100)
(C0301)
[warning] 252-252: Catching too general exception Exception
(W0718)
[warning] 230-230: Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely
(W3101)
[warning] 234-234: Use lazy % formatting in logging functions
(W1203)
[warning] 238-238: Use lazy % formatting in logging functions
(W1203)
[warning] 241-241: Use lazy % formatting in logging functions
(W1203)
[warning] 243-243: Use lazy % formatting in logging functions
(W1203)
[warning] 247-247: Use lazy % formatting in logging functions
(W1203)
[warning] 250-250: Use lazy % formatting in logging functions
(W1203)
[warning] 253-253: Use lazy % formatting in logging functions
(W1203)
echo/server/dembrane/config.py
[convention] 165-165: Line too long (137/100)
(C0301)
[convention] 166-166: Line too long (125/100)
(C0301)
[convention] 167-167: Trailing whitespace
(C0303)
[convention] 167-167: Line too long (103/100)
(C0301)
[warning] 167-167: Use lazy % formatting in logging functions
(W1203)
🔇 Additional comments (5)
echo/server/prompt_templates/default_whisper_prompt.fr.jinja (1)
1-1: LGTM! Clean French, très 100x.Perfect French prompt - "tour de table" is the idiomatic way to say it. The simplified approach is chef's kiss for transcription clarity.
echo/server/prompt_templates/default_whisper_prompt.de.jinja (1)
1-1: LGTM! Solid German, sehr gut engineering.Clean German prompt with proper grammar and natural flow. The simplified conversational approach is definitely the 100x move for transcription prompts.
echo/server/dembrane/transcribe.py (2)
205-206: LGTM! Clean conditional routing logic.The transcription routing logic correctly bypasses RunPod for English when the LiteLLM flag is enabled. This provides the flexibility mentioned in the PR objectives.
However, fix the line length:
- if ENABLE_RUNPOD_WHISPER_TRANSCRIPTION and not (language == "en" and ENABLE_ENGLISH_TRANSCRIPTION_WITH_LITELLM): + if (ENABLE_RUNPOD_WHISPER_TRANSCRIPTION and + not (language == "en" and ENABLE_ENGLISH_TRANSCRIPTION_WITH_LITELLM)):🧰 Tools
🪛 Pylint (3.3.7)
[convention] 205-205: Line too long (116/100)
(C0301)
[refactor] 205-303: Unnecessary "elif" after "return", remove the leading "el" from "elif"
(R1705)
284-285: LGTM! Clear debug logging for transcription path.The debug logging helps with troubleshooting transcription routing decisions.
echo/server/prompt_templates/default_whisper_prompt.en.jinja (1)
1-1: Verify this prompt change aligns with transcription quality expectations.You've completely replaced what was likely a detailed technical transcription prompt with a casual meeting introduction. This could significantly impact transcription quality and context understanding.
Please confirm that this simplified prompt still provides the transcription quality needed for your use case, especially given that this now routes through LiteLLM for English transcription.
- Made ENABLE_ENGLISH_TRANSCRIPTION_WITH_LITELLM optional, defaulting to false, and updated related logging. - Enhanced formatting in the queue_transcribe_audio_runpod function for better readability.
| @@ -33,7 +34,10 @@ class TranscriptionError(Exception): | |||
|
|
|||
|
|
|||
| def queue_transcribe_audio_runpod( | |||
There was a problem hiding this comment.
seperation of concerns:
this does more than "queue_transcribe_audio_runpod"
please use the higher level "transcribe_audio" fn and switch the underlying implementation when necessary?
There was a problem hiding this comment.
also maybe have a transcribe_audio_litellm fn to complement this
There was a problem hiding this comment.
queue_transcribe_audio_runpod: It just does that. It posts a request to runpod. Thats all.
There was a problem hiding this comment.
transcribe_audio_litellm: There already is a function which we are using in this case
There was a problem hiding this comment.
server/dembrane/transcribe.py/transcribe_audio_litellm
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
echo/server/dembrane/config.py(1 hunks)echo/server/dembrane/transcribe.py(6 hunks)
🧰 Additional context used
🪛 Pylint (3.3.7)
echo/server/dembrane/config.py
[convention] 169-169: Line too long (104/100)
(C0301)
echo/server/dembrane/transcribe.py
[convention] 245-245: Line too long (109/100)
(C0301)
[convention] 259-259: Line too long (102/100)
(C0301)
[refactor] 205-318: Unnecessary "elif" after "return", remove the leading "el" from "elif"
(R1705)
[warning] 220-220: Use lazy % formatting in logging functions
(W1203)
[warning] 262-262: Catching too general exception Exception
(W0718)
[warning] 240-240: Use lazy % formatting in logging functions
(W1203)
[warning] 244-246: Use lazy % formatting in logging functions
(W1203)
[warning] 249-249: Use lazy % formatting in logging functions
(W1203)
[warning] 251-251: Use lazy % formatting in logging functions
(W1203)
[warning] 255-255: Use lazy % formatting in logging functions
(W1203)
[warning] 258-260: Use lazy % formatting in logging functions
(W1203)
[warning] 263-265: Use lazy % formatting in logging functions
(W1203)
[refactor] 269-272: The if statement can be replaced with 'var = bool(test)'
(R1703)
🔇 Additional comments (3)
echo/server/dembrane/transcribe.py (3)
22-22: LGTM! Import added correctly.Clean addition of the new configuration flag import that integrates perfectly with the existing config imports.
205-208: LGTM! Solid conditional logic for English transcription routing.The conditional logic correctly implements the intended behavior - English transcription will bypass RunPod and use LiteLLM when the flag is enabled, while non-English languages continue using RunPod. The debug logging clearly indicates which transcription path is being taken.
This is a clean implementation that maintains backward compatibility while adding the new routing capability.
🧰 Tools
🪛 Pylint (3.3.7)
[refactor] 205-318: Unnecessary "elif" after "return", remove the leading "el" from "elif"
(R1705)
300-300: LGTM! Clean debug logging for LiteLLM path.Perfect addition of debug logging to clearly indicate when the LiteLLM transcription path is being used. This will be super helpful for debugging and monitoring the routing behavior.
* Enhance transcription configuration and error handling - Added ENABLE_ENGLISH_TRANSCRIPTION_WITH_LITELLM environment variable to control English transcription behavior. - Updated transcribe_conversation_chunk function to conditionally use RunPod transcription based on non en language and configuration. - Improved error handling for RunPod job status checks, including logging for unexpected responses and exceptions. - Return whisper prompt to the old prompts * Refactor transcription configuration and improve logging - Made ENABLE_ENGLISH_TRANSCRIPTION_WITH_LITELLM optional, defaulting to false, and updated related logging. - Enhanced formatting in the queue_transcribe_audio_runpod function for better readability.
2 participants
Enhance transcription configuration and error handling
Summary by CodeRabbit
New Features
Bug Fixes
Documentation