Conversation
There was a problem hiding this comment.
Pull request overview
Adds an interactive “chat mode” to latin_punct_router_eval.py and enhances output readability by color-highlighting generated continuations, plus includes a small demo script and README updates.
Changes:
- Add ANSI color helpers and generated-text highlighting in validation/example outputs.
- Introduce
--chat_modeinteractive loop to compare full vs routed translations for user-entered English. - Add
trim_demo.shand document the new mode/features inREADME.md.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| huggingface_model/gemma/270M/latin_punct_router_eval.py | Adds highlighting helper, changes translation generation to return plain + highlighted text, and introduces interactive chat mode flag/loop. |
| huggingface_model/gemma/270M/README.md | Documents highlighting and the new interactive chat mode. |
| huggingface_model/gemma/270M/trim_demo.sh | Adds a convenience script to run the evaluation with standard parameters. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -0,0 +1,13 @@ | |||
| #!/bin/bash | |||
|
|
|||
| python latin_punct_router_eval.py \ | |||
There was a problem hiding this comment.
trim_demo.sh calls python latin_punct_router_eval.py, which only works if the script is executed from this directory. If someone runs it from the repo root (or any other cwd), it will fail to locate the Python file. Consider invoking via an explicit path (e.g., relative to the script’s own directory) so the demo is runnable from anywhere.
| python latin_punct_router_eval.py \ | |
| SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" | |
| python "$SCRIPT_DIR/latin_punct_router_eval.py" \ |
| if "Spanish:" in full_text: | ||
| return full_text.split("Spanish:", 1)[1].strip() | ||
| return full_text.strip() | ||
| generated = full_text.split("Spanish:", 1)[1].strip() |
There was a problem hiding this comment.
full_text.split("Spanish:", 1) splits on the first occurrence of Spanish:. Because the 3-shot prompt itself contains multiple Spanish: lines (one per example plus the final Spanish:), this will include much of the prompt/examples in generated instead of only the model’s continuation. Use the last occurrence (e.g., rsplit("Spanish:", 1)) or otherwise slice based on the final prompt boundary so generated is just the translation for the user’s input.
| generated = full_text.split("Spanish:", 1)[1].strip() | |
| generated = full_text.rsplit("Spanish:", 1)[1].strip() |
|
This pull request adds a new interactive chat mode to the
latin_punct_router_eval.pyscript, allowing users to input English sentences and compare full vs routed Spanish translations with color-highlighted outputs. It also enhances the validation example output with generated text highlighting and introduces a demo shell script for streamlined testing.New interactive chat mode and usability improvements:
--chat_modeflag and the_run_chat_modefunction to enable an interactive mode where users can input English sentences and receive both full and routed Spanish translations, with color highlighting for user input and generated continuations. [1] [2] [3] [4]ANSI_USER,ANSI_GEN,ANSI_RESET) and the_highlight_generatedhelper to visually distinguish generated segments in both chat and validation outputs. [1] [2]Output and evaluation enhancements:
_generate_translationto return both the plain and color-highlighted generated text, and updated validation example printing to display these highlights. [1] [2] [3] [4] [5]README.mdwith documentation for the new chat mode and the highlighting features. [1] [2]Demo and reproducibility:
trim_demo.shshell script to demonstrate running the evaluation with standard parameters.