[bug] claude keeps messing up generate_sweep_config.py command

[functionstackx]

hi @Oseltamivir (prompt engineer god!!)

even after https://github.com/InferenceMAX/InferenceMAX/pull/517 prompt update, it still messes up the CLI command, can u look into fixing this? I am not sure what other prompt changes i can make

for example https://github.com/InferenceMAX/InferenceMAX/actions/runs/21231775395

usage: generate_sweep_configs.py [-h]
                                 {full-sweep,runner-model-sweep,test-config}
                                 ...
generate_sweep_configs.py: error: unrecognized arguments: --min-conc 4 --max-conc 4 --seq-len 1k1k

[functionstackx]

hmm seems like the issue is that only full sweep supports min-conc and max-conc? but ti si trying to use those flags with test-config?

[Oseltamivir]

tbh its prob cause intructions say "...instead of running full sweeps" and "Never do a full sweep without filters unless explicitly instructed." so it thinks full-sweep is actually a full sweep

[Oseltamivir]

@claude edit .github/workflows/claude.yml to resolve this misunderstanding, tell claude that generate-cli-command: full-sweep ... can be used and test-config does not accept args and only accepts a specific config keys.

Also, fix my language use when I say "However, you can analyze an ongoing run, for example if it errors, and start a new run in parallel without finishing the old run. Cancel such old runs." What I mean is that claude should wait for runs to end with an exponential backoff sleep starting at 2 minutes. At the end of each sleep command, it should check ongoing runs. If it errors or fails, start a new run and cancel the old run. It should only wait for the final run to succeed, then analyze the benchmark results.

Make the changes and create the PR for me

[Klaud-Cold]

Claude finished @Oseltamivir's task in 1m 18s —— View job • claude/issue-520-20260122-0541 • Create PR ➔

---

Completed

Tasks

• Read .github/workflows/claude.yml to understand current prompt structure
• Update prompt to clarify full-sweep vs test-config usage
• Fix the workflow monitoring language to describe exponential backoff
• Commit and push changes
• Create PR

Summary

Updated .github/workflows/claude.yml to fix the CLI command misunderstanding:

Changes:

1. Added explicit subcommand reference explaining that:

   • full-sweep accepts filter flags like --model-prefix, --framework, --precision, --runner-type, --min-conc, --max-conc, --seq-len
   • test-config accepts ONLY --config-files and --config-keys (no other flags)

2. Added example showing --min-conc, --max-conc, and --seq-len usage with full-sweep

3. Clarified that using full-sweep with filters is NOT the same as running an unfiltered sweep

4. Replaced ambiguous workflow monitoring instructions with exponential backoff behavior:

   • Start with sleep 120 (2 minutes), double each iteration
   • Check run status after each sleep
   • Cancel failed runs and start new ones
   • Only wait for final successful run before analyzing results

|