Skip to content

(feat)Rich logging#413

Open
e-dobrowolska wants to merge 2 commits intoOpenHands:mainfrom
e-dobrowolska:rich-logging
Open

(feat)Rich logging#413
e-dobrowolska wants to merge 2 commits intoOpenHands:mainfrom
e-dobrowolska:rich-logging

Conversation

@e-dobrowolska
Copy link

This PR brings an optional, richer console logging experience to all the benchmark inference scripts.

What changed

  • Added a new shared logging module: benchmarks/utils/console_logging.py
  • Conversation output is now much richer and shows tool/message/error events live in the console (via benchmarks/utils/conversation.py)
  • Switched how child process logging is set up: now uses the new centralized helper in benchmarks/utils/evaluation.py
  • Added end-of-instance summary lines (summarize_instance(...)) in all the main benchmark runner scripts:
    • benchmarks/swebench/run_infer.py
    • benchmarks/swebenchmultimodal/run_infer.py
    • benchmarks/swtbench/run_infer.py
    • benchmarks/multiswebench/run_infer.py
    • benchmarks/commit0/run_infer.py
    • benchmarks/gaia/run_infer.py
    • benchmarks/openagentsafety/run_infer.py
  • Added some docs: see benchmarks/utils/logging.md for how it looks and acts

Behavior impact

  • By default, everything works as before.
  • When enabled, you'll see:
    • a startup line for each instance
    • live updates for tools, messages, errors, etc.
    • a short end-of-instance summary
  • File logs are still written as before:
    logs/instance_<id>.log
    logs/instance_<id>.output.log

How to enable

  • export RICH_LOGGING=1

@neubig neubig requested a review from juanmichelini February 15, 2026 14:48
Copy link
Collaborator

@juanmichelini juanmichelini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested both with and without RICH_LOGGING=1, LGTM

@juanmichelini juanmichelini enabled auto-merge (squash) February 18, 2026 00:33
@juanmichelini
Copy link
Collaborator

thanks @e-dobrowolska !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments