Skip to content

Conversation

@lee-groq
Copy link
Contributor

@lee-groq lee-groq commented Dec 18, 2025

Summary

Adds image resizing flags to mathvista. -T max_dimension=... and quality=... and max_size_mb=...

Examples
-T quality=0:
Screenshot 2025-12-17 at 8 47 34 PM
-T quality=0 and -T max_dimension=100:
Screenshot 2025-12-17 at 8 48 14 PM
-T max_dimension=150:
Screenshot 2025-12-17 at 8 49 02 PM

What are you adding?

  • Bug fix (non-breaking change which fixes an issue)
  • New benchmark/evaluation
  • New model provider
  • CLI enhancement
  • Performance improvement
  • Documentation update
  • API/SDK feature
  • Integration (CI/CD, tools)
  • Export/import functionality
  • Code refactoring
  • Breaking change
  • Other

Changes Made

  • Adds image resizing flags to mathvista

Testing

  • I have run the existing test suite (pytest)
  • I have added tests for my changes
  • I have tested with multiple model providers (if applicable)
  • I have run pre-commit hooks (pre-commit run --all-files)

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (if applicable)
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Related Issues

Closes #

Additional Context


Note

Adds max_dimension, quality, and max_size_mb flags to MathVista dataset/task and implements a PIL-based image resize/re-encode pipeline.

  • Dataset Loader (src/openbench/datasets/mathvista.py):
    • Refactor record_to_sample into a factory accepting max_dimension, quality, max_size_mb and returning a converter.
    • Add PIL-based image processing: ensure RGB, optional resize to max_dimension, JPEG re-encode at quality, then size-based compression and data URI encoding.
    • get_dataset now forwards these parameters to record_to_sample.
  • Eval Task (src/openbench/evals/mathvista.py):
    • Expose and pass through max_dimension, quality, and max_size_mb to the dataset loader.

Written by Cursor Bugbot for commit 111a2ab. This will update automatically on new commits. Configure here.

@lee-groq lee-groq marked this pull request as ready for review December 18, 2025 04:58
@lee-groq lee-groq merged commit f8e643b into main Dec 18, 2025
15 of 16 checks passed
@lee-groq lee-groq deleted the lee/mathvista-image-resize-flags branch December 18, 2025 06:52
@github-actions github-actions bot mentioned this pull request Dec 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants