Support --dimensions parameter in CLI

[persiyanov]

Description

Hi there!

First off – thank you for infinity, it's a fantastic project that allows setting up OpenAI-compatible API quickly.

I was integrating it with Typesense, which recently introduced semantic search capabilities. However, I'm using Matryoshka embeddings, and Typesense doesn't support customizing the OpenAI-server request.

Hence, I had to update the infinity CLI to support setting a default value for matryoshka_dim when the server starts.

The way you can run it is:

infinity_emb v2 --model-id /me/snowflake-arctic-embed-m-v2.0 --dimensions 256 --device cpu

I'm not sure if this aligns with the roadmap of the project, but I figured I'd submit a PR anyway if others find it helpful.

Checklist

• I have read the CONTRIBUTING guidelines.
• I have added tests to cover my changes.
• I have updated the documentation (docs folder) accordingly.

[greptile-apps]

PR Summary

The PR adds a new CLI parameter, --dimensions, to set default matryoshka embedding dimensions across the Infinity API.

• Updated /libs/infinity_emb/infinity_emb/args.py to include a new dimensions field in EngineArgs.
• Modified /libs/infinity_emb/infinity_emb/cli.py to pass the --dimensions parameter for both v1 and v2 commands.
• Integrated matryoshka_dim into /libs/infinity_emb/infinity_emb/inference/batch_handler.py for embedding operations.
• Updated /docs/docs/cli_v2.md to document the new parameter.
• Adjusted /libs/infinity_emb/infinity_server.py and end-to-end tests to ensure default dimensions are applied properly.

_{💡 (1/5) You can manually trigger the bot by mentioning @greptileai in a comment!}

_{8 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile}

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.83%. Comparing base (88bad14) to head (e9fdf0d).

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #564      +/-   ##
==========================================
- Coverage   79.85%   79.83%   -0.02%     
==========================================
  Files          43       43              
  Lines        3489     3497       +8     
==========================================
+ Hits         2786     2792       +6     
- Misses        703      705       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.