Skip to content

Add quality-focused gRPC benchmark sweep#4

Open
SaiSandeepKantareddy wants to merge 3 commits intosochdb:mainfrom
SaiSandeepKantareddy:main
Open

Add quality-focused gRPC benchmark sweep#4
SaiSandeepKantareddy wants to merge 3 commits intosochdb:mainfrom
SaiSandeepKantareddy:main

Conversation

@SaiSandeepKantareddy
Copy link
Copy Markdown

@SaiSandeepKantareddy SaiSandeepKantareddy commented Apr 27, 2026

Summary

  • add a reusable gRPC quality sweep runner for staged benchmark work
  • document a quality-first benchmark methodology
  • add a retrieval/vector benchmark plan note to focus new work
  • add a server-side embedding bakeoff runner for model-vs-model quality comparisons

Why

This gives us a repeatable way to compare HNSW profiles on the same dataset and embeddings, and then compare stronger embedding models on the same evaluation path. That is the right order before considering alternative ANN families.

Validation

  • bash -n scripts/run_sochdb_grpc_quality_sweep.sh
  • bash -n scripts/run_sochdb_embedding_bakeoff.sh
  • ran the first SciFact HNSW quality sweep on the benchmark server successfully
  • started the first server-side embedding bakeoff run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant