Conversation
Signed-off-by: min.tian <min.tian.cn@gmail.com>
Signed-off-by: min.tian <min.tian.cn@gmail.com>
* feat: add hnsw support * refactor: minor fixes * feat: reformat code * fix: remove sql injections, reformat code
Signed-off-by: min.tian <min.tian.cn@gmail.com>
* Add --task-label option for cli * Fix lint issues
…#523) * Update cli.py * Update clickhouse.py * Update clickhouse.py * Update cli.py * Update config.py * remove space
…ch#521) * Add --concurrency-timeout option to avoid long time waiting, by default, it's 3600s. * Fix lint error * Update README.md, add --concurrency-timeout option
Signed-off-by: min.tian <min.tian.cn@gmail.com>
…lliztech#530) * Add a batch cli to support the batch execution of multiple cases. * Add how to use
…ch#529) * Fixing bugs in aws opensearch client Signed-off-by: Navneet Verma <navneev@amazon.com> * Added fp16 quantization type for aws_opensearch Signed-off-by: Navneet Verma <navneev@amazon.com> --------- Signed-off-by: Navneet Verma <navneev@amazon.com>
* Add Qdrant local * Add support for local setup without authentication api-key for weaviate * Expose HSNW index parameters to cli for weaviate * Add hnsw-ef parameter to Qdrant local --------- Co-authored-by: Min Tian <min.tian.cn@gmail.com>
The incorrect import is a result of previous duplicate import of the same class name and recent refactoring. zilliztech@032515f#diff-1638e273a6d7eee7b4412bf79bda9112da86d53e18d1b650b8b80a2e1df4f7e6R15
Signed-off-by: min.tian <min.tian.cn@gmail.com>
* expose lancedb index parameters to the cli interface * fix formatting issues
…t multi-client load data (zilliztech#527) * fix error of OpenSearch frontend:Validation error for AWSOpenSearchIndexConfig index_thread_qty_during_force_merge * add parameters of aws opensearch in frontend. * fiexd issue of opensearch ui and add multi-client indexing, add parameters of hnsw engine and metric-type. * fiexd issue of opensearch ui and add multi-client indexing, add parameters of hnsw engine and metric-type. * fiexd issue of opensearch ui and add multi-client indexing, add parameters of hnsw engine and metric-type. * clean logs * clean comments * clean logs * clean comments * fixed some issue, add warmup before search test only. * fixed some issue of create index and mulit-client * support faiss fp16 * clear comment * recover some code format * recover some code format * recover some code format * recover some code format * recover some code format * resolved conflict with pr: zilliztech#529
* support oceanbase * fix bug * fix bug * support IP * support multiple zone * support filter case * support ivf * support hnsw_bq and extra info * optimized formatting * optimized formatting * Revert dbCaseConfigs.py to match remote main branch * resolve review comments * Update __init__.py * Addressed review comments * Fixed code style issues
This commit marks the milestone release of VectorDBBench 1.0, introducing a wide range of new features, major enhancements, and updated benchmarks. Key changes include: - UI: Introduce a brand new homepage and navigation bar. The new design integrates powerful front-end pages for intuitive test result analysis and visualization. - Cases: Add new label-filter test cases. This allows testing search performance with metadata filters using expressions like color == "red". Initial support includes Milvus, Zilliz Cloud, Elasticsearch Cloud, Qdrant Cloud, Pinecone, and OpenSearch (AWS). - Cases: Implement new streaming test cases. These cases are designed to measure search performance while data is actively being inserted, simulating real-world "read-while-writing" scenarios. - Dataset: Add the new BioASQ dataset. This dataset is 1024-dimensional and comes in 1M and 10M sizes, enriching the diversity of our test data. - Custom Dataset: Enhance the custom dataset functionality. Users now have more flexible configuration options to simulate their own data distributions and schemas better. - New Results: Re-run and update all benchmark results for `Milvus`, `ZillizCloud`, `ElasticCloud`, `QdrantCloud`, `Pinecone`, and `OpenSearch(AWS)` to reflect their latest performance on the new test cases.
Signed-off-by: min.tian <min.tian.cn@gmail.com>
Signed-off-by: min.tian <min.tian.cn@gmail.com>
- UUID PRIMARY KEY for distributed write performance
- Connection pooling with 100+ base connections
- Comprehensive retry logic for transient errors (40001, 40003)
- C-SPANN vector index with tunable parameters
- CLI integration with full parameter support
Co-authored-by: Min Tian <min.tian.cn@gmail.com>
Signed-off-by: min.tian <min.tian.cn@gmail.com>
…illiztech#647) Two critical fixes for multi-node CockroachDB clusters: 1. Connection Timeout Handling: On multi-node v25.4 clusters, CREATE VECTOR INDEX from subprocess contexts experiences a 30-second connection timeout. The index creation continues successfully in the background. This fix detects the timeout and polls for completion (up to 5 minutes). 2. Vector Index Usage: Fixed vector_search_beam_size not being set on pooled connections, causing queries to use full table scan instead of the vector index. Now configures every connection from the pool with proper beam size. Testing: - Single-node: Works without timeout (178s index creation) - Multi-node: Successfully handles timeout and completes (131s total) - Vector index: Now properly used for all searches (verified with EXPLAIN) - Both achieve ~83% recall with good QPS Fixes issues where: - Benchmarks would fail despite successful index creation - Searches were slow due to full table scans instead of index usage
Signed-off-by: min.tian <min.tian.cn@gmail.com>
* [Bugfix] Several fixes to AliSQL Summary ======= 1. Online users cannot directly `SET GLOBAL` 2. Online instances uses strict SQL mode by default, which can affect vector insertion. * [Feature] AliSQL supports custom db name --------- Co-authored-by: wangzihao.wzh <guguang.wzh@alibaba-inc.com>
…illiztech#655) - Add frontend Streamlit UI configuration - Logo, parameter types, and input configurations - Loading and performance config for all index parameters - Add SSL/TLS support for CockroachDB Cloud clusters - Support verify-full, verify-ca, require, disable SSL modes - CLI parameters: --sslmode, --sslrootcert - SSL certificates passed to connection pool and optimize connections - Optimize connection handling for multi-node clusters - Connection pooling for concurrent searches (enables true parallelism) - Settings in connection string to reduce per-connection overhead - Dedicated connection without statement_timeout for index creation - Handles background index creation with timeout detection and polling - Handle CockroachDB-specific schema changes - Cancel running schema jobs before DROP TABLE (prevents blocking) - Transaction conflict retry logic for aborted transactions - All linting checks passing
…opensearch (zilliztech#663) * 1. Optimized logs output of load data 2. Modified create index dsl of on disk mode. * refine format
* made log util use env vars for custom log file placement * cleaned up log file config a bit It doesn't seem necessary to split the configuration of log file into log path and log name, but cleaner to just combine them.
…h#670) - Use HDR Histogram for memory-efficient latency tracking - Collect p99, p95, avg latency at each concurrency level per stage - Add Concurrent Search Performance UI with single-stage and multi-stage views - Closes zilliztech#638
- Add SCANN index type for Milvus - Build params: nlist (default: 1024), with_raw_data (default: false) - Search params: reorder_k (default: 100) - Add UI configuration for SCANN index Signed-off-by: JackLCL <chenglong.li@zilliz.com>
- Add nprobe search parameter for SCANN index (default: 64) - Enable nprobe UI configuration for SCANN_MILVUS index type Signed-off-by: JackLCL <chenglong.li@zilliz.com>
Signed-off-by: min.tian <min.tian.cn@gmail.com>
Signed-off-by: min.tian <min.tian.cn@gmail.com>
* Add Chroma cli to VectorDBBench * Fix lint issues * Add password auth functionality to chroma CLI --------- Co-authored-by: bpietrzak <>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.