Skip to content

Add test case for testing recall under churn#14

Open
wahajali wants to merge 2 commits intomainfrom
recall-under-churn
Open

Add test case for testing recall under churn#14
wahajali wants to merge 2 commits intomainfrom
recall-under-churn

Conversation

@wahajali
Copy link

This PR introduces a new task stage that calculates recall under churn conditions, allowing us to evaluate the impact of deletions and reinsertions on recall over multiple cycles. The following changes have been made:

New Task Stage for Churn:

A new stage has been added to calculate recall after performing churn (deleting and reinserting embeddings) over multiple cycles. This helps evaluate how the recall changes as embeddings are churned.
New Parameters:

p_churn: Specifies the percentage of embeddings to churn (delete and reinsert) during each cycle.
cycles: Defines the number of churn cycles to perform, allowing multiple rounds of deletion and reinsertion to simulate data changes.

These updates provide the ability to benchmark vector database performance under churn conditions, ensuring better insights into recall degradation and recovery over time.

@wahajali wahajali requested a review from Sheharyar570 October 28, 2024 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants