validator: implement test for CQL per-row TTL#430
validator: implement test for CQL per-row TTL#430smoczy123 wants to merge 1 commit intoscylladb:masterfrom
Conversation
Verify that rows inserted with USING TTL are filtered out from ANN query results after they expire. The test inserts rows with and without TTL, creates an index, waits for expiry, and asserts that only non-TTL rows are returned. Fixes: VECTOR-598 Co-authored-by: Copilot <copilot@github.com>
There was a problem hiding this comment.
Pull request overview
Adds an end-to-end validator test to ensure CQL per-row TTL rows stop appearing in ANN query results after expiry (VECTOR-598), and wires it into the validator test suite.
Changes:
- Introduce a new
ttlvalidator module containing an ANN-vs-TTL expiration test. - Register the new module in the validator’s
test_cases()list.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
crates/validator/src/ttl.rs |
New E2E test that inserts rows with/without TTL, creates an index, waits for expiry, and validates ANN results exclude expired rows. |
crates/validator/src/lib.rs |
Adds the ttl module and registers its test case. |
| const ROW_TTL_SECONDS: i32 = 5; | ||
|
|
||
| #[framed] | ||
| pub(crate) async fn new() -> TestCase<TestActors> { | ||
| let timeout = DEFAULT_TEST_TIMEOUT; | ||
| TestCase::empty() | ||
| .with_init(timeout, init) | ||
| .with_cleanup(timeout, cleanup) | ||
| .with_test( | ||
| "cql_per_row_ttl_expires_from_index", | ||
| timeout, | ||
| cql_per_row_ttl_expires_from_index, | ||
| ) | ||
| } | ||
|
|
||
| /// Test that rows inserted with CQL per-row TTL are not returned by ANN | ||
| /// queries after they expire. | ||
| /// | ||
| /// 1. Create a table and insert rows — some with a short TTL, some without. | ||
| /// 2. Create an index and verify all rows are queryable via ANN. | ||
| /// 3. Wait for the TTL to expire. | ||
| /// 4. Verify that ANN queries return only the non-TTL rows. | ||
| #[framed] | ||
| async fn cql_per_row_ttl_expires_from_index(actors: TestActors) { | ||
| info!("started"); | ||
|
|
||
| let (session, clients) = prepare_connection(&actors).await; | ||
|
|
||
| let keyspace = create_keyspace(&session).await; | ||
| let table = create_table(&session, "pk INT PRIMARY KEY, v VECTOR<FLOAT, 3>", None).await; | ||
|
|
||
| info!("Insert 3 rows with TTL and 2 rows without TTL"); | ||
| for pk in 0..3 { | ||
| session | ||
| .query_unpaged( | ||
| format!( | ||
| "INSERT INTO {table} (pk, v) VALUES ({pk}, [{v}, 0.0, 0.0]) USING TTL {ROW_TTL_SECONDS}", | ||
| v = pk as f32, | ||
| ), | ||
| (), | ||
| ) | ||
| .await | ||
| .expect("failed to insert data with TTL"); | ||
| } | ||
| for pk in 10..12 { | ||
| session | ||
| .query_unpaged( | ||
| format!( | ||
| "INSERT INTO {table} (pk, v) VALUES ({pk}, [{v}, 1.0, 1.0])", | ||
| v = pk as f32, | ||
| ), | ||
| (), | ||
| ) | ||
| .await | ||
| .expect("failed to insert data without TTL"); | ||
| } | ||
|
|
||
| let index = create_index(CreateIndexQuery::new(&session, &clients, &table, "v")).await; |
There was a problem hiding this comment.
ROW_TTL_SECONDS is only 5s, but the test creates the index after inserting the TTL rows and then waits for the index to become SERVING. On a slower run, the TTL rows can expire before the index is built / before the first index_status.count == 5 and rows_num() == 5 assertions, making the test inherently flaky (it may never observe 5 rows). Consider creating the index before inserting data (then wait for the count to reach 5), and/or increasing the TTL to comfortably exceed worst-case index build time.
| info!("Wait for TTL to expire"); | ||
| time::sleep(Duration::from_secs(ROW_TTL_SECONDS as u64 + 2)).await; |
There was a problem hiding this comment.
Consider waiting for index size to decrement - in that case maybe this sleep is not needed.
There was a problem hiding this comment.
Is this is a test for CDC? If so, should it go into cdc.rs?
Verify that rows inserted with USING TTL are filtered out from ANN query results after they expire. The test inserts rows with and without TTL, creates an index, waits for expiry, and asserts that only non-TTL rows are returned.
Fixes: VECTOR-598