fix!: null handling when using `NOT` with scalar indices by wjones127 · Pull Request #5270 · lance-format/lance

wjones127 · 2025-11-18T18:49:04Z

BREAKING CHANGE: The SearchResult struct returned by ScalarIndex::search() now wraps a NullableRowIdSet instead of a RowIdTreeMap. Scalar indices must now provide the set of row ids where the expression value is null instead of just where it is true. Additionally, the RowIdMask is now an enum instead of a struct.

This PR fixes correctness bugs that show up when (a) running a filter with NOT, (b) the column you are filtering on contains nulls, and (c) we are using a scalar index (such as btree, or bitmap). Previously, this would give the wrong answer:

import pyarrow as pa
import lance

data = pa.table({"value": [1, 5, None]})
ds = lance.write_dataset(data, "memory://")
ds.create_scalar_index("value", "BTREE")
ds.to_table(filter="NOT (value < 2)")

pyarrow.Table
value: int64
----
value: [[5,null]]

It should not include null. The reason it did is that our RowIdMask (which is output by a scalar index query) was not aware of nulls. So when it processed value < 2, it would select just row index 0. Then NOT would invert that to [1, 2], selecting both [false, null].

This PR makes RowIdMask aware of nulls. When it processes value < 2, it records selected: [0] and nulls: [2]. Then, when you invert that and then drop, you get selected: [1], giving the correct final answer of just [5].

As part of this, we also refactor RowIdMask to make allow list and deny list mutually exclusive, which simplifies some of the logic.

Fixes #4756

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

westonpace

This is great work!

Here is a first pass of questions before looking at the other files.

westonpace · 2025-11-25T22:33:40Z

+* The `SearchResult` returned by scalar indices must now output information about null values.
+  Instead of containing a `RowIdTreeMap`, it now contains a `NullableRowIdSet`. Expressions that
+  resolve to null values must be included in search results in the null set. This ensures that
+  `NOT` can be applied to index search results correctly.


It's not clear why this is in the migration guide? Are we considering SearchResult and RowIdTreeMap to be part of our public API?

Or are you trying to explain that filter results may have changed? If it's the latter I think this needs to be reworded to be more clear that this means query results will be different.

Are we considering SearchResult and RowIdTreeMap to be part of our public API?

Yeah, I'm addressing this to authors of scalar indices outside of the main package.

westonpace · 2025-11-25T23:04:53Z

+    /// Check if a row_id is selected (TRUE)
+    pub fn selected(&self, row_id: u64) -> bool {
+        self.selected.contains(row_id) && !self.nulls.contains(row_id)
+    }
+
+    /// Get the selected rows (TRUE or NULL)
+    pub fn selected_rows(&self) -> &RowIdTreeMap {
+        &self.selected
+    }


It's a little unexpected that selected checks for TRUE but selected_rows is both TRUE and NULL. Maybe we could use not_false_rows instead of selected_rows?

I'm just going to remove this, as it's not that useful. The places where we use it are just in tests and for those we really want .true_rows().

westonpace · 2025-11-25T23:05:40Z

+    selected: RowIdTreeMap,
+    nulls: RowIdTreeMap,


Can a row id be in both sets? What does that signify?

Yes. Just signifies that it's null.

I was trying to imitate the Arrow approach, where you can ignore the actual value if it's null. This makes some computations easier, since you can handle null and selected separately.

westonpace · 2025-11-25T23:06:11Z

+        self.selected.clone() - self.nulls.clone()
+    }
+
+    pub fn union_all(selections: &[Self]) -> Self {


What happens if a row id is null in one set and true in another? I guess that is a repeat of the "can a row id be in both sets" question?

Oh this is a good question. I think we should treat union-all as basically reduce(selections, (a, b) => a | b), in which case I think it should mean NULL | TRUE => TRUE. That means there is a bug here. I will fix this.

westonpace · 2025-11-25T23:17:15Z

+            (Self::AllowList(allow), Self::BlockList(block))
+            | (Self::BlockList(block), Self::AllowList(allow)) => {


I'm still a fan of simplifying things so that this case isn't allowed but we can investigate in future issue.

I'm not sure how that would be possible, unless we got rid of BlockList as an option. How do you imagine this getting simpler?

Nevermind. I was thinking of something else entirely. There is no simplification here.

westonpace · 2025-11-25T23:22:00Z

+/// This is often a result of a filter, where `selected` represents the rows that
+/// passed the filter, and `nulls` represents the rows where the filter evaluated
+/// to null. For example, in SQL `NULL > 5` evaluates to null. This is distinct
+/// from being deselected to support proper three-valued logic for NOT.


Can we go ahead and state here that NULL | TRUE = TRUE and that FALSE & NULL = FALSE?

codecov · 2025-12-04T19:48:02Z

Codecov Report

❌ Patch coverage is 94.53441% with 81 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-index/src/scalar/expression.rs	77.23%	10 Missing and 18 partials ⚠️
rust/lance-index/src/scalar/btree.rs	86.20%	2 Missing and 10 partials ⚠️
rust/lance-index/src/scalar/bitmap.rs	91.05%	4 Missing and 7 partials ⚠️
rust/lance-core/src/utils/mask.rs	98.80%	2 Missing and 4 partials ⚠️
rust/lance-index/src/scalar.rs	62.50%	6 Missing ⚠️
rust/lance-index/src/scalar/bloomfilter.rs	95.45%	2 Missing and 2 partials ⚠️
rust/lance-index/src/scalar/lance_format.rs	95.00%	0 Missing and 4 partials ⚠️
rust/lance-index/src/scalar/zonemap.rs	95.74%	2 Missing and 2 partials ⚠️
rust/lance-core/src/utils/mask/nullable.rs	99.09%	0 Missing and 3 partials ⚠️
rust/lance/src/io/exec/scalar_index.rs	81.81%	1 Missing and 1 partial ⚠️
... and 1 more

📢 Thoughts on this report? Let us know!

Scalar indexes currently treat NULL values as FALSE when evaluating filters, which violates Kleene three-valued logic. This causes bugs like `x != 5` incorrectly including rows where x is NULL. This commit adds the foundation for proper null handling: - Extended RowIdMask with null_list field to track NULL rows - Implemented Kleene logic for NOT, AND, OR operations - Updated SearchResult to carry null row information - Modified all index implementations for backward compatibility The infrastructure is complete but indexes don't yet track nulls. Follow-up commits will implement actual null tracking in BTree and Bitmap indexes. Fixes lance-format#4756 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> start on btree and bitmap get compiling Add comprehensive tests for null handling in scalar indexes Added unit tests to verify Kleene three-valued logic operations work correctly: - RowIdMask AND/OR/NOT operations with nulls - also_block and also_allow preserve null_list - Serialization/deserialization of null_list - Bitmap index returns null_list in SearchResult These tests verify the null tracking infrastructure works correctly at the Rust level. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Add tests for null handling in bitmap and btree scalar indexes Adds comprehensive tests to verify that bitmap and btree indexes correctly track and return null row IDs in query results. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Fix pyproject.toml TOML syntax error The [tool.ruff] section had a duplicate lint key which caused maturin to fail parsing the file. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Fix RowIdMask serialization to use 3 elements for null_list After adding null_list to RowIdMask, the serialization now produces 3 elements (block_list, allow_list, null_list) instead of 2. Updated serialize_to_arrow and try_from_arrow to handle 3 elements correctly. This fixes the "all columns in a record batch must have the same length" error when using scalar indexes with null tracking. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Fix RowIdMask::iter_ids() to exclude null rows The iter_ids() method was only filtering out block_list entries but not null_list entries. This caused nulls to be included in query results when they should be filtered out according to Kleene three-valued logic. Updated iter_ids() to filter out both block_list and null_list entries, ensuring that null values are never returned in iteration. Added test_iter_ids_with_nulls() to verify the fix. Note: The Python integration test still fails, indicating there may be another code path that needs fixing. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> fix few places simplify cargo clippy try to improve comment wip wip get passing other indexes pr feedback cleanup simplify fix preserve existing serialization fix fix tests Fix NOT operation to preserve certainty with inexact indices Previously, NOT operations on inexact index results (AtMost/AtLeast) would keep the same certainty variant while negating the mask. This was incorrect because: - NOT(AtMost(x)) should return AtLeast(!x), not AtMost(!x) (complement of superset is subset) - NOT(AtLeast(x)) should return AtMost(!x), not AtLeast(!x) (complement of subset is superset) This caused incorrect query results when using inexact indices like bloom filters and zonemaps with NOT operators. Changes: - Fixed NOT implementation to flip certainty variants - Removed unused `map` method that was replaced by explicit match - Added unit tests for certainty flipping - Added integration tests for NOT with nulls 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> fix bug in union all

Add local helper functions to mask.rs and nullable.rs tests: - rows() for concise RowAddrTreeMap creation - assert_mask_selects() for bulk selection assertions - nullable_set(), allow(), block() for NullableRowAddrSet/Mask construction 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Use SearchResult::at_most() helper instead of AtMost variant - Change .contains() to .selected() in tests for NullableRowAddrSet - Remove redundant .clone() calls 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

wjones127 · 2025-12-08T19:49:15Z

I've added a bunch of test to give us 100% test coverage for the mask utilities.

…t#5270) BREAKING CHANGE: The `SearchResult` struct returned by `ScalarIndex::search()` now wraps a `NullableRowIdSet` instead of a `RowIdTreeMap`. Scalar indices must now provide the set of row ids where the expression value is null instead of just where it is true. Additionally, the `RowIdMask` is now an enum instead of a struct. This PR fixes correctness bugs that show up when (a) running a filter with `NOT`, (b) the column you are filtering on contains nulls, and (c) we are using a scalar index (such as btree, or bitmap). Previously, this would give the wrong answer: ```python import pyarrow as pa import lance data = pa.table({"value": [1, 5, None]}) ds = lance.write_dataset(data, "memory://") ds.create_scalar_index("value", "BTREE") ds.to_table(filter="NOT (value < 2)") ``` ``` pyarrow.Table value: int64 ---- value: [[5,null]] ``` It should not include `null`. The reason it did is that our `RowIdMask` (which is output by a scalar index query) was not aware of nulls. So when it processed `value < 2`, it would select just row index `0`. Then `NOT` would invert that to `[1, 2]`, selecting both `[false, null]`. This PR makes `RowIdMask` aware of nulls. When it processes `value < 2`, it records `selected: [0]` and `nulls: [2]`. Then, when you invert that and then drop, you get `selected: [1]`, giving the correct final answer of just `[5]`. As part of this, we also refactor RowIdMask to make allow list and deny list mutually exclusive, which simplifies some of the logic. Fixes lance-format#4756 --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

wjones127 added the critical-fix Bugs that cause crashes, security vulnerabilities, or incorrect data. label Nov 18, 2025

github-actions Bot added bug Something isn't working python labels Nov 18, 2025

wjones127 changed the title ~~fix: null handling with scalar indices~~ fix: null handling when using NOT with scalar indices Nov 19, 2025

wjones127 changed the title ~~fix: null handling when using NOT with scalar indices~~ fix!: null handling when using NOT with scalar indices Nov 20, 2025

github-actions Bot added the breaking-change label Nov 20, 2025

wjones127 marked this pull request as ready for review November 21, 2025 16:43

wjones127 requested a review from westonpace November 21, 2025 16:44

chatgpt-codex-connector Bot reviewed Nov 21, 2025

View reviewed changes

Comment thread rust/lance-index/src/scalar/expression.rs

westonpace reviewed Nov 25, 2025

View reviewed changes

wjones127 force-pushed the null-aware-scalar-indexes branch 2 times, most recently from 772e198 to 3fceb97 Compare December 4, 2025 19:20

wjones127 force-pushed the null-aware-scalar-indexes branch from 3fceb97 to ada7686 Compare December 4, 2025 20:04

wjones127 and others added 4 commits December 8, 2025 11:33

get coverage up to 90%

6f8c2d3

improve tests

f87d998

wjones127 force-pushed the null-aware-scalar-indexes branch from 7d8521c to 56dd824 Compare December 8, 2025 19:33

westonpace approved these changes Dec 9, 2025

View reviewed changes

wjones127 merged commit 4a9a533 into lance-format:main Dec 10, 2025
27 of 28 checks passed

andrea-reale mentioned this pull request Mar 30, 2026

emilk/fix write starvation rerun-io/lance#12

Closed

		(Self::AllowList(allow), Self::BlockList(block))
		\| (Self::BlockList(block), Self::AllowList(allow)) => {

Conversation

wjones127 commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

westonpace left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

wjones127 commented Dec 8, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wjones127 commented Nov 18, 2025 •

edited

Loading

codecov Bot commented Dec 4, 2025 •

edited

Loading