Skip to content

fix: disallow wrapping auto-detected fsst in other compression#6120

Merged
westonpace merged 1 commit intolance-format:mainfrom
hamersaw:bug/disable-wrapping-fsst
Mar 6, 2026
Merged

fix: disallow wrapping auto-detected fsst in other compression#6120
westonpace merged 1 commit intolance-format:mainfrom
hamersaw:bug/disable-wrapping-fsst

Conversation

@hamersaw
Copy link
Copy Markdown
Contributor

@hamersaw hamersaw commented Mar 6, 2026

Currently, if a user manually sets a compression type (ex. zstd) we can inject FSST compression underneath. Resulting in zstd(fsst(data)) rather than zstd(data) that the user intended. This PR updates our auto-injection of FSST to only occur if the user has not specified a compression type manually.

Related to #5248

Signed-off-by: Daniel Rammer <hamersaw@protonmail.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 6, 2026

ACTION NEEDED
Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 6, 2026

PR Review

Bug fix is correct. The logic change gating auto-FSST on compression.is_none() cleanly prevents zstd(fsst(data)) when the user explicitly sets a compression scheme.

Missing test coverage (P0): Per project guidelines, all bugfixes must have corresponding tests. This PR has no test that verifies the fix. A test that sets compression to zstd on a Utf8 field with FSST-candidate data and asserts that FsstMiniBlockEncoder is NOT present in the resulting compressor would directly cover this bug.

Minor note: The fix to create_per_value (line 628) is technically unreachable for the reported bug scenario. When compression is set to zstd, the early return at line 620-623 (per_value_requested) already diverts to CompressedBufferEncoder before reaching use_fsst. Fixing it for consistency is reasonable, but worth noting it was not the active bug path.

@hamersaw hamersaw changed the title bug: fix wrapping auto-detected fsst in other compression fix: disallow wrapping auto-detected fsst in other compression Mar 6, 2026
@github-actions github-actions Bot added the bug Something isn't working label Mar 6, 2026
Copy link
Copy Markdown
Collaborator

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@westonpace westonpace merged commit c154ac8 into lance-format:main Mar 6, 2026
32 of 35 checks passed
@hamersaw hamersaw deleted the bug/disable-wrapping-fsst branch March 6, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants