24.8.8 Backport of #62966 - Add parquet bloom filters support#556

Merged

Enmk merged 110 commits intocustomizations/24.8.8from

backports/24.8.8/parquet_bloom_filters

Dec 19, 2024

Commits on Dec 12, 2024

wip, fix hashing
arthurpassos
committed
shitty serialize impl, missing arrays
arthurpassos
committed
wip
arthurpassos
committed
undo something
arthurpassos
committed
delete unnecessary stuff
arthurpassos
committed
fix tests
arthurpassos
committed
remove unused variable
arthurpassos
committed
progress
arthurpassos
committed
style
arthurpassos
committed
try to fix defines
arthurpassos
committed
add if use parquet
arthurpassos
committed
more complex test
arthurpassos
committed
update docs
arthurpassos
committed
add new setting to settingshistory
arthurpassos
committed
fix
arthurpassos
committed
add suport for in check
arthurpassos
committed
add some tests for in cehck
arthurpassos
committed
try chassert
arthurpassos
committed
fix dumb shit
arthurpassos
committed
make test file reproducible, add bloom filter to the array column and implement in and has conditions
arthurpassos
committed
fix existing tests
arthurpassos
committed
add hasall support
arthurpassos
committed
update test files so array column has higher cardinality
arthurpassos
committed
simplify tests and add tests for has array operations
arthurpassos
committed
simplify code by reducing the scope for now. Also add test for equality to function (not constant)
arthurpassos
committed
add test for nested function
arthurpassos
committed
optimize things a bit
arthurpassos
committed
hash only once approach
arthurpassos
committed
minor update
arthurpassos
committed
extract rpn build process into separate class
arthurpassos
committed
do not get parquet reader from arrow
arthurpassos
committed
early return for no row groups
arthurpassos
committed
remove a.ref
arthurpassos
committed
specify a few types instead of auto
arthurpassos
committed
add missing file
arthurpassos
committed
settings history..
arthurpassos
committed
try to use keycondition instead
arthurpassos
committed
rmv debug var
arthurpassos
committed
extern bad arguments
arthurpassos
committed
style check
arthurpassos
committed
minor typo
arthurpassos
committed
minor fix
arthurpassos
committed
drop support for array types
arthurpassos
committed
move parquet bf stuff out of keycondition
arthurpassos
committed
error codes
arthurpassos
committed
few fixes
arthurpassos
committed
remove unnecessary code
arthurpassos
committed
delete unused files
arthurpassos
committed
progress on re-using keycondition and implementing in_set
arthurpassos
committed
minor stuff
arthurpassos
committed
evaluate rpn only once
arthurpassos
committed
fix issue when in_set columns do not have bf
arthurpassos
committed
only get bloom filters for filtering columns
arthurpassos
committed
get bf reader in non optimal way
arthurpassos
committed
only calculate filtering columns if key_condition and bloom fliter is present
arthurpassos
committed
get bf reader in optimal way
arthurpassos
committed
Delete
arthurpassos
committed
some improvements, altho code looks odd
arthurpassos
committed
Update SettingsChangesHistory.cpp
arthurpassos
committed
function_unknown instead of function_true in some cases
arthurpassos
committed
put reader properties in a variable
arthurpassos
committed
process in_set even if one of the columns do not have bf
arthurpassos
committed
simplify index mapping a bit
arthurpassos
committed
vector instead of map.. need to remember order of insertion is important
arthurpassos
committed
safe guard around bf across row groups
arthurpassos
committed
progress
arthurpassos
committed
simplify getfilteringcolumns
arthurpassos
committed
remove unused type alias
arthurpassos
committed
support only basic types with no special encodingwq
arthurpassos
committed
small fixes, I need to improve testing
arthurpassos
committed
tmp
arthurpassos
committed
handle nullable on tuple
arthurpassos
committed
progress
arthurpassos
committed
fix conflicts
arthurpassos
committed
fix steyle
arthurpassos
committed
tryconvertfieldtotype
arthurpassos
committed
update tests to new dataset
arthurpassos
committed
grab Field from ordered set columns and rely on convertfieldtotype
arthurpassos
committed
tmp
arthurpassos
committed
update tests
arthurpassos
committed
add proper uint64 tests
arthurpassos
committed
add uint8 and uuid tests
arthurpassos
committed
simplify logic to use vector of uint64 instead of columnptr. Fixes msan crash
arthurpassos
committed
undo hashstring modification
arthurpassos
committed
address some coments
arthurpassos
committed
simplify maybeTrueOnBloomFilter
arthurpassos
committed
progress
arthurpassos
committed
safe guards around index mapping
arthurpassos
committed
some structural changes
arthurpassos
committed
progress
arthurpassos
committed
move setting to 24.10
arthurpassos
committed
add test for ipv6
arthurpassos
committed
rmv ssh key
arthurpassos
committed
add new test file
arthurpassos
committed
minor improvements
arthurpassos
committed
few comments adressed
arthurpassos
committed
assert parquet column size is 16 bytes when hashing ipv6
arthurpassos
committed
simplify parquet bf rpn by removing function_equals
arthurpassos
committed
fix in with invalid conversion
arthurpassos
committed
docs change
arthurpassos
committed
f_unknown if monotonic_functions not empty
arthurpassos
committed
updt
arthurpassos
committed
updt
arthurpassos
committed
updt
arthurpassos
committed
updt
arthurpassos
committed
updt
arthurpassos
committed
updt
arthurpassos
committed
updt
arthurpassos
committed
Update FormatFactory.cpp
arthurpassos
committed
Update ParquetBlockInputFormat.cpp
arthurpassos
committed