Support Iceberg Metadata Files Cache #77156 #733
Conversation
…nce_id_and_implementation_refactoring Add sequence id to iceberg metadata and iceberg avro parsing refactoring
…parsing_in_iceberg Fix path parsing in iceberg
Add `icebergTruncate` function and support it in partition pruning for Iceberg
…ime_travel_by_snapshots Iceberg time travel by snapshots
Refactor operations with Avro files in Iceberg-related code.
…_iceberg Trivial count optimization for iceberg
…ceberg Refactor some code in Iceberg Storage
Minmax iceberg
…ng_for_metadata Allow to specify metadata file for Iceberg
…-issue delta-kernel: fix progress bar, fix performance
…-pruning-in-delta-kernel Support partition pruning in DeltaLake engine
…und-data-lakes Small refactoring around data lakes
…eatadata_cache Support Iceberg Metadata Files Cache
arthurpassos
left a comment
arthurpassos
left a comment
There was a problem hiding this comment.
In order to properly review this PR, it would take a whole week if not more. All I could do was sanity check. Given the deadlines, I guess we have to trust CI/CD and merge it.
QA could help a lot on this one.
| ZoneInfoSourceFactory zone_info_source_factory = custom_factory; | ||
| } | ||
|
|
||
| DateLUTImpl::Values DateLUTImpl::lutIndexByMonthSinceEpochStartsZeroIndexing(Int32 months) const |
There was a problem hiding this comment.
Why is this being removed? What does DateLut has to do with iceberg?
There was a problem hiding this comment.
Removed in ClickHouse#77403, code in that PR totally refactored and this methods are not used anymore. PR added partition pruning for Iceberg.
https://github.com/ClickHouse/ClickHouse/pull/77403/files#r1998468379
There was a problem hiding this comment.
PR makes a refactoring in Iceberg code, and PR Iceberg metadata cache changes the same code later, so I cherry-picked it to get less conflicts.
| }; | ||
|
|
||
| template <ResultPrecision precision_> | ||
| struct ToYearNumSinceEpochImpl |
There was a problem hiding this comment.
Why do we need this?
| static constexpr bool bit_hamming_distance = IsSameOperation<Op, BitHammingDistanceImpl>::value; | ||
|
|
||
| static constexpr bool division = div_floating || int_div || int_div_or_zero || modulo; | ||
| static constexpr bool division = div_floating || int_div || int_div_or_zero || modulo || positive_modulo; |
| @@ -0,0 +1,24 @@ | |||
| #include <Functions/FunctionFactory.h> | |||
Backport of Support Iceberg Metadata Files Cache
ClickHouse#77156
And dependencies
ClickHouse#78775
ClickHouse#78486
ClickHouse#78368
ClickHouse#77318
ClickHouse#78242
ClickHouse#78021
ClickHouse#78090
ClickHouse#77916
ClickHouse#77439
ClickHouse#77403
ClickHouse#76681
ClickHouse#75010