doc: Update Iceberg cache documentation for clarity and detail #3277
+640
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ng strategies and best practices
- Add new section 'Iceberg Metadata Cache Enhancements (Since 4.0.3)' - Update Manifest Cache default to disabled (false) - Clarify that Manifest Cache does not affect data visibility - Restore original simple sections for Iceberg Table Information and Snapshot - Update disable cache instructions to distinguish 4.0.3+ from earlier versions Applied same changes as master branch to version-4.x documentation.
morningman
reviewed
Jan 27, 2026
|
|
||
| 该缓存由 `IcebergMetadataCache` 维护，每个 Iceberg Catalog 都有自己独立的实例，包含 `tableCache` 和 `viewCache` 两个缓存。 | ||
|
|
||
| 缓存的表对象（`IcebergTableCacheValue`）中还包含 Snapshot 信息，该信息按需懒加载（主要用于 MTMV 场景）。 |
Contributor
There was a problem hiding this comment.
Suggested change
| 缓存的表对象（`IcebergTableCacheValue`）中还包含 Snapshot 信息，该信息按需懒加载（主要用于 MTMV 场景）。 | |
| 缓存的表对象（`IcebergTableCacheValue`）中还包含 Snapshot 信息，该信息按需懒加载（主要用于多表物化视图场景）。 |
| - **Partition（分区）**：分区信息使用缓存的表对象的元数据（分区规范、快照）加载。缓存越旧，分区信息越滞后。 | ||
|
|
||
| :::tip | ||
| 要实时看到最新的 Schema、Snapshot 和 Partition 信息，需要禁用表缓存，设置 `iceberg.table.meta.cache.ttl-second=0`。Schema 缓存不影响使用的版本——它只是为了性能缓存已解析的结果。 |
|
|
||
| - **最短刷新时间** | ||
|
|
||
| 由 FE 配置项 `external_cache_refresh_time_minutes` 控制，单位为分钟。默认为 10 分钟。这是异步刷新，不会阻塞当前操作。 |
| 该缓存存储的是**已解析的** Iceberg Manifest 文件内容——具体是从 Manifest 文件中提取的 `DataFile` 和 `DeleteFile` 对象（而不是原始文件字节）： | ||
|
|
||
| - `DataFile` 对象：文件元数据，包括路径、分区值、统计信息等 | ||
| - `DeleteFile` 对象：Equality Delete 的删除元数据 |
| CREATE CATALOG iceberg_catalog PROPERTIES ( | ||
| 'type' = 'iceberg', | ||
| ... | ||
| 'iceberg.manifest.cache.enable' = 'true', -- 启用 Doris Manifest Cache |
Contributor
There was a problem hiding this comment.
Suggested change
| 'iceberg.manifest.cache.enable' = 'true', -- 启用 Doris Manifest Cache | |
| 'iceberg.manifest.cache.enable' = 'true', -- 启用 Iceberg Manifest Cache |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
relate prs:
Versions
Languages
Docs Checklist