[branch-0.9] Cherry pick needed commits for deps: upgrade DataFusion to 53.0, Arrow to 58 (apache#2206)#11
Merged
toutane merged 26 commits intobranch-0.9from Apr 23, 2026
Merged
Conversation
) ## Which issue does this PR close? Split off from apache#1851 - Partially fixes apache#1731. ## What changes are included in this PR? This change honors the compression setting for metadata.json file (`write.metadata.compression-codec`). ## Are these changes tested? Add unit test to verify files are gzipped when the flag is enabled. BREAKING CHANGE: Make `write_to` take `MetadataLocation` --------- Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> (cherry picked from commit b6de5db)
…thon (apache#2228) Bumps [quinn-proto](https://github.com/quinn-rs/quinn) from 0.11.13 to 0.11.14. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/quinn-rs/quinn/releases">quinn-proto's releases</a>.</em></p> <blockquote> <h2>quinn-proto 0.11.14</h2> <p><a href="https://github.com/jxs"><code>@jxs</code></a> reported a denial of service issue in quinn-proto 5 days ago:</p> <ul> <li><a href="https://github.com/quinn-rs/quinn/security/advisories/GHSA-6xvm-j4wr-6v98">https://github.com/quinn-rs/quinn/security/advisories/GHSA-6xvm-j4wr-6v98</a></li> </ul> <p>We coordinated with them to release this version to patch the issue. Unfortunately the maintainers missed these issues during code review and we did not have enough fuzzing coverage -- we regret the oversight and have added an additional fuzzing target.</p> <p>Organizations that want to participate in coordinated disclosure can contact us privately to discuss terms.</p> <h2>What's Changed</h2> <ul> <li>Fix over-permissive proto dependency edge by <a href="https://github.com/Ralith"><code>@Ralith</code></a> in <a href="https://redirect.github.com/quinn-rs/quinn/pull/2385">quinn-rs/quinn#2385</a></li> <li>0.11.x: avoid unwrapping VarInt decoding during parameter parsing by <a href="https://github.com/djc"><code>@djc</code></a> in <a href="https://redirect.github.com/quinn-rs/quinn/pull/2559">quinn-rs/quinn#2559</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/quinn-rs/quinn/commit/2c315aa7f9c2a6c1db87f8f51f40623a427c78fd"><code>2c315aa</code></a> proto: bump version to 0.11.14</li> <li><a href="https://github.com/quinn-rs/quinn/commit/8ad47f431e7deb82c08b09c2e33ef85aa88fd212"><code>8ad47f4</code></a> Use newer rustls-pki-types PEM parser API</li> <li><a href="https://github.com/quinn-rs/quinn/commit/c81c0289abe30d8437ccbf9b6304e2bc9c707cea"><code>c81c028</code></a> ci: fix workflow syntax</li> <li><a href="https://github.com/quinn-rs/quinn/commit/0050172969f7e69e136c433181330da7790d8d73"><code>0050172</code></a> ci: pin wasm-bindgen-cli version</li> <li><a href="https://github.com/quinn-rs/quinn/commit/8a6f82c58d1c565eab78f986e614223e6ed76a85"><code>8a6f82c</code></a> Take semver-compatible dependency updates</li> <li><a href="https://github.com/quinn-rs/quinn/commit/e52db4ad8df0f9720e7b0e32ecc0e48c9a93de0f"><code>e52db4a</code></a> Apply suggestions from clippy 1.91</li> <li><a href="https://github.com/quinn-rs/quinn/commit/6df7275c582ca9b7225e0ccf9f9871a55eb73155"><code>6df7275</code></a> chore: Fix <code>unnecessary_unwrap</code> clippy</li> <li><a href="https://github.com/quinn-rs/quinn/commit/c8eefa07e087b06d8f2b78ff262ce8ac952994f1"><code>c8eefa0</code></a> proto: avoid unwrapping varint decoding during parameters parsing</li> <li><a href="https://github.com/quinn-rs/quinn/commit/9723a977754c8662001b0fef97aab8f3ddf1df92"><code>9723a97</code></a> fuzz: add fuzzing target for parsing transport parameters</li> <li><a href="https://github.com/quinn-rs/quinn/commit/eaf0ef30252cef4acec21f150427e604cd4271c9"><code>eaf0ef3</code></a> Fix over-permissive proto dependency edge (<a href="https://redirect.github.com/quinn-rs/quinn/issues/2385">#2385</a>)</li> <li>Additional commits viewable in <a href="https://github.com/quinn-rs/quinn/compare/quinn-proto-0.11.13...quinn-proto-0.11.14">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-rust/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 619b98f)
## Which issue does this PR close? - Closes apache#2086 . ## What changes are included in this PR? In this pr we introduced catalog test suite in catalog-loader, which could unify the behavior of catalogs. ## Are these changes tested? Yes. --------- Co-authored-by: Ray Liu <liurenjie2008@gmail.com> (cherry picked from commit 335961a)
Bumps [datafusion](https://github.com/apache/datafusion) from 52.2.0 to 52.3.0. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/datafusion/commit/28d012a41a3017b5f682ef6b01468a7ff9a48fb7"><code>28d012a</code></a> [branch-52] Bump to 52.3.0 and changelog (<a href="https://redirect.github.com/apache/datafusion/issues/20790">#20790</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/1bd7082b798d0d55c1e90c7be1d7e3dba057c288"><code>1bd7082</code></a> [branch-52] Fix repartition from dropping data when spilling (<a href="https://redirect.github.com/apache/datafusion/issues/20672">#20672</a>) (<a href="https://redirect.github.com/apache/datafusion/issues/20777">#20777</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/9797095e152749721bec07c0944fe664acaa0849"><code>9797095</code></a> [branch-52] perf: sort replace free()->try_grow() pattern with try_resize() t...</li> <li><a href="https://github.com/apache/datafusion/commit/afc1c72a15bdd31e15a7e354e86a505be7882f08"><code>afc1c72</code></a> [branch-52] FFI_TableOptions are using default values only (<a href="https://redirect.github.com/apache/datafusion/issues/20705">#20705</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/d317d00b886bbf11cb489e4c4bdc2280b3ca9e07"><code>d317d00</code></a> [branch-52] fix: <code>HashJoin</code> panic with String dictionary keys (don't flatten ...</li> <li><a href="https://github.com/apache/datafusion/commit/72ea8ec086e59220f6b255ea565e710990ad7967"><code>72ea8ec</code></a> [branch-52] Fix constant value from stats (<a href="https://redirect.github.com/apache/datafusion/issues/20042">#20042</a>) (<a href="https://redirect.github.com/apache/datafusion/issues/20709">#20709</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/9a67de58c027e6057aa37327ae4d0192d5c45fc5"><code>9a67de5</code></a> [branch-52] Fix Arrow Spill Underrun (<a href="https://redirect.github.com/apache/datafusion/issues/20159">#20159</a>) (<a href="https://redirect.github.com/apache/datafusion/issues/20684">#20684</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/19a0fcaa276c86beda544c6e01c75f6e0639767e"><code>19a0fca</code></a> [branch-52] SortMergeJoin don't wait for all input before emitting (<a href="https://redirect.github.com/apache/datafusion/issues/20699">#20699</a>)</li> <li>See full diff in <a href="https://github.com/apache/datafusion/compare/52.2.0...52.3.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit c2b24db)
Bumps [serde_with](https://github.com/jonasbb/serde_with) from 3.17.0 to 3.18.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/jonasbb/serde_with/releases">serde_with's releases</a>.</em></p> <blockquote> <h2>serde_with v3.18.0</h2> <h3>Added</h3> <ul> <li>Support <code>OneOrMany</code> with more sequence and set types (<a href="https://redirect.github.com/jonasbb/serde_with/issues/929">#929</a>)</li> </ul> <h3>Changed</h3> <ul> <li>Bump MSRV to 1.88 due to the <code>darling</code> dependency</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/jonasbb/serde_with/commit/d50ec962c6ecad7d8972f95d7ee7cea398b7eb41"><code>d50ec96</code></a> Bump version to 3.18.0 (<a href="https://redirect.github.com/jonasbb/serde_with/issues/931">#931</a>)</li> <li><a href="https://github.com/jonasbb/serde_with/commit/984fe3252ecd47526f452e39736f70f96b503f7c"><code>984fe32</code></a> Bump version to 3.18.0</li> <li><a href="https://github.com/jonasbb/serde_with/commit/4ba41c70c7f12b2e543ae81480a50b4d76245419"><code>4ba41c7</code></a> Bump actions/upload-artifact from 6 to 7 in the github-actions group (<a href="https://redirect.github.com/jonasbb/serde_with/issues/927">#927</a>)</li> <li><a href="https://github.com/jonasbb/serde_with/commit/8fb2468ce24e822fc29cd9aa8ebb3feb3ddf1eb3"><code>8fb2468</code></a> Bump actions/upload-artifact from 6 to 7 in the github-actions group</li> <li><a href="https://github.com/jonasbb/serde_with/commit/aec0a23c15943bc4ca82d329695fabefb2b19174"><code>aec0a23</code></a> Bump MSRV to 1.88 (<a href="https://redirect.github.com/jonasbb/serde_with/issues/930">#930</a>)</li> <li><a href="https://github.com/jonasbb/serde_with/commit/25c15a2c5c53f8fa71af91d699877147568338b8"><code>25c15a2</code></a> Update time dependency to 0.3.47</li> <li><a href="https://github.com/jonasbb/serde_with/commit/93bd3f4bebec516e5608a12f09ad1859cdced9a7"><code>93bd3f4</code></a> Update test output after darling update</li> <li><a href="https://github.com/jonasbb/serde_with/commit/f825dbffb12dd758c80247259a271956f1c484b4"><code>f825dbf</code></a> Upgrade darling to 0.23.0</li> <li><a href="https://github.com/jonasbb/serde_with/commit/65cbd738f090f25d89ec4b350501b4aa5b38bd9e"><code>65cbd73</code></a> Bump MSRV to 1.88</li> <li><a href="https://github.com/jonasbb/serde_with/commit/daff02ea264c3136131bfcff079304714f359bd9"><code>daff02e</code></a> Extend OneOrMany implementation to more collection types (<a href="https://redirect.github.com/jonasbb/serde_with/issues/929">#929</a>)</li> <li>Additional commits viewable in <a href="https://github.com/jonasbb/serde_with/compare/v3.17.0...v3.18.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit dde0aea)
(cherry picked from commit d16a9bb)
Bumps [lz4_flex](https://github.com/pseitz/lz4_flex) from 0.12.0 to 0.12.1. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/PSeitz/lz4_flex/blob/main/CHANGELOG.md">lz4_flex's changelog</a>.</em></p> <blockquote> <h1>0.12.1 (2026-03-14)</h1> <h3>Security Fix</h3> <ul> <li>Fix handling of invalid match offsets during decompression <a href="https://github.com/PSeitz/lz4_flex/commit/a0b9154">#a0b9154</a> (thanks <a href="https://github.com/Marcono1234"><code>@Marcono1234</code></a>)</li> </ul> <pre><code>Invalid match offsets (offset == 0) during decompression were not properly handled, which could lead to invalid memory reads on untrusted input. Users on 0.12.x should upgrade to 0.12.1. </code></pre> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/PSeitz/lz4_flex/commit/fa48c987a88df5059a49fe7519c028d6f2b8caf4"><code>fa48c98</code></a> bump version to 0.12.1</li> <li><a href="https://github.com/PSeitz/lz4_flex/commit/a0b9154becbe22da3ce91211d7b6619c289723cf"><code>a0b9154</code></a> fix handling of invalid match offsets during decompression</li> <li>See full diff in <a href="https://github.com/pseitz/lz4_flex/compare/0.12.0...0.12.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-rust/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 16c2126)
…pache#2238) Bumps [lz4_flex](https://github.com/pseitz/lz4_flex) from 0.12.0 to 0.12.1. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/PSeitz/lz4_flex/blob/main/CHANGELOG.md">lz4_flex's changelog</a>.</em></p> <blockquote> <h1>0.12.1 (2026-03-14)</h1> <h3>Security Fix</h3> <ul> <li>Fix handling of invalid match offsets during decompression <a href="https://github.com/PSeitz/lz4_flex/commit/a0b9154">#a0b9154</a> (thanks <a href="https://github.com/Marcono1234"><code>@Marcono1234</code></a>)</li> </ul> <pre><code>Invalid match offsets (offset == 0) during decompression were not properly handled, which could lead to invalid memory reads on untrusted input. Users on 0.12.x should upgrade to 0.12.1. </code></pre> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/PSeitz/lz4_flex/commit/fa48c987a88df5059a49fe7519c028d6f2b8caf4"><code>fa48c98</code></a> bump version to 0.12.1</li> <li><a href="https://github.com/PSeitz/lz4_flex/commit/a0b9154becbe22da3ce91211d7b6619c289723cf"><code>a0b9154</code></a> fix handling of invalid match offsets during decompression</li> <li>See full diff in <a href="https://github.com/pseitz/lz4_flex/compare/0.12.0...0.12.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-rust/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: blackmwk <liurenjie1024@outlook.com> (cherry picked from commit cc256f5)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> (cherry picked from commit c1497a9)
(cherry picked from commit 0196cb1)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - publish has to be done one by one, otherwise we may see failure like this: https://github.com/apache/iceberg-rust/actions/runs/23260056698 ## What changes are included in this PR? - Change publish parallism back to 1 <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> (cherry picked from commit 7db4b01)
Removed GitHub Actions dependency update configuration. ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? Related to apache/iceberg-python#3186 Dont auto update since we now depend on github action being allowlisted by asf-infra first, https://github.com/apache/infrastructure-actions/blob/main/approved_patterns.yml <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> (cherry picked from commit cb7f78a)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? Pin `astral-sh/setup-uv` to commit SHAs from Apache's [infrastructure-actions allowlist](https://github.com/apache/infrastructure-actions/blob/07f5f9d2b05fe0ec9886e3ef0a9d79797817f0cb/approved_patterns.yml#L9) Fixes apache/infrastructure-actions#550 <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> (cherry picked from commit 99dbb16)
…atures (apache#2274) ## Which issue does this PR close? - Fix the audit check by updating `aws-lc-sys` and `rustls-webpki`. - Avoid pulling both the legacy `rustls` / Hyper 0.14 stack and the newer `default-https-client` stack through inherited AWS SDK defaults. ([AWS SDK announcement](awslabs/aws-sdk-rust#1257)) ## What changes are included in this PR? - Bump to `aws-lc-sys>=0.39.0` and `rustls-webpki>=0.103.10` to pass security audit. - Disable inherited AWS SDK default features for `aws-sdk-glue` and `aws-sdk-s3tables` - Explicitly enable `default-https-client` and `rt-tokio` - Bump the minimum `aws-sdk-glue` version to `1.85`, the first version that provides `default-https-client` ## Are these changes tested? --------- Co-authored-by: blackmwk <liurenjie1024@outlook.com> (cherry picked from commit 2820d47)
Bumps [minijinja](https://github.com/mitsuhiko/minijinja) from 2.17.1 to 2.18.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/mitsuhiko/minijinja/blob/main/CHANGELOG.md">minijinja's changelog</a>.</em></p> <blockquote> <h2>2.18.0</h2> <ul> <li>Added keyword argument support (<code>width</code>, <code>first</code>, <code>blank</code>) to the <code>indent</code> filter for Jinja2 compatibility in Rust and Go. <a href="https://redirect.github.com/mitsuhiko/minijinja/issues/864">#864</a></li> <li>Added support for dotted integer lookup (for example <code>foo.0</code>) in Rust and Go for Jinja compatibility. <a href="https://redirect.github.com/mitsuhiko/minijinja/issues/881">#881</a></li> <li>Added support for dotted filter and test names (including <code>foo . bar . baz</code>) for Jinja compatibility. <a href="https://redirect.github.com/mitsuhiko/minijinja/issues/879">#879</a></li> <li>Fixed string escape handling to preserve unknown escapes (such as <code>\s</code>) for Jinja compatibility in Rust and Go. <a href="https://redirect.github.com/mitsuhiko/minijinja/issues/880">#880</a></li> <li>Improved generic performance across template parsing, compilation, and rendering.</li> <li>Fixed <code>minijinja-cabi</code> ownership and pointer-safety issues that could leak <code>mj_value</code> values on error paths.</li> <li>Added high-priority <code>minijinja-cabi</code> APIs for callback-based functions/filters/tests, globals, loaders, path joining, auto-escape configuration, and fuel limits.</li> <li>Switched <code>minijinja-cabi</code> header maintenance to manual source-based syncing and removed cbindgen-based generation tooling.</li> <li>Added lightweight C smoke tests for <code>minijinja-cabi</code> (via <code>make -C minijinja-cabi test</code>) with coverage across all exported C ABI functions, and wired them into top-level testing and CI.</li> <li>Added <code>render_captured</code> and <code>render_captured_to</code> methods on <code>Template</code> which return a <code>Captured</code> type holding the rendered output and the template state.</li> <li>Added <code>into_output</code> method on <code>Captured</code> to consume and return the output string.</li> <li>Deprecated <code>render_and_return_state</code>, <code>eval_to_state</code>, and <code>render_to_write</code> in favor of the new <code>render_captured</code> / <code>render_captured_to</code> / <code>Captured</code> API.</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/mitsuhiko/minijinja/commit/92f114d1fd62525b2b4dc1adb77ae1e83c1214a9"><code>92f114d</code></a> release 2.18.0</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/80d30a7526a0119981a1664fab8036b7e64c0d14"><code>80d30a7</code></a> refactor(vendor): prune unused self_cell API surface</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/50ce37a18ad368f22b4c40ff2b3355895ff58556"><code>50ce37a</code></a> fix: typos</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/24891e10c207846fa264c0f8eca930045bbb5fca"><code>24891e1</code></a> feat(filters): add kwargs support to indent filter for Jinja2 parity</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/4cca670f8a346832771d2a567f778b5dc4058156"><code>4cca670</code></a> refactor: deprecate render_to_write in favor of render_captured_to</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/ac88f8e619e0b7d5a4e23819ed5d2ebc046029c6"><code>ac88f8e</code></a> fix: correct typo render_capturedd_to -> render_captured_to</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/710137b2626cfae81b1eb935ea4c9df2435c053d"><code>710137b</code></a> chore: remove dead_code allow and unused MutBorrow from vendored self_cell</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/39d00e61a9f7246b7015dcf655f11159cde1d8cd"><code>39d00e6</code></a> feat: Added new capture methods for state</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/42b0d089333363b8bd667ec99ab67ff7977ef6d4"><code>42b0d08</code></a> feat: vendor self_cell and make loader default</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/cc12ae0812b8d85dd5963cfa373971fb0b1ff6da"><code>cc12ae0</code></a> fix: make cabi compatible with older rustc</li> <li>Additional commits viewable in <a href="https://github.com/mitsuhiko/minijinja/compare/minijinja-go/v2.17.1...minijinja-go/v2.18.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: blackmwk <liurenjie1024@outlook.com> (cherry picked from commit dc0a3fa)
Bumps [datafusion](https://github.com/apache/datafusion) from 52.3.0 to 52.4.0. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/datafusion/commit/e5bad58716cf74612ff3b245010411425063c3ec"><code>e5bad58</code></a> [branch-52] Update version to 52.4.0 and update changelog (<a href="https://redirect.github.com/apache/datafusion/issues/21004">#21004</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/e034c6b0b103c674c4576644007b30480565bec3"><code>e034c6b</code></a> [branch-52] Update to use lz4_flex 0.12.1 and quinn-proto 0.11.14 (<a href="https://redirect.github.com/apache/datafusion/issues/21009">#21009</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/664099b60640097a982e63174a96d8828fe1dc0d"><code>664099b</code></a> [branch-52] fix: InList Dictionary filter pushdown type mismatch (<a href="https://redirect.github.com/apache/datafusion/issues/20962">#20962</a>) (<a href="https://redirect.github.com/apache/datafusion/issues/2">#2</a>...</li> <li><a href="https://github.com/apache/datafusion/commit/74aaa65001afd7bc649f471bcf634d52744c46fd"><code>74aaa65</code></a> [branch-52] chore: Ignore RUSTSEC-2024-0014 (<a href="https://redirect.github.com/apache/datafusion/issues/20862">#20862</a>) (<a href="https://redirect.github.com/apache/datafusion/issues/21020">#21020</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/5881edec5d937036891bbec9e7cb01837d9155a5"><code>5881ede</code></a> [branch-52] fix: SanityCheckPlan error with window functions and NVL filter (...</li> <li><a href="https://github.com/apache/datafusion/commit/7e20eb7ddb3acf8174af7adec52859e28333d570"><code>7e20eb7</code></a> [branch-52] perf: Cache num_output_rows in sort merge join to avoid O(n) reco...</li> <li><a href="https://github.com/apache/datafusion/commit/e5547e2772fbaed693e7472f38feab690a7fe3ef"><code>e5547e2</code></a> [branch-52] Fix duplicate group keys after hash aggregation spill (<a href="https://redirect.github.com/apache/datafusion/issues/20724">#20724</a>) (#...</li> <li><a href="https://github.com/apache/datafusion/commit/2947378e9ef9dbdda75b4ff047edcfc1a06ef0d2"><code>2947378</code></a> [branch-52] fix: disable dynamic filter pushdown for non min/max aggregates (...</li> <li><a href="https://github.com/apache/datafusion/commit/41acbf8e4bb4ac15003bd5365661e6b17551f7f0"><code>41acbf8</code></a> [branch-52] fix: Return <code>probe_side.len()</code> for RightMark/Anti count(*) querie...</li> <li><a href="https://github.com/apache/datafusion/commit/a5f6fbb4cd89a47e1036986abe201def15542093"><code>a5f6fbb</code></a> [branch-52] fix: interval analysis error when have two filterexec that inner ...</li> <li>Additional commits viewable in <a href="https://github.com/apache/datafusion/compare/52.3.0...52.4.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: blackmwk <liurenjie1024@outlook.com> (cherry picked from commit 958ae65)
(cherry picked from commit 1e2ffb9)
…ing wrong version (apache#2277) (cherry picked from commit 4c08d35)
…python (apache#2278) Addresses the security advisory GHSA-pwjx-qhcg-rvj4 for rustls-webpki < 0.103.10 in the Python bindings lockfile. This is a rebase of apache#2268 onto main which already includes the root Cargo.lock audit fix from apache#2274 (aws-lc-sys >= 0.39.0). ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? ## Are these changes tested? ci. (cherry picked from commit 1f1ba53)
…ache#2281) Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.11.0 to 1.11.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/tokio-rs/bytes/releases">bytes's releases</a>.</em></p> <blockquote> <h2>Bytes v1.11.1</h2> <h1>1.11.1 (February 3rd, 2026)</h1> <ul> <li>Fix integer overflow in <code>BytesMut::reserve</code></li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md">bytes's changelog</a>.</em></p> <blockquote> <h1>1.11.1 (February 3rd, 2026)</h1> <ul> <li>Fix integer overflow in <code>BytesMut::reserve</code></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/tokio-rs/bytes/commit/417dccdeff249e0c011327de7d92e0d6fbe7cc43"><code>417dccd</code></a> Release bytes v1.11.1 (<a href="https://redirect.github.com/tokio-rs/bytes/issues/820">#820</a>)</li> <li><a href="https://github.com/tokio-rs/bytes/commit/d0293b0e35838123c51ca5dfdf468ecafee4398f"><code>d0293b0</code></a> Merge commit from fork</li> <li>See full diff in <a href="https://github.com/tokio-rs/bytes/compare/v1.11.0...v1.11.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-rust/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 44e73f6)
…che#2282) Bumps [time](https://github.com/time-rs/time) from 0.3.44 to 0.3.47. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/time-rs/time/releases">time's releases</a>.</em></p> <blockquote> <h2>v0.3.47</h2> <p>See the <a href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a> for details.</p> <h2>v0.3.46</h2> <p>See the <a href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a> for details.</p> <h2>v0.3.45</h2> <p>See the <a href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a> for details.</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">time's changelog</a>.</em></p> <blockquote> <h2>0.3.47 [2026-02-05]</h2> <h3>Security</h3> <ul> <li> <p>The possibility of a stack exhaustion denial of service attack when parsing RFC 2822 has been eliminated. Previously, it was possible to craft input that would cause unbounded recursion. Now, the depth of the recursion is tracked, causing an error to be returned if it exceeds a reasonable limit.</p> <p>This attack vector requires parsing user-provided input, with any type, using the RFC 2822 format.</p> </li> </ul> <h3>Compatibility</h3> <ul> <li>Attempting to format a value with a well-known format (i.e. RFC 3339, RFC 2822, or ISO 8601) will error at compile time if the type being formatted does not provide sufficient information. This would previously fail at runtime. Similarly, attempting to format a value with ISO 8601 that is only configured for parsing (i.e. <code>Iso8601::PARSING</code>) will error at compile time.</li> </ul> <h3>Added</h3> <ul> <li>Builder methods for format description modifiers, eliminating the need for verbose initialization when done manually.</li> <li><code>date!(2026-W01-2)</code> is now supported. Previously, a space was required between <code>W</code> and <code>01</code>.</li> <li><code>[end]</code> now has a <code>trailing_input</code> modifier which can either be <code>prohibit</code> (the default) or <code>discard</code>. When it is <code>discard</code>, all remaining input is ignored. Note that if there are components after <code>[end]</code>, they will still attempt to be parsed, likely resulting in an error.</li> </ul> <h3>Changed</h3> <ul> <li>More performance gains when parsing.</li> </ul> <h3>Fixed</h3> <ul> <li>If manually formatting a value, the number of bytes written was one short for some components. This has been fixed such that the number of bytes written is always correct.</li> <li>The possibility of integer overflow when parsing an owned format description has been effectively eliminated. This would previously wrap when overflow checks were disabled. Instead of storing the depth as <code>u8</code>, it is stored as <code>u32</code>. This would require multiple gigabytes of nested input to overflow, at which point we've got other problems and trivial mitigations are available by downstream users.</li> </ul> <h2>0.3.46 [2026-01-23]</h2> <h3>Added</h3> <ul> <li>All possible panics are now documented for the relevant methods.</li> <li>The need to use <code>#[serde(default)]</code> when using custom <code>serde</code> formats is documented. This applies only when deserializing an <code>Option<T></code>.</li> <li><code>Duration::nanoseconds_i128</code> has been made public, mirroring <code>std::time::Duration::from_nanos_u128</code>.</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/time-rs/time/commit/d5144cd2874862d46466c900910cd8577d066019"><code>d5144cd</code></a> v0.3.47 release</li> <li><a href="https://github.com/time-rs/time/commit/f6206b050fd54817d8872834b4d61f605570e89b"><code>f6206b0</code></a> Guard against integer overflow in release mode</li> <li><a href="https://github.com/time-rs/time/commit/1c63dc7985b8fa26bd8c689423cc56b7a03841ee"><code>1c63dc7</code></a> Avoid denial of service when parsing Rfc2822</li> <li><a href="https://github.com/time-rs/time/commit/5940df6e72efb63d246ca1ca59a0f836ad32ad8a"><code>5940df6</code></a> Add builder methods to avoid verbose construction</li> <li><a href="https://github.com/time-rs/time/commit/00881a4da1bc5a6cb6313052e5017dbd7daa40f0"><code>00881a4</code></a> Manually format macros everywhere</li> <li><a href="https://github.com/time-rs/time/commit/bb723b6d826e46c174d75cd08987061984b0ceb7"><code>bb723b6</code></a> Add <code>trailing_input</code> modifier to <code>end</code></li> <li><a href="https://github.com/time-rs/time/commit/31c4f8e0b56e6ae24fe0d6ef0e492b6741dda783"><code>31c4f8e</code></a> Permit <code>W12</code> in <code>date!</code> macro</li> <li><a href="https://github.com/time-rs/time/commit/490a17bf306576850f33a86d3ca95d96db7b1dcd"><code>490a17b</code></a> Mark error paths in well-known formats as cold</li> <li><a href="https://github.com/time-rs/time/commit/6cb1896a600be1538ecfab8f233fe9cfe9fa8951"><code>6cb1896</code></a> Optimize <code>Rfc2822</code> parsing</li> <li><a href="https://github.com/time-rs/time/commit/6d264d59c25e3da0453c3defebf4640b0086a006"><code>6d264d5</code></a> Remove erroneous <code>#[inline(never)]</code> attributes</li> <li>Additional commits viewable in <a href="https://github.com/time-rs/time/compare/v0.3.44...v0.3.47">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-rust/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 9e9a2b1)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - N/A. ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> - Add DataFusion Comet to the list of users with a description. ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> - N/A. (cherry picked from commit 89a9c90)
…ache#2026) Add Core Encryption Primitives for Iceberg Encryption Support. Part of apache#2034 ## Summary This PR introduces the foundational cryptographic primitives needed for implementing encryption in iceberg-rust, providing AES-GCM encryption operations that match the Java implementation's behavior and data format. ## Motivation Iceberg's Java implementation supports table-level encryption to protect sensitive data at rest. To achieve feature parity and ensure interoperability between Java and Rust implementations, we need to build encryption support from the ground up. This PR provides the core cryptographic operations that will serve as the foundation for the complete encryption feature. ## Changes New Module: encryption Added a new encryption module with core AES-GCM cryptographic operations: - encryption/crypto.rs - Core encryption implementation - EncryptionAlgorithm enum supporting AES-128-GCM as this is the only algorithm currently supported in arrow parquet - SecureKey struct with automatic memory zeroization for security - AesGcmEncryptor providing encrypt/decrypt operations with AAD support Key Features 1. Java-Compatible Format: Ciphertext format matches Java's implementation exactly: [12-byte nonce][encrypted data][16-byte GCM authentication tag] 1. This ensures files encrypted by Java can be decrypted by Rust and vice versa. 2. Secure Key Handling: Uses the zeroize crate to automatically clear encryption keys from memory when dropped, preventing key material from lingering in memory. 3. Additional Authenticated Data (AAD): Full support for AAD to ensure integrity of associated metadata that isn't encrypted. 4. Comprehensive Testing: 8 tests covering: - Round-trip encryption/decryption for both AES-128 and AES-256 - AAD validation - Empty plaintext handling - Tamper detection - Format compatibility verification Dependencies Added - aes-gcm = "0.10" - Industry-standard AES-GCM implementation - zeroize = "1.7" - Secure memory cleanup for encryption keys Compatibility This implementation directly corresponds to Java's https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/encryption/Ciphers.java: | Java Class | Rust Implementation | |-----------------------------|------------------------------------------| | Ciphers.AesGcmEncryptor | AesGcmEncryptor::encrypt() | | Ciphers.AesGcmDecryptor | AesGcmEncryptor::decrypt() | | EncryptionAlgorithm.AES_GCM | EncryptionAlgorithm::Aes128Gcm| Testing Future Work This PR is the first in a series to implement full encryption support. Upcoming PRs will add: 1. Table properties for encryption configuration 2. Key management interfaces (KeyManagementClient trait) 3. EncryptionManager implementation 4. Native Parquet encryption integration 5. AWS KMS support 6. Integration with Table and FileIO Review Notes - This PR is intentionally minimal and self-contained - No existing code paths are modified - this is purely additive - The module is public but won't be used until future PRs wire it up - Format compatibility with Java has been prioritized to ensure interoperability ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. apache#2035 ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? Yes <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> (cherry picked from commit fea5906)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes apache#2133 ## What changes are included in this PR? - Add catalog/utils.rs to provide helpers to delete table data using file_io and table_metadata - Add new API `purge_table` to `Catalog` trait and add default implementation - Implement purge_table for S3TableCatalog and RestCatalog <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? Added new tests in table_suite <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> (cherry picked from commit 56fda82)
## Which issue does this PR close? - Closes #. ## What changes are included in this PR? - Bump DataFusion to 53.0.0, Arrow/Parquet to 58, sqllogictest to 0.29, pyo3 to 0.28. - Adapt to DataFusion 53 API changes in physical plan executors and python bindings. - Update SLT expected test output. ## Are these changes tested? Existing tests. --------- Co-authored-by: Xander <zander181@googlemail.com> (cherry picked from commit 477a1e5)
notfilippo
approved these changes
Apr 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry picks all the commits from apache#2225 to apache#2206