[branch-0.9] Cherry pick needed commits for: Split arrow reader into smaller modules (apache#2358) #14
Merged
toutane merged 34 commits intobranch-0.9from Apr 30, 2026
Merged
Conversation
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? Relates to apache/iceberg#15742 This PR - Add "ASF allowlist check" - Pin commit for codeql.yml (zizmor recommended) - Add back Github Action auto-update for dependabot (reverts apache#2267) - Add cooldown to dependabot (zizmor recommended) - `Swatinem/rust-cache@v2` -> `swatinem/rust-cache@v2` (fix case sensitivity) [asf infra allowlist uses lowercase](https://github.com/apache/infrastructure-actions/blob/fae466bc0d9821859a623cbc7648c750ff359ec6/approved_patterns.yml#L271) We can add back dependabot for github action because the "ASF allowlist check" will now alert when an action is not allowed (failures will no longer be silent) <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> (cherry picked from commit aff502d)
## What changes are included in this PR? - Make `convert_filters_to_predicate` public in the DataFusion integration to allow external usage of the filter conversion logic. ## Are these changes tested? - This is a visibility change (`pub use`) and does not introduce new logic. Co-authored-by: Denis Semenov <d.s.semenov@vkteam.ru> (cherry picked from commit 3212c31)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> Relates to apache/iceberg#15742 Follow up to apache#2289 ## What changes are included in this PR? Fix github workflow based on zizmor recommendation for security best practice <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> Yes ``` ➜ iceberg-rust git:(kevinjqliu/zizmor-fix) uvx --from zizmor zizmor --offline .github/ 🌈 zizmor v1.23.1 INFO audit: zizmor: 🌈 completed .github/actions/get-msrv/action.yml INFO audit: zizmor: 🌈 completed .github/actions/overwrite-package-version/action.yml INFO audit: zizmor: 🌈 completed .github/actions/setup-builder/action.yml INFO audit: zizmor: 🌈 completed .github/dependabot.yml INFO audit: zizmor: 🌈 completed .github/workflows/audit.yml INFO audit: zizmor: 🌈 completed .github/workflows/bindings_python_ci.yml INFO audit: zizmor: 🌈 completed .github/workflows/ci.yml INFO audit: zizmor: 🌈 completed .github/workflows/ci_typos.yml INFO audit: zizmor: 🌈 completed .github/workflows/codeql.yml INFO audit: zizmor: 🌈 completed .github/workflows/publish.yml INFO audit: zizmor: 🌈 completed .github/workflows/release_python.yml INFO audit: zizmor: 🌈 completed .github/workflows/release_python_nightly.yml INFO audit: zizmor: 🌈 completed .github/workflows/stale.yml INFO audit: zizmor: 🌈 completed .github/workflows/website.yml No findings to report. Good job! (1 ignored, 37 suppressed) ``` (cherry picked from commit 6ee5e71)
…tion test (apache#2294) ## Which issue does this PR close? - Closes apache#352. ## What changes are included in this PR? - Replaces hardcoded `-1` with `EMPTY_SNAPSHOT_ID` constant in table metadata deserialization. - Adds `test_empty_snapshot_id_is_normalized_to_none` to verify that the Java-style `-1` sentinel for `current-snapshot-id` is normalized to `None` during deserialization. - Removes the public `UNASSIGNED_SNAPSHOT_ID` constant and moving it to a private constant scoped to the manifest writer module. ## Are these changes tested? Adds a test `test_empty_snapshot_id_is_normalized_to_none` verifying the deserialization normalization. (cherry picked from commit 14f2e14)
## Which issue does this PR close? This is an intermediate PR for apache#1731 I'm splitting out changes from apache#1851 to the compression codec to make it easier to review. Once we decide on approach here and merge it I'll update apache#1851 accordingly. ## What changes are included in this PR? - Add optional compression level to gzip and zstd (needed for when avro compression usage). - Add Snappy as a compression codec (also will be used for Avro) - Manually code up some previously auto-generated methods as a result. AI helped with an initial version of this PR. ## Are these changes tested? Additional unit tests (cherry picked from commit d8011a0)
(cherry picked from commit ff6c700)
(cherry picked from commit dc3a2d5)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes apache#2241 ## What changes are included in this PR? - Add `Ancestors` to help scan past snapshots - Moved existing util to the new utils mod <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? Yes <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> (cherry picked from commit 626de2e)
## Which issue does this PR close? - Closes apache#2323 . ## What changes are included in this PR? ## Are these changes tested? ci (cherry picked from commit 59c8f4b)
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.44.0 to 1.45.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/crate-ci/typos/releases">crate-ci/typos's releases</a>.</em></p> <blockquote> <h2>v1.45.0</h2> <h2>[1.45.0] - 2026-04-01</h2> <h3>Features</h3> <ul> <li>Updated the dictionary with the <a href="https://redirect.github.com/crate-ci/typos/issues/1509">March 2026</a> changes</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/crate-ci/typos/blob/master/CHANGELOG.md">crate-ci/typos's changelog</a>.</em></p> <blockquote> <h1>Change Log</h1> <p>All notable changes to this project will be documented in this file.</p> <p>The format is based on <a href="https://keepachangelog.com/">Keep a Changelog</a> and this project adheres to <a href="https://semver.org/">Semantic Versioning</a>.</p> <!-- raw HTML omitted --> <h2>[Unreleased] - ReleaseDate</h2> <h2>[1.45.0] - 2026-04-01</h2> <h3>Features</h3> <ul> <li>Updated the dictionary with the <a href="https://redirect.github.com/crate-ci/typos/issues/1509">March 2026</a> changes</li> </ul> <h2>[1.44.0] - 2026-02-27</h2> <h3>Features</h3> <ul> <li>Updated the dictionary with the <a href="https://redirect.github.com/crate-ci/typos/issues/1488">February 2026</a> changes</li> </ul> <h2>[1.43.5] - 2026-02-16</h2> <h3>Fixes</h3> <ul> <li><em>(pypi)</em> Hopefully fix the sdist build</li> </ul> <h2>[1.43.4] - 2026-02-09</h2> <h3>Fixes</h3> <ul> <li>Don't correct <code>pincher</code></li> </ul> <h2>[1.43.3] - 2026-02-06</h2> <h3>Fixes</h3> <ul> <li><em>(action)</em> Adjust how typos are reported to github</li> </ul> <h2>[1.43.2] - 2026-02-05</h2> <h3>Fixes</h3> <ul> <li>Don't correct <code>certifi</code> in Python</li> </ul> <h2>[1.43.1] - 2026-02-03</h2> <h3>Fixes</h3> <ul> <li>Don't correct <code>consts</code></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/crate-ci/typos/commit/02ea592e44b3a53c302f697cddca7641cd051c3d"><code>02ea592</code></a> chore: Release</li> <li><a href="https://github.com/crate-ci/typos/commit/b859c0df7f391deba73030f79b957e62b4d81dc6"><code>b859c0d</code></a> chore: Release</li> <li><a href="https://github.com/crate-ci/typos/commit/6fd32cee0c74337fd419788965d8f1c96d6f87a5"><code>6fd32ce</code></a> docs: Update changelog</li> <li><a href="https://github.com/crate-ci/typos/commit/7626d89232db5fc0c23e290107cf420047e297e9"><code>7626d89</code></a> Merge pull request <a href="https://redirect.github.com/crate-ci/typos/issues/1530">#1530</a> from crate-ci/renovate/j178-prek-action-2.x</li> <li><a href="https://github.com/crate-ci/typos/commit/2c9510cd3b9756411e186a1ff1d0ee9a0bb61895"><code>2c9510c</code></a> Merge pull request <a href="https://redirect.github.com/crate-ci/typos/issues/1532">#1532</a> from epage/march</li> <li><a href="https://github.com/crate-ci/typos/commit/265b88f6490711bebac4bb109149f1f06c2255e9"><code>265b88f</code></a> feat(dict): March updates</li> <li><a href="https://github.com/crate-ci/typos/commit/5baf2ce236258ea037d71f364aa7a13eb1d2cf70"><code>5baf2ce</code></a> chore(deps): Update compatible (<a href="https://redirect.github.com/crate-ci/typos/issues/1529">#1529</a>)</li> <li><a href="https://github.com/crate-ci/typos/commit/0442cb7c40aa99aa5a9ec99ac4c3f0c654260f34"><code>0442cb7</code></a> chore(deps): Update j178/prek-action action to v2</li> <li><a href="https://github.com/crate-ci/typos/commit/8f11c0dc0f31c780c45b3dd5b72ff4b48a350b75"><code>8f11c0d</code></a> Merge pull request <a href="https://redirect.github.com/crate-ci/typos/issues/1524">#1524</a> from epage/update</li> <li><a href="https://github.com/crate-ci/typos/commit/ecdbfab467d6037f796c649e7616aee4a6a16c5a"><code>ecdbfab</code></a> chore: Update dependencies</li> <li>Additional commits viewable in <a href="https://github.com/crate-ci/typos/compare/631208b7aac2daa8b707f55e7331f9112b0e062d...02ea592e44b3a53c302f697cddca7641cd051c3d">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: blackmwk <liurenjie1024@outlook.com> (cherry picked from commit c6c0ce7)
…e#2319) Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.70.0 to 2.73.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/taiki-e/install-action/releases">taiki-e/install-action's releases</a>.</em></p> <blockquote> <h2>2.73.0</h2> <ul> <li> <p>Introduce <a href="https://blog.yossarian.net/2025/11/21/We-should-all-be-using-dependency-cooldowns">dependency cooldown</a> when installing with <code>taiki-e/install-action@<tool_name></code>, <code>tool: <tool_name>@latest</code>, or <code>tool: <tool_name>@<omitted_version></code> to mitigate the risk of supply chain attacks by default. (<a href="https://redirect.github.com/taiki-e/install-action/pull/1666">#1666</a>)</p> <p>This action without this cooldown already takes a few hours to a few days for new releases to be reflected (as with other common package managers that verify checksums or signatures), so this should not affect most users.</p> <p>See the <a href="https://github.com/taiki-e/install-action#security">"Security" section in readme</a> for more details.</p> </li> <li> <p>Improve robustness for network failure.</p> </li> <li> <p>Documentation improvements.</p> </li> </ul> <h2>2.72.0</h2> <ul> <li> <p>Support <code>cargo-xwin</code>. (<a href="https://redirect.github.com/taiki-e/install-action/pull/1659">#1659</a>, thanks <a href="https://github.com/daxpedda"><code>@daxpedda</code></a>)</p> </li> <li> <p>Support trailing comma in <code>tool</code> input option.</p> </li> <li> <p>Update <code>tombi@latest</code> to 0.9.14.</p> </li> </ul> <h2>2.71.3</h2> <ul> <li> <p>Update <code>wasm-tools@latest</code> to 1.246.2.</p> </li> <li> <p>Update <code>mise@latest</code> to 2026.4.3.</p> </li> </ul> <h2>2.71.2</h2> <ul> <li> <p>Implement workaround for <a href="https://redirect.github.com/actions/partner-runner-images/issues/169">windows-11-arm runner bug</a> which sometimes causes installation failure. (<a href="https://redirect.github.com/taiki-e/install-action/pull/1657">#1657</a>)</p> <p>This addresses an issue that was attempted to be worked around in 2.71.0 but was insufficient.</p> </li> <li> <p>Update <code>mise@latest</code> to 2026.4.1.</p> </li> <li> <p>Update <code>uv@latest</code> to 0.11.3.</p> </li> </ul> <h2>2.71.1</h2> <ul> <li> <p>Fix a regression that caused an execution policy violation on self-hosted Windows runner due to use of non-default <code>powershell</code> shell, introduced in 2.71.0.</p> </li> <li> <p>Update <code>dprint@latest</code> to 0.53.2.</p> </li> </ul> <h2>2.71.0</h2> <ul> <li> <p>Support <code>wasm-tools</code>. (<a href="https://redirect.github.com/taiki-e/install-action/pull/1642">#1642</a>, thanks <a href="https://github.com/crepererum"><code>@crepererum</code></a>)</p> </li> <li> <p>Support <code>covgate</code>. (<a href="https://redirect.github.com/taiki-e/install-action/pull/1613">#1613</a>, thanks <a href="https://github.com/jesse-black"><code>@jesse-black</code></a>)</p> </li> <li> <p>Implement potential workaround for <a href="https://redirect.github.com/actions/partner-runner-images/issues/169">windows-11-arm runner bug</a> which sometimes causes issue that the action successfully completes but the tool is not installed. (<a href="https://redirect.github.com/taiki-e/install-action/pull/1647">#1647</a>)</p> </li> <li> <p>Update <code>typos@latest</code> to 1.45.0.</p> </li> <li> <p>Update <code>mise@latest</code> to 2026.4.0.</p> </li> <li> <p>Update <code>cargo-careful@latest</code> to 0.4.10.</p> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md">taiki-e/install-action's changelog</a>.</em></p> <blockquote> <h1>Changelog</h1> <p>All notable changes to this project will be documented in this file.</p> <p>This project adheres to <a href="https://semver.org">Semantic Versioning</a>.</p> <!-- raw HTML omitted --> <h2>[Unreleased]</h2> <ul> <li> <p>Update <code>vacuum@latest</code> to 0.25.6.</p> </li> <li> <p>Update <code>gungraun-runner@latest</code> to 0.18.1.</p> </li> </ul> <h2>[2.75.7] - 2026-04-11</h2> <ul> <li> <p>Update <code>covgate@latest</code> to 0.1.4.</p> </li> <li> <p>Update <code>wasm-bindgen@latest</code> to 0.2.118.</p> </li> </ul> <h2>[2.75.6] - 2026-04-11</h2> <ul> <li> <p>Update <code>mise@latest</code> to 2026.4.8.</p> </li> <li> <p>Update <code>cargo-deny@latest</code> to 0.19.1.</p> </li> </ul> <h2>[2.75.5] - 2026-04-10</h2> <ul> <li> <p>Update <code>biome@latest</code> to 2.4.11.</p> </li> <li> <p>Update <code>wasmtime@latest</code> to 43.0.1.</p> </li> <li> <p>Update <code>uv@latest</code> to 0.11.6.</p> </li> <li> <p>Update <code>mise@latest</code> to 2026.4.7.</p> </li> <li> <p>Update <code>gungraun-runner@latest</code> to 0.18.0.</p> </li> </ul> <h2>[2.75.4] - 2026-04-10</h2> <ul> <li> <p>Enhance security when cargo-binstall fallback is enabled. (<a href="https://github.com/taiki-e/install-action/commit/08a38582e35739e1fda5e4cd298e348536a1ddaa">08a38582</a>, <a href="https://github.com/taiki-e/install-action/commit/ba626b4d71edcae17f6ca956b7b81e925b2d24ef">ba626b4d</a>)</p> </li> <li> <p>Update <code>martin@latest</code> to 1.5.0.</p> </li> <li> <p>Update <code>uv@latest</code> to 0.11.5.</p> </li> <li> <p>Update <code>syft@latest</code> to 1.42.4.</p> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/taiki-e/install-action/commit/0abfcd587b70a713fdaa7fb502c885e2112acb15"><code>0abfcd5</code></a> Release 2.75.7</li> <li><a href="https://github.com/taiki-e/install-action/commit/cf81fe383b77d63972a04bb1e84d163e13d3a584"><code>cf81fe3</code></a> Update <code>covgate@latest</code> to 0.1.4</li> <li><a href="https://github.com/taiki-e/install-action/commit/e8aec859b7eb3ba8572ade244032d91139e4b75f"><code>e8aec85</code></a> Update <code>wasm-bindgen@latest</code> to 0.2.118</li> <li><a href="https://github.com/taiki-e/install-action/commit/d053c7f943c3a577e98f342e9e28edf09931644c"><code>d053c7f</code></a> Release 2.75.6</li> <li><a href="https://github.com/taiki-e/install-action/commit/9019f2ebb5fcc4be56ccca93d779a8a14f83bb42"><code>9019f2e</code></a> Update <code>mise@latest</code> to 2026.4.8</li> <li><a href="https://github.com/taiki-e/install-action/commit/1b07fb74591cf228b32613bd9ca519dc027ae914"><code>1b07fb7</code></a> Update <code>cargo-deny@latest</code> to 0.19.1</li> <li><a href="https://github.com/taiki-e/install-action/commit/be1e8a863739de1370a81e1c5ad4babbab50c914"><code>be1e8a8</code></a> Update vacuum manifest</li> <li><a href="https://github.com/taiki-e/install-action/commit/1cc7501bf80d38541e25e6a90f1f1c10054297c9"><code>1cc7501</code></a> Update gungraun-runner manifest</li> <li><a href="https://github.com/taiki-e/install-action/commit/44afe623bee1f92436bd2e2d30d56626792cd0f0"><code>44afe62</code></a> Update covgate manifest</li> <li><a href="https://github.com/taiki-e/install-action/commit/7a4939c09608b2a1986b484eca1d16fd0db8ebef"><code>7a4939c</code></a> Release 2.75.5</li> <li>Additional commits viewable in <a href="https://github.com/taiki-e/install-action/compare/0fde6d128a3d980ceac30be8c8b8739abd963b81...0abfcd587b70a713fdaa7fb502c885e2112acb15">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: blackmwk <liurenjie1024@outlook.com> (cherry picked from commit 37d19b8)
Bumps [aws-sdk-s3tables](https://github.com/awslabs/aws-sdk-rust) from 1.53.0 to 1.54.0. <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/awslabs/aws-sdk-rust/commits">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: blackmwk <liurenjie1024@outlook.com> (cherry picked from commit d472bf8)
## Which issue does this PR close? - Closes #. ## What changes are included in this PR? Our current ci failed due to an audit issue, see: apache#2321 . This pr fix the ci failure. ## Are these changes tested? CI. (cherry picked from commit 99ffd68)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.50.0 to 1.51.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/tokio-rs/tokio/releases">tokio's releases</a>.</em></p> <blockquote> <h2>Tokio v1.51.0</h2> <h1>1.51.0 (April 3rd, 2026)</h1> <h3>Added</h3> <ul> <li>net: implement <code>get_peer_cred</code> on Hurd (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7989">#7989</a>)</li> <li>runtime: add <code>tokio::runtime::worker_index()</code> (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7921">#7921</a>)</li> <li>runtime: add runtime name (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7924">#7924</a>)</li> <li>runtime: stabilize <code>LocalRuntime</code> (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7557">#7557</a>)</li> <li>wasm: add wasm32-wasip2 networking support (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7933">#7933</a>)</li> </ul> <h3>Changed</h3> <ul> <li>runtime: steal tasks from the LIFO slot (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7431">#7431</a>)</li> </ul> <h3>Fixed</h3> <ul> <li>docs: do not show "Available on non-loom only." doc label (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7977">#7977</a>)</li> <li>macros: improve overall macro hygiene (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7997">#7997</a>)</li> <li>sync: fix <code>notify_waiters</code> priority in <code>Notify</code> (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7996">#7996</a>)</li> <li>sync: fix panic in <code>Chan::recv_many</code> when called with non-empty vector on closed channel (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7991">#7991</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/tokio/issues/7431">#7431</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7431">tokio-rs/tokio#7431</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7557">#7557</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7557">tokio-rs/tokio#7557</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7921">#7921</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7921">tokio-rs/tokio#7921</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7924">#7924</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7924">tokio-rs/tokio#7924</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7933">#7933</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7933">tokio-rs/tokio#7933</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7977">#7977</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7977">tokio-rs/tokio#7977</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7989">#7989</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7989">tokio-rs/tokio#7989</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7991">#7991</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7991">tokio-rs/tokio#7991</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7996">#7996</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7996">tokio-rs/tokio#7996</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7997">#7997</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7997">tokio-rs/tokio#7997</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/tokio-rs/tokio/commit/0af06b7bab12c58161b1d0ae79bdf4452305d42f"><code>0af06b7</code></a> chore: prepare Tokio v1.51.0 (<a href="https://redirect.github.com/tokio-rs/tokio/issues/8005">#8005</a>)</li> <li><a href="https://github.com/tokio-rs/tokio/commit/01a7f1dfabc93293743701074752ff0d8e787595"><code>01a7f1d</code></a> chore: prepare tokio-macros v2.7.0 (<a href="https://redirect.github.com/tokio-rs/tokio/issues/8004">#8004</a>)</li> <li><a href="https://github.com/tokio-rs/tokio/commit/eeb55c733ba9a83c51d08b1629dca6a5ec0f4b2b"><code>eeb55c7</code></a> runtime: steal tasks from the LIFO slot (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7431">#7431</a>)</li> <li><a href="https://github.com/tokio-rs/tokio/commit/1fc450aefba4b05cdff9b7825ca5e39cccb3780e"><code>1fc450a</code></a> runtime: stabilize <code>LocalRuntime</code> (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7557">#7557</a>)</li> <li><a href="https://github.com/tokio-rs/tokio/commit/324218f9bbdc26e4bb527d036613826824f3078b"><code>324218f</code></a> Merge tag 'tokio-1.47.4' (<a href="https://redirect.github.com/tokio-rs/tokio/issues/8003">#8003</a>)</li> <li><a href="https://github.com/tokio-rs/tokio/commit/aa65d0d0b8ea6eec80985b9d231390f137493071"><code>aa65d0d</code></a> chore: prepare Tokio v1.47.4 (<a href="https://redirect.github.com/tokio-rs/tokio/issues/8002">#8002</a>)</li> <li><a href="https://github.com/tokio-rs/tokio/commit/bf18ed452d6aae438e84ae008a01a74776abdc19"><code>bf18ed4</code></a> sync: fix panic in <code>Chan::recv_many</code> when called with non-empty vector on clo...</li> <li><a href="https://github.com/tokio-rs/tokio/commit/43134f1e5784993eb4fb3863933d74ac9e28f598"><code>43134f1</code></a> wasm: add wasm32-wasip2 networking support (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7933">#7933</a>)</li> <li><a href="https://github.com/tokio-rs/tokio/commit/b4c3246d330379430937bdbb5e1b0c37282ae23e"><code>b4c3246</code></a> macros: improve overall macro hygiene (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7997">#7997</a>)</li> <li><a href="https://github.com/tokio-rs/tokio/commit/7947fa4bd79d7345aa7e6b189fc1fbb6983a4351"><code>7947fa4</code></a> rt: add runtime name (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7924">#7924</a>)</li> <li>Additional commits viewable in <a href="https://github.com/tokio-rs/tokio/compare/tokio-1.50.0...tokio-1.51.0">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 575ebcd)
Bumps [minijinja](https://github.com/mitsuhiko/minijinja) from 2.18.0 to 2.19.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/mitsuhiko/minijinja/blob/main/CHANGELOG.md">minijinja's changelog</a>.</em></p> <blockquote> <h2>2.19.0</h2> <ul> <li>Fixed strict undefined behavior for comparison operators (such as <code>==</code>), string concatenation (<code>~</code>), and undefined needles in the <code>in</code> operator to better match Jinja2. <a href="https://redirect.github.com/mitsuhiko/minijinja/issues/886">#886</a> <a href="https://redirect.github.com/mitsuhiko/minijinja/issues/888">#888</a></li> <li>Fixed the <code>default</code> filter in strict undefined mode so an explicitly passed undefined fallback argument errors instead of being treated like a missing argument. <a href="https://redirect.github.com/mitsuhiko/minijinja/issues/887">#887</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/mitsuhiko/minijinja/commit/f15dc1e703ec58bc9cb50353322afb7890e3e016"><code>f15dc1e</code></a> chore(release): 2.19.0</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/e04d2764b27824aaa1befe8b3d666c810661418a"><code>e04d276</code></a> fix(undefined): align strict undefined behavior with Jinja2</li> <li>See full diff in <a href="https://github.com/mitsuhiko/minijinja/compare/minijinja-go/v2.18.0...minijinja-go/v2.19.0">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 8e884ef)
## Which issue does this PR close? - Closes apache#2312 ## What changes are included in this PR? Changes the s3tables default scheme to "s3://" ## Are these changes tested? With this change, I can read files resolved through an s3table catalog now. (cherry picked from commit b06b573)
## Which issue does this PR close? N/A ## What changes are included in this PR? The Java Iceberg SDK defaults [`s3.path-style-access` to `false`](https://github.com/apache/iceberg/blob/main/aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java#L238-L240) i.e. virtual-host-style addressing is the spec default. opendal's `S3Config::default()`, which the opendal storage backend inherits from, uses path-style. As a result, any user building an iceberg-rust `FileIO` against AWS S3 or an S3-compatible service has to explicitly set `s3.path-style-access=false` to get the same behavior they would get from Java out of the box — and several S3-compatible endpoints only accept virtual-host-style URLs and fail outright with `SecondLevelDomainForbidden` (or equivalent) under the current default. ## Are these changes tested? Yes: - `cargo test -p iceberg --lib io::storage::config::s3::tests` - `cargo test -p iceberg-storage-opendal --lib s3::tests (cherry picked from commit d6692fc)
## Which issue does this PR close? - Closes apache#2327 apache#2328 apache#2329 ## What changes are included in this PR? Upgrade rnd version, use recommended api. ## Are these changes tested? ut. (cherry picked from commit 3fe7be1)
…pache#2333) ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? - update codeql version from 4.35.1 to 4.35.2 - update comment tag <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> (cherry picked from commit 5334dcb)
(cherry picked from commit e285c79)
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 7.3.1 to 8.0.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/astral-sh/setup-uv/releases">astral-sh/setup-uv's releases</a>.</em></p> <blockquote> <h2>v8.0.0 🌈 Immutable releases and secure tags</h2> <h1>This is the first immutable release of <code>setup-uv</code> 🥳</h1> <p>All future releases are also immutable, if you want to know more about what this means checkout <a href="https://docs.github.com/en/code-security/concepts/supply-chain-security/immutable-releases">the docs</a>.</p> <p>This release also has two breaking changes</p> <h2>New format for <code>manifest-file</code></h2> <p>The previously deprecated way of defining a custom version manifest to control which <code>uv</code> versions are available and where to download them from got removed. The functionality is still there but you have to use the <a href="https://github.com/astral-sh/setup-uv/blob/main/docs/customization.md#format">new format</a>.</p> <h2>No more major and minor tags</h2> <p>To increase <strong>security</strong> even more we will <strong>stop publishing minor tags</strong>. You won't be able to use <code>@v8</code> or <code>@v8.0</code> any longer. We do this because pinning to major releases opens up users to supply chain attacks like what happened to <a href="https://unit42.paloaltonetworks.com/github-actions-supply-chain-attack/">tj-actions</a>.</p> <blockquote> <p>[!TIP] Use the immutable tag as a version <code>astral-sh/setup-uv@v8.0.0</code> Or even better the githash <code>astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57</code></p> </blockquote> <h2>🚨 Breaking changes</h2> <ul> <li>Remove update-major-minor-tags workflow <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/826">#826</a>)</li> <li>Remove deprecrated custom manifest <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/813">#813</a>)</li> </ul> <h2>🧰 Maintenance</h2> <ul> <li>Shortcircuit latest version from manifest <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/828">#828</a>)</li> <li>Simplify inputs.ts <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/827">#827</a>)</li> <li>Bump release-drafter to v7.1.1 <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/825">#825</a>)</li> <li>Refactor inputs <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/823">#823</a>)</li> <li>Replace inline compile args with tsconfig <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/824">#824</a>)</li> <li>chore: update known checksums for 0.11.2 @<a href="https://github.com/apps/github-actions">github-actions[bot]</a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/821">#821</a>)</li> <li>chore: update known checksums for 0.11.1 @<a href="https://github.com/apps/github-actions">github-actions[bot]</a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/817">#817</a>)</li> <li>chore: update known checksums for 0.11.0 @<a href="https://github.com/apps/github-actions">github-actions[bot]</a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/815">#815</a>)</li> <li>Fix latest-version workflow check <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/812">#812</a>)</li> <li>chore: update known checksums for 0.10.11/0.10.12 @<a href="https://github.com/apps/github-actions">github-actions[bot]</a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/811">#811</a>)</li> </ul> <h2>v7.6.0 🌈 Fetch uv from Astral's mirror by default</h2> <h2>Changes</h2> <p>We now default to download uv from <code>releases.astral.sh</code>. This means by default we don't hit the GitHub API at all and shouldn't see any rate limits and timeouts any more.</p> <h2>🚀 Enhancements</h2> <ul> <li>Fetch uv from Astral's mirror by default <a href="https://github.com/zsol"><code>@zsol</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/809">#809</a>)</li> </ul> <h2>🧰 Maintenance</h2> <ul> <li>Switch to ESM for source and test, use CommonJS for dist <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/806">#806</a>)</li> <li>chore: update known checksums for 0.10.10 @<a href="https://github.com/apps/github-actions">github-actions[bot]</a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/804">#804</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/astral-sh/setup-uv/commit/cec208311dfd045dd5311c1add060b2062131d57"><code>cec2083</code></a> Shortcircuit latest version from manifest (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/828">#828</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/4dd8ab45206a76f8c1dfe399fa88df10a7264f27"><code>4dd8ab4</code></a> Simplify inputs.ts (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/827">#827</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/7fdbe7cf0c8ef50cfd0878eed7b5180abc6b53c7"><code>7fdbe7c</code></a> Remove update-major-minor-tags workflow (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/826">#826</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/485abd05e5c74a247f0a309e333d2433ab9a353a"><code>485abd0</code></a> Bump release-drafter to v7.1.1 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/825">#825</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/f82eb19c06057c455674b2602e0139fd906f1428"><code>f82eb19</code></a> Refactor inputs (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/823">#823</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/868d1f74d9d862d7b40219546bfe35299c6dd452"><code>868d1f7</code></a> Replace inline compile args with tsconfig (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/824">#824</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/447e6d02b15d65b3247cce2d6019f11957285d11"><code>447e6d0</code></a> chore: update known checksums for 0.11.2 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/821">#821</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/5c62c5926145985eec91f09e2e0a75f40daed929"><code>5c62c59</code></a> chore: update known checksums for 0.11.1 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/817">#817</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/e1a7373adb857afd2a70b971e8ebdacc64ed27d0"><code>e1a7373</code></a> chore: update known checksums for 0.11.0 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/815">#815</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/89709315bb3bd4bf0f4b1db4b710e99009087ab5"><code>8970931</code></a> Remove deprecrated custom manifest (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/813">#813</a>)</li> <li>Additional commits viewable in <a href="https://github.com/astral-sh/setup-uv/compare/5a095e7a2014a4212f075830d4f7277575a9d098...cec208311dfd045dd5311c1add060b2062131d57">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shawn Chang <yxchang@amazon.com> (cherry picked from commit d118aa8)
…n/loading plus reading/writing (apache#2315) ## Which issue does this PR close? - Extends apache#2313 with a test. In particular, I've written a test that verifies creating a table and writing to it, and then loading that table from the catalog and reading from it using the default storage backend. ## What changes are included in this PR? This adds a test that currently fails and will be passing once apache#2313 is merged. ## Are these changes tested? This change is only a new test. Signed-off-by: Daniel Carl Jones <djonesoa@amazon.com> (cherry picked from commit 2890a61)
…r schemas with nested types (apache#2307) ## Which issue does this PR close? - Closes apache#2306. - Downstream issue: apache/datafusion-comet#3860 ## What changes are included in this PR? `build_fallback_field_id_map` iterated over Parquet leaf columns instead of top-level fields when building the field ID to column index mapping for migrated files (no embedded field IDs). When nested types (struct, list, map) precede a primitive column, they expand into multiple leaves, causing the mapping to diverge from `add_fallback_field_ids_to_arrow_schema` which correctly assigns ordinal IDs to top-level Arrow fields. This made predicates on columns after nested types resolve to a leaf inside the group, crashing with "Leaf column `id` in predicates isn't a root column in Parquet schema". The fix iterates `root_schema().get_fields()` directly, assigning ordinal IDs only to top-level fields. For non-primitive fields (struct/list/map), it uses `get_column_root_idx` to advance past their leaf columns. This mirrors iceberg-java's `ParquetSchemaUtil.addFallbackIds()`, which iterates `fileSchema.getFields()` assigning ordinal IDs to top-level fields. Also renames "Leave column" to "Leaf column" in error messages. ## Are these changes tested? - An integration test (`test_predicate_on_migrated_file_with_nested_types`) writes a Parquet file without field IDs containing struct, list, and map columns before an `id` column, then reads with a predicate on `id`. This reproduces the exact crash before the fix. Test data is constructed with `serde_arrow` for readability. - [Apache DataFusion Comet](https://github.com/apache/datafusion-comet) used the repro test in [apache/datafusion-comet#3860](apache/datafusion-comet#3860) and it passes with this change: apache/datafusion-comet#3872 --------- Co-authored-by: blackmwk <liurenjie1024@outlook.com> (cherry picked from commit 5ea6f4c)
…e#2301) ## Which issue does this PR close? - Closes apache#2299. ## What changes are included in this PR? - Add `coerce_int96_timestamps()` to patch the Arrow schema before reading, using arrow-rs's schema hint mechanism (`ArrowReaderOptions::with_schema`) to read INT96 columns at the resolution specified by the Iceberg table schema - `timestamp`/`timestamptz` → microsecond, `timestamp_ns`/`timestamptz_ns` → nanosecond, per the [Iceberg spec](https://iceberg.apache.org/spec/#primitive-types) - Falls back to microsecond when no field ID is available (matching Iceberg Java's `TimestampInt96Reader` behavior) - Applied after all three schema resolution branches (with field IDs, name mapping, positional fallback) so the fix covers both native and migrated tables - Handles INT96 inside nested types (structs, lists, maps) via `ArrowSchemaVisitor` traversal - Visitor and tests live in a standalone `arrow/int96.rs` module to keep `reader.rs` manageable - Made `visit_schema` in `arrow/schema.rs` `pub(crate)` so the coercion visitor can reuse the existing traversal ## Are these changes tested? - `test_read_int96_timestamps_with_field_ids` — files with embedded field IDs (branch 1) - `test_read_int96_timestamps_without_field_ids` — migrated files without field IDs (branches 2/3) - `test_read_int96_timestamps_in_struct` — INT96 inside a struct field - `test_read_int96_timestamps_in_list` — INT96 inside a list field (3-level Parquet LIST encoding) - `test_read_int96_timestamps_in_map` — INT96 as map values - All tests use dates outside the i64 nanosecond range (~1677-2262) to confirm the overflow is avoided - [Apache DataFusion Comet](https://github.com/apache/datafusion-comet) used the repro test in [apache/datafusion-comet#3856](apache/datafusion-comet#3856) and it passes with this change: apache/datafusion-comet#3857 (cherry picked from commit a2f067d)
…ecryption (apache#2286) (cherry picked from commit 4e8cf7f)
…e impls (apache#2338) ## Which issue does this PR close? - Closes apache#2245 - Related apache#2231 ## What changes are included in this PR? - Remove configured_scheme field from OpenDalStorage::{S3,Azdls} - Make S3 storage use the scheme in the file paths, allowing for custom S3-compatible schemes like minio:// - Use `HashMap<Scheme, Arc<OpenDalStorage>>` so aliases share a storage instance - Added new unit tests and removed some now-obsolete scheme-mismatch tests ## Break Change We are removing a struct field from public types, so this would need to be release in 0.10.0 ```rust // Before OpenDalStorageFactory::S3 { configured_scheme: "s3a".to_string(), customized_credential_load: None, } OpenDalStorageFactory::Azdls { configured_scheme: AzureStorageScheme::Abfss, } // After OpenDalStorageFactory::S3 { customized_credential_load: None } OpenDalStorageFactory::Azdls ``` ## Are these changes tested? Beyond the unit tests, I ran these integration tests. ```sh docker compose -f dev/docker-compose.yaml up -d --wait # requires unset on any AWS_ env vars cargo test -p iceberg-integration-tests cargo test -p iceberg-catalog-hms --test hms_catalog_test cargo test -p iceberg-catalog-loader cargo test -p iceberg-storage-opendal --features opendal-s3 --test file_io_s3_test ``` (cherry picked from commit fda82a2)
) Bumps [PyO3/maturin-action](https://github.com/pyo3/maturin-action) from 1.50.1 to 1.51.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pyo3/maturin-action/releases">PyO3/maturin-action's releases</a>.</em></p> <blockquote> <h2>v1.51.0</h2> <h2>What's Changed</h2> <ul> <li>Initial Android build support by <a href="https://github.com/messense"><code>@messense</code></a> in <a href="https://redirect.github.com/PyO3/maturin-action/pull/426">PyO3/maturin-action#426</a></li> <li>Support native riscv64 builds on manylinux 2_39 by <a href="https://github.com/weiji14"><code>@weiji14</code></a> in <a href="https://redirect.github.com/PyO3/maturin-action/pull/429">PyO3/maturin-action#429</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/weiji14"><code>@weiji14</code></a> made their first contribution in <a href="https://redirect.github.com/PyO3/maturin-action/pull/429">PyO3/maturin-action#429</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/PyO3/maturin-action/compare/v1.50.1...v1.51.0">https://github.com/PyO3/maturin-action/compare/v1.50.1...v1.51.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/PyO3/maturin-action/commit/e83996d129638aa358a18fbd1dfb82f0b0fb5d3b"><code>e83996d</code></a> Bump version to 1.51.0</li> <li><a href="https://github.com/PyO3/maturin-action/commit/36eb3d01a9199f87fbd6ddf7038b03a03daea226"><code>36eb3d0</code></a> Update versions-manifest.json (<a href="https://redirect.github.com/pyo3/maturin-action/issues/435">#435</a>)</li> <li><a href="https://github.com/PyO3/maturin-action/commit/bd90123931ca0ba261b88b3893dd9609ed72eda3"><code>bd90123</code></a> Update versions-manifest.json (<a href="https://redirect.github.com/pyo3/maturin-action/issues/434">#434</a>)</li> <li><a href="https://github.com/PyO3/maturin-action/commit/ef46725b6d4e993a8fb2fe367bf1f1e3a643e4d5"><code>ef46725</code></a> Bump brace-expansion (<a href="https://redirect.github.com/pyo3/maturin-action/issues/433">#433</a>)</li> <li><a href="https://github.com/PyO3/maturin-action/commit/72c5fe112fe88001c8c5e1a7c89766b7d55f7405"><code>72c5fe1</code></a> Bump actions/setup-node from 6.2.0 to 6.3.0 (<a href="https://redirect.github.com/pyo3/maturin-action/issues/432">#432</a>)</li> <li><a href="https://github.com/PyO3/maturin-action/commit/94507410305a8e2ef07fd58e260bddc52cac4eac"><code>9450741</code></a> Bump docker/setup-qemu-action from 3.7.0 to 4.0.0 (<a href="https://redirect.github.com/pyo3/maturin-action/issues/430">#430</a>)</li> <li><a href="https://github.com/PyO3/maturin-action/commit/d8dc7d43e879b97c55e21052a6dc5377dd7438f7"><code>d8dc7d4</code></a> Bump zizmorcore/zizmor-action from 0.5.0 to 0.5.2 (<a href="https://redirect.github.com/pyo3/maturin-action/issues/431">#431</a>)</li> <li><a href="https://github.com/PyO3/maturin-action/commit/0bca1d2400d75ecf79351a0e7bad20426228ea10"><code>0bca1d2</code></a> Bump picomatch from 4.0.3 to 4.0.4 (<a href="https://redirect.github.com/pyo3/maturin-action/issues/428">#428</a>)</li> <li><a href="https://github.com/PyO3/maturin-action/commit/3f475a463fd27ce2440c13988f39fd1312dc9381"><code>3f475a4</code></a> Support native riscv64 builds on manylinux 2_39 (<a href="https://redirect.github.com/pyo3/maturin-action/issues/429">#429</a>)</li> <li><a href="https://github.com/PyO3/maturin-action/commit/5f46978ce3318baa8b414bc7da24a147233222ee"><code>5f46978</code></a> Bump flatted from 3.3.3 to 3.4.2 (<a href="https://redirect.github.com/pyo3/maturin-action/issues/427">#427</a>)</li> <li>Additional commits viewable in <a href="https://github.com/pyo3/maturin-action/compare/04ac600d27cdf7a9a280dadf7147097c42b757ad...e83996d129638aa358a18fbd1dfb82f0b0fb5d3b">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 1c7eb65)
…#2345) Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 7.0.0 to 7.0.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/upload-artifact/releases">actions/upload-artifact's releases</a>.</em></p> <blockquote> <h2>v7.0.1</h2> <h2>What's Changed</h2> <ul> <li>Update the readme with direct upload details by <a href="https://github.com/danwkennedy"><code>@danwkennedy</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/795">actions/upload-artifact#795</a></li> <li>Readme: bump all the example versions to v7 by <a href="https://github.com/danwkennedy"><code>@danwkennedy</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/796">actions/upload-artifact#796</a></li> <li>Include changes in typespec/ts-http-runtime 0.3.5 by <a href="https://github.com/yacaovsnc"><code>@yacaovsnc</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/797">actions/upload-artifact#797</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/upload-artifact/compare/v7...v7.0.1">https://github.com/actions/upload-artifact/compare/v7...v7.0.1</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/actions/upload-artifact/commit/043fb46d1a93c77aae656e7c1c64a875d1fc6a0a"><code>043fb46</code></a> Merge pull request <a href="https://redirect.github.com/actions/upload-artifact/issues/797">#797</a> from actions/yacaovsnc/update-dependency</li> <li><a href="https://github.com/actions/upload-artifact/commit/634250c1388765ea7ed0f053e636f1f399000b94"><code>634250c</code></a> Include changes in typespec/ts-http-runtime 0.3.5</li> <li><a href="https://github.com/actions/upload-artifact/commit/e454baaac2be505c9450e11b8f3215c6fc023ce8"><code>e454baa</code></a> Readme: bump all the example versions to v7 (<a href="https://redirect.github.com/actions/upload-artifact/issues/796">#796</a>)</li> <li><a href="https://github.com/actions/upload-artifact/commit/74fad66b98a6d799dc004d3353ccd0e6f6b2530e"><code>74fad66</code></a> Update the readme with direct upload details (<a href="https://redirect.github.com/actions/upload-artifact/issues/795">#795</a>)</li> <li>See full diff in <a href="https://github.com/actions/upload-artifact/compare/bbbca2ddaa5d8feaa63e36b76fdaad77386f024f...043fb46d1a93c77aae656e7c1c64a875d1fc6a0a">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 02496d3)
## Which issue does this PR close? - Closes apache#2347 . ## What changes are included in this PR? This change performs datum conversion for FixedSizedBinaryArray types from fixed binary primitive types. It closely follows the previously added support for Uuid types. ## Are these changes tested? Yes (cherry picked from commit c1538de)
## Which issue does this PR close? Keep up with latest release from datafusion: apache/datafusion#21079 - Closes #. ## What changes are included in this PR? chore: bump datafusion to 53.1.0 <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> (cherry picked from commit 1b64009)
…s/python (apache#2352) Bumps [rustls-webpki](https://github.com/rustls/webpki) from 0.103.10 to 0.103.12. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/rustls/webpki/releases">rustls-webpki's releases</a>.</em></p> <blockquote> <h2>0.103.12</h2> <p>This release fixes two bugs in name constraint enforcement:</p> <ul> <li><strong>GHSA-965h-392x-2mh5</strong>: name constraints for URI names were ignored and therefore accepted. URI name constraints are now rejected unconditionally. Note this library does not provide an API for asserting URI names, and URI name constraints are otherwise not implemented.</li> <li><strong>GHSA-xgp8-3hg3-c2mh</strong>: permitted subtree name constraints for DNS names were accepted for certificates asserting a wildcard name. This was incorrect because, given a name constraint of <code>accept.example.com</code>, <code>*.example.com</code> could feasibly allow a name of <code>reject.example.com</code> which is outside the constraint. This is very similar to <a href="https://go.dev/issue/76442">CVE-2025-61727</a>.</li> </ul> <p>Since name constraints are restrictions on otherwise properly-issued certificates, these bugs are reachable only after signature verification and require misissuance to exploit.</p> <h2>What's Changed</h2> <ul> <li>Prepare 0.103.12 by <a href="https://github.com/djc"><code>@djc</code></a> in <a href="https://redirect.github.com/rustls/webpki/pull/470">rustls/webpki#470</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/rustls/webpki/compare/v/0.103.11...v/0.103.12">https://github.com/rustls/webpki/compare/v/0.103.11...v/0.103.12</a></p> <h2>0.103.11</h2> <p>In response to <a href="https://redirect.github.com/rustls/webpki/issues/464">#464</a>, we've slightly relaxed requirements for <code>anchor_from_trust_cert()</code> to ignore unknown extensions even if they're marked as critical. This only affects parsing a <code>TrustAnchor</code> from DER, for which most extensions are ignored anyway.</p> <h2>What's Changed</h2> <ul> <li>Backport parsing trust anchors with unknown critical extensions to 0.103 by <a href="https://github.com/djc"><code>@djc</code></a> in <a href="https://redirect.github.com/rustls/webpki/pull/466">rustls/webpki#466</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/rustls/webpki/commit/27131d476e2b68a537e629d6d012bef8dad6efd3"><code>27131d4</code></a> Bump version to 0.103.12</li> <li><a href="https://github.com/rustls/webpki/commit/6ecb8769cde2246e761e058709421c14a7dee6b1"><code>6ecb876</code></a> Clean up stuttery enum variant names</li> <li><a href="https://github.com/rustls/webpki/commit/318b3e6e03ca2bc21600ca6bb0d0c6439b9e6aeb"><code>318b3e6</code></a> Ignore wildcard labels when matching name constraints</li> <li><a href="https://github.com/rustls/webpki/commit/12196229a327d3d670798688254bd3ea24aba24b"><code>1219622</code></a> Rewrite constraint matching to avoid permissive catch-all branch</li> <li><a href="https://github.com/rustls/webpki/commit/57bc62ce538c2d0d31d44b3eb8c58e6a0a764b47"><code>57bc62c</code></a> Bump version to 0.103.11</li> <li><a href="https://github.com/rustls/webpki/commit/d0fa01ee0a76b7585c13ec43de5854955146ffef"><code>d0fa01e</code></a> Allow parsing trust anchors with unknown criticial extensions</li> <li>See full diff in <a href="https://github.com/rustls/webpki/compare/v/0.103.10...v/0.103.12">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-rust/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit ad44fc3)
…e#2351) ## Which issue does this PR close? Not tied to a specific issue - found during an audit of pushdown filter gaps. ## What changes are included in this PR? `PredicateConverter::is_nan` and `not_nan` were never actually implemented - `is_nan` returned `always_true` (matches every row) and `not_nan` returned `always_false` (matches no rows). Every other predicate in `PredicateConverter` projects the column from the batch and runs an arrow compute kernel, but these two just returned constants. This adds a `compute_is_nan` helper that downcasts to `Float32Array`/`Float64Array` and checks each value with `f.is_nan()`, preserving nulls. Non-float types return all false. `is_nan` and `not_nan` now use it the same way `is_null`/`not_null` use `arrow::is_null`/`is_not_null`. ## Are these changes tested? Yes, test added (cherry picked from commit 1bed7b6)
…-0104 (apache#2356) ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> (cherry picked from commit 4b4ffd0)
## Which issue does this PR close? - Closes apache#2309 ## What changes are included in this PR? Split arrow reader module into smaller onces so that it would be easier to maintain. I didn't do any extra changes on purpose to make the pr easier to read. ## Are these changes tested? ut. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> (cherry picked from commit 4b0b352)
gabotechs
approved these changes
Apr 30, 2026
gh-worker-dd-devflow-36fce6
Bot
added
mergequeue-status: queued
mergequeue-status: in_progress
and removed
mergequeue-status: queued
labels
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry picked all commits from apache#2289 to apache#2358