Merge upstream 20260123 #37
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Which issue does this PR close? - Closes #. ## What changes are included in this PR? Minor updates to `README.md` text. ## Are these changes tested? N/A
## Which issue does this PR close? - Closes #. ## What changes are included in this PR? Just a nit, I like to split up the make command so i can rerun it later Before: <img width="751" height="148" alt="Screenshot 2026-01-12 at 9 36 24 AM" src="https://github.com/user-attachments/assets/392fd55c-2a7a-4b57-9eb5-e5b32e04f6bd" /> After: <img width="742" height="181" alt="Screenshot 2026-01-12 at 9 36 49 AM" src="https://github.com/user-attachments/assets/b6413d7e-72cd-47a2-bc2c-972ec4bd7b36" /> ## Are these changes tested?
## Which issue does this PR close? - Closes #. ## What changes are included in this PR? Add a `make site` command to build and serve the iceberg-rust website. Update README to match ## Are these changes tested?
…che#2008) Implement schema validation in project_with_partition to ensure the input schema matches the Iceberg table schema before calculating partition values. This prevents subtle bugs from schema mismatches and provides clear error messages when schemas don't match. Changes: - Add helper functions to recursively strip metadata from Arrow schemas - Implement schema validation that compares input schema with expected Iceberg table schema, ignoring metadata differences - Add comprehensive tests for metadata stripping and schema validation - Closes apache#1752 The implementation follows the approach suggested in issue apache#1752: - Recursively visits schema and removes metadata from all fields - Compares cleaned schemas using Arrow's built-in equality operator - Returns helpful error messages showing both schemas on mismatch ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? ## Are these changes tested? Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
## Which issue does this PR close? - Closes apache#1905 ## What changes are included in this PR? - Implement `register_table` in `IcebergSchemaProvider` - Added a new slt: `create_table.slt` - Updated existing slts Note: This does NOT cover the syntax `CREATE TABLE ... AS VALUES` ## Are these changes tested? Yes
## Which issue does this PR close? - Closes apache#2015. ## What changes are included in this PR? Update copyright year from `2023-2024` to `2023-2026` in NOTICE files. ## Are these changes tested? No code changes, only documentation update.
…apache#1974) ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? I noticed this workflow will run on my forked repo periodically. Let's disable it for forked repos ## Are these changes tested?
## Which issue does this PR close? - Related to - apache#1314 - apache#172 ## What changes are included in this PR? ## Are these changes tested? --------- Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? Remove mentions of release version from the "downloads" page (https://rust.iceberg.apache.org/download.html) So we dont have to update it after every release :) Rendered  hyperlinks referenced in screenshot: https://dlcdn.apache.org/iceberg/ https://github.com/apache/iceberg-rust/releases https://archive.apache.org/dist/iceberg/ <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
## Which issue does this PR close? - Closes apache#2041. ## What changes are included in this PR? Upgrade spark to use 3.5.8 which is available in https://dlcdn.apache.org/spark/ 3.5.7 is gone, causing our CI to fail ## Are these changes tested?
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes apache#2041 ## What changes are included in this PR? We made some upgrades to the Spark Dockerfile in pyiceberg (apache/iceberg-python#2540) (which i think rust's Dockerfile copied over previously). Porting those changes over: - Use `apache/spark` as base image (should be faster than downloading spark from apache cdn) - Upgrade to spark 4.0 - Use Spark connect for provisioning <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? Yes <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? Similar to apache/iceberg-python#2885 Add the `--build` flag to `docker compose up`. This will trigger docker to rebuild if the docker-compose file or the Dockerfile is changed. Without `--build`, changes to `Dockerfile` will not trigger a new image build and we might accidentally still be using the old image thinking its updated <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> --------- Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
Bumps [rust_decimal](https://github.com/paupino/rust-decimal) from 1.39.0 to 1.40.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/paupino/rust-decimal/releases">rust_decimal's releases</a>.</em></p> <blockquote> <h2>1.40.0</h2> <h2>What's Changed</h2> <ul> <li>Fixed Issue <a href="https://redirect.github.com/paupino/rust-decimal/issues/760">#760</a> serde_json roundtrip on specific values (0.0, 1.0, 2.0,...) by <a href="https://github.com/hugobast"><code>@hugobast</code></a> in <a href="https://redirect.github.com/paupino/rust-decimal/pull/761">paupino/rust-decimal#761</a></li> <li>Allow the usage of stack-based strings by <a href="https://github.com/c410-f3r"><code>@c410-f3r</code></a> in <a href="https://redirect.github.com/paupino/rust-decimal/pull/762">paupino/rust-decimal#762</a></li> <li>Fix: RUSTSEC-2026-0001 by <a href="https://github.com/sebadob"><code>@sebadob</code></a> in <a href="https://redirect.github.com/paupino/rust-decimal/pull/770">paupino/rust-decimal#770</a></li> <li>Version 1.40.0 by <a href="https://github.com/paupino"><code>@paupino</code></a> in <a href="https://redirect.github.com/paupino/rust-decimal/pull/771">paupino/rust-decimal#771</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/hugobast"><code>@hugobast</code></a> made their first contribution in <a href="https://redirect.github.com/paupino/rust-decimal/pull/761">paupino/rust-decimal#761</a></li> <li><a href="https://github.com/sebadob"><code>@sebadob</code></a> made their first contribution in <a href="https://redirect.github.com/paupino/rust-decimal/pull/770">paupino/rust-decimal#770</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/paupino/rust-decimal/compare/1.39.0...1.40.0">https://github.com/paupino/rust-decimal/compare/1.39.0...1.40.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/paupino/rust-decimal/commit/96933153ea066ea8b96c7cbfb8e83a308c02e04f"><code>9693315</code></a> Version 1.40.0</li> <li><a href="https://github.com/paupino/rust-decimal/commit/f55091ce5ec7de27066f7860a60c9740c48d0f43"><code>f55091c</code></a> Fix: RUSTSEC-2026-0001</li> <li><a href="https://github.com/paupino/rust-decimal/commit/bab09af2ea82b50d81ce3c8be2f0a8185672d91a"><code>bab09af</code></a> Merge pull request <a href="https://redirect.github.com/paupino/rust-decimal/issues/762">#762</a> from c410-f3r/aaaaaaaaaaa</li> <li><a href="https://github.com/paupino/rust-decimal/commit/69fe199660cbff00fcfa2f994bbab26964c86346"><code>69fe199</code></a> Change calling method of doc test</li> <li><a href="https://github.com/paupino/rust-decimal/commit/83e2ce7c1029a1a4b06ee44ac5500cfb4afbe531"><code>83e2ce7</code></a> Address comments</li> <li><a href="https://github.com/paupino/rust-decimal/commit/f650296e531d653dcc5b44225e451afe592d0925"><code>f650296</code></a> Allow the usage of stack-based strings</li> <li><a href="https://github.com/paupino/rust-decimal/commit/db61f68aaf8340bd6e2b6ba37b878d0d36beaa75"><code>db61f68</code></a> Merge pull request <a href="https://redirect.github.com/paupino/rust-decimal/issues/761">#761</a> from hugobast/v1</li> <li><a href="https://github.com/paupino/rust-decimal/commit/3fd6ea2fb03f6c290cb380d6d38b13b7ffe2d3d9"><code>3fd6ea2</code></a> chore: convert to a set of test cases</li> <li><a href="https://github.com/paupino/rust-decimal/commit/fcbab2d2b0fc3b44840688018d235296f9e1054d"><code>fcbab2d</code></a> fix: address float to string loss of precision</li> <li>See full diff in <a href="https://github.com/paupino/rust-decimal/compare/1.39.0...1.40.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
Bumps the arrow-parquet group with 2 updates: [arrow-arith](https://github.com/apache/arrow-rs) and [arrow-cast](https://github.com/apache/arrow-rs). Updates `arrow-arith` from 57.1.0 to 57.2.0 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/apache/arrow-rs/releases">arrow-arith's releases</a>.</em></p> <blockquote> <h2>arrow 57.2.0</h2> <!-- raw HTML omitted --> <h1>Changelog</h1> <h2><a href="https://github.com/apache/arrow-rs/tree/57.2.0">57.2.0</a> (2026-01-07)</h2> <p><a href="https://github.com/apache/arrow-rs/compare/57.1.0...57.2.0">Full Changelog</a></p> <p><strong>Breaking changes:</strong></p> <ul> <li>Seal Array trait <a href="https://redirect.github.com/apache/arrow-rs/pull/9092">#9092</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>] (<a href="https://github.com/tustvold">tustvold</a>)</li> <li>[Variant] Unify the CastOptions usage in parquet-variant-compute <a href="https://redirect.github.com/apache/arrow-rs/pull/8984">#8984</a> (<a href="https://github.com/klion26">klion26</a>)</li> </ul> <p><strong>Implemented enhancements:</strong></p> <ul> <li>[parquet] further relax <code>LevelInfoBuilder::types_compatible</code> for <code>ArrowWriter</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/9098">#9098</a></li> <li>Update arrow-row documentation with Union encoding <a href="https://redirect.github.com/apache/arrow-rs/issues/9084">#9084</a></li> <li>Add code examples for min and max compute functions <a href="https://redirect.github.com/apache/arrow-rs/issues/9055">#9055</a></li> <li>Add <code>append_n</code> to bytes view builder API <a href="https://redirect.github.com/apache/arrow-rs/issues/9034">#9034</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Move <code>RunArray::get_physical_indices</code> to <code>RunEndBuffer</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/9025">#9025</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Allow quote style in csv writer <a href="https://redirect.github.com/apache/arrow-rs/issues/9003">#9003</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>IPC support for ListView <a href="https://redirect.github.com/apache/arrow-rs/issues/9002">#9002</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Implement <code>BinaryArrayType</code> for <code>&FixedSizeBinaryArray</code>s <a href="https://redirect.github.com/apache/arrow-rs/issues/8992">#8992</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: implement num-traits for i256 <a href="https://redirect.github.com/apache/arrow-rs/issues/8976">#8976</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Support for <code>Arc<str></code> in <code>ParquetRecordWriter</code> derive macro <a href="https://redirect.github.com/apache/arrow-rs/issues/8972">#8972</a></li> <li>[arrow-avro] suggest switching from xz to liblzma <a href="https://redirect.github.com/apache/arrow-rs/issues/8970">#8970</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: add i256::trailing_zeros <a href="https://redirect.github.com/apache/arrow-rs/issues/8968">#8968</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: make i256::leading_zeros public <a href="https://redirect.github.com/apache/arrow-rs/issues/8965">#8965</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Add spark like <code>ignoreLeadingWhiteSpace</code> and <code>ignoreTrailingWhiteSpace</code> options to the csv writer <a href="https://redirect.github.com/apache/arrow-rs/issues/8961">#8961</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Add round trip benchmark for Parquet writer/reader <a href="https://redirect.github.com/apache/arrow-rs/issues/8955">#8955</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li> <li>Support performant <code>interleave</code> for List/LargeList <a href="https://redirect.github.com/apache/arrow-rs/issues/8952">#8952</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] Support array access when parsing <code>VariantPath</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8946">#8946</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md">arrow-arith's changelog</a>.</em></p> <blockquote> <h2><a href="https://github.com/apache/arrow-rs/tree/57.2.0">57.2.0</a> (2026-01-07)</h2> <p><a href="https://github.com/apache/arrow-rs/compare/57.1.0...57.2.0">Full Changelog</a></p> <p><strong>Breaking changes:</strong></p> <ul> <li>Seal Array trait <a href="https://redirect.github.com/apache/arrow-rs/pull/9092">#9092</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>] (<a href="https://github.com/tustvold">tustvold</a>)</li> <li>[Variant] Unify the CastOptions usage in parquet-variant-compute <a href="https://redirect.github.com/apache/arrow-rs/pull/8984">#8984</a> (<a href="https://github.com/klion26">klion26</a>)</li> </ul> <p><strong>Implemented enhancements:</strong></p> <ul> <li>[parquet] further relax <code>LevelInfoBuilder::types_compatible</code> for <code>ArrowWriter</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/9098">#9098</a></li> <li>Update arrow-row documentation with Union encoding <a href="https://redirect.github.com/apache/arrow-rs/issues/9084">#9084</a></li> <li>Add code examples for min and max compute functions <a href="https://redirect.github.com/apache/arrow-rs/issues/9055">#9055</a></li> <li>Add <code>append_n</code> to bytes view builder API <a href="https://redirect.github.com/apache/arrow-rs/issues/9034">#9034</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Move <code>RunArray::get_physical_indices</code> to <code>RunEndBuffer</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/9025">#9025</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Allow quote style in csv writer <a href="https://redirect.github.com/apache/arrow-rs/issues/9003">#9003</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>IPC support for ListView <a href="https://redirect.github.com/apache/arrow-rs/issues/9002">#9002</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Implement <code>BinaryArrayType</code> for <code>&FixedSizeBinaryArray</code>s <a href="https://redirect.github.com/apache/arrow-rs/issues/8992">#8992</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: implement num-traits for i256 <a href="https://redirect.github.com/apache/arrow-rs/issues/8976">#8976</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Support for <code>Arc<str></code> in <code>ParquetRecordWriter</code> derive macro <a href="https://redirect.github.com/apache/arrow-rs/issues/8972">#8972</a></li> <li>[arrow-avro] suggest switching from xz to liblzma <a href="https://redirect.github.com/apache/arrow-rs/issues/8970">#8970</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: add i256::trailing_zeros <a href="https://redirect.github.com/apache/arrow-rs/issues/8968">#8968</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: make i256::leading_zeros public <a href="https://redirect.github.com/apache/arrow-rs/issues/8965">#8965</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Add spark like <code>ignoreLeadingWhiteSpace</code> and <code>ignoreTrailingWhiteSpace</code> options to the csv writer <a href="https://redirect.github.com/apache/arrow-rs/issues/8961">#8961</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Add round trip benchmark for Parquet writer/reader <a href="https://redirect.github.com/apache/arrow-rs/issues/8955">#8955</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li> <li>Support performant <code>interleave</code> for List/LargeList <a href="https://redirect.github.com/apache/arrow-rs/issues/8952">#8952</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] Support array access when parsing <code>VariantPath</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8946">#8946</a></li> <li>Some panic!s could be represented as unimplemented!s <a href="https://redirect.github.com/apache/arrow-rs/issues/8932">#8932</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] easier way to construct a shredded schema <a href="https://redirect.github.com/apache/arrow-rs/issues/8922">#8922</a></li> <li>Support <code>DataType::ListView</code> and <code>DataType::LargeListView</code> in <code>ArrayData::new_null</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8908">#8908</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Add <code>GenericListViewArray::from_iter_primitive</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8906">#8906</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] Unify the cast option usage in ParquentVariant <a href="https://redirect.github.com/apache/arrow-rs/issues/8873">#8873</a></li> <li>Blog post about efficient filter representation in Parquet filter pushdown <a href="https://redirect.github.com/apache/arrow-rs/issues/8843">#8843</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li> <li>Add comparison support for Union arrays in the <code>cmp</code> kernel <a href="https://redirect.github.com/apache/arrow-rs/issues/8837">#8837</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] Support array shredding into <code>List/LargeList/ListView/LargeListView</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8830">#8830</a></li> <li>Support <code>Union</code> data types for row format <a href="https://redirect.github.com/apache/arrow-rs/issues/8828">#8828</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>FFI support for ListView <a href="https://redirect.github.com/apache/arrow-rs/issues/8819">#8819</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] Support more Arrow Datatypes from Variant primitive types <a href="https://redirect.github.com/apache/arrow-rs/issues/8805">#8805</a></li> <li><code>FixedSizeBinaryBuilder</code> supports <code>append_array</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8750">#8750</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Implement special case <code>zip</code> with scalar for Utf8View <a href="https://redirect.github.com/apache/arrow-rs/issues/8724">#8724</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[geometry] Wire up arrow reader/writer for <code>GEOMETRY</code> and <code>GEOGRAPHY</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8717">#8717</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li> </ul> <p><strong>Fixed bugs:</strong></p> <ul> <li>Soundness Bug in <code>try_binary</code> when <code>Array</code> is implemented incorrectly in external crate <a href="https://redirect.github.com/apache/arrow-rs/issues/9106">#9106</a></li> <li>casting <code>Dict(_, LargeUtf8)</code> to <code>Utf8View</code> (<code>StringViewArray</code>) panics <a href="https://redirect.github.com/apache/arrow-rs/issues/9101">#9101</a></li> <li>wrong results for null count of <code>nullif</code> kernel <a href="https://redirect.github.com/apache/arrow-rs/issues/9085">#9085</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>] [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Empty first line in some code examples <a href="https://redirect.github.com/apache/arrow-rs/issues/9063">#9063</a></li> <li>GenericByteViewArray::slice is not zero-copy but ought to be <a href="https://redirect.github.com/apache/arrow-rs/issues/9014">#9014</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/arrow-rs/commit/9e822e050016e659028741499388d454a74cf23c"><code>9e822e0</code></a> Update version to <code>57.2.0</code>, add CHANGELOG (<a href="https://redirect.github.com/apache/arrow-rs/issues/9103">#9103</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/28f66f94e0e7f0e2b27686207a91e40b60e3b957"><code>28f66f9</code></a> Add Union encoding documentation (<a href="https://redirect.github.com/apache/arrow-rs/issues/9102">#9102</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/a8346be02a240788b1246d847bc8dfec21274306"><code>a8346be</code></a> Minor: make it clear cache array reader is not cloning arrays (<a href="https://redirect.github.com/apache/arrow-rs/issues/9057">#9057</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/721f373fba7493c873a64421e32f1f67114ac130"><code>721f373</code></a> Seal Array trait (<a href="https://redirect.github.com/apache/arrow-rs/issues/9092">#9092</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/2b179b805f62e927771f764f8ae92151249e9edd"><code>2b179b8</code></a> feat(parquet): relax type compatility check in parquet ArrowWriter (<a href="https://redirect.github.com/apache/arrow-rs/issues/9099">#9099</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/10a976fc03af32d26f9e4bf4dcc420cb1a455ef0"><code>10a976f</code></a> chore: increase row count and batch size for more deterministic tests (<a href="https://redirect.github.com/apache/arrow-rs/issues/9088">#9088</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/2507946be697d511689a8f59b6a7db45ef40854b"><code>2507946</code></a> Add <code>FlightInfo::with_endpoints</code> method (<a href="https://redirect.github.com/apache/arrow-rs/issues/9075">#9075</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/068a7e44f0881ffe542225c5dd8cb354b74a9615"><code>068a7e4</code></a> Add <code>DataType::is_decimal</code> (<a href="https://redirect.github.com/apache/arrow-rs/issues/9100">#9100</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/b1dfb697babcb614040ea2ae17e842e9db557f69"><code>b1dfb69</code></a> Fix row slice bug in Union column decoding with many columns (<a href="https://redirect.github.com/apache/arrow-rs/issues/9000">#9000</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/a9d6e92664996a875bab2b5df60704781a396e6c"><code>a9d6e92</code></a> [Variant] Move <code>ArrayVariantToArrowRowBuilder</code> to <code>variant_to_arrow</code> (<a href="https://redirect.github.com/apache/arrow-rs/issues/9094">#9094</a>)</li> <li>Additional commits viewable in <a href="https://github.com/apache/arrow-rs/compare/57.1.0...57.2.0">compare view</a></li> </ul> </details> <br /> Updates `arrow-cast` from 57.1.0 to 57.2.0 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/apache/arrow-rs/releases">arrow-cast's releases</a>.</em></p> <blockquote> <h2>arrow 57.2.0</h2> <!-- raw HTML omitted --> <h1>Changelog</h1> <h2><a href="https://github.com/apache/arrow-rs/tree/57.2.0">57.2.0</a> (2026-01-07)</h2> <p><a href="https://github.com/apache/arrow-rs/compare/57.1.0...57.2.0">Full Changelog</a></p> <p><strong>Breaking changes:</strong></p> <ul> <li>Seal Array trait <a href="https://redirect.github.com/apache/arrow-rs/pull/9092">#9092</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>] (<a href="https://github.com/tustvold">tustvold</a>)</li> <li>[Variant] Unify the CastOptions usage in parquet-variant-compute <a href="https://redirect.github.com/apache/arrow-rs/pull/8984">#8984</a> (<a href="https://github.com/klion26">klion26</a>)</li> </ul> <p><strong>Implemented enhancements:</strong></p> <ul> <li>[parquet] further relax <code>LevelInfoBuilder::types_compatible</code> for <code>ArrowWriter</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/9098">#9098</a></li> <li>Update arrow-row documentation with Union encoding <a href="https://redirect.github.com/apache/arrow-rs/issues/9084">#9084</a></li> <li>Add code examples for min and max compute functions <a href="https://redirect.github.com/apache/arrow-rs/issues/9055">#9055</a></li> <li>Add <code>append_n</code> to bytes view builder API <a href="https://redirect.github.com/apache/arrow-rs/issues/9034">#9034</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Move <code>RunArray::get_physical_indices</code> to <code>RunEndBuffer</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/9025">#9025</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Allow quote style in csv writer <a href="https://redirect.github.com/apache/arrow-rs/issues/9003">#9003</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>IPC support for ListView <a href="https://redirect.github.com/apache/arrow-rs/issues/9002">#9002</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Implement <code>BinaryArrayType</code> for <code>&FixedSizeBinaryArray</code>s <a href="https://redirect.github.com/apache/arrow-rs/issues/8992">#8992</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: implement num-traits for i256 <a href="https://redirect.github.com/apache/arrow-rs/issues/8976">#8976</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Support for <code>Arc<str></code> in <code>ParquetRecordWriter</code> derive macro <a href="https://redirect.github.com/apache/arrow-rs/issues/8972">#8972</a></li> <li>[arrow-avro] suggest switching from xz to liblzma <a href="https://redirect.github.com/apache/arrow-rs/issues/8970">#8970</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: add i256::trailing_zeros <a href="https://redirect.github.com/apache/arrow-rs/issues/8968">#8968</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: make i256::leading_zeros public <a href="https://redirect.github.com/apache/arrow-rs/issues/8965">#8965</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Add spark like <code>ignoreLeadingWhiteSpace</code> and <code>ignoreTrailingWhiteSpace</code> options to the csv writer <a href="https://redirect.github.com/apache/arrow-rs/issues/8961">#8961</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Add round trip benchmark for Parquet writer/reader <a href="https://redirect.github.com/apache/arrow-rs/issues/8955">#8955</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li> <li>Support performant <code>interleave</code> for List/LargeList <a href="https://redirect.github.com/apache/arrow-rs/issues/8952">#8952</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] Support array access when parsing <code>VariantPath</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8946">#8946</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md">arrow-cast's changelog</a>.</em></p> <blockquote> <h2><a href="https://github.com/apache/arrow-rs/tree/57.2.0">57.2.0</a> (2026-01-07)</h2> <p><a href="https://github.com/apache/arrow-rs/compare/57.1.0...57.2.0">Full Changelog</a></p> <p><strong>Breaking changes:</strong></p> <ul> <li>Seal Array trait <a href="https://redirect.github.com/apache/arrow-rs/pull/9092">#9092</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>] (<a href="https://github.com/tustvold">tustvold</a>)</li> <li>[Variant] Unify the CastOptions usage in parquet-variant-compute <a href="https://redirect.github.com/apache/arrow-rs/pull/8984">#8984</a> (<a href="https://github.com/klion26">klion26</a>)</li> </ul> <p><strong>Implemented enhancements:</strong></p> <ul> <li>[parquet] further relax <code>LevelInfoBuilder::types_compatible</code> for <code>ArrowWriter</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/9098">#9098</a></li> <li>Update arrow-row documentation with Union encoding <a href="https://redirect.github.com/apache/arrow-rs/issues/9084">#9084</a></li> <li>Add code examples for min and max compute functions <a href="https://redirect.github.com/apache/arrow-rs/issues/9055">#9055</a></li> <li>Add <code>append_n</code> to bytes view builder API <a href="https://redirect.github.com/apache/arrow-rs/issues/9034">#9034</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Move <code>RunArray::get_physical_indices</code> to <code>RunEndBuffer</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/9025">#9025</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Allow quote style in csv writer <a href="https://redirect.github.com/apache/arrow-rs/issues/9003">#9003</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>IPC support for ListView <a href="https://redirect.github.com/apache/arrow-rs/issues/9002">#9002</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Implement <code>BinaryArrayType</code> for <code>&FixedSizeBinaryArray</code>s <a href="https://redirect.github.com/apache/arrow-rs/issues/8992">#8992</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: implement num-traits for i256 <a href="https://redirect.github.com/apache/arrow-rs/issues/8976">#8976</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Support for <code>Arc<str></code> in <code>ParquetRecordWriter</code> derive macro <a href="https://redirect.github.com/apache/arrow-rs/issues/8972">#8972</a></li> <li>[arrow-avro] suggest switching from xz to liblzma <a href="https://redirect.github.com/apache/arrow-rs/issues/8970">#8970</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: add i256::trailing_zeros <a href="https://redirect.github.com/apache/arrow-rs/issues/8968">#8968</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>arrow-buffer: make i256::leading_zeros public <a href="https://redirect.github.com/apache/arrow-rs/issues/8965">#8965</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Add spark like <code>ignoreLeadingWhiteSpace</code> and <code>ignoreTrailingWhiteSpace</code> options to the csv writer <a href="https://redirect.github.com/apache/arrow-rs/issues/8961">#8961</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Add round trip benchmark for Parquet writer/reader <a href="https://redirect.github.com/apache/arrow-rs/issues/8955">#8955</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li> <li>Support performant <code>interleave</code> for List/LargeList <a href="https://redirect.github.com/apache/arrow-rs/issues/8952">#8952</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] Support array access when parsing <code>VariantPath</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8946">#8946</a></li> <li>Some panic!s could be represented as unimplemented!s <a href="https://redirect.github.com/apache/arrow-rs/issues/8932">#8932</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] easier way to construct a shredded schema <a href="https://redirect.github.com/apache/arrow-rs/issues/8922">#8922</a></li> <li>Support <code>DataType::ListView</code> and <code>DataType::LargeListView</code> in <code>ArrayData::new_null</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8908">#8908</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Add <code>GenericListViewArray::from_iter_primitive</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8906">#8906</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] Unify the cast option usage in ParquentVariant <a href="https://redirect.github.com/apache/arrow-rs/issues/8873">#8873</a></li> <li>Blog post about efficient filter representation in Parquet filter pushdown <a href="https://redirect.github.com/apache/arrow-rs/issues/8843">#8843</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li> <li>Add comparison support for Union arrays in the <code>cmp</code> kernel <a href="https://redirect.github.com/apache/arrow-rs/issues/8837">#8837</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] Support array shredding into <code>List/LargeList/ListView/LargeListView</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8830">#8830</a></li> <li>Support <code>Union</code> data types for row format <a href="https://redirect.github.com/apache/arrow-rs/issues/8828">#8828</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>FFI support for ListView <a href="https://redirect.github.com/apache/arrow-rs/issues/8819">#8819</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[Variant] Support more Arrow Datatypes from Variant primitive types <a href="https://redirect.github.com/apache/arrow-rs/issues/8805">#8805</a></li> <li><code>FixedSizeBinaryBuilder</code> supports <code>append_array</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8750">#8750</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Implement special case <code>zip</code> with scalar for Utf8View <a href="https://redirect.github.com/apache/arrow-rs/issues/8724">#8724</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>[geometry] Wire up arrow reader/writer for <code>GEOMETRY</code> and <code>GEOGRAPHY</code> <a href="https://redirect.github.com/apache/arrow-rs/issues/8717">#8717</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li> </ul> <p><strong>Fixed bugs:</strong></p> <ul> <li>Soundness Bug in <code>try_binary</code> when <code>Array</code> is implemented incorrectly in external crate <a href="https://redirect.github.com/apache/arrow-rs/issues/9106">#9106</a></li> <li>casting <code>Dict(_, LargeUtf8)</code> to <code>Utf8View</code> (<code>StringViewArray</code>) panics <a href="https://redirect.github.com/apache/arrow-rs/issues/9101">#9101</a></li> <li>wrong results for null count of <code>nullif</code> kernel <a href="https://redirect.github.com/apache/arrow-rs/issues/9085">#9085</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>] [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li> <li>Empty first line in some code examples <a href="https://redirect.github.com/apache/arrow-rs/issues/9063">#9063</a></li> <li>GenericByteViewArray::slice is not zero-copy but ought to be <a href="https://redirect.github.com/apache/arrow-rs/issues/9014">#9014</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/arrow-rs/commit/9e822e050016e659028741499388d454a74cf23c"><code>9e822e0</code></a> Update version to <code>57.2.0</code>, add CHANGELOG (<a href="https://redirect.github.com/apache/arrow-rs/issues/9103">#9103</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/28f66f94e0e7f0e2b27686207a91e40b60e3b957"><code>28f66f9</code></a> Add Union encoding documentation (<a href="https://redirect.github.com/apache/arrow-rs/issues/9102">#9102</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/a8346be02a240788b1246d847bc8dfec21274306"><code>a8346be</code></a> Minor: make it clear cache array reader is not cloning arrays (<a href="https://redirect.github.com/apache/arrow-rs/issues/9057">#9057</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/721f373fba7493c873a64421e32f1f67114ac130"><code>721f373</code></a> Seal Array trait (<a href="https://redirect.github.com/apache/arrow-rs/issues/9092">#9092</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/2b179b805f62e927771f764f8ae92151249e9edd"><code>2b179b8</code></a> feat(parquet): relax type compatility check in parquet ArrowWriter (<a href="https://redirect.github.com/apache/arrow-rs/issues/9099">#9099</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/10a976fc03af32d26f9e4bf4dcc420cb1a455ef0"><code>10a976f</code></a> chore: increase row count and batch size for more deterministic tests (<a href="https://redirect.github.com/apache/arrow-rs/issues/9088">#9088</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/2507946be697d511689a8f59b6a7db45ef40854b"><code>2507946</code></a> Add <code>FlightInfo::with_endpoints</code> method (<a href="https://redirect.github.com/apache/arrow-rs/issues/9075">#9075</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/068a7e44f0881ffe542225c5dd8cb354b74a9615"><code>068a7e4</code></a> Add <code>DataType::is_decimal</code> (<a href="https://redirect.github.com/apache/arrow-rs/issues/9100">#9100</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/b1dfb697babcb614040ea2ae17e842e9db557f69"><code>b1dfb69</code></a> Fix row slice bug in Union column decoding with many columns (<a href="https://redirect.github.com/apache/arrow-rs/issues/9000">#9000</a>)</li> <li><a href="https://github.com/apache/arrow-rs/commit/a9d6e92664996a875bab2b5df60704781a396e6c"><code>a9d6e92</code></a> [Variant] Move <code>ArrayVariantToArrowRowBuilder</code> to <code>variant_to_arrow</code> (<a href="https://redirect.github.com/apache/arrow-rs/issues/9094">#9094</a>)</li> <li>Additional commits viewable in <a href="https://github.com/apache/arrow-rs/compare/57.1.0...57.2.0">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Which issue does this PR close? - Closes apache#2038 ## What changes are included in this PR? Get MSRV by parsing `rust-version` from `Cargo.toml`. This is to avoid forgetting to update MSRV version in the github workflow. ## Are these changes tested?
…he#2011) ## Which issue does this PR close? - Closes apache#2007. ## What changes are included in this PR? Implements Iceberg spec requirement to reuse partition field IDs when adding specs with equivalent fields (same source_id + transform). - Add Hash trait to Transform enum for HashMap keys - Add field ID reuse logic in TableMetadataBuilder.add_partition_spec() ## Are these changes tested? Added a test to cover partition field id reuse when new specs are added --------- Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
…pache#2049) ## Which issue does this PR close? - Closes apache#2038 This unblocks us to publish python binding to pypi ## What changes are included in this PR? Need to manually trigger `.github/workflows/release_python.yml` to publish python binding to pypi. This PR fixes the manual flow by allowing to specify `RELEASE_TAG` as input. Added an extra validation for the manual workflow to check the version against the library version in `bindings/python/Cargo.toml` ## Are these changes tested?
…oid waker churn and add determinism to FileScanTask processing (apache#2020) ## Which issue does this PR close? - N/A. ## What changes are included in this PR? - Due to the way Comet maps DataFusion `SessionContext`, the tokio runtime, and Spark Tasks, we see frequent waker churn when concurrency is set to 1 in the `ArrowReader`. This adds a fast path that does not use `try_flatten_unordered` and its internal `replace_waker` calls. - This also prevents tasks from being reordered at runtime. Several Iceberg Java tests expect specific query results without an `ORDER BY`, so this enables those tests to keep working when concurrency is set to 1. See apache/datafusion-comet#3051 and <img width="3804" height="754" alt="flamegraph" src="https://github.com/user-attachments/assets/26b93e85-5835-4bf4-b7f1-b136face940d" /> ## Are these changes tested? New test for determinism, also running the entire Iceberg Java Spark suite via Comet in apache/datafusion-comet#3051. --------- Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
…shdown (apache#2048) ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? Add support for converting Binary and LargeBinary DataFusion ScalarValue types to Iceberg Datum, enabling binary predicates to be pushed down to the Iceberg storage layer. This conversion allows SQL queries with binary hex literals (X'...') to push predicates down to Iceberg, improving query performance by filtering data at the storage level rather than in DataFusion. The integration test verifies that binary predicates are successfully pushed down end-to-end: - Without conversion: predicate stays in FilterExec with predicate:[] - With conversion: predicate pushed to IcebergTableScan Other scalar types (Boolean, Timestamp, Decimal) were investigated but excluded because they are not reachable through practical usage: - Boolean: DataFusion aggressively optimizes comparisons (e.g., x=true becomes just x) before reaching the converter - Timestamp/Decimal: SQL literals are converted to strings/other types before reaching the converter ## Are these changes tested? --------- Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
## Which issue does this PR close? - Closes apache#2060 ## What changes are included in this PR? - Support DROP TABLE syntax by implementing `SchemaProvider::deregister_table` ## Are these changes tested? Added sqllogictests
## Which issue does this PR close? - Closes apache#2052 ## What changes are included in this PR? - Rename Storage to OpenDalStorage so we can introduce Storage trait later ## Are these changes tested? Relies on existing tests
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.