Update Apache.Arrow reference to v0.14. #167
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Looks like TPCH build is failing. |
Member
Author
|
I'll fix it tomorrow. |
Member
Author
|
Should be fixed now. |
Contributor
|
Looks like E2E is failing now. Does it repro locally? |
Member
Author
The root cause for the failure is https://issues.apache.org/jira/browse/ARROW-5908. I'm working on getting a PR up for that fix. If I can get it into the next service release of Arrow, I will update dotnet/spark with it. If not, there is a simple workaround I can do in the tests until we get the fix. |
wesm
pushed a commit
to apache/arrow
that referenced
this pull request
Jul 11, 2019
Ensure 8-byte alignment on each buffer in a RecordBatch as specified in https://arrow.apache.org/docs/format/Layout.html#requirements-goals-and-non-goals >It is required to have all the contiguous memory buffers in an IPC payload aligned at 8-byte boundaries. In other words, each buffer must start at an aligned 8-byte offset. Additionally, each buffer should be padded to a multiple of 8 bytes. /cc @pgovind @stephentoub @imback82 @wesm - If possible, can we also include this patch in the next release (0.14.1 or 0.15.0)? We hit this issue trying to update .NET for Apache Spark to the latest Arrow release - dotnet/spark#167. Author: Eric Erhardt <eric.erhardt@microsoft.com> Closes #4851 from eerhardt/FixWriterPadding and squashes the following commits: 76807e9 <Eric Erhardt> PR feedback 7ecda78 <Eric Erhardt> Ensure 8-byte alignment on each buffer in a RecordBatch.
wesm
pushed a commit
to apache/arrow
that referenced
this pull request
Jul 13, 2019
Ensure 8-byte alignment on each buffer in a RecordBatch as specified in https://arrow.apache.org/docs/format/Layout.html#requirements-goals-and-non-goals >It is required to have all the contiguous memory buffers in an IPC payload aligned at 8-byte boundaries. In other words, each buffer must start at an aligned 8-byte offset. Additionally, each buffer should be padded to a multiple of 8 bytes. /cc @pgovind @stephentoub @imback82 @wesm - If possible, can we also include this patch in the next release (0.14.1 or 0.15.0)? We hit this issue trying to update .NET for Apache Spark to the latest Arrow release - dotnet/spark#167. Author: Eric Erhardt <eric.erhardt@microsoft.com> Closes #4851 from eerhardt/FixWriterPadding and squashes the following commits: 76807e9 <Eric Erhardt> PR feedback 7ecda78 <Eric Erhardt> Ensure 8-byte alignment on each buffer in a RecordBatch.
Member
Author
|
Apache Arrow is planning a The fixes in the patched version are: |
Member
Author
kou
pushed a commit
to apache/arrow-dotnet
that referenced
this pull request
Jul 30, 2025
Ensure 8-byte alignment on each buffer in a RecordBatch as specified in https://arrow.apache.org/docs/format/Layout.html#requirements-goals-and-non-goals >It is required to have all the contiguous memory buffers in an IPC payload aligned at 8-byte boundaries. In other words, each buffer must start at an aligned 8-byte offset. Additionally, each buffer should be padded to a multiple of 8 bytes. /cc @pgovind @stephentoub @imback82 @wesm - If possible, can we also include this patch in the next release (0.14.1 or 0.15.0)? We hit this issue trying to update .NET for Apache Spark to the latest Arrow release - dotnet/spark#167. Author: Eric Erhardt <eric.erhardt@microsoft.com> Closes #4851 from eerhardt/FixWriterPadding and squashes the following commits: 76807e938 <Eric Erhardt> PR feedback 7ecda78c6 <Eric Erhardt> Ensure 8-byte alignment on each buffer in a RecordBatch.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updating to the latest Arrow version which allows us to simplify some code, and brings in the latest perf fixes.