ARROW-10801: [Rust] [Flight] Support sending FlightData for Dictionaries with that of a RecordBatch #8826
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nevi-me
reviewed
Dec 4, 2020
Contributor
nevi-me
left a comment
nevi-me
left a comment
There was a problem hiding this comment.
This looks great, have 1 comment that needs addressing.
carols10cents
force-pushed
the
refactor-file-stream-writer
branch
from
dca9d09 to
78a913a
Compare
alamb
reviewed
Dec 7, 2020
Contributor
alamb
left a comment
alamb
left a comment
There was a problem hiding this comment.
For what it is worth, I reviewed this code and while I am not an expert in Flight so can't speak for the correctness, the structure seems clear and readable. Nice work @carols10cents !
nevi-me
reviewed
Dec 12, 2020
| message.add_header_type(ipc::MessageHeader::Schema); | ||
| message.add_bodyLength(0); | ||
| message.add_header(schema); | ||
| // TODO: custom metadata |
Contributor
There was a problem hiding this comment.
We should attach JIRAs to the TODOs so we don't lose track of them
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains a series of refactorings to extract code from the IPC
FileWriterandStreamWriterthat generatesEncodedDatainstances. I highly recommend reviewing commit-by-commit, I tried to make the changes fairly mechanical and easy to understand at each step.With those refactorings, it becomes easier to reuse the code with Flight too, so that when we're generating
FlightDatafor aRecordBatch, we get theFlightDatafor the batch's dictionaries too.This will help in getting the Flight integration tests that include dictionaries to pass... I'm also having some unrelated protocol problems over in this PR I'm working on but this work will help there.