ARROW-12043: [Rust] [Parquet] Write FSB arrays #9771
Conversation
jorgecarleitao
left a comment
jorgecarleitao
left a comment
There was a problem hiding this comment.
LGTM. Great addition, thanks a lot @nevi-me !
| ArrowDataType::FixedSizeBinary(_) => { | ||
| let mut col_writer = get_col_writer(&mut row_group_writer)?; | ||
| write_leaf( | ||
| &mut col_writer, | ||
| array, | ||
| levels.pop().expect("Levels exhausted"), | ||
| )?; | ||
| row_group_writer.close_column(col_writer)?; | ||
| Ok(()) | ||
| } |
There was a problem hiding this comment.
I can't spot the difference between this and all cases above. Is there any?
There was a problem hiding this comment.
lol, there isn't. Someone (won't name them) copied this chunk separately, thinking that it'll be different. Then they forgot to merge it when they realised that there was no difference.
I'll fix it shortly
| "Attempting to write an Arrow type that is not yet implemented" | ||
| .to_string(), |
There was a problem hiding this comment.
| "Attempting to write an Arrow type that is not yet implemented" | |
| .to_string(), | |
| format!("Attempting to write an Arrow type {?} to parquet that is not yet implemented", array.data_type()) |
( I realize the error message is not changed by this PR I just saw it when looking)
nevi-me
force-pushed
the
ARROW-12043
branch
3 times, most recently
from
1e91b45 to
5bf4197
Compare
Codecov Report
@@ Coverage Diff @@
## master #9771 +/- ##
==========================================
- Coverage 82.59% 82.41% -0.18%
==========================================
Files 248 252 +4
Lines 58294 58977 +683
==========================================
+ Hits 48149 48608 +459
- Misses 10145 10369 +224
Continue to review full report at Codecov.
|
4 participants
Minor change to compute the levels for FSB arrays and write them out. Added a roundtrip test.