Arrow: test case for fixed type #13700
Conversation
|
@nastra: In #3029 you were working on the Fixed type, but abandoned it, because
OTOH, the Fixed type seems to be supported by the Arrow reader. Thanks, |
Back then I wasn't sure whether we could just remove |
| * Types.FixedType} and {@link Types.DecimalType} See | ||
| * https://github.com/apache/iceberg/issues/2485 and | ||
| * <li>Data types: {@link Types.ListType}, {@link Types.MapType}, {@link Types.StructType}, and | ||
| * {@link Types.DecimalType} See https://github.com/apache/iceberg/issues/2485 and |
There was a problem hiding this comment.
decimal type support also exists already, so we can remove that comment
There was a problem hiding this comment.
Yes, that's right, I remove this comment in my other PR: #13562
| rec.setField("uuid_nullable", uuid); | ||
| rec.setField("decimal", new BigDecimal("14.0" + i % 10)); | ||
| rec.setField("decimal_nullable", new BigDecimal("14.0" + i % 10)); | ||
| rec.setField("fixed", new byte[] {1, 2, 3, 4, 5, 6, (byte) i}); |
There was a problem hiding this comment.
nit: I'd maybe just define a string and convert it to its binary representation (same as we do a few lines above for bytes:
rec.setField("fixed", ("abc" + i).getBytes(StandardCharsets.UTF_8));
rec.setField("fixed_nullable", ("abcdef" + i).getBytes(StandardCharsets.UTF_8));
| rec.setField("uuid_nullable", uuid); | ||
| rec.setField("decimal", new BigDecimal("14.20")); | ||
| rec.setField("decimal_nullable", new BigDecimal("14.20")); | ||
| rec.setField("fixed", new byte[] {1, 2, 3, 4, 5, 6, 7}); |
There was a problem hiding this comment.
| rec.setField("fixed", new byte[] {1, 2, 3, 4, 5, 6, 7}); | |
| rec.setField("fixed", "abcd".getBytes(StandardCharsets.UTF_8)); |
| Types.NestedField.required(26, "decimal", Types.DecimalType.of(9, 2)), | ||
| Types.NestedField.optional(27, "decimal_nullable", Types.DecimalType.of(9, 2))); | ||
| Types.NestedField.optional(27, "decimal_nullable", Types.DecimalType.of(9, 2)), | ||
| Types.NestedField.required(28, "fixed", Types.FixedType.ofLength(7)), |
There was a problem hiding this comment.
nit: maybe also use different lengths for both fields
arrow/src/test/java/org/apache/iceberg/arrow/vectorized/TestArrowReader.java
Outdated
Show resolved
Hide resolved
arrow/src/test/java/org/apache/iceberg/arrow/vectorized/TestArrowReader.java
Outdated
Show resolved
Hide resolved
3 participants
No description provided.