When a DataFile is built through the derived builder (DataFileBuilder) and no split_offsets are given, this is still serialised as [] instead of null.
E.g. converting the Avro file to JSON it looks something like this:
...
"split_offsets": {
"array": []
},I suspect this is because the value within the DataFile is a Vec<i64> instead of an Option<Vec<i64>>, so we get an empty Vec instead of nothing.
I think this Option is the correct approach because the spec also states that split_offsets are optional and it looks like they're optional in the Java implementation too.
When a
DataFileis built through the derived builder (DataFileBuilder) and nosplit_offsetsare given, this is still serialised as[]instead ofnull.E.g. converting the Avro file to JSON it looks something like this:
I suspect this is because the value within the
DataFileis aVec<i64>instead of anOption<Vec<i64>>, so we get an emptyVecinstead of nothing.I think this
Optionis the correct approach because the spec also states thatsplit_offsetsare optional and it looks like they're optional in the Java implementation too.