Describe the bug
After #474, ColumnReader.loadVector initiates CometDictionary after re-import arrays. It is reported by @andygrove (#473 (comment)) that after the patch, there is obvious regression when comparing with Spark.
It is because, although re-importing dictionary values is fast, CometDictionary has an initialization step to copy values from imported dictionary values to an internal Java array. We need to re-use previous CometDictionary's copied Java arrays.
Steps to reproduce
No response
Expected behavior
No response
Additional context
No response
Describe the bug
After #474,
ColumnReader.loadVectorinitiatesCometDictionaryafter re-import arrays. It is reported by @andygrove (#473 (comment)) that after the patch, there is obvious regression when comparing with Spark.It is because, although re-importing dictionary values is fast,
CometDictionaryhas an initialization step to copy values from imported dictionary values to an internal Java array. We need to re-use previousCometDictionary's copied Java arrays.Steps to reproduce
No response
Expected behavior
No response
Additional context
No response