[WIP] Vectorized Reads using Arrow#313
[WIP] Vectorized Reads using Arrow#313prodeezy wants to merge 15 commits intoapache:vectorized-readfrom
Conversation
…rizedSparkParquetReaders.
|
Hey @prodeezy, thanks for updating and creating the pull request. Looks like the build is breaking and while we don't need exhaustive reviews, it would be nice to keep the branch in a buildable state. I pulled it down and it looks like the dependency locks are preventing. Could you just add the updated version locks to pull in arrow (i.e. There are a number of checkstyle violations, but I think we can look past those for now (unless you want to tackle them with the version locks). |
|
will do. i have a different branch "vectorized-read" in my local repo with latest code from master merged in. will update locks and fix checkstyle errors and create another pr. closing this for now. sorry bout the confusion. |
|
raised #319 as a replacement with the updated version locks and checkstyle errors fixed. |
Just a WIP POC right now so is not expected to be merged. This is just for comments.
Things I intend to add:
Iceberg's Reader needs to choose Row-wise or Vectorized reading based on config.
issue#311
Update: #319 is the new PR