-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-17773] [Input/Output] Add VoidObjectInspector #15337
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
ok to test |
|
@seyfe This seems like a good idea. Could you add a little test to see if this is working? |
|
Test build #66290 has finished for PR 15337 at commit
|
|
@hvanhovell. Thanks for the suggestion. Updated the HiveInspectorSuite, so 3 tests failed with below error: After applying the fix, all HiveInspectorSuite tests passed. |
|
Test build #66299 has finished for PR 15337 at commit
|
|
Thanks - merging in master/2.0. |
|
Actually there is a conflict with back porting this into branch-2.0. Do you mind making a pull request against branch-2.0 too? |
Added VoidObjectInspector to the list of PrimitiveObjectInspectors (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) Executing following query was failing. select SOME_UDAF*(a.arr) from ( select Array(null) as arr from dim_one_row ) a After the fix, I am getting the correct output: res0: Array[org.apache.spark.sql.Row] = Array([null]) Author: Ergin Seyfe <eseyfe@fb.com> Closes apache#15337 from seyfe/add_void_object_inspector.
This is the PR for branch2.0: PR #15337 Added VoidObjectInspector to the list of PrimitiveObjectInspectors (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) Executing following query was failing. select SOME_UDAF*(a.arr) from ( select Array(null) as arr from dim_one_row ) a After the fix, I am getting the correct output: res0: Array[org.apache.spark.sql.Row] = Array([null]) Author: Ergin Seyfe <eseyfefb.com> Closes #15337 from seyfe/add_void_object_inspector. Author: Ergin Seyfe <eseyfe@fb.com> Closes #15345 from seyfe/add_void_object_inspector_2.0.
What changes were proposed in this pull request?
Added VoidObjectInspector to the list of PrimitiveObjectInspectors
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
Executing following query was failing.
select SOME_UDAF*(a.arr)
from (
select Array(null) as arr from dim_one_row
) a
After the fix, I am getting the correct output:
res0: Array[org.apache.spark.sql.Row] = Array([null])