-
Notifications
You must be signed in to change notification settings - Fork 4.8k
HIVE-1360. It has been a long-standing request for UDFs to be able to #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
access parameter values. This not only enables significant performance improvement possibilities, it also allows for fundamentally richer behavior, such as allowing the output type of a UDF to depend on its inputs. The strategy in this diff is to introduce the notion of a ConstantObjectInspector, like a regular ObjectInspector except that it encapsulates a constant value and knows what this constant value is. These COIs are created through a factory method by ExprNodeConstantDesc during plan generation hence UDFs will be able to capture these constant values during the initialize phase. Furthermore, because these ConstantObjectInspectors are simply subinterfaces of ObjectInspector, UDFs which are not "constant-aware" receive ObjectInspectors which also implement the same interfaces they are used to, so no special handling needs to be done for existing UDFs. An example UDF which uses this new functionality is also included in this diff. NAMED_STRUCT is like STRUCT except that it also allows users to specify the *names* of the fields of the struct, something previously not possible because the names of the fields must be known at compile time.
access parameter values. This not only enables significant performance improvement possibilities, it also allows for fundamentally richer behavior, such as allowing the output type of a UDF to depend on its inputs. The strategy in this diff is to introduce the notion of a ConstantObjectInspector, like a regular ObjectInspector except that it encapsulates a constant value and knows what this constant value is. These COIs are created through a factory method by ExprNodeConstantDesc during plan generation hence UDFs will be able to capture these constant values during the initialize phase. Furthermore, because these ConstantObjectInspectors are simply subinterfaces of ObjectInspector, UDFs which are not "constant-aware" receive ObjectInspectors which also implement the same interfaces they are used to, so no special handling needs to be done for existing UDFs. An example UDF which uses this new functionality is also included in this diff. NAMED_STRUCT is like STRUCT except that it also allows users to specify the *names* of the fields of the struct, something previously not possible because the names of the fields must be known at compile time.
jhartlaub
referenced
this pull request
in jhartlaub/hive
May 6, 2013
Fixed NULL handling problem for timestamps, backport HIVE-4122
mattyb149
referenced
this pull request
in mattyb149/hive
May 7, 2013
Added support for Hive Server 1 and 2
jfota
pushed a commit
to mapr/hive
that referenced
this pull request
Oct 31, 2013
11490: Disable map join optimization for tables not managed by hive
chenchun
pushed a commit
to MTDATA/hive
that referenced
this pull request
Jan 26, 2014
* commit '4e37373a5631a3eb4c0932a5edffaea6441cb7b5': DATA-4457
asfgit
pushed a commit
that referenced
this pull request
Oct 23, 2014
#2 (Chao via Xuefu) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1633887 13f79535-47bb-0310-9956-ffa450edef68
asfgit
pushed a commit
that referenced
this pull request
Oct 23, 2014
#2 (Chao via Xuefu) git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1633911 13f79535-47bb-0310-9956-ffa450edef68
asfgit
pushed a commit
that referenced
this pull request
Oct 23, 2014
…n't guarantee order #2 (Chao via Xuefu) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1633913 13f79535-47bb-0310-9956-ffa450edef68
asfgit
pushed a commit
that referenced
this pull request
Oct 23, 2014
#2 (Chao via Xuefu) merged from trunk r1633911 git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1633916 13f79535-47bb-0310-9956-ffa450edef68
asfgit
pushed a commit
that referenced
this pull request
Oct 29, 2014
git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1635272 13f79535-47bb-0310-9956-ffa450edef68
asfgit
pushed a commit
that referenced
this pull request
Nov 26, 2014
…e order #2 (Chao via Xuefu) git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1641830 13f79535-47bb-0310-9956-ffa450edef68
asfgit
pushed a commit
that referenced
this pull request
Dec 19, 2014
git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1646883 13f79535-47bb-0310-9956-ffa450edef68
asfgit
pushed a commit
that referenced
this pull request
Jul 21, 2017
…W__ID struct (Matt McCline, reviewed by Teddy Choi) try #2
omalley
referenced
this pull request
in omalley/hive
Nov 9, 2017
SirOibaf
referenced
this pull request
in SirOibaf/hive
Apr 3, 2019
…W__ID struct (Matt McCline, reviewed by Teddy Choi) try logicalclocks#2
SirOibaf
referenced
this pull request
in SirOibaf/hive
Apr 8, 2019
…W__ID struct (Matt McCline, reviewed by Teddy Choi) try logicalclocks#2
kristopherkane
added a commit
to kristopherkane/hive
that referenced
this pull request
Jun 18, 2019
leongu-tc
pushed a commit
to leongu-tc/hive
that referenced
this pull request
Jul 27, 2019
This was referenced Sep 8, 2019
szlta
added a commit
to szlta/hive
that referenced
this pull request
Mar 23, 2020
Change-Id: If7500706a9960789bb79d302cb2a0bb6e91150bc
szlta
added a commit
to szlta/hive
that referenced
this pull request
Apr 6, 2020
Change-Id: If7500706a9960789bb79d302cb2a0bb6e91150bc
szlta
added a commit
to szlta/hive
that referenced
this pull request
Apr 9, 2020
Change-Id: If7500706a9960789bb79d302cb2a0bb6e91150bc
szlta
added a commit
to szlta/hive
that referenced
this pull request
Apr 14, 2020
Change-Id: If7500706a9960789bb79d302cb2a0bb6e91150bc
szlta
added a commit
to szlta/hive
that referenced
this pull request
Apr 17, 2020
Change-Id: If7500706a9960789bb79d302cb2a0bb6e91150bc
Contributor
|
Issue fixed through manual patch process: |
szlta
added a commit
to szlta/hive
that referenced
this pull request
Jul 31, 2020
Change-Id: I80a95a9c3b7eda49abfa3a8f7f5aaf15ca3fc4d6
InvisibleProgrammer
pushed a commit
to InvisibleProgrammer/hive
that referenced
this pull request
Sep 16, 2023
# This is the 1st commit message: Start to refactor Comment out classes to work on to be able to build the module Refator TestCopyUtils.java Before refactor: TestReplDumpTask Refactor TestReplDumpTask.java Before refactor: TestAtlasLoadTask Refactor TestAtlasLoadTask.java Refator TestRangerDumpTask.java Before refactor: TestCompactionHeartbeatService Refactor TestCompactionHeartbeatService.java Before refactor: TestRetryable Refactor TestRetryable.java Before refactor: TestRangerLoadTask Refactor: RangerLoadTask Before refactor: TestAtlasDumpTask Refactor: AtlasDumpTask Before refactor: TestPrimaryToReplicaResourceFunction Refactor TestPrimaryToReplicaResourceFunction Before refactor: TestExportService Refactor: TestExportService HIVE-26522: Added test for HIVE-22033 regarding delegation token renewal (apache#3585) HIVE-26676: Count distinct in subquery returning wrong results (Steve Carlin, reviewed by Alessandro Solimando, Aman Sinha, Krisztian Kasa) HIVE-26736: Authorization failure for nested Views having WITH clause. (apache#3760). (Ayush Saxena, reviewed by Denys Kuzmenko) HIVE-26628: Iceberg table is created when running explain ctas command (Krisztian Kasa, reviewed by Denys Kuzmenko) HIVE-26734: Iceberg: Add an option to allow positional delete files without actual row data. (apache#3758). (Ayush Saxena, reviewed by Adam Szita, Denys Kuzmenko) HIVE-26524: Use Calcite to remove sections of a query plan known never produces rows - ADDENDUM (Krisztian Kasa, reviewed by Stamatis Zampetakis) HIVE-26740: HS2 makes direct connections to HMS backend DB due to Compaction/StatsUpdater (apache#3765) (Adam Szita, reviewed by Zhihua Deng) HIVE-26631: Remove unused Thrift config parameters login.timeout and exponential.backoff.slot.length (xiuzhu9527 reviewed by Stamatis Zampetakis) Closes apache#3672 HIVE-26747: Remove implementor from HiveRelNode (Krisztian Kasa, reviewed by Stamatis Zampetakis) HIVE-26747: Remove implementor from HiveRelNode (Krisztian Kasa, reviewed by Stamatis Zampetakis) ADDENDUM HIVE-26745: HPL unable to handle Decimal or null values in hplsql mode (apache#3769) (Adam Szita, reviewed by Attila Magyar and Denys Kuzmenko) HIVE-26722: HiveFilterSetOpTransposeRule incorrectly prunes UNION ALL operands. (apache#3748). (Alessandro Solimando, reviewed by Ayush Saxena, Simhadri Govindappa) HIVE-26746: Request tracking: change to X-Request-ID header (apache#3770) (Laszlo Bodor reviewed by Zhihua Deng) HIVE-26624: Set repl.background.enable on target after failover completion (Vinit Patni, reviewed by László Pintér, Teddy Choi) Co-authored-by: vpatni <vpatni@cloudera.com> HIVE-26712: HCatMapReduceTest writes test files in project base directory instead of build directory. (apache#3738) (Chris Nauroth reviewed by Ayush Saxena) HIVE-26726: Tinyint column with windowing fn crashes at runtime (Steve Carlin, reviewed by Aman Sinha, Krisztian Kasa) HIVE-26680: Make CMV use Direct Insert Semantics (Sourabh Badhya, reviewed by Denys Kuzmenko, Laszlo Vegh) Closes apache#3715 HIVE-26243: Add vectorized implementation of the 'ds_kll_sketch' UDAF (Alessandro Solimando, reviewed by Denys Kuzmenko, Zoltan Haindrich) Closes apache#3317 HIVE-26761: Add result sorting to complex_alias.q (apache#3783) (Balazs Cseh reviewed by Laszlo Bodor) HIVE-26759: Update SHOW COMPACTIONS query to support Postgres HMS (Akshat Mathur, reviewed by Denys Kuzmenko, Zsolt Miskolczi) Closes 3782 HIVE-26765: Hive Ranger URL policy for insert overwrite directory denies access when fully qualified paths are passed (apache#3790) (Simhadri Govindappa, reviewed by Adam Szita) Small refactors Fix bug # This is the commit message apache#2: Remove uneccessary constructor
kasakrisz
referenced
this pull request
in kasakrisz/hive
Dec 8, 2023
kasakrisz
referenced
this pull request
in kasakrisz/hive
Dec 11, 2023
mladjan-gadzic
pushed a commit
to mladjan-gadzic/hive
that referenced
this pull request
May 14, 2024
Co-authored-by: George Jahad <george@georgejahad.com>
mudit1289
pushed a commit
to mudit1289/hive
that referenced
this pull request
Jun 26, 2024
FDG-10213:- backporting HIVE-21215
difin
pushed a commit
to difin/hive
that referenced
this pull request
Sep 6, 2024
difin
pushed a commit
to difin/hive
that referenced
this pull request
Sep 6, 2024
kasakrisz
referenced
this pull request
in kasakrisz/hive
Dec 6, 2024
kasakrisz
referenced
this pull request
in kasakrisz/hive
Dec 7, 2024
kasakrisz
referenced
this pull request
in kasakrisz/hive
Dec 9, 2024
kasakrisz
referenced
this pull request
in kasakrisz/hive
Jan 8, 2025
difin
pushed a commit
to difin/hive
that referenced
this pull request
Mar 27, 2025
difin
pushed a commit
to difin/hive
that referenced
this pull request
Mar 28, 2025
difin
pushed a commit
to difin/hive
that referenced
this pull request
Mar 28, 2025
deniskuzZ
added a commit
to ayushtkn/hive
that referenced
this pull request
Sep 10, 2025
InvisibleProgrammer
pushed a commit
to InvisibleProgrammer/hive
that referenced
this pull request
Nov 25, 2025
InvisibleProgrammer
pushed a commit
to InvisibleProgrammer/hive
that referenced
this pull request
Jan 9, 2026
InvisibleProgrammer
pushed a commit
to InvisibleProgrammer/hive
that referenced
this pull request
Jan 11, 2026
InvisibleProgrammer
pushed a commit
to InvisibleProgrammer/hive
that referenced
this pull request
Jan 19, 2026
InvisibleProgrammer
pushed a commit
to InvisibleProgrammer/hive
that referenced
this pull request
Jan 23, 2026
InvisibleProgrammer
pushed a commit
to InvisibleProgrammer/hive
that referenced
this pull request
Jan 27, 2026
InvisibleProgrammer
pushed a commit
to InvisibleProgrammer/hive
that referenced
this pull request
Jan 27, 2026
InvisibleProgrammer
pushed a commit
to InvisibleProgrammer/hive
that referenced
this pull request
Feb 2, 2026
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
access parameter values. This not only enables significant performance
improvement possibilities, it also allows for fundamentally richer
behavior, such as allowing the output type of a UDF to depend on its
inputs.
The strategy in this diff is to introduce the notion of a
ConstantObjectInspector, like a regular ObjectInspector except that it
encapsulates a constant value and knows what this constant value is.
These COIs are created through a factory method by ExprNodeConstantDesc
during plan generation hence UDFs will be able to capture these constant
values during the initialize phase. Furthermore, because these
ConstantObjectInspectors are simply subinterfaces of ObjectInspector,
UDFs which are not "constant-aware" receive ObjectInspectors which
also implement the same interfaces they are used to, so no special
handling needs to be done for existing UDFs.
An example UDF which uses this new functionality is also included in
this diff. NAMED_STRUCT is like STRUCT except that it also allows users
to specify the names of the fields of the struct, something previously
not possible because the names of the fields must be known at compile
time.