[SQL] [SPARK-8064] Fixes HiveSparkSubmitSuite for Hive 1.2.1 upgrade #7868
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…sses into a hive package
…ts pass instead of timing out: undetected regression?
…naryCLIService on port"
…ve dependency of hive-exec
…ependency of hive-exec
…essages to say "w/Hive 1.2.1" in console output
…in the raised exception; this ensures that the stdout/stderr is included in jenkins reports, so it becomes possible to diagnose the cause.
…e rest of the work; can be backported to earlier versions
…TOK_UNIONDISTINCT
…ure that jersey server is on classpath on hadoop versions < 2.6
…till no obvious reason why jersey server code in spark-assembly isn't being picked up -it hasn't been shaded
…nd/or error string patterns seen in log.
…ight to map of serde options
…nings. It's all autogenerated code, so still ugly.
-things that aren't needed are excluded -sql/hive built with ivy is in sync with the maven reconciliation policy, rather than latest-first
…ed references in the hive- packages; this keeps mvn and ivy resolution compatible, as the reconciliation policy is "by hand"
… that automatically.
… This one has the protobuf-shaded hive-exec jar
…inal (?) hive-exec spark
Contributor
Author
|
Enumerating all time |
liancheng
force-pushed
the
hive-1.2.1/hiveconf-timevar-fix
branch
from
e4556c1 to
6a92bb0
Compare
Contributor
Author
|
@steveloughran Changing default values of these |
|
Test build #39427 has finished for PR 7868 at commit
|
|
Test build #39428 has finished for PR 7868 at commit
|
Contributor
Author
|
Closing this since it's merged into PR #7191. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a fork of PR #7191 authored by @steveloughran. This PR fixes
HiveSparkSubmitSuitetest failures.Hive 0.14.0+ adds time suffixes (
s,ms, andd, etc.) to the original default values of a bunch ofHiveConfoptions that define various timeouts (see HIVE-5799). This failsHiveContextinitialization when we are connecting to a lower version of Hive (e.g., 0.12.0 inHiveSparkSubmitSuite), because these options are expected to be integral values in lower versions of Hive.This PR enumerates all such options and converts their values to corresponding
Longvalues according to their output time unit before passing them to the metastore Hive client of lower versions.Please refer to this failed Jenkins build for more details about the failed test case.