[SPARK-28957][SQL] Copy any "spark.hive.foo=bar" spark properties into hadoop conf as "hive.foo=bar"

[WeichenXu123]

What changes were proposed in this pull request?

Copy any "spark.hive.foo=bar" spark properties into hadoop conf as "hive.foo=bar"

Why are the changes needed?

Providing spark side config entry for hive configurations.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

UT.

[SparkQA]

Test build #110036 has finished for PR 25661 at commit 036035c.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #110038 has finished for PR 25661 at commit 2d8ea81.

• This patch fails build dependency tests.
• This patch merges cleanly.
• This patch adds no public classes.

[WeichenXu123]

Jenkins retest this please.

[SparkQA]

Test build #110078 has finished for PR 25661 at commit 2d8ea81.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds no public classes.

[felixcheung]

probably ok... I was a bit worry that existing app having spark.hive.* might now break, but probably low chance that is happening?

[SparkQA]

Test build #110086 has finished for PR 25661 at commit 9f6b4f0.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[WeichenXu123]

@cloud-fan What do you think ?
Should we override existing spark.hive config ?

[cloud-fan]

I like this proposal. It's much better than looking at all hive configs and re-implementing it in Spark. (see how InsertIntoHiveTable handles "hive.exec.dynamic.partition and hive.exec.dynamic.partition.mode).

@WeichenXu123 can you resolve the conflicts?

[cloud-fan]

Can we also update configuration.md#custom-hadoophive-configuration?

[SparkQA]

Test build #111280 has finished for PR 25661 at commit b0e80e7.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #111323 has finished for PR 25661 at commit a119a6c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[cloud-fan]

thanks, merging to master!

[dongjoon-hyun]

nit. This function description seems to become behind, but, PR looks good.
Thanks!