[SPARK-33743]change TimestampType match to datetime2 instead of datetime for MsSQLServerDialect

[luxu1-ms]

SPARK-33743 is to change datetime datatype mapping in JDBC mssqldialect.

What changes were proposed in this pull request?

override def getJDBCType(dt: DataType): Option[JdbcType] = dt match {
case TimestampType => Some(JdbcType("DATETIME2", java.sql.Types.TIMESTAMP))

Why are the changes needed?

Spark datetime type is timestamp type. This supports a microsecond resolution.
Sql supports 2 date time types:

datetime can support only milli seconds resolution (0 to 999).
datetime2 is extension of datetime , is compatible with datetime and supports 0 to 9999999 sub second resolution.
datetime2 (Transact-SQL) - SQL Server | Microsoft Docs
datetime (Transact-SQL) - SQL Server | Microsoft Docs

Currently MsSQLServerDialect maps timestamp type to datetime. Datetime only allows 3 digits of microseconds. This implies results in errors when writing timestamp with more than 3 digits of microseconds to sql server table. We want to map timestamp to datetime2, which is compatible with datetime but allows 7 digits of microseconds.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Unit tests were updated and passed in JDBCSuit.scala.
E2E test done with SQL Server.

[AmplabJenkins]

Can one of the admins verify this patch?

[darrenpricedcww]

Hi Team,

Any ETA on getting this one complete?

Thanks,
Darren

[philipp-t]

we also have a lot of conflicts with Datetime2 in MSSQL, is their a plan if this fix is released soon??

[JochenMSFT]

when could we close / merge the PR?

[github-actions]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!