BigQuery timestamp parsing regression (are restrictive timestamp parsing changes intentional?)

[TJC]

Environment details

• OS: Linux (Ubuntu Bionic and Alpine 3.8)
• Java version: 11.0.2 and 8.191.12
• google-cloud-java version(s): 1.62.0

Overview of issue

In commit fc8438b the BigQuery date parsing format string was changed from
yyyy-MM-dd HH:mm:ss.SSSSSSZZ to
yyyy-MM-dd HH:mm:ss.SSSSSSxxx

This has broken backwards compatibility in two ways.

• with strings that were using Z or +0000 for the timezone component.
• with strings that were using less than six decimal places of sub-second accuracy

Was this change desirable? I don't understand why you would intentionally make the timestamp parsing as restrictive as this.

In the code sample below, all the example date strings will parse fine in the older BigQuery SDK versions, but fail in the current version.

Code sample

(code is in Scala but that isn't relevant to the issue)

val ourTime = "2019-02-14 12:34:45.938993Z"
val ourTime2 = "2019-02-14 12:34:45.938993+0000"
val ourTime3 = "2019-02-14 12:34:45.102+00:00"

val jodaFormat =
  org.joda.time.format.DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss.SSSSSSZZ")

val threetenFormat =
  org.threeten.bp.format.DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.SSSSSSxxx")


jodaFormat.parseDateTime(ourTime)
jodaFormat.parseDateTime(ourTime2)
jodaFormat.parseDateTime(ourTime3)
// success!

threetenFormat.parse(ourTime)
// fails!
threetenFormat.parse(ourTime2)
// fails!
threetenFormat.parse(ourTime3)
// fails!

[sduskis]

Sorry about that. We converted from Joda to threeten.bp, not just yyyy-MM-dd HH:mm:ss.SSSSSSZZ to
yyyy-MM-dd HH:mm:ss.SSSSSSxxx. It looks like we needed more unit tests before the conversion.

I'll see if I can figure out how to make threeten behave more like joda (although any help would be greatly appreciated).

[sduskis]

@ajaaym, help would be appreciated.

[ajaaym]

@sduskis working on this.