feat(duckdb): preserve time zone and timestamp in DATE_TRUNC()

[toriwei]

BigQuery's TIMESTAMP_TRUNC() maps to DuckDB's DATE_TRUNC()

BigQuery's function can be used like TIMESTAMP_TRUNC(timestamp, date_granularity, timezone), which transpiles incorrectly into DuckDB.

BigQuery:
SELECT TIMESTAMP_TRUNC(TIMESTAMP '2024-03-15 14:35:47.123456', DAY, 'America/New_York') AS trunc_day_ny
Result: 2024-03-15 04:00:00 # BigQuery's results are shown in UTC.

Current DuckDB transpilation/execution:
D SET TimeZone = 'UTC'; 
D SELECT DATE_TRUNC('DAY', CAST('2024-03-15 14:35:47.123456' AS TIMESTAMPTZ)) AS trunc_day_ny;
Result: 2024-03-15 00:00:00+00 

BigQuery Behavior

• converts UTC timestamp to given time_zone
• truncates timestamp to start of date granularity
• converts timestamp back to UTC

DuckDB Changes
The inner AT TIME ZONE ensures datetime is in the specified timezone before truncating.

D SET TimeZone = 'UTC';
D SELECT DATE_TRUNC('DAY', CAST('2024-03-15 14:35:47.123456' AS TIMESTAMPTZ) AT TIME ZONE 'America/New_York') as trunc_day_ny;
┌──────────────┐
│ trunc_day_ny │
│     date     │
├──────────────┤
│ 2024-03-15   │
└──────────────┘

We add the outerAT TIME ZONE to preserve the timestamp component.

D SET TimeZone = 'UTC';
D SELECT DATE_TRUNC('DAY', CAST('2024-03-15 14:35:47.123456' AS TIMESTAMPTZ) AT TIME ZONE 'America/New_York') AT TIME ZONE 'America/New_York' as trunc_day_ny;
┌──────────────────────────┐
│       trunc_day_ny       │
│ timestamp with time zone │
├──────────────────────────┤
│ 2024-03-15 04:00:00+00   │
└──────────────────────────┘

[georgesittas]

@toriwei some of these tests seem to be failing because of a time zone discrepancy:

(sqlmesh) ➜  playground bq query "SELECT TIMESTAMP '2024-03-15 14:35:00'"
+---------------------+
|         f0_         |
+---------------------+
| 2024-03-15 14:35:00 |
+---------------------+
(sqlmesh) ➜  playground sqlglot --read bigquery --write duckdb "SELECT TIMESTAMP '2024-03-15 14:35:00'"
SELECT CAST('2024-03-15 14:35:00' AS TIMESTAMPTZ)
(sqlmesh) ➜  playground duckdb -c "SELECT CAST('2024-03-15 14:35:00' AS TIMESTAMPTZ)"
┌─────────────────────────────────────────────────────────┐
│ CAST('2024-03-15 14:35:00' AS TIMESTAMP WITH TIME ZONE) │
│                timestamp with time zone                 │
├─────────────────────────────────────────────────────────┤
│ 2024-03-15 14:35:00+02                                  │
└─────────────────────────────────────────────────────────┘

BigQuery returns 14:35:00 which is UTC but DuckDB returns 14:35:00+02 which is in the local time zone. I'm in UTC+2, hence the difference; you may observe different results.

[georgesittas]

Great work @toriwei, thanks for improving the test coverage!