[Bug]: JDBC javasdk_date:v1 decode error

[ffernandez92]

What happened?

When using the ReadFromJdbc transform in Beam YAML, I encountered a ValueError with the following stack trace:

ValueError: Error apply transform "ReadFromJdbc" at line 4: Failed to decode schema due to an issue with Field proto:
2024-12-23 16:03:49.144 ∅∅∅
name: "day"
2024-12-23 16:03:49.144 ∅∅∅
type {
2024-12-23 16:03:49.144 ∅∅∅
logical_type {
2024-12-23 16:03:49.144 ∅∅∅
urn: "beam:logical_type:javasdk_date:v1"
2024-12-23 16:03:49.144 ∅∅∅
payload: "\202SNAPPY\000\000\000\000\001\000\000\000..."
2024-12-23 16:03:49.145 ∅∅∅
representation {
2024-12-23 16:03:49.145 ∅∅∅
logical_type {

The error seems to be related to decoding a schema field of type logical_type with the URN beam:logical_type:javasdk_date:v1

Steps to reproduce:

1 - Create a YAML to read from JDBC:

pipeline:
  type: chain
  transforms:
    - type: ReadFromJdbc
      config:
        url: trino_connection
        driver_class_name: io.trino.jdbc.TrinoDriver
        query: SELECT day FROM test_table
        driver_jars: gs://trino_jdbc.jar
        username: xxxxx
        password: xxxxxx

day is type DATE (91) in Trino

2 - Run the pipeline.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[Abacn]

Jdbc data time type has been a long standing issue in Python JdbcIO. A workaround could be cast to string in the query

SELECT CAST(day AS VARCHAR(20)); FROM test_table

and use a Map to transfer to python date type downstream

[gcarson30]

I'm running into this same issue using the Dataflow flex templates, even though it works with the DirectRunner. It would be nice to have a better solution.

[mataralhawiti]

@gcarson30 what Beam version are you running? are you using cross-language transform?
I'm hitting this issue again after I upgraded to 2.65.0, and it's happing with 2.66.0.
The same exact code would run without any issue 2.64.0

[Abacn]

cc: @shunping

[shunping]

@gcarson30 what Beam version are you running? are you using cross-language transform? I'm hitting this issue again after I upgraded to 2.65.0, and it's happing with 2.66.0. The same exact code would run without any issue 2.64.0

My recent changes on logical type were pushed after 2.65.0 was released. So there could be due to a different issue here.

[shunping]

I am able to reproduce the problem using a simple yaml file as #33442 (comment), even on 2.64.0. Working on a fix right now.