Skip to content

Use MillisInstant by default when Python JdbcIO is used #29182

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Abacn wants to merge 1 commit into from
Closed

Use MillisInstant by default when Python JdbcIO is used #29182

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions sdks/python/apache_beam/io/external/xlang_jdbcio_it_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,6 @@
from apache_beam.testing.test_pipeline import TestPipeline
from apache_beam.testing.util import assert_that
from apache_beam.testing.util import equal_to
from apache_beam.typehints.schemas import LogicalType
from apache_beam.typehints.schemas import MillisInstant
from apache_beam.utils.timestamp import Timestamp

# pylint: disable=wrong-import-order, wrong-import-position, ungrouped-imports
Expand Down Expand Up @@ -189,10 +187,6 @@ def test_xlang_jdbc_write_read(self, database):
classpath=classpath,
))

# Register MillisInstant logical type to override the mapping from Timestamp
# originally handled by MicrosInstant.
LogicalType.register_logical_type(MillisInstant)

with TestPipeline() as p:
p.not_use_test_runner_api = True
result = (
Expand Down
7 changes: 7 additions & 0 deletions sdks/python/apache_beam/io/jdbc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,9 @@ def __init__(
expansion_service or default_io_expansion_service(classpath),
)

# TODO(https://github.com/apache/beam/issues/28359) The following logical type
# definitions and registrations are workaround for #28359. Remove them when
# switched to portable Data and Time type for JdbcIO.

@LogicalType.register_logical_type
class JdbcDateType(LogicalType[datetime.date, MillisInstant, str]):
Expand Down Expand Up @@ -455,3 +458,7 @@ def argument(self):
@classmethod
def _from_typing(cls, typ):
return cls()

# Register MillisInstant logical type to override the mapping from Timestamp
# originally handled by MicrosInstant.
LogicalType.register_logical_type(MillisInstant)