Skip to content

Conversation

@Abacn
Copy link
Contributor

@Abacn Abacn commented Dec 2, 2022

Fixes #24415

Reverts #23969

#23969 introduced

  • a one direction mapping from LogicalType(Date) to Avro Logical Type Data
  • and a bidirectional mapping between LoficalType(SqlTypes.TIMESTAMP) Avro Logical type TimestampMicros

However, SqlTypes.TIMESTAMP (portable micros_instant) is not yet supported in Beam SQL (though its comments say Beam LogicalType corresponding to ZetaSQL TIMESTAMP type, which is not true yet)

@github-actions github-actions bot added the java label Dec 2, 2022
@Abacn
Copy link
Contributor Author

Abacn commented Dec 2, 2022

R: @apilloud
CC: @kennknowles

@github-actions
Copy link
Contributor

github-actions bot commented Dec 2, 2022

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

@Abacn
Copy link
Contributor Author

Abacn commented Dec 2, 2022

This is a regression of Beam 2.44.0. Though It should be safe to make mapping of LoficalType(SqlTypes.TIMESTAMP) -> Avro Logical type TimestampMicros one directional as Date, however, defer this to future releases.

@Abacn
Copy link
Contributor Author

Abacn commented Dec 2, 2022

R: @robertwb

@Abacn
Copy link
Contributor Author

Abacn commented Dec 2, 2022

Java Precommit fails known flakes

org.apache.beam.runners.samza.runtime.AsyncDoFnRunnerTest.testPipelineWithState
org.apache.beam.sdk.io.pulsar.PulsarIOTest.testReadFromSimpleTopic

#21714; #23745 (same test class) not relevant to this PR

@Abacn
Copy link
Contributor Author

Abacn commented Dec 2, 2022

Kotlin example actually passed but not updated to GitHub UI: https://ci-beam.apache.org/job/beam_PreCommit_Kotlin_Examples_Commit/2800/

Will get this in and need a cherry pick to release-2.44.0 branch

@Abacn Abacn merged commit d873220 into master Dec 2, 2022
@Abacn Abacn deleted the revert-23969-sqlttypes branch December 2, 2022 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT

2 participants