Skip to content

[BEAM-10224] Test group by and aggregation on DATE and TIME type #12120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
kennknowles merged 1 commit into from
Jul 8, 2020
Merged

[BEAM-10224] Test group by and aggregation on DATE and TIME type #12120

kennknowles merged 1 commit into from
Jul 8, 2020

Conversation

@robinyqiu
Copy link
Contributor

@robinyqiu robinyqiu commented Jun 29, 2020 •
edited
Loading

r: @TheNeuralBit @kennknowles

cc: @ZijieSong946 @apilloud @reuvenlax

Post-Commit Tests Status (on master branch)

Lang SDK Dataflow Flink Samza Spark Twister2
Go Build Status --- Build Status --- Build Status ---
Java Build Status Build Status
Build Status		 Build Status
Build Status
Build Status
Build Status		 Build Status Build Status
Build Status
Build Status		 Build Status
Python Build Status
Build Status
Build Status
Build Status		 Build Status
Build Status
Build Status		 Build Status
Build Status		 --- Build Status ---
XLang --- --- Build Status --- --- Build Status

Pre-Commit Tests Status (on master branch)

--- Java Python Go Website
Non-portable Build Status Build Status
Build Status		 Build Status Build Status
Portable --- Build Status --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

@robinyqiu robinyqiu changed the title [BEAM-10224] Test Aggregation on DATE and TIME type [BEAM-10224] Test group by and aggregation on DATE and TIME type Jun 29, 2020
@robinyqiu
Copy link
Contributor Author

robinyqiu commented Jun 29, 2020

retest this please

@robinyqiu
Copy link
Contributor Author

robinyqiu commented Jun 29, 2020

The new ZetaSQL DATE and TIME types use Beam logical types. Currently Beam SQL transforms operate on the base types of logical types. This produces incorrect result because input types are often expected. In addition, after #11074 values of input type are stored in Rows. I think we should leverage that change to avoid unnecessary conversion to base types for better performance, instead of forcing the use of base types.

@robinyqiu robinyqiu requested a review from kennknowles June 29, 2020 23:46
kennknowles
kennknowles approved these changes Jul 6, 2020
View reviewed changes
Copy link
Member

@kennknowles kennknowles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I just reviewed the diffs, which seem straightforward. If there is anything else I should be thinking about LMK.

kennknowles
kennknowles approved these changes Jul 8, 2020
View reviewed changes
Copy link
Member

@kennknowles kennknowles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this LGTM and no further comment, I'll merge.

@kennknowles kennknowles merged commit f65a187 into apache:master Jul 8, 2020
@robinyqiu robinyqiu deleted the date-time branch July 17, 2020 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kennknowles kennknowles kennknowles approved these changes

@TheNeuralBit TheNeuralBit Awaiting requested review from TheNeuralBit

Assignees

No one assigned

Labels

extensions java sql

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@robinyqiu @kennknowles