-
Notifications
You must be signed in to change notification settings - Fork 4k
ARROW-10924: [C++] Validate temporal data in ValidateArrayFull #12014
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
359b29e
Adding validation for Date64Type, Time32Type, and Time64Type
JabariBooker fd2e08d
Merge branch 'master' of https://github.com/apache/arrow
JabariBooker b13946b
Corrected type usage for Time64Type check
JabariBooker bd1209e
Resolving formatting issues and adding constexpr's
JabariBooker aeedda5
Using kCamelCase for constants
JabariBooker 9400fbb
Renaming time constants
JabariBooker ec369c4
Merge branch 'master' into ARROW-10924
68f62a5
Merge branch 'master' into ARROW-10924
JabariBooker f68c2a8
Moving constexpr's and making small grammatical edits
JabariBooker 827b6a4
More grammar corrections and updating array test constant for Date64S…
JabariBooker 4ace496
Added random generation handling for temporal types
JabariBooker b7b9d16
Merge branch 'master' into ARROW-10924
JabariBooker 36c3dfc
Formatting code properly
JabariBooker 6d7189b
Changing test constants to reflect new restrictions
JabariBooker d8be683
Added new test buffers for date64
JabariBooker 997c7a1
Fixed random number generation and constants for testing date64
JabariBooker e409c73
Changes to the last unit tests for date64 restrictions
JabariBooker 27411f2
Missing changes to constant from last commit
JabariBooker 75febd1
Updated data generation for integration tests; other minor changes
JabariBooker e1ba73a
Added flag for date64 validation in integration tests, set to false
JabariBooker cc7192f
Added integration test flag for TIME32/64 types
JabariBooker File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1616,14 +1616,12 @@ TEST(GroupBy, MinMaxTypes) { | |
| types.insert(types.end(), NumericTypes().begin(), NumericTypes().end()); | ||
| types.insert(types.end(), TemporalTypes().begin(), TemporalTypes().end()); | ||
| types.push_back(month_interval()); | ||
| for (const auto& ty : types) { | ||
| SCOPED_TRACE(ty->ToString()); | ||
| auto in_schema = schema({field("argument0", ty), field("key", int64())}); | ||
| auto table = TableFromJSON(in_schema, {R"([ | ||
|
|
||
| const std::vector<std::string> default_table = {R"([ | ||
| [1, 1], | ||
| [null, 1] | ||
| ])", | ||
| R"([ | ||
| R"([ | ||
| [0, 2], | ||
| [null, 3], | ||
| [3, 4], | ||
|
|
@@ -1632,11 +1630,54 @@ TEST(GroupBy, MinMaxTypes) { | |
| [3, 1], | ||
| [0, 2] | ||
| ])", | ||
| R"([ | ||
| R"([ | ||
| [0, 2], | ||
| [1, null], | ||
| [null, 3] | ||
| ])"}); | ||
| ])"}; | ||
|
|
||
| const std::vector<std::string> date64_table = {R"([ | ||
| [86400000, 1], | ||
| [null, 1] | ||
| ])", | ||
| R"([ | ||
| [0, 2], | ||
| [null, 3], | ||
| [259200000, 4], | ||
| [432000000, 4], | ||
| [345600000, null], | ||
| [259200000, 1], | ||
| [0, 2] | ||
| ])", | ||
| R"([ | ||
| [0, 2], | ||
| [86400000, null], | ||
| [null, 3] | ||
| ])"}; | ||
|
|
||
| const std::string default_expected = | ||
| R"([ | ||
| [{"min": 1, "max": 3}, 1], | ||
| [{"min": 0, "max": 0}, 2], | ||
| [{"min": null, "max": null}, 3], | ||
| [{"min": 3, "max": 5}, 4], | ||
| [{"min": 1, "max": 4}, null] | ||
| ])"; | ||
|
|
||
| const std::string date64_expected = | ||
| R"([ | ||
| [{"min": 86400000, "max": 259200000}, 1], | ||
| [{"min": 0, "max": 0}, 2], | ||
| [{"min": null, "max": null}, 3], | ||
| [{"min": 259200000, "max": 432000000}, 4], | ||
| [{"min": 86400000, "max": 345600000}, null] | ||
| ])"; | ||
|
|
||
| for (const auto& ty : types) { | ||
| SCOPED_TRACE(ty->ToString()); | ||
| auto in_schema = schema({field("argument0", ty), field("key", int64())}); | ||
| auto table = | ||
| TableFromJSON(in_schema, (ty->name() == "date64") ? date64_table : default_table); | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: can check |
||
|
|
||
| ASSERT_OK_AND_ASSIGN( | ||
| Datum aggregated_and_grouped, | ||
|
|
@@ -1652,13 +1693,7 @@ TEST(GroupBy, MinMaxTypes) { | |
| field("hash_min_max", struct_({field("min", ty), field("max", ty)})), | ||
| field("key_0", int64()), | ||
| }), | ||
| R"([ | ||
| [{"min": 1, "max": 3}, 1], | ||
| [{"min": 0, "max": 0}, 2], | ||
| [{"min": null, "max": null}, 3], | ||
| [{"min": 3, "max": 5}, 4], | ||
| [{"min": 1, "max": 4}, null] | ||
| ])"), | ||
| (ty->name() == "date64") ? date64_expected : default_expected), | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: ditto here |
||
| aggregated_and_grouped, | ||
| /*verbose=*/true); | ||
| } | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.