Skip to content

ARROW-6243: [C++][Dataset] Filter expressions #5157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bkietz wants to merge 28 commits into from
Closed

ARROW-6243: [C++][Dataset] Filter expressions #5157

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
f316578
add basic filter expressions
bkietz Aug 13, 2019
6ccd636
add execution of filter expressions using compute kernels
bkietz Aug 13, 2019
9282284
simplify filter testing
bkietz Aug 14, 2019
a67b330
add comments, more tests, simplify operator overloads
bkietz Aug 14, 2019
06b25be
move expression evaluation to a free function
bkietz Aug 14, 2019
3499fd2
add an expression simplification test
bkietz Aug 14, 2019
3d355ff
break up assumption logic, add function expression factories
bkietz Aug 15, 2019
55aa452
implement more robust null handling
bkietz Aug 15, 2019
60d9e08
fix factory linkage, factory fns deal in shared_ptrs exclusively
bkietz Aug 15, 2019
523186d
add support for evaluation of trivial expressions, tests
bkietz Aug 21, 2019
02d94a6
use explicit enumeration of comparison results
bkietz Aug 21, 2019
5899067
break OperatorExpression into multiple classes
bkietz Aug 21, 2019
e9af935
lint fixes
bkietz Aug 22, 2019
f9e0c08
remove unused Empty() method
bkietz Aug 22, 2019
5bae590
re-enable Invert
bkietz Aug 22, 2019
cb56906
rename FieldRef -> Field, and_ -> all, or_ -> any, add comments to Ex…
bkietz Aug 22, 2019
d7f3ed2
simplify Expression::Equals
bkietz Aug 22, 2019
24323fc
implement NotExpression::ToString
bkietz Aug 22, 2019
d54ca06
Expressions evaluate to Datums
bkietz Aug 22, 2019
46fa3fb
add Expression::Validate implementations
bkietz Aug 22, 2019
cab17b1
amend doccomments
bkietz Aug 25, 2019
3ac299b
use strongly typed nulls
bkietz Aug 25, 2019
7c01f76
construct correct scalartype
bkietz Aug 25, 2019
ca155c6
add explicit std::move, msvc doesn't like defining operator and
bkietz Aug 26, 2019
0e366bb
rename all/any to and/or
bkietz Aug 28, 2019
9494bab
refactor And, Or to binary
bkietz Sep 2, 2019
539c287
rename fieldRef to field_ref, comments
bkietz Sep 2, 2019
fda4742
give MSVC a little help to avoid instantiating impossible constructors
bkietz Sep 9, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cpp/build-support/run_cpplint.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
-whitespace/comments
-readability/casting
-readability/todo
-readability/alt_tokens
-build/header_guard
-build/c++11
-runtime/references
Expand Down
5 changes: 4 additions & 1 deletion cpp/src/arrow/compute/kernels/compare.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,10 @@ Status Compare(FunctionContext* context, const Datum& left, const Datum& right,
DCHECK(out);

auto type = left.type();
DCHECK(type->Equals(right.type()));
if (!type->Equals(right.type())) {
return Status::TypeError("Cannot compare data of differing type ", *type, " vs ",
*right.type());
}
// Requires that both types are equal.
auto fn = MakeCompareFunction(context, *type, options);
if (fn == nullptr) {
Expand Down
33 changes: 17 additions & 16 deletions cpp/src/arrow/dataset/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ arrow_install_all_headers("arrow/dataset")
# pkg-config support
arrow_add_pkg_config("arrow-dataset")

set(ARROW_DATASET_SRCS dataset.cc file_base.cc scanner.cc)
set(ARROW_DATASET_SRCS dataset.cc file_base.cc filter.cc scanner.cc)
set(ARROW_DATASET_LINK_STATIC arrow_static)
set(ARROW_DATASET_LINK_SHARED arrow_shared)

Expand Down Expand Up @@ -79,22 +79,23 @@ function(ADD_ARROW_DATASET_TEST REL_TEST_NAME)
set(LABELS "arrow_dataset")
endif()

if(NOT WIN32)
add_arrow_test(${REL_TEST_NAME}
EXTRA_LINK_LIBS
${ARROW_DATASET_TEST_LINK_LIBS}
PREFIX
${PREFIX}
LABELS
${LABELS}
${ARG_UNPARSED_ARGUMENTS})
endif()
add_arrow_test(${REL_TEST_NAME}
EXTRA_LINK_LIBS
${ARROW_DATASET_TEST_LINK_LIBS}
PREFIX
${PREFIX}
LABELS
${LABELS}
${ARG_UNPARSED_ARGUMENTS})
endfunction()

add_arrow_dataset_test(dataset_test)
add_arrow_dataset_test(file_test)
add_arrow_dataset_test(scanner_test)
if(NOT WIN32)
add_arrow_dataset_test(dataset_test)
add_arrow_dataset_test(file_test)
add_arrow_dataset_test(filter_test)
add_arrow_dataset_test(scanner_test)

if(ARROW_PARQUET)
add_arrow_dataset_test(file_parquet_test)
if(ARROW_PARQUET)
add_arrow_dataset_test(file_parquet_test)
endif()
endif()
1 change: 1 addition & 0 deletions cpp/src/arrow/dataset/api.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,5 @@
#include "arrow/dataset/file_base.h"
#include "arrow/dataset/file_csv.h"
#include "arrow/dataset/file_feather.h"
#include "arrow/dataset/filter.h"
#include "arrow/dataset/scanner.h"
Loading