[Python] raise error message when passing invalid filter in parquet reading

[asfimport]

From https://stackoverflow.com/questions/56522977/using-predicates-to-filter-rows-from-pyarrow-parquet-parquetdataset

For example, when specifying a column in the filter which is a normal column and not a key in your partitioned folder hierarchy, the filter gets silently ignored. It would be nice to get an error message for this.
Reproducible example:

df = pd.DataFrame({'a': [0, 0, 1, 1], 'b': [0, 1, 0, 1], 'c': [1, 2, 3, 4]})
table = pa.Table.from_pandas(df)
pq.write_to_dataset(table, 'test_parquet_row_filters', partition_cols=['a'])
# filter on 'a' (partition column) -> works
pq.read_table('test_parquet_row_filters', filters=[('a', '=', 1)]).to_pandas()
# filter on normal column (in future could do row group filtering) -> silently does nothing
pq.read_table('test_parquet_row_filters', filters=[('b', '=', 1)]).to_pandas()

Reporter: Joris Van den Bossche / @jorisvandenbossche
Assignee: Joris Van den Bossche / @jorisvandenbossche

Related issues:

• [Python][Dataset] Support using dataset API in pyarrow.parquet with a minimal ParquetDataset shim (depends upon)

PRs and other links:

• GitHub Pull Request #7052

_{Note: This issue was originally created as ARROW-5572. Please see the migration documentation for further details.}

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
This works now correctly with the new Datasets API, since we can filter on both partition keys as "normal" columns.

So once we use the datasets API under the hood in pyarrow.parquet (ARROW-8039), this issue will be resolved.

[asfimport]

Wes McKinney / @wesm:
Issue resolved by pull request 7052
#7052