Skip to content

PARQUET-2165: Remove deprecated PathGlobPattern class#979

Open
steveloughran wants to merge 1 commit intoapache:masterfrom
steveloughran:PARQUET-2165-PathGlobPattern
Open

PARQUET-2165: Remove deprecated PathGlobPattern class#979
steveloughran wants to merge 1 commit intoapache:masterfrom
steveloughran:PARQUET-2165-PathGlobPattern

Conversation

@steveloughran
Copy link
Contributor

Remove the deprecated classes PathGlobPattern and
DeprecatedFieldProjectionFilter so that Parquet will
compile against hadoop 3.x.

If a thrift reader is configured to use the now-deleted filter,
by setting the filter in "parquet.thrift.column.filter",
a ThriftProjectionException will be thrown.

Jira

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

It modifies the test TestParquetToThriftReadWriteAndProjection to switch to the strict filter in all test cases where the old one was being used.

*these tests now all fail with ThriftProjectionException: No columns have been selected

I could cut the tests "obsolete" but it would seem to me that moving the tests to the strict filter would be better. I will just need help doing this.

Commits

  • My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain Javadoc that explain what it does

Remove the deprecated classes PathGlobPattern and
DeprecatedFieldProjectionFilter so that Parquet will
compile against hadoop 3.x.

If a thrift reader is configured to use the now-deleted filter,
by setting the filter in "parquet.thrift.column.filter",
a ThriftProjectionException will be thrown.
@Fokko
Copy link
Contributor

Fokko commented Oct 17, 2023

@steveloughran is this something that you still want to get in?

@Fokko
Copy link
Contributor

Fokko commented Oct 17, 2023

Seems to conflict with #1076

@steveloughran
Copy link
Contributor Author

This needs to be disposed of for building against more recent hadoop versions, so someone has to do it..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants