PARQUET-2165: Remove deprecated PathGlobPattern class#979
Open
steveloughran wants to merge 1 commit intoapache:masterfrom
Open
PARQUET-2165: Remove deprecated PathGlobPattern class#979steveloughran wants to merge 1 commit intoapache:masterfrom
steveloughran wants to merge 1 commit intoapache:masterfrom
Conversation
Remove the deprecated classes PathGlobPattern and DeprecatedFieldProjectionFilter so that Parquet will compile against hadoop 3.x. If a thrift reader is configured to use the now-deleted filter, by setting the filter in "parquet.thrift.column.filter", a ThriftProjectionException will be thrown.
Contributor
|
@steveloughran is this something that you still want to get in? |
Contributor
|
Seems to conflict with #1076 |
4 tasks
Contributor
Author
|
This needs to be disposed of for building against more recent hadoop versions, so someone has to do it.. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remove the deprecated classes PathGlobPattern and
DeprecatedFieldProjectionFilter so that Parquet will
compile against hadoop 3.x.
If a thrift reader is configured to use the now-deleted filter,
by setting the filter in "parquet.thrift.column.filter",
a ThriftProjectionException will be thrown.
Jira
Tests
It modifies the test
TestParquetToThriftReadWriteAndProjectionto switch to the strict filter in all test cases where the old one was being used.*these tests now all fail with
ThriftProjectionException: No columns have been selectedI could cut the tests "obsolete" but it would seem to me that moving the tests to the strict filter would be better. I will just need help doing this.
Commits
Documentation