Skip to content

Conversation

@JoshRosen
Copy link
Contributor

Spark 1.6 extends BaseRelation with a new API which allows data sources to tell Spark which filters they handle, allowing Spark to eliminate its own defensive filtering for filters that are handled by the data source (see apache/spark#9399 for more details).

This patch implements this new API in spark-redshift and adds tests.

@JoshRosen JoshRosen added this to the 0.5.3 milestone Nov 24, 2015
@codecov-io
Copy link

Current coverage is 95.08%

Merging #128 into master will increase coverage by +0.01% as of c72ee35

@@            master    #128   diff @@
======================================
  Files           12      12       
  Stmts          508     509     +1
  Branches       123     123       
  Methods          0       0       
======================================
+ Hit            483     484     +1
  Partial          0       0       
  Missed          25      25       

Review entire Coverage Diff as of c72ee35

Powered by Codecov. Updated on successful CI builds.

@rxin
Copy link
Contributor

rxin commented Nov 24, 2015

LGTM

@JoshRosen JoshRosen closed this in f6ff246 Nov 24, 2015
@JoshRosen JoshRosen deleted the support-filter-skipping-in-spark-1.6 branch November 24, 2015 23:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants