Skip to content

Simplify LikeFilter implementation of getBitmapIndex, estimateSelectivity.#3910

Merged
leventov merged 4 commits intoapache:masterfrom
gianm:cleanup-like-filter
Feb 8, 2017
Merged

Simplify LikeFilter implementation of getBitmapIndex, estimateSelectivity.#3910
leventov merged 4 commits intoapache:masterfrom
gianm:cleanup-like-filter

Conversation

@gianm
Copy link
Copy Markdown
Contributor

@gianm gianm commented Feb 6, 2017

Follow up to #3848. I think the simplification is worth the extra box which should be cheap.

cc @jihoonson / @leventov -- what do you think? Similar changes could be made to BoundFilter and InFilter if this one looks good.

LikeFilter:

  • Reduce code duplication, and simplify methods, at the cost of incurring an extra box
    of ImmutableBitmap into a SingletonImmutableList. I think this is fine, since this
    should be cheap and the code path is not hot (just once per filter).

Filters:

  • Make estimateSelectivity public since it seems intended that they be used by Filter
    implementations, and Filters from extensions may want to use them too. Removed
    @VisibleForTesting for the same reason.
  • Rename one of the estimatePredicateSelectivity overloads to estimateSelectivity, since
    predicates aren't involved.

…vity.

LikeFilter:
- Reduce code duplication, and simplify methods, at the cost of incurring an extra box
  of ImmutableBitmap into a SingletonImmutableList. I think this is fine, since this
  should be cheap and the code path is not hot (just once per filter).

Filters:
- Make estimateSelectivity public since it seems intended that they be used by Filter
  implementations, and Filters from extensions may want to use them too. Removed
  @VisibleForTesting for the same reason.
- Rename one of the estimatePredicateSelectivity overloads to estimateSelectivity, since
  predicates aren't involved.
@gianm gianm added this to the 0.10.0 milestone Feb 6, 2017
@fjy
Copy link
Copy Markdown
Contributor

fjy commented Feb 6, 2017

👍

@leventov leventov self-assigned this Feb 7, 2017
* @param bitmaps the bitmaps
*
* @return unioned bitmap
*/
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic should just be part of every BitmapFactory.union() implementation. No ambiguity, impossible to choose "wrong" method and miss some optimization.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, will move it in.

} else {
// fallback
return ImmutableList.of(
Filters.matchPredicate(
Copy link
Copy Markdown
Member

@leventov leventov Feb 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When getBitmapIterable() is called from estimateSelectivity(), this fallback is nonsense, because the whole point of estimateSelectivity() is to avoid materializing this bitmap. It could be fixed by adding a method like Iterable<ImmutableBitmap> Filters.predicateMatchingBitmaps().

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha, good point. Will fix.

@gianm
Copy link
Copy Markdown
Contributor Author

gianm commented Feb 7, 2017

thx @leventov, pushed an update.

public ImmutableBitmap union(Iterable<ImmutableBitmap> b)
throws ClassCastException
{
if (b instanceof List) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can check for Collection here

@Override
public ImmutableBitmap union(Iterable<ImmutableBitmap> b)
{
if (b instanceof List) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed list to collection.

@jihoonson
Copy link
Copy Markdown
Contributor

I'll look at this patch soon.

@jihoonson
Copy link
Copy Markdown
Contributor

LGTM

@leventov leventov merged commit 97765fd into apache:master Feb 8, 2017
@gianm gianm deleted the cleanup-like-filter branch February 9, 2017 02:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants