Skip to content

Add time interval dim filter and retention analysis example#3315

Merged
gianm merged 3 commits intoapache:masterfrom
jon-wei:interval_filter
Aug 5, 2016
Merged

Add time interval dim filter and retention analysis example#3315
gianm merged 3 commits intoapache:masterfrom
jon-wei:interval_filter

Conversation

@jon-wei
Copy link
Copy Markdown
Contributor

@jon-wei jon-wei commented Aug 3, 2016

This PR adds a new DimFilter that provides a convenient syntax for range filtering on long millisecond timestamp columns.

The new IntervalDimFilter accepts an array of ISO 8601 intervals, converts them into startMillis/endMillis ranges.

Its toFilter() method converts the IntervalDimFilter to an OR of BoundFilters from those millisecond ranges.

The PR also adds a test/docs demonstrating a basic retention analysis example, using the new time interval filter.

@jon-wei jon-wei added the Feature label Aug 3, 2016
@jon-wei jon-wei added this to the 0.9.2 milestone Aug 3, 2016
final byte[][] intervalsBytes = new byte[intervals.size()][];
int intervalsBytesSize = 0;
int index = 0;
for (Interval interval : intervals) {
Copy link
Copy Markdown
Contributor

@gianm gianm Aug 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intervals can be stored as start/end millis pairs, which should simplify the cache key generation.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gianm the filter now stores a list of start/end pairs for cache-key and filter conversion purposes

@nishantmonu51
Copy link
Copy Markdown
Member

@jon-wei, great example.

👍

@fjy
Copy link
Copy Markdown
Contributor

fjy commented Aug 5, 2016

👍

1 similar comment
@gianm
Copy link
Copy Markdown
Contributor

gianm commented Aug 5, 2016

👍

@gianm gianm merged commit decefb7 into apache:master Aug 5, 2016
@jon-wei jon-wei deleted the interval_filter branch October 6, 2017 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants