Skip to content

TimeBoundary, DataSourceMetadata filterSegments flawed with overlapping segments #6177

@gianm

Description

@gianm

The "interval" on a LogicalSegment corresponds to the timeline object holder interval, meaning that with overlapping segments, it is only the queryable part. This means that the code in filterSegments of both timeBoundary and dataSourceMetadata is flawed. It looks like this in both of them:

    final T min = query.isMaxTime() ? null : segments.get(0);
    final T max = query.isMinTime() ? null : segments.get(segments.size() - 1);

    return Lists.newArrayList(
        Iterables.filter(
            segments,
            input -> (min != null && input.getInterval().overlaps(min.getInterval())) ||
                   (max != null && input.getInterval().overlaps(max.getInterval()))
        )
    );

But because of how "interval" works on the LogicalSegments, input.getInterval() for two different LogicalSegments will never overlap. This causes erroneous results in cases like this:

  • Segment A is 2017/2018 (YEAR granularity) and has data up through May 2017.
  • Segment B is 2017-08-01/2017-08-02 (DAY granularity) and has data for August 1 2017.

In this case, three LogicalSegments are returned:

  • 2017/2017-08-01
  • 2017-08-01/2017-08-02
  • 2017-08-02/2018

And the filterSegments methods on these two query types will only inspect the last LogicalSegment, meaning they'll miss the actual max time, which is in the 2017-08-01/2017-08-02 holder.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions