support multiple intervals in dataSource inputSpec#1988
Conversation
dd8643a to
7924d00
Compare
37a0499 to
60bcd10
Compare
|
👍 |
There was a problem hiding this comment.
This could potentially return the same segments multiple times, I think. Is that bad?
There was a problem hiding this comment.
I'm thinking of the case where two of the intervals in the list both partially overlap the same segment.
There was a problem hiding this comment.
Or even if two intervals in the list overlap each other.
There was a problem hiding this comment.
good catch.
but, it wouldn't matter because list of intervals is always list of "disjoint" intervals ensured by calling JodaUtils.condenseIntervals(..). In case of 2 intervals overlapping same segment, this would have given same segment twice, which again wouldn't matter because caller uses "windowed" segments appropriately.
however, that looks weird from api perspective so updated the code to remove duplicates and also updated the test case to verify same.
60bcd10 to
49f6cd7
Compare
|
@himanshug looks like a legitimate ci failure. Could you fix that and squash the commits? 👍 from me after that, everything else looks good |
49f6cd7 to
61aaa09
Compare
|
@gianm ah that happened due to rebase not updating a new class. fixed |
support multiple intervals in dataSource inputSpec
so that users can read data from multiple intervals from input dataSource