Skip to content

fix kafka index task doesn't resume when recieve duplicate request#6990

Merged
clintropolis merged 2 commits intoapache:masterfrom
QiuMM:duplicate
Feb 12, 2019
Merged

fix kafka index task doesn't resume when recieve duplicate request#6990
clintropolis merged 2 commits intoapache:masterfrom
QiuMM:duplicate

Conversation

@QiuMM
Copy link
Copy Markdown
Member

@QiuMM QiuMM commented Feb 2, 2019

When the kafka index task recieve duplicate setEndOffsets request, it returns and doesn't call resume() , this will cause the task pausing forever.

@QiuMM QiuMM added the Bug label Feb 2, 2019
@fjy fjy added this to the 0.14.0 milestone Feb 3, 2019
Copy link
Copy Markdown
Member

@clintropolis clintropolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I see what's going on here and that this makes sense, 👍

Is it a major hassle to add a test for this?

@jihoonson
Copy link
Copy Markdown
Contributor

Hi @QiuMM, would you elaborate more on this case? When and how can it happen that a kafka task is paused for the same endOffsets twice or more?

@QiuMM
Copy link
Copy Markdown
Member Author

QiuMM commented Feb 12, 2019

@jihoonson I am not sure how it happen, I just observed one kafka index task pasuing forever and found log like this: Ignoring duplicate request, end offsets already set for sequences.

@jihoonson
Copy link
Copy Markdown
Contributor

@QiuMM thanks. I guess it might happen if no data has been ingested since the first endOffsets update when the second pause & update endOffsets are requested. I just wanted to check that my guess is correct. The latest change looks good to me.

Copy link
Copy Markdown
Member

@clintropolis clintropolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a test, LGTM 👍

@clintropolis clintropolis merged commit d0abf5c into apache:master Feb 12, 2019
clintropolis pushed a commit to clintropolis/druid that referenced this pull request Feb 12, 2019
…pache#6990)

* fix kafka index task doesn't resume when recieve duplicate request

* add unit test
jihoonson pushed a commit that referenced this pull request Feb 13, 2019
…6990) (#7055)

* fix kafka index task doesn't resume when recieve duplicate request

* add unit test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants