Skip to content

KAFKA-7316 Fix Streams Scala filter recursive call#5538

Merged
guozhangwang merged 1 commit intoapache:trunkfrom
joan38:fix-filter
Aug 23, 2018
Merged

KAFKA-7316 Fix Streams Scala filter recursive call#5538
guozhangwang merged 1 commit intoapache:trunkfrom
joan38:fix-filter

Conversation

@joan38
Copy link
Copy Markdown
Contributor

@joan38 joan38 commented Aug 20, 2018

Due to lack of conversion to kstream Predicate, existing filter method in KTable.scala would result in StackOverflowError.

This PR fixes the bug and adds testing for it.

@vvcephei
Copy link
Copy Markdown
Contributor

See also: #5543

@mjsax mjsax added the streams label Aug 21, 2018
@joan38 joan38 changed the title Fix Streams Scala filter recursive call KAFKA-7316 Fix Streams Scala filter recursive call Aug 21, 2018
@guozhangwang
Copy link
Copy Markdown
Contributor

@vvcephei could you take a look since now #5543 has been merged.

@joan38 joan38 force-pushed the fix-filter branch 2 times, most recently from 0ab1de4 to 0171c33 Compare August 22, 2018 01:17
@joan38
Copy link
Copy Markdown
Contributor Author

joan38 commented Aug 22, 2018

This PR is good to go.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What should the value be? I think it's null, right? Can we put that in the test?

Otherwise, it seems like it might actually be spitting out 1, and we wouldn't know.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It used to be there but it returns null for a Long and scalatest doesn't seem to work with that. I tried to .asInstanceOf[Any] or null.asInstanceOf[Long] but it's doesn't seems to work. Any idea?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even weirder, the null.asInstanceOf[Long] works on my machine but not on CI...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I guess you can always skip the "fancy" assertion and just do if (record.value != null) { throw new AssertionError(); }

Elegant? No. But it should work ;)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or something higher up the chain, like assert(record.value === null)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idea shows me this warning Comparing unrelated types: Long and Null

Copy link
Copy Markdown
Contributor Author

@joan38 joan38 Aug 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying out this:

record.value shouldBe (null: java.lang.Long)

Works locally, let's see on CI.

@vvcephei
Copy link
Copy Markdown
Contributor

@joan38, Thanks again for the patch and the test.

I left one comment on the tests, otherwise, it looks good to me!

@vvcephei
Copy link
Copy Markdown
Contributor

Also, a note that this change should also be cherry-picked to 2.0, which is affected.

@vvcephei
Copy link
Copy Markdown
Contributor

One more thing: @joan38, Can you add a brief description to the PR?

@joan38
Copy link
Copy Markdown
Contributor Author

joan38 commented Aug 22, 2018

Done

@joan38
Copy link
Copy Markdown
Contributor Author

joan38 commented Aug 23, 2018

@vvcephei worked!

@joan38
Copy link
Copy Markdown
Contributor Author

joan38 commented Aug 23, 2018

@guozhangwang I think we are all good on this one. Ready for merge.

Copy link
Copy Markdown
Contributor

@guozhangwang guozhangwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@guozhangwang guozhangwang merged commit d8f9f27 into apache:trunk Aug 23, 2018
guozhangwang pushed a commit that referenced this pull request Aug 23, 2018
Due to lack of conversion to kstream Predicate, existing filter method in KTable.scala would result in StackOverflowError.

This PR fixes the bug and adds testing for it.

Reviewers: Guozhang Wang <guozhang@confluent.io>, John Roesler <john@confluent.io>
@guozhangwang
Copy link
Copy Markdown
Contributor

Cherry-picked to 2.0 again.

@joan38 joan38 deleted the fix-filter branch August 23, 2018 16:27
@joan38
Copy link
Copy Markdown
Contributor Author

joan38 commented Aug 23, 2018

Thanks @guozhangwang !

pengxiaolong pushed a commit to pengxiaolong/kafka that referenced this pull request Jun 14, 2019
Due to lack of conversion to kstream Predicate, existing filter method in KTable.scala would result in StackOverflowError.

This PR fixes the bug and adds testing for it.

Reviewers: Guozhang Wang <guozhang@confluent.io>, John Roesler <john@confluent.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants