Skip to content

[fix][io][branch-2.11] Not restart instance when kafka source poll ex…#20817

Merged
Technoboy- merged 1 commit intoapache:branch-2.11from
shibd:fix_211_kafka_source
Jul 17, 2023
Merged

[fix][io][branch-2.11] Not restart instance when kafka source poll ex…#20817
Technoboy- merged 1 commit intoapache:branch-2.11from
shibd:fix_211_kafka_source

Conversation

@shibd
Copy link
Copy Markdown
Member

@shibd shibd commented Jul 16, 2023

Motivation

When Kafka source poll message exception, it will throw it to the function framework, and not trigger the restart function instance.

#20795 uses a new method on PulsarSource #20791 to fix it on master branch. To maintain compatibility, we can't cherry-pick it to the historical version, so the class is implemented in the Kafka source connector.

Modifications

  • Let consumer.subscribe run on the start thread, which can quickly fail when subscribe exception.
  • Add new class KafkaPushSource on Kafka source connector.
  • When Kafka consumer exception, it will call notifyError to report the exception to the io connector framework.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

@shibd shibd added the type/bug The PR fixed a bug or issue reported a bug label Jul 16, 2023
@shibd shibd requested review from RobertIndie and poorbarcode July 16, 2023 13:34
@shibd shibd self-assigned this Jul 16, 2023
@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Jul 16, 2023
@Technoboy- Technoboy- merged commit ac1e0de into apache:branch-2.11 Jul 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cherry-picked/branch-2.11 doc-not-needed Your PR changes do not impact docs ready-to-test release/2.11.3 type/bug The PR fixed a bug or issue reported a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants