Skip to content

KAFKA-14017: Implement new KIP-618 APIs in FileStreamSourceConnector#12355

Merged
C0urante merged 1 commit intoapache:trunkfrom
C0urante:kafka-14017
Dec 1, 2022
Merged

KAFKA-14017: Implement new KIP-618 APIs in FileStreamSourceConnector#12355
C0urante merged 1 commit intoapache:trunkfrom
C0urante:kafka-14017

Conversation

@C0urante
Copy link
Copy Markdown
Contributor

@C0urante C0urante commented Jun 27, 2022

Jira

Implements the new SourceConnector::exactlyOnceSupport method in the file source connector

  • When reading from stdin, returns ExactlyOnceSupport.UNSUPPORTED as we do not track offsets
  • When reading from a file, returns ExactlyOnceSupport.SUPPORTED as we do track offsets and, as long as the only modifications to the file after the connector is created are appends, those offsets should be perfectly accurate

Does not implement the new SourceConnector::canDefineTransactionBoundaries method as the default is to return ConnectorTransactionBoundaries.UNSUPPORTED, which is correct for this connector as it is incapable of defining its own transaction boundaries

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@C0urante
Copy link
Copy Markdown
Contributor Author

C0urante commented Oct 12, 2022

@showuon Would you mind giving this a pass? Should be a quick one :)

Copy link
Copy Markdown
Member

@mimaison mimaison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @C0urante

@C0urante
Copy link
Copy Markdown
Contributor Author

C0urante commented Dec 1, 2022

Thanks Mickael

@C0urante C0urante merged commit e11b515 into apache:trunk Dec 1, 2022
@C0urante C0urante deleted the kafka-14017 branch December 1, 2022 16:20
@lucasbru
Copy link
Copy Markdown
Member

lucasbru commented Dec 1, 2022

Are you sure this change compiles? Doesn't build for me

@C0urante
Copy link
Copy Markdown
Contributor Author

C0urante commented Dec 1, 2022

@lucasbru There's a compilation failure introduced by a change that landed on trunk between when the last commit for this PR was filed, and when it was merged. There is a fix PR already available which should be merged shortly: #12938. Apologies for the mistake!

hachikuji pushed a commit that referenced this pull request Dec 1, 2022
Fix compilation failure introduced in #12355.

Reviewers: Mickael Maison <mickael.maison@gmail.com>
a0x8o added a commit to a0x8o/kafka that referenced this pull request Dec 1, 2022
Fix compilation failure introduced in apache/kafka#12355.

Reviewers: Mickael Maison <mickael.maison@gmail.com>
guozhangwang pushed a commit to guozhangwang/kafka that referenced this pull request Jan 25, 2023
…pache#12355)

Reviewers: Yash Mayya <yash.mayya@gmail.com>,  Maison <mickael.maison@gmail.com>
guozhangwang pushed a commit to guozhangwang/kafka that referenced this pull request Jan 25, 2023
Fix compilation failure introduced in apache#12355.

Reviewers: Mickael Maison <mickael.maison@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants