Skip to content

KAFKA-5998: fix checkpointableOffsets handling#7027

Closed
vvcephei wants to merge 5 commits intoapache:2.2from
vvcephei:KAFKA-5998-fix-checkpointableOffsets-handling
Closed

KAFKA-5998: fix checkpointableOffsets handling#7027
vvcephei wants to merge 5 commits intoapache:2.2from
vvcephei:KAFKA-5998-fix-checkpointableOffsets-handling

Conversation

@vvcephei
Copy link
Copy Markdown
Contributor

@vvcephei vvcephei commented Jul 2, 2019

  • each task should write only the checkpoint offsets for changelog/partitions that it owns
  • check upon loading checkpointableOffsets and also immediately upon writing checkpointableOffsets that the task actually owns the partitions in question. This will prevent checkpoint file corruption in the future, and also help people clear out their corrupted checkpoint files so that they can be rebuild correctly.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Jul 2, 2019

This PR is targeted at 2.2 so that @pkleindl can verify the fix. I'll move it to trunk later.

I still need to write/update the integration tests for this condition (and actually run them to make sure I didn't break anything else).

@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Jul 3, 2019

I've rebased on trunk and opened #7030 for actual review and merging. This PR will just get closed once @pkleindl confirms the fix.

@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Jul 3, 2019

Closed in favor of #7030

@vvcephei vvcephei closed this Jul 3, 2019
@vvcephei vvcephei deleted the KAFKA-5998-fix-checkpointableOffsets-handling branch July 3, 2019 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant