Skip to content

MINOR: AbstractIndex.close should unmap#5757

Merged
ijuma merged 2 commits intoapache:trunkfrom
ijuma:unmap-index-on-close
Oct 11, 2018
Merged

MINOR: AbstractIndex.close should unmap#5757
ijuma merged 2 commits intoapache:trunkfrom
ijuma:unmap-index-on-close

Conversation

@ijuma
Copy link
Copy Markdown
Member

@ijuma ijuma commented Oct 8, 2018

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@ijuma ijuma requested review from junrao and lindong28 October 8, 2018 00:55
@ijuma
Copy link
Copy Markdown
Member Author

ijuma commented Oct 8, 2018

@lindong28 @junrao Is there a reason why we don't do this?

@lindong28
Copy link
Copy Markdown
Member

lindong28 commented Oct 8, 2018

@ijuma Thanks for fixing this. LGTM.

AbstractIndex.closeHandler was added in the JBOD KIP-112. In case a disk is considered faulty due to IOException, we can not call close() because close() does IO operation on the disk. In order to unmount the fault disk without shutting down the broker, we need to unmap the reference to all files on the disk by calling closeHandler(). Given the motivation of adding closeHandler(), we can see that its motivation is separate from the usage of close(), which means that we do not need to change the implementation of close() assuming our Kafka server implementation is already working well in the non-JBOD mode. This is why we didn't do this previously.

If I understand it right, it is still safe (with current Kafka implementation) not to close handler in close(). But it is more elegant and intuitive to call closeHandler() in close().

@ijuma
Copy link
Copy Markdown
Member Author

ijuma commented Oct 8, 2018

@lindong28, my claim is that close should have been updated when we added the unmap call to deleteIfExists. To make this more obvious, I have updated deleteIfExists to call closeHandler as well and moved the comment previously existing there to closeHandler.

@lindong28
Copy link
Copy Markdown
Member

@ijuma Yeah I certainly agree with your observation and the updated code is better.

@ijuma
Copy link
Copy Markdown
Member Author

ijuma commented Oct 8, 2018

Thanks @lindong28. I'll wait for @junrao to confirm before merging.

Copy link
Copy Markdown
Contributor

@junrao junrao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ijuma : Thanks for the patch. LGTM

@ijuma
Copy link
Copy Markdown
Member Author

ijuma commented Oct 11, 2018

Merging to trunk and 2.1 branches.

@ijuma ijuma merged commit 8d52b7e into apache:trunk Oct 11, 2018
@ijuma ijuma deleted the unmap-index-on-close branch October 11, 2018 00:22
ijuma added a commit that referenced this pull request Oct 11, 2018
Reviewers: Dong Lin <lindong28@gmail.com>, Jun Rao <junrao@gmail.com>
lindong28 added a commit that referenced this pull request Oct 21, 2018
smccauliff pushed a commit to linkedin/kafka that referenced this pull request Jun 13, 2019
…pache#5757)

TICKET =
LI_DESCRIPTION = proactive cherry-pick
EXIT_CRITERIA = HASH [8d52b7e]
ORIGINAL_DESCRIPTION =

Reviewers: Dong Lin <lindong28@gmail.com>, Jun Rao <junrao@gmail.com>
(cherry picked from commit 8d52b7e)
pengxiaolong pushed a commit to pengxiaolong/kafka that referenced this pull request Jun 14, 2019
Reviewers: Dong Lin <lindong28@gmail.com>, Jun Rao <junrao@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants