HBASE-29292 Automatically reassign regions when WALs and ZK go missing #6968
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nihal Jain <nihaljain@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Charles Connell <cconnell@hubspot.com>
…ache#6153) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Pankaj <pankajkumar@apache.org> Signed-off-by: Istvan Toth <stoty@apache.org>
…pache#6146) * Sync code as per irb 1.4.2 * Also provide option to try irb's new functionalities for colorize and autocomplete Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> (cherry picked from commit 6788ff4)
… from output tarball (apache#6157) (apache#6159) Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit 98e7e1b)
…che#6162) (apache#6164) Signed-off-by: Pankaj <pankajkumar@apache.org> (cherry picked from commit 41dd87c)
…ache#6174) Signed-off-by: Duo Zhang <zhangduo@apache.org>
…eTransition if procedure is initialised by next Active master (apache#6136) Added masterActiveTime as fencing token for remote procedures Signed-off-by: Duo Zhang <zhangduo@apache.org> Reviewed-by: Aman Poonia <aman.poonia.29@gmail.com>
…nalSslHandler (apache#6181) Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
…apache#6175) Signed-off-by: Pankaj Kumar<pankajkumar@apache.org> Reviewed-by: Vineet Kumar Maheshwari <vineet.4008@gmail.com>
…rable state for the given backup (apache#6088) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
…lete (apache#6089) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
Signed-off-by: Nihal Jain <nihaljain@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> Reviewed-by: Vineet Kumar Maheshwari <vineet.4008@gmail.com>
…ks in a defined order (apache#6168)" to branch-2 (apache#6200) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Charles Connell <cconnell@hubspot.com>
… callbac…" (apache#6203) This reverts commit 18f6734.
…fined order (apache#6168) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Charles Connell <cconnell@hubspot.com>
…ks a connection to ZooKeeper (apache#6207) Signed-off-by: Duo Zhang <zhangduo@apache.org> Co-authored-by: rodenli <rodenli@tencent.com>
…procedure.jsp while Master is initializing (apache#6210) Signed-off-by: Duo Zhang <zhangduo@apache.org>
…apache#6187) (apache#6214) Co-authored-by: WangXin <1458451310@qq.com> Co-authored-by: wangxin <wangxin9702@gmail.com> Signed-off-by: Nihal Jain <nihaljain@apache.org> (cherry picked from commit b161ad5)
…#6205) (apache#6211) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> (cherry picked from commit 35c7fc0)
… cache. (apache#6183) (apache#6198) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
…ile unassigning a region (apache#6197) (apache#6217) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> (cherry picked from commit 888e4dd)
Co-authored-by: Ray Mattingly <rmattingly@hubspot.com> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Co-authored-by: Ray Mattingly <rmattingly@hubspot.com> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
…pache#6218) (apache#5607) - Also handle changes required for hadoop-2 assembly Signed-off-by: Istvan Toth <stoty@apache.org> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> (cherry picked from commit b6394b1)
…pache#6075) Signed-off-by: Duo Zhang <zhangduo@apache.org>
…addendum: sync protobuf 3 version to hbase-thidparty)
…om persistence (apache#6182) (apache#6219) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
…apache#6226) (apache#6238) Co-authored-by: wangxin <wangxin9702@gmail.com> Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit 3390195) Co-authored-by: WangXin <1458451310@qq.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
…StoreScanner (apache#6901) Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit abc8b43)
…sisted (apache#6916) (apache#6910) Signed-off-by: Andrew Purtell <apurtell@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Reid Chan <reidchan@apache.org> Signed-off-by: gvprathyusha6 <70918688+gvprathyusha6@users.noreply.github.com>
… as input and heap ByteBuffer as output, or vice versa (apache#6806) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ray Mattingly <rmattingly@apache.org> The HBase backup mechanism keeps track of which HFiles were bulk loaded, so they can be included in incremental backups. Before this ticket, these bulk load records were only deleted when an incremental backup is created. This commit adds 2 more locations: 1) after a full backup. Since a full backup already captures all data, this meant that unnecessary HFiles were being included in the next incremental backup. 2) after a table delete/truncate/CF-deletion. Previously, if an HFile was loaded before a table was cleared, the next incremental backup would effectively still include the HFile. This lead to incorrect data being restored. This commit also completely refactors & simplifies the test for this functionality. Co-authored-by: DieterDP <90392398+DieterDP-ng@users.noreply.github.com>
…t ClassLoader (apache#6918) Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit bd30367)
apache#6857) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Charles Connell <cconnell@hubspot.com>
…le (apache#6842) (apache#6929) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Signed-off-by: Nihal Jain <nihaljain@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…ributes field (apache#6823) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
…operties (apache#6948) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> (cherry picked from commit ae40069)
…Changed behavior (apache#6900) Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit 99bd5b5)
… support (apache#6953) (apache#6957) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
apache#6924) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> (cherry picked from commit 5dafa9e)
Co-authored-by: Josh Elser <elserj@apache.org> Co-authored-by: Sergey Soldatov <ssa@apache.org>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
Contributor
Author
|
let me reopen a new one against master #6981 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
sorry for the long delay from the comments of #2114 that peter asked changes for HBASE-24282.
This patch addresses the previous comments by introducing a new feature flag,
hbase.master.assign.regions.on.unknown.servers, to handle cases where an HBase cluster is restarted after WAL and ZooKeeper nodes have been removed. This is particularly useful when recreating an HBase cluster on cloud storage, where HDFS and ZooKeeper are always initialized fresh.