Skip to content

(#5267) remove path check when start BE #5268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
morningman merged 1 commit into from
Jan 24, 2021
Merged

(#5267) remove path check when start BE #5268

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion be/src/olap/data_dir.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -687,7 +687,7 @@ OLAPStatus DataDir::load() {
int64_t tablet_id, int32_t schema_hash,
const std::string& value) -> bool {
OLAPStatus status = _tablet_manager->load_tablet_from_meta(this, tablet_id, schema_hash,
value, false, false);
value, false, false, false, false);
if (status != OLAP_SUCCESS && status != OLAP_ERR_TABLE_ALREADY_DELETED_ERROR
&& status != OLAP_ERR_ENGINE_INSERT_OLD_TABLET) {
// load_tablet_from_meta() may return OLAP_ERR_TABLE_ALREADY_DELETED_ERROR
Expand Down
15 changes: 11 additions & 4 deletions be/src/olap/tablet_manager.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -788,7 +788,7 @@ TabletSharedPtr TabletManager::find_best_tablet_to_compaction(

OLAPStatus TabletManager::load_tablet_from_meta(DataDir* data_dir, TTabletId tablet_id,
TSchemaHash schema_hash, const string& meta_binary,
bool update_meta, bool force, bool restore) {
bool update_meta, bool force, bool restore, bool check_path) {
WriteLock wlock(_get_tablets_shard_lock(tablet_id));
TabletMetaSharedPtr tablet_meta(new TabletMeta());
OLAPStatus status = tablet_meta->deserialize(meta_binary);
Expand Down Expand Up @@ -826,8 +826,15 @@ OLAPStatus TabletManager::load_tablet_from_meta(DataDir* data_dir, TTabletId tab
return OLAP_ERR_TABLE_CREATE_FROM_HEADER_ERROR;
}

// check if the tablet path exists since the path maybe deleted by gc thread
if (!Env::Default()->path_exists(tablet->tablet_path()).ok()) {
// NOTE: method load_tablet_from_meta could be called by two cases as below
// case 1: BE start;
// case 2: Clone Task/Restore
// For case 1 doesn't need path check because BE is just starting and not ready,
// just check tablet meta status to judge whether tablet is delete is enough.
// For case 2, If a tablet has just been copied to local BE,
// it may be cleared by gc-thread(see perform_path_gc_by_tablet) because the tablet meta may not be loaded to memory.
// So clone task should check path and then failed and retry in this case.
if (check_path && !Env::Default()->path_exists(tablet->tablet_path()).ok()) {
LOG(WARNING) << "tablet path not exists, create tablet failed, path="
<< tablet->tablet_path();
return OLAP_ERR_TABLE_ALREADY_DELETED_ERROR;
Expand Down Expand Up @@ -896,7 +903,7 @@ OLAPStatus TabletManager::load_tablet_from_dir(DataDir* store, TTabletId tablet_
string meta_binary;
tablet_meta->serialize(&meta_binary);
RETURN_NOT_OK_LOG(
load_tablet_from_meta(store, tablet_id, schema_hash, meta_binary, true, force, restore),
load_tablet_from_meta(store, tablet_id, schema_hash, meta_binary, true, force, restore, true),
strings::Substitute("fail to load tablet. header_path=$0", header_path));

return OLAP_SUCCESS;
Expand Down
2 changes: 1 addition & 1 deletion be/src/olap/tablet_manager.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ class TabletManager {
// where we should change tablet status from shutdown back to running
OLAPStatus load_tablet_from_meta(DataDir* data_dir, TTabletId tablet_id,
TSchemaHash schema_hash, const std::string& header,
bool update_meta, bool force = false, bool restore = false);
bool update_meta, bool force = false, bool restore = false, bool check_path=true);

OLAPStatus load_tablet_from_dir(DataDir* data_dir, TTabletId tablet_id, SchemaHash schema_hash,
const std::string& schema_hash_path, bool force = false,
Expand Down