[Improvement]remove path check when start be #5267
Description
Question
When I upgrade doris be from 0.12 to 0.13-11,
I found be starts very slowly;
LOG is as below:
BE start to load tablet time:
I0107 20:26:00.874502 35609 data_dir.cpp:670] load rowset from meta finished, data dir: /data6/olap
I0107 20:26:00.874577 35609 data_dir.cpp:675] begin loading tablet from meta
BE finish to load table time:
I0107 20:40:56.520390 35165 storage_engine.cpp:105] success to init storage engine.
I0107 20:40:53.236734 35609 data_dir.cpp:714] load tablet from meta finished, loaded tablet: 225803, error tablet: 0, path: /data6/olap
It costs about 14min.
Use perf tool to sample time-consuming, result is as below:
- 76.55% 0.23% palo_be palo_be [.] doris::TabletManager::load_tablet_from_meta ▒
- 76.33% doris::TabletManager::load_tablet_from_meta ▒
- 23.34% __GI___access ▒
- 22.47% system_call_fastpath ▒
- 22.43% sys_access ▒
- 22.33% sys_faccessat ▒
- 21.51% user_path_at ▒
- 21.48% user_path_at_empty ▒
- 21.22% filename_lookup ▒
- 21.09% path_lookupat ▒
+ 11.59% lookup_slow ▒
+ 8.71% link_path_walk ▒
+ 17.83% doris::TabletMeta::create_from_file ▒
+ 10.79% pthread_once ▒
+ 8.42% google::protobuf::MessageLite::ParseFromCodedStream ▒
+ 8.16% doris::Tablet::_contains_rowset ▒
+ 5.28% doris::TabletManager::_add_tablet_unlocked ▒
+ 0.81% pthread_rwlock_wrlock
The main sys call in load_tablet_from_meta is path check.
After I removed path check when start be, LOG as below
The time which BE begins to load tablet meta:
I0111 17:06:43.250362 33369 data_dir.cpp:675] begin loading tablet from meta
The time which BE load tablet meta finished:
I0111 17:07:37.121706 33371 data_dir.cpp:714] load tablet from meta finished, loaded tablet: 224436, error tablet: 0, path: /data8/olap
It costs about one minute;
By the way, the BE has about 1.75 million tablet