[Proposal] Change some default config in be to make doris more friendly to new user and make brpc_num_threads configurable to get better performance for qps #3696
Description
Now, there are many rpcs between be and fe, and some default config in be may cause poor performance for query or stream load. In our production environment, we find that when we execute stream load in doris, the txn can be easily publish timeout and query can be easily rpc timeout, http request to be can be easily no response.
And we read the code for server in be, and we find the num of thrift server threads is too small to handle so many rpc which include exec_plan_fragment and submit_tasks at the same time, and http server use reactor model to process request, when the stream load performance is poor (server thread would be blocked in rpc or other blocked function) and the num of threads in http server is small, it will cause http server can not handle http request in time.
Finally, we do some experiments after change the some config in be, and we find that query and stream load both get good performance and http server can response in time.
we use thread pool in java client to make 10000 stream load requests to fe with 5 be instance. Every request has one row data, and we have 500 tables in on db, every table will be inserted 20 rows. the total time cost for java client is 35398ms, and some stream load results are as following:
{
"TxnId": 276003,
"Label": "86292a00-511b-47de-82e2-6dfa9c2d219d",
"Status": "Success",
"Message": "OK",
"NumberTotalRows": 1,
"NumberLoadedRows": 1,
"NumberFilteredRows": 0,
"NumberUnselectedRows": 0,
"LoadBytes": 12,
"LoadTimeMs": 31,
"BeginTxnMs": 0,
"ReadBufferMs": 0,
"WriteBufferMs": 12,
"StreamLoadPutMs": 0,
"CommitAndPublishMs": 15
}
{
"TxnId": 276006,
"Label": "7d6023a4-5fb5-4551-9327-4c758f231851",
"Status": "Success",
"Message": "OK",
"NumberTotalRows": 1,
"NumberLoadedRows": 1,
"NumberFilteredRows": 0,
"NumberUnselectedRows": 0,
"LoadBytes": 12,
"LoadTimeMs": 31,
"BeginTxnMs": 0,
"ReadBufferMs": 0,
"WriteBufferMs": 12,
"StreamLoadPutMs": 0,
"CommitAndPublishMs": 15
}
{
"TxnId": 275256,
"Label": "3b29cd47-2aa8-4255-9d37-df6148570e93",
"Status": "Success",
"Message": "OK",
"NumberTotalRows": 1,
"NumberLoadedRows": 1,
"NumberFilteredRows": 0,
"NumberUnselectedRows": 0,
"LoadBytes": 11,
"LoadTimeMs": 34,
"BeginTxnMs": 0,
"ReadBufferMs": 0,
"WriteBufferMs": 12,
"StreamLoadPutMs": 0,
"CommitAndPublishMs": 18
}
load all time 35398