[opt](s3) auto retry when meeting 429 error (#35397) #38176

morningman · 2024-07-21T08:19:37Z

bp #35397

doris-robot · 2024-07-21T08:19:42Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions · 2024-07-21T08:27:08Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-07-21T08:33:13Z

clang-tidy review says "All clean, LGTM! 👍"

morningman · 2024-07-21T08:45:43Z

run buildall

github-actions · 2024-07-21T09:08:47Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-07-21T09:08:49Z

PR approved by anyone and no changes requested.

- Add 2 new BE config - `s3_read_base_wait_time_ms` and `s3_read_max_wait_time_ms` When meet s3 429 error, the "get" request will sleep `s3_read_base_wait_time_ms (*1, *2, *3, *4)` ms get try again. The max sleep time is s3_read_max_wait_time_ms and the max retry time is max_s3_client_retry - Add more metrics for s3 file reader - `s3_file_reader_too_many_request`: counter of 429 error. - `s3_file_reader_s3_get_request`: the QPS of s3 get request. - `TotalGetRequest`: Get request counter in profile - `TooManyRequestErr`: 429 error counter in profile - `TooManyRequestSleepTime`: Sum of sleep time after 429 error in profile - `TotalBytesRead`: Total bytes read from s3 in profile

morningman · 2024-07-21T09:20:39Z

run buildall

github-actions · 2024-07-21T09:27:29Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-21T09:53:56Z

TPC-H: Total hot run time: 50183 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b5dbef43cbc5431a8029d3504bc6574b99eda7d4, data reload: false

------ Round 1 ----------------------------------
q1	18078	4475	4375	4375
q2	2082	169	155	155
q3	10485	1929	1943	1929
q4	10369	1241	1322	1241
q5	8653	4005	3959	3959
q6	241	127	148	127
q7	2073	1638	1630	1630
q8	9288	2749	2729	2729
q9	11089	10498	10326	10326
q10	8702	3528	3532	3528
q11	424	256	238	238
q12	483	303	315	303
q13	18356	3964	4082	3964
q14	347	328	337	328
q15	509	481	469	469
q16	686	577	584	577
q17	1153	960	989	960
q18	7278	7013	6847	6847
q19	1782	1651	1608	1608
q20	525	314	316	314
q21	4445	4135	4139	4135
q22	534	441	448	441
Total cold run time: 117582 ms
Total hot run time: 50183 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4417	4297	4307	4297
q2	322	231	223	223
q3	4237	4151	4149	4149
q4	2764	2730	2765	2730
q5	7268	7157	7148	7148
q6	237	123	118	118
q7	3254	2906	2897	2897
q8	4400	4527	4517	4517
q9	17472	17113	17036	17036
q10	4289	4291	4289	4289
q11	771	706	703	703
q12	1047	842	859	842
q13	7091	3769	3749	3749
q14	452	426	449	426
q15	503	470	453	453
q16	740	709	677	677
q17	3924	3846	3931	3846
q18	8790	8747	8801	8747
q19	1763	1710	1661	1661
q20	2388	2120	2126	2120
q21	8519	8537	8613	8537
q22	1052	968	979	968
Total cold run time: 85700 ms
Total hot run time: 80133 ms

doris-robot · 2024-07-21T10:06:49Z

TPC-DS: Total hot run time: 205012 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b5dbef43cbc5431a8029d3504bc6574b99eda7d4, data reload: false

query1	937	427	386	386
query2	6549	2794	2754	2754
query3	6919	213	210	210
query4	20148	18150	17982	17982
query5	19747	6569	6602	6569
query6	351	212	227	212
query7	4658	309	317	309
query8	423	413	415	413
query9	3181	2728	2665	2665
query10	417	334	298	298
query11	11402	10721	10719	10719
query12	124	74	76	74
query13	5606	721	721	721
query14	18455	13696	13416	13416
query15	365	249	261	249
query16	6437	303	261	261
query17	1359	1563	886	886
query18	2255	429	415	415
query19	232	153	156	153
query20	81	81	80	80
query21	188	96	91	91
query22	5246	4883	4898	4883
query23	32714	31798	31952	31798
query24	6822	6525	6473	6473
query25	532	423	443	423
query26	532	170	164	164
query27	1861	300	301	300
query28	6188	2388	2343	2343
query29	2914	2655	2746	2655
query30	244	166	173	166
query31	921	749	752	749
query32	72	67	63	63
query33	413	258	259	258
query34	858	502	491	491
query35	1119	972	943	943
query36	1344	1143	1209	1143
query37	93	60	61	60
query38	3115	2938	2949	2938
query39	1395	1351	1316	1316
query40	212	98	100	98
query41	50	44	44	44
query42	87	78	86	78
query43	707	703	729	703
query44	1148	728	731	728
query45	247	242	238	238
query46	1240	988	974	974
query47	1844	1767	1703	1703
query48	1030	718	752	718
query49	625	368	383	368
query50	884	644	631	631
query51	4803	4696	4711	4696
query52	108	80	93	80
query53	463	327	318	318
query54	2702	2502	2468	2468
query55	83	79	88	79
query56	236	248	210	210
query57	1201	1064	1065	1064
query58	226	218	186	186
query59	4440	4146	3703	3703
query60	224	216	223	216
query61	99	94	98	94
query62	798	471	474	471
query63	499	346	349	346
query64	2591	1572	1542	1542
query65	3629	3555	3574	3555
query66	812	385	391	385
query67	16103	15766	17358	15766
query68	8974	677	663	663
query69	589	351	347	347
query70	1718	1609	1609	1609
query71	416	310	317	310
query72	6515	3534	3525	3525
query73	739	330	341	330
query74	6299	5884	5923	5884
query75	5292	3755	3751	3751
query76	5462	1187	1197	1187
query77	925	259	263	259
query78	12765	12395	11819	11819
query79	12710	651	632	632
query80	863	406	426	406
query81	494	235	238	235
query82	723	103	101	101
query83	178	139	140	139
query84	257	72	71	71
query85	932	343	344	343
query86	349	310	314	310
query87	3282	3022	3021	3021
query88	5538	2350	2333	2333
query89	409	311	285	285
query90	2640	231	220	220
query91	197	140	143	140
query92	57	53	53	53
query93	5533	571	542	542
query94	1291	211	205	205
query95	1106	1073	1048	1048
query96	677	338	333	333
query97	6670	6397	6457	6397
query98	193	187	175	175
query99	3125	848	848	848
Total cold run time: 320226 ms
Total hot run time: 205012 ms

doris-robot · 2024-07-21T10:12:16Z

ClickBench: Total hot run time: 30.88 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b5dbef43cbc5431a8029d3504bc6574b99eda7d4, data reload: false

query1	0.03	0.02	0.02
query2	0.06	0.03	0.02
query3	0.25	0.04	0.05
query4	1.81	0.11	0.07
query5	0.54	0.52	0.51
query6	1.23	0.63	0.63
query7	0.02	0.01	0.01
query8	0.03	0.03	0.02
query9	0.52	0.50	0.48
query10	0.55	0.54	0.56
query11	0.12	0.09	0.09
query12	0.11	0.10	0.09
query13	0.62	0.62	0.61
query14	0.81	0.79	0.80
query15	0.79	0.76	0.77
query16	0.37	0.37	0.37
query17	0.99	1.01	0.99
query18	0.24	0.24	0.25
query19	1.91	1.85	1.86
query20	0.02	0.01	0.01
query21	15.46	0.56	0.55
query22	1.95	2.19	2.16
query23	17.35	1.00	0.94
query24	5.89	0.60	1.08
query25	0.37	0.09	0.06
query26	0.57	0.15	0.14
query27	0.04	0.04	0.04
query28	7.57	0.79	0.71
query29	12.61	2.28	2.18
query30	0.56	0.52	0.48
query31	2.82	0.40	0.38
query32	3.36	0.50	0.48
query33	3.09	3.08	3.11
query34	15.25	4.83	4.81
query35	4.87	4.90	4.88
query36	1.07	1.00	1.02
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.22 s
Total hot run time: 30.88 s

doris-robot · 2024-07-21T10:15:30Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit b5dbef43cbc5431a8029d3504bc6574b99eda7d4 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

morningman · 2024-07-21T13:51:48Z

run buildall

github-actions · 2024-07-21T13:58:52Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-21T14:24:35Z

TPC-H: Total hot run time: 49938 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ab59851ae4af3582a18081b55c96728fd1f437ed, data reload: false

------ Round 1 ----------------------------------
q1	17596	4429	4356	4356
q2	2068	157	151	151
q3	10467	1886	1912	1886
q4	10308	1265	1312	1265
q5	8404	3915	3959	3915
q6	229	153	126	126
q7	2044	1609	1609	1609
q8	9268	2704	2690	2690
q9	10934	10359	10467	10359
q10	8636	3529	3499	3499
q11	424	247	236	236
q12	473	300	307	300
q13	18324	3971	4022	3971
q14	347	349	322	322
q15	503	456	457	456
q16	664	570	578	570
q17	1131	946	962	946
q18	7306	6812	6885	6812
q19	1808	1647	1636	1636
q20	565	294	318	294
q21	4438	4103	4100	4100
q22	538	439	446	439
Total cold run time: 116475 ms
Total hot run time: 49938 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4311	4345	4377	4345
q2	320	229	223	223
q3	4188	4141	4161	4141
q4	2775	2740	2743	2740
q5	7201	7117	7108	7108
q6	236	120	122	120
q7	3247	2858	2860	2858
q8	4350	4590	4443	4443
q9	17414	16966	17219	16966
q10	4217	4223	4250	4223
q11	764	723	696	696
q12	1042	862	895	862
q13	6288	3780	3764	3764
q14	453	424	449	424
q15	500	465	451	451
q16	731	675	682	675
q17	3901	3879	3882	3879
q18	8758	8801	8828	8801
q19	1703	1718	1688	1688
q20	2384	2126	2118	2118
q21	8642	8376	8426	8376
q22	1039	987	988	987
Total cold run time: 84464 ms
Total hot run time: 79888 ms

doris-robot · 2024-07-21T14:37:17Z

TPC-DS: Total hot run time: 203469 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ab59851ae4af3582a18081b55c96728fd1f437ed, data reload: false

query1	917	425	376	376
query2	6551	2724	2612	2612
query3	6918	223	209	209
query4	20231	18099	18033	18033
query5	19727	6532	6529	6529
query6	288	224	232	224
query7	4169	296	309	296
query8	427	403	398	398
query9	3121	2704	2631	2631
query10	407	319	309	309
query11	11334	10835	10746	10746
query12	120	78	73	73
query13	5606	694	702	694
query14	17742	13409	13172	13172
query15	359	247	243	243
query16	6460	286	267	267
query17	1740	1460	880	880
query18	2325	407	411	407
query19	203	148	154	148
query20	83	79	79	79
query21	192	96	91	91
query22	5177	5002	4998	4998
query23	32369	31876	31987	31876
query24	6959	6513	6472	6472
query25	525	426	421	421
query26	524	165	160	160
query27	1866	295	302	295
query28	6252	2373	2338	2338
query29	2902	2691	2813	2691
query30	252	166	170	166
query31	915	761	738	738
query32	72	67	60	60
query33	389	265	246	246
query34	863	477	488	477
query35	1124	934	915	915
query36	1294	1172	1317	1172
query37	87	61	61	61
query38	3101	3011	2901	2901
query39	1394	1319	1332	1319
query40	205	99	97	97
query41	51	43	44	43
query42	90	83	85	83
query43	685	765	699	699
query44	1126	711	723	711
query45	252	241	227	227
query46	1228	923	995	923
query47	1830	1675	1738	1675
query48	1018	721	713	713
query49	615	366	366	366
query50	858	584	598	584
query51	4729	4648	4660	4648
query52	90	87	75	75
query53	442	337	326	326
query54	2656	2449	2489	2449
query55	96	86	87	86
query56	245	219	211	211
query57	1229	1128	1162	1128
query58	212	198	205	198
query59	4257	3884	3730	3730
query60	230	214	200	200
query61	100	93	95	93
query62	820	456	467	456
query63	488	340	337	337
query64	2464	1551	1434	1434
query65	3627	3549	3548	3548
query66	791	386	384	384
query67	19010	15493	16394	15493
query68	8276	660	643	643
query69	569	355	347	347
query70	1665	1338	1305	1305
query71	397	312	310	310
query72	6689	3571	3515	3515
query73	732	333	319	319
query74	6279	5816	5928	5816
query75	4544	3732	3693	3693
query76	4639	1184	1194	1184
query77	555	250	258	250
query78	12835	11669	11755	11669
query79	8192	636	649	636
query80	2373	407	407	407
query81	519	234	236	234
query82	1495	99	96	96
query83	186	134	131	131
query84	260	71	73	71
query85	1426	332	339	332
query86	375	303	291	291
query87	3272	3042	3066	3042
query88	5223	2344	2358	2344
query89	362	301	282	282
query90	1745	211	210	210
query91	191	143	139	139
query92	57	54	53	53
query93	4734	552	567	552
query94	887	208	210	208
query95	1081	1061	1041	1041
query96	645	328	328	328
query97	6571	6523	6368	6368
query98	198	178	170	170
query99	2982	843	899	843
Total cold run time: 314383 ms
Total hot run time: 203469 ms

doris-robot · 2024-07-21T14:42:46Z

ClickBench: Total hot run time: 30.57 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ab59851ae4af3582a18081b55c96728fd1f437ed, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.26	0.05	0.05
query4	1.80	0.07	0.06
query5	0.54	0.52	0.52
query6	1.33	0.62	0.62
query7	0.01	0.00	0.00
query8	0.04	0.02	0.02
query9	0.52	0.51	0.49
query10	0.55	0.54	0.54
query11	0.12	0.09	0.09
query12	0.11	0.09	0.09
query13	0.64	0.61	0.61
query14	0.78	0.79	0.82
query15	0.79	0.77	0.76
query16	0.38	0.39	0.38
query17	1.01	1.01	1.02
query18	0.24	0.26	0.25
query19	1.87	1.86	1.89
query20	0.02	0.01	0.00
query21	15.49	0.56	0.55
query22	2.05	2.54	1.57
query23	16.71	0.99	0.86
query24	5.90	0.88	1.12
query25	0.36	0.12	0.07
query26	0.63	0.17	0.15
query27	0.03	0.04	0.04
query28	7.25	0.76	0.74
query29	12.68	2.31	2.14
query30	0.64	0.54	0.53
query31	2.81	0.39	0.38
query32	3.39	0.50	0.49
query33	3.07	3.07	3.10
query34	15.27	4.84	4.80
query35	4.89	4.85	4.86
query36	1.05	1.02	1.02
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.02
query42	0.02	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 103.7 s
Total hot run time: 30.57 s

doris-robot · 2024-07-21T14:45:58Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ab59851ae4af3582a18081b55c96728fd1f437ed with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.9 seconds inserted 10000000 Rows, about 456K ops/s

morningman · 2024-07-22T03:55:55Z

run buildall

github-actions · 2024-07-22T04:03:46Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-22T04:15:10Z

TPC-H: Total hot run time: 50171 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bb98f90d46c5a17a55081371981d745ecc49ce44, data reload: false

------ Round 1 ----------------------------------
q1	17992	4442	4341	4341
q2	2066	160	147	147
q3	10279	1927	1937	1927
q4	10117	1256	1318	1256
q5	8435	3949	3959	3949
q6	239	150	124	124
q7	2061	1611	1621	1611
q8	9567	2742	2708	2708
q9	13999	10572	10566	10566
q10	8617	3490	3538	3490
q11	407	254	247	247
q12	476	304	303	303
q13	18360	3999	4036	3999
q14	349	322	321	321
q15	503	462	460	460
q16	677	574	570	570
q17	1137	931	900	900
q18	7347	6905	6787	6787
q19	1766	1688	1595	1595
q20	518	324	321	321
q21	4363	4166	4103	4103
q22	521	449	446	446
Total cold run time: 119796 ms
Total hot run time: 50171 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4373	4280	4296	4280
q2	324	230	220	220
q3	4215	4145	4141	4141
q4	2735	2748	2741	2741
q5	7205	7184	7083	7083
q6	239	121	120	120
q7	3165	2826	2844	2826
q8	4413	4462	4505	4462
q9	17279	17125	16939	16939
q10	4242	4271	4223	4223
q11	770	714	688	688
q12	1035	861	860	860
q13	7350	3761	3729	3729
q14	451	435	428	428
q15	509	458	458	458
q16	755	682	672	672
q17	3877	3821	3851	3821
q18	8894	8766	8740	8740
q19	1735	1658	1683	1658
q20	2389	2127	2103	2103
q21	8611	8490	8502	8490
q22	1055	948	959	948
Total cold run time: 85621 ms
Total hot run time: 79630 ms

doris-robot · 2024-07-22T04:23:39Z

TeamCity be ut coverage result:
Function Coverage: 37.81% (8117/21467)
Line Coverage: 29.48% (66544/225744)
Region Coverage: 28.95% (34291/118466)
Branch Coverage: 24.83% (17619/70960)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bb98f90d46c5a17a55081371981d745ecc49ce44_bb98f90d46c5a17a55081371981d745ecc49ce44/report/index.html

doris-robot · 2024-07-22T04:27:51Z

TPC-DS: Total hot run time: 203694 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bb98f90d46c5a17a55081371981d745ecc49ce44, data reload: false

query1	938	420	377	377
query2	6532	2791	2597	2597
query3	6922	209	205	205
query4	19985	18031	18088	18031
query5	19743	6566	6557	6557
query6	304	216	226	216
query7	4163	299	295	295
query8	428	436	377	377
query9	3076	2709	2616	2616
query10	429	343	306	306
query11	11367	10679	10717	10679
query12	128	78	69	69
query13	5606	698	705	698
query14	17544	13688	13298	13298
query15	363	247	234	234
query16	6479	284	278	278
query17	1484	1538	877	877
query18	2252	404	400	400
query19	208	159	150	150
query20	73	77	77	77
query21	193	102	98	98
query22	5404	5153	5097	5097
query23	32438	31916	31955	31916
query24	7027	6517	6494	6494
query25	520	418	429	418
query26	534	162	170	162
query27	1878	291	294	291
query28	6153	2370	2324	2324
query29	2893	2787	2586	2586
query30	243	168	176	168
query31	922	748	747	747
query32	70	62	63	62
query33	403	253	246	246
query34	854	476	475	475
query35	1132	916	952	916
query36	1376	1057	1236	1057
query37	88	62	62	62
query38	3078	2913	2895	2895
query39	1361	1335	1329	1329
query40	204	97	97	97
query41	45	43	45	43
query42	83	82	91	82
query43	676	662	768	662
query44	1122	721	710	710
query45	246	234	235	234
query46	1231	946	966	946
query47	1864	1829	2078	1829
query48	1014	700	724	700
query49	619	367	379	367
query50	871	593	588	588
query51	4838	4669	4661	4661
query52	99	90	96	90
query53	442	314	325	314
query54	2640	2453	2498	2453
query55	99	80	88	80
query56	219	215	204	204
query57	1156	1274	1095	1095
query58	205	206	203	203
query59	4427	3706	4155	3706
query60	202	206	227	206
query61	99	99	110	99
query62	821	463	493	463
query63	483	341	331	331
query64	2585	1537	1439	1439
query65	3672	3572	3556	3556
query66	780	383	378	378
query67	16592	15526	15179	15179
query68	10521	684	644	644
query69	582	334	358	334
query70	2161	1438	1512	1438
query71	424	318	322	318
query72	6501	3467	3466	3466
query73	743	330	342	330
query74	6437	5919	5871	5871
query75	5439	3717	3738	3717
query76	6807	1169	1242	1169
query77	1167	261	256	256
query78	12883	12140	11977	11977
query79	6557	629	646	629
query80	1042	402	444	402
query81	487	235	239	235
query82	678	102	98	98
query83	176	140	136	136
query84	265	71	72	71
query85	891	338	334	334
query86	339	291	326	291
query87	3220	3027	3078	3027
query88	4420	2347	2344	2344
query89	374	309	292	292
query90	1946	208	225	208
query91	182	143	138	138
query92	63	54	53	53
query93	5412	563	544	544
query94	718	209	217	209
query95	1123	1070	1041	1041
query96	644	333	338	333
query97	6626	6366	6474	6366
query98	188	171	171	171
query99	3031	857	917	857
Total cold run time: 313972 ms
Total hot run time: 203694 ms

doris-robot · 2024-07-22T04:33:21Z

ClickBench: Total hot run time: 30.92 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bb98f90d46c5a17a55081371981d745ecc49ce44, data reload: false

query1	0.03	0.02	0.02
query2	0.06	0.03	0.02
query3	0.25	0.04	0.04
query4	1.81	0.06	0.06
query5	0.54	0.53	0.52
query6	1.26	0.63	0.63
query7	0.01	0.00	0.01
query8	0.04	0.02	0.03
query9	0.53	0.49	0.47
query10	0.54	0.55	0.53
query11	0.13	0.08	0.08
query12	0.12	0.10	0.09
query13	0.62	0.62	0.61
query14	0.79	0.79	0.78
query15	0.77	0.77	0.75
query16	0.36	0.35	0.36
query17	1.02	0.99	1.02
query18	0.22	0.22	0.24
query19	1.91	1.82	1.77
query20	0.02	0.01	0.01
query21	15.44	0.55	0.55
query22	2.14	2.12	2.18
query23	17.45	1.00	1.12
query24	5.99	0.63	1.09
query25	0.36	0.09	0.06
query26	0.58	0.17	0.16
query27	0.05	0.03	0.05
query28	7.43	0.76	0.77
query29	12.65	2.32	2.32
query30	0.60	0.50	0.53
query31	2.80	0.39	0.37
query32	3.40	0.50	0.50
query33	3.10	3.08	3.07
query34	15.27	4.80	4.80
query35	4.86	4.87	4.84
query36	1.04	1.01	1.01
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.01	0.02
query43	0.03	0.02	0.01
Total cold run time: 104.57 s
Total hot run time: 30.92 s

doris-robot · 2024-07-22T04:36:35Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit bb98f90d46c5a17a55081371981d745ecc49ce44 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.6 seconds inserted 10000000 Rows, about 462K ops/s

…38805) related to #38176 comment out _s3_stats.*

) bp apache#35397

…pache#38805) related to apache#38176 comment out _s3_stats.*

morningman force-pushed the 21_bp35396 branch from e74e3f1 to 9110ddc Compare July 21, 2024 08:25

morningman added the dev/2.0.x label Jul 21, 2024

gavinchou approved these changes Jul 21, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 21, 2024

github-actions bot added the reviewed label Jul 21, 2024

morningman force-pushed the 21_bp35396 branch from 9110ddc to b5dbef4 Compare July 21, 2024 09:19

fix

ab59851

fix

bb98f90

morningman merged commit acbe603 into apache:branch-2.0 Jul 22, 2024

morningman removed the dev/2.0.x label Jul 22, 2024

xiaokang mentioned this pull request Aug 2, 2024

[fix](s3 profile) try to fix S3FileReader used after free coredump #38805

Merged

xiaokang added a commit that referenced this pull request Aug 2, 2024

[fix](s3 profile) try to fix S3FileReader used after free coredump (#…

e90d615

…38805) related to #38176 comment out _s3_stats.*

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024

[opt](s3) auto retry when meeting 429 error (apache#35397) (apache#38176

7635b9a

) bp apache#35397

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024

[fix](s3 profile) try to fix S3FileReader used after free coredump (a…

5357eca

…pache#38805) related to apache#38176 comment out _s3_stats.*

[opt](s3) auto retry when meeting 429 error (#35397) #38176

[opt](s3) auto retry when meeting 429 error (#35397) #38176

Uh oh!

Conversation

morningman commented Jul 21, 2024

Uh oh!

doris-robot commented Jul 21, 2024

Uh oh!

github-actions bot commented Jul 21, 2024

Uh oh!

github-actions bot commented Jul 21, 2024

Uh oh!

morningman commented Jul 21, 2024

Uh oh!

github-actions bot commented Jul 21, 2024

Uh oh!

github-actions bot commented Jul 21, 2024

Uh oh!

morningman commented Jul 21, 2024

Uh oh!

github-actions bot commented Jul 21, 2024

Uh oh!

doris-robot commented Jul 21, 2024

Uh oh!

doris-robot commented Jul 21, 2024

Uh oh!

doris-robot commented Jul 21, 2024

Uh oh!

doris-robot commented Jul 21, 2024

Uh oh!

morningman commented Jul 21, 2024

Uh oh!

github-actions bot commented Jul 21, 2024

Uh oh!

doris-robot commented Jul 21, 2024

Uh oh!

doris-robot commented Jul 21, 2024

Uh oh!

doris-robot commented Jul 21, 2024

Uh oh!

doris-robot commented Jul 21, 2024

Uh oh!

morningman commented Jul 22, 2024

Uh oh!

github-actions bot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants