Skip to content

Conversation

@github-actions
Copy link
Contributor

Cherry-picked from #46054

…ndex storage format is v1 (#46054)

Problem Summary:
Wrong inverted index file path for inverted index storage format v1 when
making segment compaction.
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Dec 30, 2024
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41111 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 92e06562edc6a4f886ab7052542bc83bf93b3cb6, data reload: false

------ Round 1 ----------------------------------
q1	17604	7422	7266	7266
q2	2062	174	173	173
q3	10604	1064	1144	1064
q4	10576	772	723	723
q5	7726	2918	2882	2882
q6	245	152	149	149
q7	978	617	620	617
q8	9361	1983	2026	1983
q9	6548	6384	6400	6384
q10	7045	2293	2286	2286
q11	467	281	264	264
q12	417	219	224	219
q13	17783	3011	3008	3008
q14	251	212	210	210
q15	563	509	504	504
q16	695	604	615	604
q17	996	550	609	550
q18	7237	6607	6729	6607
q19	1385	1130	1051	1051
q20	554	230	217	217
q21	4219	3448	3316	3316
q22	1136	1034	1069	1034
Total cold run time: 108452 ms
Total hot run time: 41111 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7294	7311	7240	7240
q2	332	234	241	234
q3	2965	2898	3004	2898
q4	2053	1880	1780	1780
q5	5757	5789	5761	5761
q6	237	146	149	146
q7	2213	1848	1800	1800
q8	3332	3616	3504	3504
q9	8929	8971	8977	8971
q10	3595	3551	3518	3518
q11	615	521	511	511
q12	816	640	627	627
q13	9240	3228	3145	3145
q14	327	271	280	271
q15	565	527	528	527
q16	717	677	668	668
q17	1870	1645	1666	1645
q18	8213	7620	7528	7528
q19	1679	1534	1529	1529
q20	2104	1873	1901	1873
q21	5523	5392	5453	5392
q22	1128	1086	1030	1030
Total cold run time: 69504 ms
Total hot run time: 60598 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198351 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 92e06562edc6a4f886ab7052542bc83bf93b3cb6, data reload: false

query1	1281	930	908	908
query2	6219	2046	2059	2046
query3	10978	4399	4332	4332
query4	66635	29260	23413	23413
query5	4913	462	478	462
query6	428	195	196	195
query7	5533	315	315	315
query8	318	243	234	234
query9	8433	2684	2687	2684
query10	459	272	264	264
query11	17062	15226	15888	15226
query12	150	97	106	97
query13	1454	450	436	436
query14	10618	7564	7393	7393
query15	205	195	188	188
query16	7099	479	533	479
query17	1119	593	595	593
query18	1809	345	332	332
query19	208	168	162	162
query20	124	114	122	114
query21	70	50	46	46
query22	4620	4483	4342	4342
query23	34712	34539	34616	34539
query24	6124	2961	2991	2961
query25	542	433	422	422
query26	665	177	172	172
query27	1938	307	307	307
query28	4553	2523	2497	2497
query29	702	487	448	448
query30	257	170	160	160
query31	1033	827	861	827
query32	64	56	56	56
query33	463	293	278	278
query34	942	524	515	515
query35	850	740	771	740
query36	1104	983	984	983
query37	118	69	74	69
query38	4116	4102	4103	4102
query39	1525	1626	1485	1485
query40	139	80	82	80
query41	49	46	47	46
query42	115	103	100	100
query43	537	500	506	500
query44	1223	856	827	827
query45	188	171	169	169
query46	1148	746	741	741
query47	2018	1907	1930	1907
query48	486	383	391	383
query49	721	408	384	384
query50	841	438	428	428
query51	7270	7169	7150	7150
query52	100	86	96	86
query53	278	187	181	181
query54	569	458	440	440
query55	74	77	75	75
query56	265	237	226	226
query57	1236	1111	1099	1099
query58	207	200	206	200
query59	3173	3112	2783	2783
query60	277	268	257	257
query61	110	105	111	105
query62	772	668	674	668
query63	218	193	201	193
query64	1455	679	671	671
query65	3288	3222	3259	3222
query66	712	298	303	298
query67	16018	15732	15617	15617
query68	4156	571	577	571
query69	434	261	261	261
query70	1211	1095	1141	1095
query71	363	257	258	257
query72	6480	4051	4075	4051
query73	746	349	342	342
query74	10420	8992	9107	8992
query75	3350	2649	2679	2649
query76	1842	1169	1045	1045
query77	490	274	269	269
query78	10633	9586	9607	9586
query79	1704	613	606	606
query80	1063	441	423	423
query81	545	249	244	244
query82	211	118	112	112
query83	168	141	149	141
query84	292	86	75	75
query85	981	300	297	297
query86	404	294	292	292
query87	4416	4297	4319	4297
query88	4103	2402	2374	2374
query89	402	299	289	289
query90	1999	187	187	187
query91	182	146	147	146
query92	67	49	50	49
query93	2395	558	563	558
query94	889	307	296	296
query95	363	269	270	269
query96	630	272	281	272
query97	3376	3194	3203	3194
query98	215	208	196	196
query99	1609	1313	1258	1258
Total cold run time: 319069 ms
Total hot run time: 198351 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.9 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 92e06562edc6a4f886ab7052542bc83bf93b3cb6, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.06
query4	1.63	0.10	0.10
query5	0.53	0.52	0.49
query6	1.15	0.72	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.05
query9	0.57	0.50	0.49
query10	0.54	0.54	0.54
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.61	0.59	0.60
query14	2.98	2.99	2.93
query15	0.91	0.82	0.82
query16	0.38	0.36	0.37
query17	0.97	1.05	1.01
query18	0.23	0.22	0.23
query19	2.01	1.88	2.00
query20	0.01	0.01	0.01
query21	15.35	0.60	0.61
query22	2.21	2.73	2.23
query23	16.93	1.06	0.77
query24	3.07	1.48	2.23
query25	0.20	0.18	0.07
query26	0.65	0.13	0.14
query27	0.04	0.04	0.05
query28	9.29	1.12	1.08
query29	12.59	3.22	3.20
query30	0.25	0.06	0.07
query31	2.85	0.39	0.38
query32	3.26	0.46	0.48
query33	3.00	3.04	3.00
query34	16.94	4.53	4.55
query35	4.56	4.56	4.53
query36	0.66	0.48	0.50
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.04
Total cold run time: 105.55 s
Total hot run time: 33.9 s

@airborne12 airborne12 merged commit 46d3281 into branch-3.0 Dec 30, 2024
20 of 21 checks passed
@airborne12 airborne12 deleted the auto-pick-46054-branch-3.0 branch December 30, 2024 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants