Skip to content

branch-3.0: [Chore](function) change hash set usage of all functions #43289 #46975

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
github-actions wants to merge 1 commit into from
Closed

branch-3.0: [Chore](function) change hash set usage of all functions #43289 #46975

github-actions wants to merge 1 commit into from

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Jan 14, 2025

Cherry-picked from #43289

@BiteTheDDDDt
[Chore](function) change hash set usage of all functions (#43289)
12f0926 
remove all ck hash_set usage and use phmap_set
some test results have changed because the order in which data is stored
within the hash table has changed.
@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 14, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Jan 14, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 14, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jan 14, 2025

TPC-H: Total hot run time: 41339 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 12f092643591f7d257e02f87820cf9cc34a24d3e, data reload: false

------ Round 1 ----------------------------------
q1	17678	7465	7321	7321
q2	2068	166	163	163
q3	10657	1165	1231	1165
q4	10736	746	745	745
q5	7808	2928	2804	2804
q6	245	147	146	146
q7	991	620	622	620
q8	9763	2023	2102	2023
q9	6811	6534	6562	6534
q10	7581	2310	2323	2310
q11	476	261	272	261
q12	419	220	213	213
q13	18146	3051	3012	3012
q14	241	204	204	204
q15	583	522	517	517
q16	698	609	632	609
q17	995	644	565	565
q18	7346	6636	6596	6596
q19	1452	1131	1132	1131
q20	491	200	201	200
q21	4026	3310	3229	3229
q22	1053	992	971	971
Total cold run time: 110264 ms
Total hot run time: 41339 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7274	7220	7260	7220
q2	337	234	223	223
q3	3128	2959	3054	2959
q4	2040	1780	1801	1780
q5	5638	5760	5691	5691
q6	219	137	138	137
q7	2213	1799	1827	1799
q8	3385	3559	3496	3496
q9	8848	8869	8845	8845
q10	3585	3502	3583	3502
q11	607	508	491	491
q12	846	602	574	574
q13	7579	3136	3175	3136
q14	312	269	269	269
q15	575	523	519	519
q16	698	672	669	669
q17	1848	1604	1593	1593
q18	8260	7709	7619	7619
q19	1685	1515	1482	1482
q20	2028	1822	1802	1802
q21	5192	5250	5167	5167
q22	1093	1002	981	981
Total cold run time: 67390 ms
Total hot run time: 59954 ms

@doris-robot
Copy link

doris-robot commented Jan 14, 2025

TPC-DS: Total hot run time: 191786 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 12f092643591f7d257e02f87820cf9cc34a24d3e, data reload: false

query1	958	385	389	385
query2	6514	2134	2010	2010
query3	6719	216	217	216
query4	34155	23620	23537	23537
query5	4376	475	459	459
query6	282	180	191	180
query7	4651	313	311	311
query8	285	217	224	217
query9	9488	2682	2654	2654
query10	463	270	256	256
query11	18220	15443	15240	15240
query12	159	101	106	101
query13	1667	437	430	430
query14	9781	7127	7293	7127
query15	309	174	182	174
query16	8213	467	484	467
query17	1688	583	589	583
query18	2165	321	318	318
query19	370	166	155	155
query20	119	108	113	108
query21	214	104	107	104
query22	4558	4407	4345	4345
query23	34819	33942	33684	33684
query24	11615	2826	2859	2826
query25	690	382	398	382
query26	1817	170	183	170
query27	2833	344	346	344
query28	7886	2441	2409	2409
query29	1033	432	435	432
query30	336	166	163	163
query31	1022	787	798	787
query32	91	58	57	57
query33	771	292	279	279
query34	972	492	517	492
query35	862	730	698	698
query36	1110	950	957	950
query37	193	71	74	71
query38	4153	3836	3784	3784
query39	1465	1431	1437	1431
query40	277	97	99	97
query41	51	51	47	47
query42	109	105	102	102
query43	526	471	484	471
query44	1249	818	817	817
query45	182	164	173	164
query46	1146	710	720	710
query47	1947	1786	1825	1786
query48	503	385	384	384
query49	1289	388	380	380
query50	818	415	414	414
query51	7149	7176	7125	7125
query52	103	89	85	85
query53	250	183	185	183
query54	1191	467	460	460
query55	78	79	76	76
query56	265	241	247	241
query57	1249	1090	1161	1090
query58	239	207	220	207
query59	3341	3163	3104	3104
query60	289	245	253	245
query61	111	107	105	105
query62	896	657	662	657
query63	219	192	187	187
query64	5187	679	696	679
query65	3268	3214	3223	3214
query66	1432	333	331	331
query67	15927	15504	15653	15504
query68	4717	572	579	572
query69	452	265	270	265
query70	1153	1120	1114	1114
query71	345	261	259	259
query72	6378	4086	3987	3987
query73	758	344	359	344
query74	10375	9033	9001	9001
query75	3408	2642	2663	2642
query76	2911	1045	1076	1045
query77	440	282	269	269
query78	10339	9606	9588	9588
query79	1082	606	595	595
query80	623	436	447	436
query81	521	237	244	237
query82	211	118	119	118
query83	162	156	138	138
query84	233	85	77	77
query85	984	307	294	294
query86	335	300	298	298
query87	4347	4294	4280	4280
query88	3797	2417	2390	2390
query89	386	291	291	291
query90	2020	185	184	184
query91	187	148	145	145
query92	62	52	52	52
query93	1077	563	569	563
query94	770	291	294	291
query95	353	268	256	256
query96	614	285	276	276
query97	3349	3107	3150	3107
query98	219	202	196	196
query99	1490	1275	1288	1275
Total cold run time: 300411 ms
Total hot run time: 191786 ms

@doris-robot
Copy link

doris-robot commented Jan 14, 2025

ClickBench: Total hot run time: 33.06 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 12f092643591f7d257e02f87820cf9cc34a24d3e, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.04	0.03
query3	0.24	0.07	0.06
query4	1.63	0.10	0.10
query5	0.53	0.51	0.53
query6	1.14	0.73	0.72
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.56	0.51	0.50
query10	0.54	0.56	0.56
query11	0.14	0.10	0.11
query12	0.14	0.12	0.11
query13	0.61	0.59	0.60
query14	3.06	2.93	3.08
query15	0.90	0.82	0.82
query16	0.39	0.39	0.38
query17	1.00	1.07	1.00
query18	0.24	0.22	0.21
query19	1.92	1.80	2.01
query20	0.01	0.01	0.02
query21	15.35	0.58	0.57
query22	2.72	2.59	2.19
query23	16.93	1.06	0.71
query24	2.80	0.78	1.39
query25	0.14	0.12	0.11
query26	0.39	0.14	0.14
query27	0.05	0.05	0.04
query28	11.09	1.11	1.08
query29	12.57	3.30	3.25
query30	0.24	0.05	0.06
query31	2.86	0.40	0.39
query32	3.23	0.46	0.46
query33	2.97	3.06	3.07
query34	16.82	4.44	4.47
query35	4.55	4.52	4.50
query36	0.68	0.49	0.50
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.12	0.12
query41	0.08	0.02	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.09 s
Total hot run time: 33.06 s

@dataroaring dataroaring deleted the auto-pick-43289-branch-3.0 branch January 19, 2025 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hello-stephen @doris-robot @BiteTheDDDDt @dataroaring