Skip to content

[fix](memory) Fix BE compile in Mac with no prefix Jemalloc #34886

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
zy-kkk merged 1 commit into from
May 16, 2024
Merged

[fix](memory) Fix BE compile in Mac with no prefix Jemalloc #34886

zy-kkk merged 1 commit into from
May 16, 2024

Conversation

@xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented May 15, 2024

Proposed changes

compile jemalloc on mac have default prefix je_, so default use prefix jemalloc to ensure code uniformity.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

doris-robot commented May 15, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented May 15, 2024

run buildall

@doris-robot
Copy link

doris-robot commented May 15, 2024

TPC-H: Total hot run time: 41021 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d4fd669ab5d55714cd1d54db76b518b72d1ddc2b, data reload: false

------ Round 1 ----------------------------------
q1	17622	4385	4275	4275
q2	2011	189	200	189
q3	10407	1229	1192	1192
q4	10171	812	788	788
q5	7479	2723	2635	2635
q6	213	131	132	131
q7	1019	589	568	568
q8	9227	2097	2062	2062
q9	9080	6621	6522	6522
q10	8944	3725	3745	3725
q11	444	243	247	243
q12	458	227	221	221
q13	18604	2969	2932	2932
q14	265	221	218	218
q15	529	464	472	464
q16	515	387	388	387
q17	962	744	789	744
q18	8127	7553	7548	7548
q19	1643	1559	1535	1535
q20	515	324	311	311
q21	5234	4051	4120	4051
q22	376	287	280	280
Total cold run time: 113845 ms
Total hot run time: 41021 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4336	4289	4237	4237
q2	374	262	267	262
q3	3037	2795	2702	2702
q4	1900	1604	1616	1604
q5	5266	5307	5328	5307
q6	209	124	125	124
q7	2266	1859	1924	1859
q8	3211	3335	3330	3330
q9	8407	8364	8394	8364
q10	3836	3681	3694	3681
q11	611	491	503	491
q12	762	612	615	612
q13	16269	2967	2941	2941
q14	274	262	261	261
q15	520	483	479	479
q16	472	423	409	409
q17	1776	1480	1477	1477
q18	7720	7707	7372	7372
q19	4720	1605	1561	1561
q20	1984	1780	1745	1745
q21	5130	4805	5056	4805
q22	560	494	511	494
Total cold run time: 73640 ms
Total hot run time: 54117 ms

@doris-robot
Copy link

doris-robot commented May 15, 2024

TPC-DS: Total hot run time: 187533 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d4fd669ab5d55714cd1d54db76b518b72d1ddc2b, data reload: false

query1	910	382	371	371
query2	6444	2520	2455	2455
query3	6659	208	211	208
query4	23548	21257	21112	21112
query5	4217	422	414	414
query6	262	178	172	172
query7	4589	290	290	290
query8	250	193	183	183
query9	8473	2418	2400	2400
query10	450	259	279	259
query11	14860	14087	14195	14087
query12	134	90	90	90
query13	1641	394	374	374
query14	10094	7894	8581	7894
query15	253	170	171	170
query16	8145	275	271	271
query17	1844	582	554	554
query18	2021	290	282	282
query19	268	151	155	151
query20	98	88	88	88
query21	201	137	138	137
query22	5055	4850	4865	4850
query23	34280	33546	33771	33546
query24	11981	2898	2964	2898
query25	722	383	368	368
query26	1759	159	151	151
query27	2869	325	334	325
query28	7045	2064	2050	2050
query29	1025	610	613	610
query30	311	175	176	175
query31	1004	777	719	719
query32	95	53	53	53
query33	741	257	236	236
query34	1045	494	493	493
query35	874	687	672	672
query36	1090	925	960	925
query37	191	74	72	72
query38	2889	2828	2815	2815
query39	1625	1551	1554	1551
query40	274	128	122	122
query41	45	43	43	43
query42	105	94	94	94
query43	568	543	550	543
query44	1205	704	735	704
query45	277	262	252	252
query46	1089	716	704	704
query47	1978	1907	1899	1899
query48	369	299	293	293
query49	1187	396	400	396
query50	769	381	396	381
query51	6971	6820	6772	6772
query52	101	95	94	94
query53	351	289	282	282
query54	941	428	426	426
query55	72	73	71	71
query56	240	218	229	218
query57	1253	1122	1175	1122
query58	223	204	206	204
query59	3361	3216	3294	3216
query60	248	240	244	240
query61	92	116	85	85
query62	676	484	476	476
query63	315	285	279	279
query64	9765	7401	7379	7379
query65	3159	3069	3052	3052
query66	1362	361	377	361
query67	15898	15075	14904	14904
query68	4789	532	556	532
query69	488	314	301	301
query70	1222	1112	1115	1112
query71	410	265	278	265
query72	7176	2589	2362	2362
query73	715	322	319	319
query74	6661	6148	6040	6040
query75	3505	2658	2691	2658
query76	2929	995	975	975
query77	397	264	273	264
query78	10628	10249	9979	9979
query79	2337	530	522	522
query80	848	437	430	430
query81	530	241	242	241
query82	785	96	98	96
query83	275	214	169	169
query84	238	85	86	85
query85	1334	271	255	255
query86	441	302	290	290
query87	3311	3067	3077	3067
query88	4185	2426	2425	2425
query89	470	373	399	373
query90	1967	187	190	187
query91	124	97	98	97
query92	60	49	48	48
query93	1830	497	492	492
query94	1200	177	183	177
query95	394	309	301	301
query96	588	271	269	269
query97	3150	3005	3022	3005
query98	237	220	213	213
query99	1137	901	913	901
Total cold run time: 287217 ms
Total hot run time: 187533 ms

@doris-robot
Copy link

doris-robot commented May 15, 2024

ClickBench: Total hot run time: 30.37 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d4fd669ab5d55714cd1d54db76b518b72d1ddc2b, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.06	0.06
query4	1.66	0.08	0.08
query5	0.49	0.48	0.52
query6	1.13	0.72	0.71
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.50
query10	0.55	0.54	0.54
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	0.78	0.78	0.79
query15	0.81	0.82	0.81
query16	0.36	0.36	0.36
query17	0.96	0.96	1.02
query18	0.23	0.25	0.23
query19	1.81	1.74	1.71
query20	0.02	0.01	0.02
query21	15.58	0.67	0.66
query22	3.98	7.33	1.86
query23	18.38	1.34	1.24
query24	1.38	0.30	0.24
query25	0.15	0.08	0.07
query26	0.25	0.16	0.16
query27	0.08	0.08	0.08
query28	13.42	1.01	1.00
query29	13.18	3.25	3.25
query30	0.24	0.05	0.06
query31	2.86	0.39	0.37
query32	3.30	0.47	0.46
query33	2.85	2.84	2.82
query34	17.08	4.44	4.41
query35	4.43	4.55	4.50
query36	0.65	0.46	0.46
query37	0.18	0.16	0.15
query38	0.15	0.14	0.14
query39	0.04	0.04	0.04
query40	0.17	0.14	0.14
query41	0.09	0.05	0.04
query42	0.06	0.04	0.05
query43	0.04	0.03	0.03
Total cold run time: 109.19 s
Total hot run time: 30.37 s

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented May 16, 2024

comment run performance

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 16, 2024
@github-actions
Copy link
Contributor

github-actions bot commented May 16, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented May 16, 2024

PR approved by anyone and no changes requested.

zy-kkk
zy-kkk approved these changes May 16, 2024
View reviewed changes
Copy link
Member

@zy-kkk zy-kkk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zy-kkk
Copy link
Member

zy-kkk commented May 16, 2024

run performance

@zy-kkk zy-kkk merged commit 75fee8f into apache:master May 16, 2024
@doris-robot
Copy link

doris-robot commented May 16, 2024

TPC-DS: Total hot run time: 187168 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d4fd669ab5d55714cd1d54db76b518b72d1ddc2b, data reload: false

query1	906	378	367	367
query2	6464	2330	2297	2297
query3	6646	217	220	217
query4	23245	21258	21260	21258
query5	4144	418	412	412
query6	262	171	183	171
query7	4581	293	280	280
query8	239	182	196	182
query9	8500	2334	2325	2325
query10	426	244	256	244
query11	14922	14214	14278	14214
query12	140	91	87	87
query13	1662	367	368	367
query14	10455	8096	7607	7607
query15	248	167	168	167
query16	8156	260	258	258
query17	1877	552	561	552
query18	2055	269	270	269
query19	197	149	147	147
query20	92	88	91	88
query21	197	138	131	131
query22	5064	4895	4893	4893
query23	34056	33585	33563	33563
query24	10898	2985	2978	2978
query25	617	386	363	363
query26	1172	165	154	154
query27	3012	312	339	312
query28	7575	2000	2005	2000
query29	857	620	606	606
query30	281	170	174	170
query31	953	768	745	745
query32	100	52	52	52
query33	745	252	242	242
query34	1069	483	479	479
query35	796	674	691	674
query36	1052	906	942	906
query37	135	70	73	70
query38	2876	2776	2736	2736
query39	1575	1554	1567	1554
query40	208	132	126	126
query41	48	75	45	45
query42	104	95	96	95
query43	591	531	532	531
query44	1202	722	728	722
query45	267	250	221	221
query46	1092	744	709	709
query47	1961	1869	1855	1855
query48	386	290	291	290
query49	1067	379	390	379
query50	756	393	369	369
query51	6870	6809	6749	6749
query52	104	94	90	90
query53	354	285	284	284
query54	859	437	415	415
query55	76	74	75	74
query56	245	225	212	212
query57	1223	1136	1149	1136
query58	218	204	205	204
query59	3376	3232	3232	3232
query60	250	232	233	232
query61	104	89	87	87
query62	684	482	454	454
query63	318	284	286	284
query64	8841	7387	7425	7387
query65	3197	3125	3100	3100
query66	1407	353	370	353
query67	15441	15040	15030	15030
query68	4570	516	522	516
query69	487	298	309	298
query70	1156	1152	1137	1137
query71	388	276	263	263
query72	7462	2545	2367	2367
query73	698	324	324	324
query74	6525	6158	6190	6158
query75	3426	2642	2578	2578
query76	2639	981	962	962
query77	404	265	272	265
query78	10409	10121	10015	10015
query79	2357	514	507	507
query80	1724	440	439	439
query81	524	242	244	242
query82	694	97	96	96
query83	326	173	174	173
query84	271	86	83	83
query85	1846	263	265	263
query86	499	293	292	292
query87	3279	3078	3149	3078
query88	4247	2297	2305	2297
query89	486	380	376	376
query90	1979	180	185	180
query91	120	98	96	96
query92	57	48	47	47
query93	2488	502	483	483
query94	1257	176	176	176
query95	385	291	295	291
query96	586	276	264	264
query97	3194	3014	2998	2998
query98	249	225	219	219
query99	1266	899	883	883
Total cold run time: 285494 ms
Total hot run time: 187168 ms

@doris-robot
Copy link

doris-robot commented May 16, 2024

ClickBench: Total hot run time: 30.74 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d4fd669ab5d55714cd1d54db76b518b72d1ddc2b, data reload: false

query1	0.03	0.03	0.03
query2	0.09	0.04	0.04
query3	0.24	0.05	0.04
query4	1.68	0.07	0.08
query5	0.49	0.49	0.52
query6	1.12	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.52	0.49	0.50
query10	0.54	0.54	0.52
query11	0.16	0.10	0.11
query12	0.14	0.12	0.12
query13	0.60	0.59	0.61
query14	0.76	0.78	0.79
query15	0.84	0.80	0.80
query16	0.36	0.36	0.35
query17	0.96	0.94	1.00
query18	0.22	0.24	0.23
query19	1.79	1.75	1.69
query20	0.01	0.01	0.02
query21	15.71	0.66	0.65
query22	4.18	6.94	2.39
query23	18.28	1.37	1.30
query24	1.89	0.24	0.20
query25	0.14	0.08	0.08
query26	0.28	0.18	0.17
query27	0.08	0.08	0.07
query28	13.30	1.00	1.02
query29	12.99	3.27	3.22
query30	0.24	0.06	0.06
query31	2.87	0.39	0.38
query32	3.30	0.47	0.48
query33	2.81	2.89	2.83
query34	17.25	4.43	4.38
query35	4.45	4.45	4.48
query36	0.65	0.46	0.45
query37	0.17	0.15	0.15
query38	0.16	0.15	0.14
query39	0.04	0.03	0.04
query40	0.16	0.14	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.03
Total cold run time: 109.76 s
Total hot run time: 30.74 s

dataroaring pushed a commit that referenced this pull request May 16, 2024
@xinyiZzz @dataroaring
[fix](memory) Fix BE compile in Mac with no prefix Jemalloc (#34886)
761ec49 
compile jemalloc on mac have default prefix `je_`, so default use prefix jemalloc to ensure code uniformity.
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request May 31, 2024
@xinyiZzz
Revert "[fix](memory) Fix BE compile in Mac with no prefix Jemalloc (a…
7611bad 
…pache#34886)"

This reverts commit 75fee8f.
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request May 31, 2024
@xinyiZzz
Revert "[fix](memory) Fix BE compile in Mac with no prefix Jemalloc (a…
6e3fed9 
…pache#34886)"

This reverts commit 75fee8f.
@xinyiZzz xinyiZzz mentioned this pull request Dec 17, 2024
Closed
16 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@924060929 924060929 924060929 approved these changes

@zy-kkk zy-kkk zy-kkk approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@xinyiZzz @doris-robot @zy-kkk @924060929 @dataroaring