Skip to content

Conversation

@morningman
Copy link
Contributor

Reverts #47913
Before this PR, the BE's JVM heap size does not follow the JAVA_OPTS_FOR_JDK17 in be.conf,
it use default value, which is 1/4 of physical memory.
After this PR, the BE's JVM heap size is set to 2GB, and it cause some test pipeline OOM.
So I revert this PR temporarily to try investigate the usage of BE's JVM first.

@Thearas
Copy link
Contributor

Thearas commented Feb 18, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31468 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 162b5df80a871ddcfb1a0bb3d9444c45b7d0372e, data reload: false

------ Round 1 ----------------------------------
q1	17581	5135	5089	5089
q2	2045	302	168	168
q3	10397	1346	703	703
q4	10215	1017	537	537
q5	7512	2407	2354	2354
q6	187	163	137	137
q7	894	744	620	620
q8	9287	1281	1077	1077
q9	4945	4548	4634	4548
q10	6829	2320	1888	1888
q11	472	291	267	267
q12	352	357	227	227
q13	17753	3646	3048	3048
q14	224	220	210	210
q15	526	469	457	457
q16	629	616	583	583
q17	562	877	333	333
q18	6668	6256	6177	6177
q19	1203	950	533	533
q20	308	331	195	195
q21	2794	2133	2007	2007
q22	363	334	310	310
Total cold run time: 101746 ms
Total hot run time: 31468 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5119	5135	5120	5120
q2	229	332	246	246
q3	2145	2657	2315	2315
q4	1436	1835	1359	1359
q5	4221	4135	4113	4113
q6	208	163	123	123
q7	1846	1794	1655	1655
q8	2583	2671	2589	2589
q9	7225	7138	7119	7119
q10	2970	3153	2752	2752
q11	593	515	482	482
q12	661	731	638	638
q13	3459	3951	3295	3295
q14	289	318	267	267
q15	523	485	451	451
q16	618	681	629	629
q17	1168	1585	1318	1318
q18	7701	7311	7217	7217
q19	772	841	816	816
q20	1940	2060	1890	1890
q21	5348	4911	4984	4911
q22	641	596	536	536
Total cold run time: 51695 ms
Total hot run time: 49841 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 182995 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 162b5df80a871ddcfb1a0bb3d9444c45b7d0372e, data reload: false

query1	971	380	381	380
query2	6542	1821	1808	1808
query3	6788	214	212	212
query4	26373	23228	23114	23114
query5	4320	649	487	487
query6	302	198	180	180
query7	4597	494	293	293
query8	288	250	236	236
query9	8645	2532	2510	2510
query10	446	319	257	257
query11	16114	15158	14945	14945
query12	154	101	101	101
query13	1638	516	384	384
query14	9014	6551	6172	6172
query15	208	189	167	167
query16	7144	606	433	433
query17	904	694	535	535
query18	1933	384	307	307
query19	207	188	144	144
query20	120	113	117	113
query21	205	117	100	100
query22	4259	4394	4065	4065
query23	33740	32826	32884	32826
query24	7770	2375	2374	2374
query25	533	451	382	382
query26	1238	269	150	150
query27	2192	491	335	335
query28	3996	2415	2393	2393
query29	771	530	419	419
query30	230	187	156	156
query31	919	864	765	765
query32	76	67	60	60
query33	553	342	295	295
query34	806	852	495	495
query35	787	812	740	740
query36	978	1004	915	915
query37	129	101	78	78
query38	4168	4144	4103	4103
query39	1456	1389	1401	1389
query40	211	121	110	110
query41	65	56	62	56
query42	123	106	117	106
query43	502	510	465	465
query44	1277	807	814	807
query45	184	173	160	160
query46	873	1038	666	666
query47	1761	1774	1733	1733
query48	370	413	320	320
query49	812	542	434	434
query50	679	745	420	420
query51	4190	4172	4107	4107
query52	118	109	93	93
query53	235	254	189	189
query54	501	508	416	416
query55	89	81	85	81
query56	276	297	256	256
query57	1147	1137	1090	1090
query58	256	227	243	227
query59	2614	2833	2730	2730
query60	278	285	264	264
query61	121	119	116	116
query62	799	749	668	668
query63	234	196	196	196
query64	4455	1008	699	699
query65	3220	3156	3128	3128
query66	1139	402	307	307
query67	15923	15770	15425	15425
query68	7255	770	516	516
query69	478	284	309	284
query70	1211	1126	1126	1126
query71	386	295	263	263
query72	5732	3540	3514	3514
query73	751	738	354	354
query74	9162	8900	8898	8898
query75	3147	3154	2679	2679
query76	3160	1164	736	736
query77	480	380	283	283
query78	10042	10134	9316	9316
query79	1909	805	580	580
query80	647	539	453	453
query81	510	281	239	239
query82	198	132	95	95
query83	167	177	154	154
query84	293	104	74	74
query85	743	344	334	334
query86	374	301	297	297
query87	4421	4641	4320	4320
query88	2887	2214	2206	2206
query89	382	314	287	287
query90	1972	191	190	190
query91	203	140	110	110
query92	74	62	58	58
query93	1657	1013	577	577
query94	677	418	297	297
query95	350	270	253	253
query96	479	542	265	265
query97	2769	2872	2730	2730
query98	219	200	211	200
query99	1275	1412	1220	1220
Total cold run time: 267976 ms
Total hot run time: 182995 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.16 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 162b5df80a871ddcfb1a0bb3d9444c45b7d0372e, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.04
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.43	0.41	0.41
query6	1.17	0.67	0.67
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.59	0.51	0.53
query10	0.56	0.58	0.56
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.63	0.60	0.60
query14	2.69	2.72	2.73
query15	0.91	0.86	0.85
query16	0.37	0.39	0.37
query17	0.99	1.03	1.01
query18	0.22	0.20	0.19
query19	1.94	1.83	2.04
query20	0.01	0.02	0.01
query21	15.35	0.88	0.55
query22	0.75	1.17	0.66
query23	14.99	1.36	0.59
query24	12.52	1.02	0.32
query25	0.32	0.11	0.08
query26	0.80	0.19	0.13
query27	0.05	0.04	0.04
query28	6.12	0.79	0.43
query29	12.53	3.93	3.30
query30	0.26	0.10	0.07
query31	2.81	0.58	0.39
query32	3.23	0.54	0.46
query33	2.98	2.99	2.99
query34	15.77	5.16	4.59
query35	4.53	4.55	4.56
query36	0.65	0.50	0.48
query37	0.09	0.06	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.18	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.97 s
Total hot run time: 30.16 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 43.93% (11472/26114)
Line Coverage: 33.93% (96663/284897)
Region Coverage: 32.64% (49462/151557)
Branch Coverage: 28.31% (24869/87856)
Coverage Report: http://coverage.selectdb-in.cc/coverage/162b5df80a871ddcfb1a0bb3d9444c45b7d0372e_162b5df80a871ddcfb1a0bb3d9444c45b7d0372e/report/index.html

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 18, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@kaka11chen kaka11chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit c7acec4 into master Feb 18, 2025
31 of 34 checks passed
lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
…PTS" (apache#48017)

Reverts apache#47913
Before this PR, the BE's JVM heap size does not follow the
JAVA_OPTS_FOR_JDK17 in be.conf,
it use default value, which is 1/4 of physical memory.
After this PR, the BE's JVM heap size is set to 2GB, and it cause some
test pipeline OOM.
So I revert this PR temporarily to try investigate the usage of BE's JVM
first.
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
…PTS" (apache#48017)

Reverts apache#47913
Before this PR, the BE's JVM heap size does not follow the
JAVA_OPTS_FOR_JDK17 in be.conf,
it use default value, which is 1/4 of physical memory.
After this PR, the BE's JVM heap size is set to 2GB, and it cause some
test pipeline OOM.
So I revert this PR temporarily to try investigate the usage of BE's JVM
first.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants