Skip to content

[fix] (compaction) add error log and fix time series #38791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
airborne12 merged 3 commits into from
Aug 6, 2024
Merged

[fix] (compaction) add error log and fix time series #38791

airborne12 merged 3 commits into from
Aug 6, 2024

Conversation

@csun5285
Copy link
Contributor

@csun5285 csun5285 commented Aug 2, 2024

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

doris-robot commented Aug 2, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@csun5285
Copy link
Contributor Author

csun5285 commented Aug 2, 2024

run buildall

@github-actions github-actions bot added the doing label Aug 2, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@csun5285
Copy link
Contributor Author

csun5285 commented Aug 2, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

qidaye
qidaye reviewed Aug 6, 2024
View reviewed changes
@csun5285
Copy link
Contributor Author

csun5285 commented Aug 6, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Aug 6, 2024

TPC-H: Total hot run time: 41922 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7fbfc01d86f3a8e89e849919682774cf691140ce, data reload: false

------ Round 1 ----------------------------------
q1	18179	4198	4117	4117
q2	2520	209	201	201
q3	11046	1333	1383	1333
q4	10860	810	928	810
q5	7952	2980	3008	2980
q6	225	139	143	139
q7	1067	666	615	615
q8	9442	1904	1950	1904
q9	8447	6568	6600	6568
q10	8739	3834	3858	3834
q11	425	252	249	249
q12	407	225	223	223
q13	17749	2942	2969	2942
q14	276	249	241	241
q15	523	474	485	474
q16	492	412	400	400
q17	952	927	942	927
q18	8237	7402	7425	7402
q19	1388	1220	1213	1213
q20	556	319	337	319
q21	5374	4797	4756	4756
q22	362	275	281	275
Total cold run time: 115218 ms
Total hot run time: 41922 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4122	4011	4028	4011
q2	346	229	221	221
q3	3007	3008	2978	2978
q4	1868	1875	1847	1847
q5	5236	5270	5250	5250
q6	219	135	132	132
q7	2073	1673	1714	1673
q8	3170	3263	3210	3210
q9	8330	8313	8273	8273
q10	3775	3843	3867	3843
q11	560	462	457	457
q12	721	541	541	541
q13	10497	2942	2991	2942
q14	289	265	264	264
q15	512	476	478	476
q16	447	405	418	405
q17	1722	1680	1707	1680
q18	7735	7505	7322	7322
q19	1658	1669	1659	1659
q20	1992	1804	1761	1761
q21	5555	5159	5190	5159
q22	504	441	456	441
Total cold run time: 64338 ms
Total hot run time: 54545 ms

@doris-robot
Copy link

doris-robot commented Aug 6, 2024

TPC-DS: Total hot run time: 169129 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7fbfc01d86f3a8e89e849919682774cf691140ce, data reload: false

query1	922	372	362	362
query2	6482	1742	1776	1742
query3	6670	217	231	217
query4	20176	17457	17544	17457
query5	4309	520	527	520
query6	291	188	187	187
query7	4609	309	293	293
query8	256	194	199	194
query9	8526	2391	2393	2391
query10	447	279	273	273
query11	10595	10235	10066	10066
query12	130	88	84	84
query13	1621	382	369	369
query14	8474	6947	6103	6103
query15	211	160	169	160
query16	7098	481	479	479
query17	928	581	542	542
query18	1916	282	297	282
query19	189	146	149	146
query20	89	86	84	84
query21	215	106	101	101
query22	4168	4241	4080	4080
query23	33747	33248	33103	33103
query24	10369	3070	3027	3027
query25	677	383	387	383
query26	1803	150	149	149
query27	3024	279	280	279
query28	7031	1991	1970	1970
query29	1345	421	405	405
query30	279	151	149	149
query31	934	748	763	748
query32	98	60	54	54
query33	707	315	318	315
query34	910	472	482	472
query35	827	746	715	715
query36	1041	886	875	875
query37	294	78	80	78
query38	2892	2764	2794	2764
query39	864	808	818	808
query40	282	113	113	113
query41	47	47	47	47
query42	123	101	101	101
query43	466	455	418	418
query44	1162	726	728	726
query45	209	181	178	178
query46	1101	812	795	795
query47	1817	1747	1776	1747
query48	360	290	286	286
query49	1192	472	440	440
query50	901	430	422	422
query51	6828	6673	6722	6673
query52	102	90	88	88
query53	247	186	180	180
query54	647	446	451	446
query55	77	74	78	74
query56	271	287	257	257
query57	1154	1074	1043	1043
query58	273	269	278	269
query59	2902	2484	2360	2360
query60	309	286	287	286
query61	102	96	101	96
query62	927	669	671	669
query63	218	181	182	181
query64	5954	1944	1878	1878
query65	3165	3105	3113	3105
query66	1433	338	336	336
query67	15255	14893	14768	14768
query68	4311	578	586	578
query69	441	320	295	295
query70	1118	1084	1087	1084
query71	424	279	284	279
query72	7190	2867	2663	2663
query73	759	328	327	327
query74	6086	5747	5683	5683
query75	3371	2787	2737	2737
query76	2452	1222	1275	1222
query77	506	331	341	331
query78	9481	8986	8886	8886
query79	2813	551	548	548
query80	1213	529	529	529
query81	576	226	232	226
query82	869	134	133	133
query83	249	202	176	176
query84	281	88	89	88
query85	1519	388	366	366
query86	494	302	320	302
query87	3291	3117	3134	3117
query88	3938	2414	2414	2414
query89	379	296	297	296
query90	1878	196	199	196
query91	141	112	117	112
query92	61	53	54	53
query93	2161	632	624	624
query94	925	323	312	312
query95	388	278	273	273
query96	622	284	282	282
query97	3228	3095	3141	3095
query98	215	215	196	196
query99	1617	1316	1298	1298
Total cold run time: 265685 ms
Total hot run time: 169129 ms

@doris-robot
Copy link

doris-robot commented Aug 6, 2024

ClickBench: Total hot run time: 31.28 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7fbfc01d86f3a8e89e849919682774cf691140ce, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.07	0.06
query5	0.49	0.49	0.48
query6	1.15	0.73	0.72
query7	0.02	0.01	0.01
query8	0.04	0.05	0.05
query9	0.58	0.52	0.52
query10	0.57	0.56	0.59
query11	0.16	0.12	0.12
query12	0.16	0.12	0.13
query13	0.61	0.61	0.60
query14	0.78	0.81	0.80
query15	0.96	0.88	0.90
query16	0.36	0.36	0.37
query17	1.02	0.99	1.02
query18	0.22	0.22	0.21
query19	1.89	1.75	1.74
query20	0.01	0.01	0.01
query21	15.40	0.77	0.66
query22	3.85	6.74	2.32
query23	18.14	1.46	1.34
query24	2.27	0.23	0.21
query25	0.19	0.08	0.07
query26	0.33	0.22	0.22
query27	0.46	0.23	0.23
query28	13.16	1.02	0.98
query29	12.55	3.32	3.28
query30	0.25	0.06	0.06
query31	2.87	0.42	0.42
query32	3.25	0.52	0.50
query33	2.98	3.01	2.97
query34	15.43	4.31	4.26
query35	4.33	4.33	4.35
query36	0.68	0.48	0.50
query37	0.21	0.16	0.16
query38	0.16	0.16	0.14
query39	0.04	0.04	0.03
query40	0.17	0.14	0.13
query41	0.09	0.05	0.05
query42	0.05	0.05	0.05
query43	0.04	0.05	0.04
Total cold run time: 107.95 s
Total hot run time: 31.28 s

xiaokang
xiaokang approved these changes Aug 6, 2024
View reviewed changes
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Aug 6, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

PR approved by anyone and no changes requested.

airborne12
airborne12 approved these changes Aug 6, 2024
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@airborne12 airborne12 merged commit 3d0cec5 into apache:master Aug 6, 2024
airborne12 pushed a commit to airborne12/apache-doris that referenced this pull request Aug 7, 2024
@csun5285 @airborne12
[fix] (compaction) add error log and fix time series (apache#38791)
ea14b0c 
## Proposed changes


Issue Number: close #xxx

<!--Describe your changes.-->
This was referenced Aug 7, 2024
Merged
Merged
airborne12 pushed a commit that referenced this pull request Aug 9, 2024
@csun5285
[fix] (compaction) fix time series (#38791) (#39052)
5db4184 
## Proposed changes

pick from master #38791
airborne12 pushed a commit that referenced this pull request Aug 9, 2024
@csun5285
[fix] (compaction) fix time series (#38791) (#39053)
f4a98a7 
## Proposed changes

pick from master #38791
dataroaring pushed a commit that referenced this pull request Aug 11, 2024
@csun5285 @dataroaring
[fix] (compaction) add error log and fix time series (#38791)
780c9ff 
## Proposed changes


Issue Number: close #xxx

<!--Describe your changes.-->
@xiaokang xiaokang removed the doing label Aug 14, 2024
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
@csun5285 @dataroaring
[fix] (compaction) add error log and fix time series (#38791)
dfbbd3d 
## Proposed changes


Issue Number: close #xxx

<!--Describe your changes.-->
GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Aug 27, 2024
@csun5285
[fix] (compaction) fix time series (apache#38791) (apache#39053)
194d845 
## Proposed changes

pick from master apache#38791
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@qidaye qidaye qidaye left review comments

@xiaokang xiaokang xiaokang approved these changes

@airborne12 airborne12 airborne12 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.15-merged dev/2.1.6-merged dev/3.0.2-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@csun5285 @doris-robot @xiaokang @airborne12 @qidaye @dataroaring