Skip to content

[fix](variable) modify @@auto_commit column type to BIGINT in Nereids. #33887

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 1 commit into from
Apr 19, 2024
Merged

[fix](variable) modify @@auto_commit column type to BIGINT in Nereids. #33887

yiguolei merged 1 commit into from
Apr 19, 2024

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Apr 19, 2024

Proposed changes

#33282 only change the type in old planner.
This PR modify it in Nereids

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

doris-robot commented Apr 19, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 19, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Apr 19, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 19, 2024

PR approved by anyone and no changes requested.

@morningman
Copy link
Contributor Author

morningman commented Apr 19, 2024

run buildall

@morningman morningman added usercase Important user case type label and removed dev/2.0.x labels Apr 19, 2024
@doris-robot
Copy link

doris-robot commented Apr 19, 2024

TPC-H: Total hot run time: 38503 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0a2a48d083e1a220b36b379e0d84e8df12bb9c1a, data reload: false

------ Round 1 ----------------------------------
q1	17608	4390	4254	4254
q2	2021	183	190	183
q3	10451	1116	1141	1116
q4	10188	746	761	746
q5	7509	2719	2646	2646
q6	215	133	131	131
q7	1008	594	607	594
q8	9235	2027	2024	2024
q9	7452	6563	6511	6511
q10	8555	3517	3559	3517
q11	469	229	230	229
q12	485	226	216	216
q13	17776	2955	2972	2955
q14	273	234	229	229
q15	532	490	484	484
q16	509	388	382	382
q17	954	730	668	668
q18	7235	6805	6779	6779
q19	6468	1522	1491	1491
q20	654	325	299	299
q21	3485	2758	2743	2743
q22	371	306	308	306
Total cold run time: 113453 ms
Total hot run time: 38503 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4352	4249	4204	4204
q2	374	276	266	266
q3	3011	2712	2802	2712
q4	1880	1547	1570	1547
q5	5286	5319	5296	5296
q6	214	122	122	122
q7	2199	1857	1890	1857
q8	3179	3397	3322	3322
q9	8636	8591	8688	8591
q10	4081	3874	3987	3874
q11	629	491	508	491
q12	812	620	644	620
q13	16739	3197	3177	3177
q14	319	285	292	285
q15	536	500	480	480
q16	504	460	455	455
q17	1854	1563	1500	1500
q18	8010	7948	7801	7801
q19	1652	1571	1542	1542
q20	2044	1870	1842	1842
q21	5115	4904	4963	4904
q22	548	471	476	471
Total cold run time: 71974 ms
Total hot run time: 55359 ms

@doris-robot
Copy link

doris-robot commented Apr 19, 2024

TPC-DS: Total hot run time: 184962 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0a2a48d083e1a220b36b379e0d84e8df12bb9c1a, data reload: false

query1	889	357	351	351
query2	6157	2672	2404	2404
query3	6672	210	208	208
query4	22809	21314	21267	21267
query5	4136	412	408	408
query6	281	186	214	186
query7	4576	286	276	276
query8	249	189	187	187
query9	8533	2316	2276	2276
query10	405	231	240	231
query11	14744	14179	14112	14112
query12	136	87	83	83
query13	1633	370	357	357
query14	9228	8069	7542	7542
query15	286	193	185	185
query16	8150	264	267	264
query17	1925	576	560	560
query18	2057	282	273	273
query19	321	155	158	155
query20	93	86	85	85
query21	205	128	122	122
query22	5102	4848	4860	4848
query23	34117	33256	33278	33256
query24	11098	2979	2975	2975
query25	593	403	412	403
query26	710	160	147	147
query27	2257	366	372	366
query28	6165	2061	2068	2061
query29	868	616	614	614
query30	320	185	175	175
query31	987	760	761	760
query32	90	50	52	50
query33	664	244	243	243
query34	895	485	477	477
query35	853	723	714	714
query36	1089	931	949	931
query37	117	79	71	71
query38	3522	3305	3316	3305
query39	1617	1613	1575	1575
query40	171	129	123	123
query41	48	42	40	40
query42	106	113	95	95
query43	594	529	570	529
query44	1094	753	739	739
query45	301	265	292	265
query46	1096	725	742	725
query47	2021	1914	1920	1914
query48	382	305	311	305
query49	843	399	393	393
query50	806	404	396	396
query51	6931	6761	6789	6761
query52	91	89	87	87
query53	340	271	270	270
query54	300	222	224	222
query55	79	72	71	71
query56	240	223	222	222
query57	1220	1130	1121	1121
query58	219	196	192	192
query59	3518	3206	3043	3043
query60	269	228	234	228
query61	90	86	85	85
query62	607	441	442	441
query63	301	278	273	273
query64	5024	3998	3823	3823
query65	3044	3005	3023	3005
query66	753	320	329	320
query67	15388	14870	15000	14870
query68	6148	534	534	534
query69	537	297	298	297
query70	1248	1153	1158	1153
query71	1454	1255	1262	1255
query72	6506	2593	2414	2414
query73	722	316	313	313
query74	6852	6472	6345	6345
query75	3701	2653	2669	2653
query76	3983	1123	1001	1001
query77	627	262	260	260
query78	10849	10130	10333	10130
query79	4535	523	531	523
query80	1185	431	424	424
query81	510	242	232	232
query82	1361	98	89	89
query83	198	168	164	164
query84	260	88	78	78
query85	1244	267	260	260
query86	447	298	304	298
query87	3501	3296	3261	3261
query88	4712	2319	2332	2319
query89	494	367	365	365
query90	1910	178	181	178
query91	119	93	94	93
query92	54	45	45	45
query93	5344	507	502	502
query94	1029	181	175	175
query95	391	295	294	294
query96	608	261	262	261
query97	3166	2939	2970	2939
query98	232	218	211	211
query99	1229	835	865	835
Total cold run time: 285581 ms
Total hot run time: 184962 ms

@doris-robot
Copy link

doris-robot commented Apr 19, 2024

ClickBench: Total hot run time: 30.63 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0a2a48d083e1a220b36b379e0d84e8df12bb9c1a, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.03
query3	0.23	0.05	0.05
query4	1.66	0.11	0.10
query5	0.52	0.50	0.51
query6	1.46	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.55	0.55	0.55
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.60	0.59	0.59
query14	0.75	0.76	0.79
query15	0.82	0.79	0.81
query16	0.37	0.36	0.37
query17	1.00	1.00	0.95
query18	0.22	0.24	0.25
query19	1.85	1.65	1.71
query20	0.02	0.01	0.01
query21	15.39	0.66	0.65
query22	4.50	6.02	2.31
query23	18.29	1.46	1.27
query24	1.40	0.42	0.20
query25	0.14	0.07	0.10
query26	0.26	0.17	0.16
query27	0.08	0.07	0.08
query28	13.37	1.00	0.97
query29	12.59	3.27	3.25
query30	0.26	0.06	0.06
query31	2.88	0.38	0.37
query32	3.27	0.46	0.46
query33	2.81	2.82	2.79
query34	17.03	4.41	4.46
query35	4.47	4.53	4.48
query36	0.65	0.47	0.46
query37	0.17	0.15	0.15
query38	0.15	0.14	0.15
query39	0.04	0.03	0.04
query40	0.18	0.14	0.13
query41	0.08	0.04	0.05
query42	0.05	0.04	0.05
query43	0.03	0.04	0.03
Total cold run time: 109.17 s
Total hot run time: 30.63 s

@doris-robot
Copy link

doris-robot commented Apr 19, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 0a2a48d083e1a220b36b379e0d84e8df12bb9c1a with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.6 seconds inserted 10000000 Rows, about 735K ops/s

@yiguolei yiguolei merged commit b82b8dd into apache:master Apr 19, 2024
yiguolei pushed a commit that referenced this pull request Apr 19, 2024
@morningman @Doris-Extras
[fix](variable) modify @@auto_commit column type to BIGINT in Nereids. …
00adad8 
…#33887
dataroaring pushed a commit that referenced this pull request Apr 22, 2024
@morningman @dataroaring
[fix](variable) modify @@auto_commit column type to BIGINT in Nereids. …
b7468d8 
…#33887
morningman added a commit to morningman/doris that referenced this pull request Jun 20, 2024
@morningman
[fix](variable) modify @@auto_commit column type to BIGINT in Nereids. …
0469dd6 
…apache#33887
@morningman morningman mentioned this pull request Jun 20, 2024
Merged
morningman added a commit that referenced this pull request Jun 20, 2024
@morningman
[fix](variable) modify @@auto_commit column type to BIGINT (#36584)
132b9d6 
bp #33887 #33282
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@morningman
[fix](variable) modify @@auto_commit column type to BIGINT (apache#36584
e187329 
)

bp apache#33887 apache#33282
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.12-merged dev/2.1.3-merged dev/3.0.0-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@morningman @doris-robot @yiguolei @morrySnow @xiaokang @zy-kkk @dataroaring