Skip to content

branch-3.0: [improve](binlog) Allow commit txn without waiting txn publish #48961 #49265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Mar 20, 2025
Merged

branch-3.0: [improve](binlog) Allow commit txn without waiting txn publish #48961 #49265

dataroaring merged 1 commit into from
Mar 20, 2025

Conversation

@w41ter
Copy link
Contributor

@w41ter w41ter commented Mar 19, 2025

cherry pick from #48961

@w41ter w41ter requested a review from dataroaring as a code owner March 19, 2025 10:04
@w41ter
Copy link
Contributor Author

w41ter commented Mar 19, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Mar 19, 2025

TeamCity cloud ut coverage result:
Function Coverage: 83.00% (1074/1294)
Line Coverage: 65.93% (17723/26883)
Region Coverage: 65.31% (8719/13350)
Branch Coverage: 55.28% (4702/8506)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bbe4f665891c8bb0ae5555773caf923f82baa09a_bbe4f665891c8bb0ae5555773caf923f82baa09a_cloud/report/index.html

@doris-robot
Copy link

doris-robot commented Mar 19, 2025

TPC-H: Total hot run time: 40469 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bbe4f665891c8bb0ae5555773caf923f82baa09a, data reload: false

------ Round 1 ----------------------------------
q1	17582	6833	6764	6764
q2	2078	175	176	175
q3	10636	1108	1191	1108
q4	10584	736	753	736
q5	7764	2907	2834	2834
q6	222	133	133	133
q7	981	620	632	620
q8	9364	2009	2088	2009
q9	6603	6466	6434	6434
q10	6948	2288	2305	2288
q11	462	278	268	268
q12	402	220	214	214
q13	17788	2978	3021	2978
q14	230	204	212	204
q15	505	473	480	473
q16	665	593	611	593
q17	979	579	614	579
q18	7348	6744	6717	6717
q19	1401	1046	1118	1046
q20	462	208	197	197
q21	4062	3385	3141	3141
q22	1109	1013	958	958
Total cold run time: 108175 ms
Total hot run time: 40469 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6618	6590	6578	6578
q2	332	237	228	228
q3	2925	2797	3013	2797
q4	2055	1808	1806	1806
q5	5772	5750	5810	5750
q6	214	127	129	127
q7	2249	1858	1825	1825
q8	3404	3557	3566	3557
q9	8873	8943	8893	8893
q10	3595	3576	3565	3565
q11	589	513	501	501
q12	827	653	619	619
q13	8618	3230	3123	3123
q14	303	271	274	271
q15	507	469	463	463
q16	673	644	648	644
q17	1849	1671	1590	1590
q18	8363	7933	7751	7751
q19	1680	1636	1502	1502
q20	2118	1893	1872	1872
q21	5590	5365	5194	5194
q22	1147	1027	1012	1012
Total cold run time: 68301 ms
Total hot run time: 59668 ms

@doris-robot
Copy link

doris-robot commented Mar 19, 2025

TPC-DS: Total hot run time: 198001 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bbe4f665891c8bb0ae5555773caf923f82baa09a, data reload: false

query1	1281	915	916	915
query2	6282	2063	2006	2006
query3	10786	4497	4260	4260
query4	60892	29299	23844	23844
query5	5225	468	452	452
query6	410	175	181	175
query7	5443	322	309	309
query8	312	225	233	225
query9	8342	2681	2668	2668
query10	459	281	263	263
query11	17467	15375	16010	15375
query12	163	100	102	100
query13	1411	447	443	443
query14	11354	7310	7857	7310
query15	215	185	184	184
query16	7186	454	519	454
query17	1222	590	597	590
query18	1848	314	322	314
query19	211	163	163	163
query20	122	110	111	110
query21	212	110	103	103
query22	4660	4369	4672	4369
query23	35077	34048	34444	34048
query24	6156	2998	2948	2948
query25	540	425	423	423
query26	655	175	183	175
query27	1952	372	365	365
query28	4527	2485	2490	2485
query29	730	436	441	436
query30	239	168	161	161
query31	1014	821	821	821
query32	77	60	60	60
query33	415	288	297	288
query34	926	501	513	501
query35	881	747	757	747
query36	1109	986	946	946
query37	118	70	71	70
query38	4232	4115	4022	4022
query39	1514	1457	1457	1457
query40	203	101	97	97
query41	49	46	49	46
query42	117	103	102	102
query43	541	494	486	486
query44	1225	849	833	833
query45	196	171	171	171
query46	1159	747	731	731
query47	1959	1924	1900	1900
query48	484	390	387	387
query49	721	383	389	383
query50	838	427	426	426
query51	7344	7193	7300	7193
query52	106	97	94	94
query53	265	191	193	191
query54	563	481	471	471
query55	83	77	79	77
query56	277	261	249	249
query57	1290	1152	1143	1143
query58	218	207	210	207
query59	3347	2932	2829	2829
query60	268	259	255	255
query61	123	105	107	105
query62	769	668	662	662
query63	217	184	185	184
query64	1395	663	641	641
query65	3283	3189	3230	3189
query66	685	300	291	291
query67	16040	15655	15510	15510
query68	4155	603	592	592
query69	432	287	261	261
query70	1180	1129	1123	1123
query71	325	263	260	260
query72	6460	4060	4091	4060
query73	766	358	356	356
query74	10090	8895	9026	8895
query75	3375	2651	2691	2651
query76	2040	1049	1090	1049
query77	480	275	279	275
query78	10558	9709	9529	9529
query79	1417	615	612	612
query80	869	434	431	431
query81	508	245	243	243
query82	1289	94	86	86
query83	254	157	143	143
query84	290	84	77	77
query85	900	296	290	290
query86	319	292	304	292
query87	4506	4263	4279	4263
query88	3787	2427	2389	2389
query89	428	293	290	290
query90	1945	188	187	187
query91	185	147	151	147
query92	63	48	52	48
query93	1965	558	557	557
query94	701	299	281	281
query95	355	255	256	255
query96	605	284	281	281
query97	3340	3152	3157	3152
query98	215	199	213	199
query99	1557	1289	1308	1289
Total cold run time: 314789 ms
Total hot run time: 198001 ms

@doris-robot
Copy link

doris-robot commented Mar 19, 2025

ClickBench: Total hot run time: 32.9 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bbe4f665891c8bb0ae5555773caf923f82baa09a, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.04	0.05
query3	0.23	0.05	0.06
query4	1.65	0.08	0.08
query5	0.53	0.52	0.51
query6	1.13	0.74	0.74
query7	0.02	0.02	0.01
query8	0.04	0.05	0.05
query9	0.56	0.49	0.51
query10	0.56	0.55	0.56
query11	0.19	0.13	0.12
query12	0.16	0.13	0.12
query13	0.63	0.61	0.60
query14	2.77	2.74	2.89
query15	0.91	0.84	0.84
query16	0.37	0.37	0.38
query17	1.07	1.05	0.99
query18	0.20	0.20	0.21
query19	1.90	1.84	2.04
query20	0.02	0.02	0.01
query21	15.35	0.67	0.66
query22	4.21	7.82	1.96
query23	18.20	1.40	1.39
query24	2.26	0.23	0.22
query25	0.16	0.09	0.08
query26	0.27	0.18	0.18
query27	0.08	0.08	0.08
query28	13.18	0.60	0.57
query29	12.63	3.36	3.32
query30	0.24	0.05	0.05
query31	2.85	0.40	0.41
query32	3.22	0.49	0.48
query33	3.01	3.02	3.06
query34	16.83	4.52	4.54
query35	4.60	4.50	4.50
query36	0.67	0.48	0.51
query37	0.19	0.17	0.16
query38	0.17	0.15	0.14
query39	0.05	0.04	0.04
query40	0.18	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.06
query43	0.05	0.04	0.05
Total cold run time: 111.61 s
Total hot run time: 32.9 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Mar 19, 2025

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 38.87% (10166/26154)
Line Coverage 30.29% (86666/286090)
Region Coverage 29.35% (44540/151734)
Branch Coverage 25.88% (22655/87546)

dataroaring
dataroaring approved these changes Mar 20, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 555d782 into apache:branch-3.0 Mar 20, 2025
22 of 24 checks passed
@w41ter w41ter deleted the pick_48961_to_origin_branch-3.0 branch March 20, 2025 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@w41ter @doris-robot @hello-stephen @dataroaring