Skip to content

[fix] (streamload) fixed the issue of data loss due to concurrency when importing data from streamload (#48948) #49666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Mar 31, 2025
Merged

[fix] (streamload) fixed the issue of data loss due to concurrency when importing data from streamload (#48948) #49666

dataroaring merged 1 commit into from
Mar 31, 2025

Conversation

@liaoxin01
Copy link
Contributor

@liaoxin01 liaoxin01 commented Mar 30, 2025

Cherry-picked from #48948

@ghkang98 @liaoxin01
[fix] (streamload) fixed the issue of data loss due to concurrency wh…
c3965d8 
…en importing data from streamload (apache#48948)

This is mainly to solve the multithreading problem caused by
inconsistent visible order of EOS and data_queue variables in doris's
streamload function and asyn_result_writer in the process_block process
due to the compilation reordering of the ARM system or the weak memory
order problem, which leads to data loss.

Problem Summary:

Mainly in the arm architecture, streamload has data loss problems. The
transaction of importing data can be executed and submitted normally,
but the NumberTotalRowshe NumberFilterRows in the returned load result
are both zero
[Uploading stream_load_lost_data.docx…]()
((https://github.com/user-attachments/files/19201955/stream_load.docx))

Co-authored-by: lik40 <lik40@chinatelecom.cn>
@liaoxin01 liaoxin01 requested a review from dataroaring as a code owner March 30, 2025 02:55
@Thearas
Copy link
Contributor

Thearas commented Mar 30, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

dataroaring
dataroaring approved these changes Mar 30, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Mar 30, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Mar 30, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Mar 30, 2025

PR approved by anyone and no changes requested.

@liaoxin01
Copy link
Contributor Author

liaoxin01 commented Mar 30, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Mar 30, 2025

TPC-H: Total hot run time: 40111 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c3965d84782319ee665169c4ea7b77a24e6b94bc, data reload: false

------ Round 1 ----------------------------------
q1	17566	6944	6657	6657
q2	2077	176	166	166
q3	10590	1086	1248	1086
q4	10512	763	729	729
q5	7753	2873	2783	2783
q6	228	134	135	134
q7	1004	642	617	617
q8	9355	1935	2090	1935
q9	6720	6473	6472	6472
q10	7041	2255	2277	2255
q11	474	268	269	268
q12	406	220	215	215
q13	17805	3031	2987	2987
q14	234	208	215	208
q15	504	473	461	461
q16	655	589	574	574
q17	1001	617	556	556
q18	7408	6692	6624	6624
q19	1404	1077	947	947
q20	474	206	204	204
q21	4006	3247	3364	3247
q22	1100	1001	986	986
Total cold run time: 108317 ms
Total hot run time: 40111 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6629	6629	6590	6590
q2	330	227	222	222
q3	2917	2796	2977	2796
q4	2090	1798	1789	1789
q5	5732	6050	5787	5787
q6	213	136	134	134
q7	2238	1817	1794	1794
q8	3402	3571	3495	3495
q9	8800	8831	8929	8831
q10	3603	3543	3569	3543
q11	599	501	492	492
q12	804	612	587	587
q13	9865	3183	3181	3181
q14	297	267	288	267
q15	523	454	480	454
q16	679	652	648	648
q17	1872	1670	1615	1615
q18	8337	7702	7880	7702
q19	1660	1610	1542	1542
q20	2070	1890	1867	1867
q21	5647	5304	5348	5304
q22	1146	1046	1015	1015
Total cold run time: 69453 ms
Total hot run time: 59655 ms

@doris-robot
Copy link

doris-robot commented Mar 30, 2025

TPC-DS: Total hot run time: 198578 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c3965d84782319ee665169c4ea7b77a24e6b94bc, data reload: false

query1	1282	919	918	918
query2	6229	2084	2100	2084
query3	10919	4451	4304	4304
query4	61155	33057	23710	23710
query5	4629	457	446	446
query6	394	178	177	177
query7	5450	304	309	304
query8	292	216	206	206
query9	8277	2623	2600	2600
query10	466	278	260	260
query11	16138	15167	15665	15167
query12	158	108	102	102
query13	1423	477	445	445
query14	10841	7139	6874	6874
query15	214	176	189	176
query16	7229	476	546	476
query17	1206	585	592	585
query18	1854	334	328	328
query19	217	171	164	164
query20	118	120	109	109
query21	203	106	105	105
query22	4822	4805	4677	4677
query23	34363	34027	34249	34027
query24	6122	2950	3034	2950
query25	514	403	399	399
query26	658	172	170	170
query27	1773	351	351	351
query28	3983	2514	2439	2439
query29	671	459	431	431
query30	248	165	165	165
query31	1015	853	863	853
query32	65	59	58	58
query33	403	298	301	298
query34	933	505	513	505
query35	851	746	743	743
query36	1112	965	964	964
query37	116	69	70	69
query38	4152	4077	3951	3951
query39	1547	1482	1468	1468
query40	198	100	103	100
query41	47	50	52	50
query42	111	100	104	100
query43	541	503	494	494
query44	1199	838	827	827
query45	184	172	168	168
query46	1148	755	709	709
query47	2086	1955	1978	1955
query48	475	393	404	393
query49	725	406	411	406
query50	845	429	428	428
query51	7412	7316	7281	7281
query52	107	95	97	95
query53	259	184	183	183
query54	578	475	470	470
query55	79	78	77	77
query56	266	257	242	242
query57	1337	1165	1098	1098
query58	219	206	202	202
query59	3248	3074	3109	3074
query60	273	255	256	255
query61	110	109	112	109
query62	769	662	686	662
query63	222	187	184	184
query64	1513	770	741	741
query65	3234	3219	3252	3219
query66	675	312	311	311
query67	16076	15781	15751	15751
query68	4065	613	584	584
query69	433	302	270	270
query70	1208	1139	1114	1114
query71	368	268	269	268
query72	6536	4193	4020	4020
query73	759	347	363	347
query74	10288	9204	9294	9204
query75	3361	2640	2673	2640
query76	2101	1041	1072	1041
query77	508	278	279	278
query78	10597	9632	9495	9495
query79	2088	612	620	612
query80	1362	421	427	421
query81	524	237	238	237
query82	996	88	92	88
query83	162	142	139	139
query84	283	82	76	76
query85	969	304	295	295
query86	423	293	298	293
query87	4462	4313	4340	4313
query88	4108	2425	2388	2388
query89	417	290	292	290
query90	1984	186	188	186
query91	184	150	149	149
query92	62	51	50	50
query93	2536	589	565	565
query94	827	291	308	291
query95	362	260	266	260
query96	626	279	280	279
query97	3323	3254	3252	3252
query98	217	204	198	198
query99	1559	1293	1303	1293
Total cold run time: 313676 ms
Total hot run time: 198578 ms

@doris-robot
Copy link

doris-robot commented Mar 30, 2025

ClickBench: Total hot run time: 31.1 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c3965d84782319ee665169c4ea7b77a24e6b94bc, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.54	0.51	0.50
query6	1.15	0.73	0.73
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.57	0.50	0.52
query10	0.56	0.56	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.61	0.59
query14	2.73	2.87	2.73
query15	0.89	0.81	0.83
query16	0.38	0.39	0.39
query17	1.07	1.04	1.03
query18	0.25	0.22	0.20
query19	1.91	1.78	2.11
query20	0.02	0.01	0.01
query21	15.37	0.60	0.58
query22	2.55	1.76	1.16
query23	17.06	0.99	0.90
query24	3.49	0.39	1.57
query25	0.30	0.20	0.09
query26	0.35	0.14	0.14
query27	0.05	0.04	0.05
query28	10.37	0.49	0.44
query29	12.91	3.29	3.27
query30	0.25	0.06	0.06
query31	2.87	0.40	0.39
query32	3.23	0.47	0.47
query33	2.98	3.00	3.01
query34	16.56	4.53	4.55
query35	4.53	4.55	4.61
query36	0.70	0.48	0.47
query37	0.10	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.02	0.03
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 107.04 s
Total hot run time: 31.1 s

@doris-robot
Copy link

doris-robot commented Mar 30, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/22) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 39.06% (10228/26185)
Line Coverage 30.42% (87225/286721)
Region Coverage 29.49% (44868/152146)
Branch Coverage 26.03% (22855/87814)

@dataroaring dataroaring merged commit 09a7a35 into apache:branch-3.0 Mar 31, 2025
20 of 24 checks passed
@gavinchou gavinchou mentioned this pull request Apr 23, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@liaoxin01 @Thearas @doris-robot @dataroaring @ghkang98