Skip to content

branch-3.0: [test](cloud-mow)Add update big delete bitmap case #47097 #47767

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Feb 12, 2025
Merged

branch-3.0: [test](cloud-mow)Add update big delete bitmap case #47097 #47767

dataroaring merged 1 commit into from
Feb 12, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Feb 11, 2025

Cherry-picked from #47097

@hust-hhb
[test](cloud-mow)Add update big delete bitmap case (#47097)
7933e5f 
This case is used to test the scenario of updating a large number of
delete bitmap when doing compaction, if delete bitmap size over the fdb
limit(10MB), updating delete bitmap will fail, pr #46969 fix this
problem, before the fix, this case will fail with error msg "Transaction
exceeds byte limit" on metaservice.
@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 11, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Feb 11, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 11, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Feb 11, 2025

TPC-H: Total hot run time: 40765 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7933e5f75a23e2d63ef33eb65be932f62b41debb, data reload: false

------ Round 1 ----------------------------------
q1	17573	7333	7252	7252
q2	2081	175	162	162
q3	10578	1055	1197	1055
q4	10574	753	726	726
q5	7750	2861	2819	2819
q6	243	150	145	145
q7	966	611	600	600
q8	9352	1914	2054	1914
q9	6605	6396	6413	6396
q10	7013	2296	2399	2296
q11	468	272	272	272
q12	398	213	212	212
q13	17775	2977	2997	2977
q14	241	208	206	206
q15	574	525	531	525
q16	665	572	603	572
q17	1003	613	563	563
q18	7247	6813	6759	6759
q19	1390	1038	1058	1038
q20	472	202	194	194
q21	4001	3247	3091	3091
q22	1089	1013	991	991
Total cold run time: 108058 ms
Total hot run time: 40765 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7268	7197	7237	7197
q2	335	235	230	230
q3	2868	2933	2937	2933
q4	2051	1850	1824	1824
q5	5699	5703	5718	5703
q6	230	139	144	139
q7	2266	1774	1863	1774
q8	3351	3554	3472	3472
q9	8791	8852	8825	8825
q10	3609	3600	3554	3554
q11	609	519	499	499
q12	796	639	603	603
q13	10079	3204	3205	3204
q14	307	279	274	274
q15	584	526	523	523
q16	698	665	639	639
q17	1824	1628	1592	1592
q18	8178	7742	7755	7742
q19	1651	1609	1506	1506
q20	2143	1870	1881	1870
q21	5444	5377	5465	5377
q22	1140	1056	1011	1011
Total cold run time: 69921 ms
Total hot run time: 60491 ms

@doris-robot
Copy link

doris-robot commented Feb 11, 2025

TPC-DS: Total hot run time: 198491 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7933e5f75a23e2d63ef33eb65be932f62b41debb, data reload: false

query1	1296	879	886	879
query2	6223	2062	1966	1966
query3	10877	4470	4508	4470
query4	66531	28689	23601	23601
query5	5018	467	446	446
query6	410	176	174	174
query7	5534	313	318	313
query8	300	217	231	217
query9	8675	2665	2669	2665
query10	459	269	261	261
query11	17167	15357	15857	15357
query12	162	100	106	100
query13	1458	465	428	428
query14	10420	7177	7349	7177
query15	207	177	185	177
query16	7242	481	512	481
query17	1094	589	593	589
query18	1910	334	325	325
query19	225	156	172	156
query20	120	110	111	110
query21	211	104	107	104
query22	4820	4640	4621	4621
query23	34239	34311	34336	34311
query24	6275	3027	3023	3023
query25	549	428	438	428
query26	687	178	175	175
query27	1879	357	349	349
query28	4109	2442	2411	2411
query29	736	480	439	439
query30	237	159	164	159
query31	1035	823	846	823
query32	66	55	59	55
query33	406	290	297	290
query34	978	525	515	515
query35	870	726	702	702
query36	1133	985	975	975
query37	126	79	75	75
query38	4102	3956	4070	3956
query39	1515	1476	1469	1469
query40	193	104	101	101
query41	50	46	49	46
query42	110	102	101	101
query43	538	494	500	494
query44	1215	827	831	827
query45	181	163	166	163
query46	1196	747	751	747
query47	2064	1901	1924	1901
query48	502	388	387	387
query49	753	391	398	391
query50	881	432	430	430
query51	7380	7356	7100	7100
query52	100	91	91	91
query53	264	183	183	183
query54	574	458	443	443
query55	76	76	76	76
query56	266	257	257	257
query57	1262	1140	1090	1090
query58	223	204	212	204
query59	3109	2929	2880	2880
query60	283	259	253	253
query61	150	157	107	107
query62	848	731	729	729
query63	224	200	188	188
query64	1387	696	668	668
query65	3243	3229	3184	3184
query66	701	302	305	302
query67	16135	15637	15564	15564
query68	3959	600	581	581
query69	435	268	271	268
query70	1225	1144	1107	1107
query71	350	256	265	256
query72	6367	4081	4071	4071
query73	803	351	351	351
query74	10569	8950	9358	8950
query75	3348	2669	2640	2640
query76	1890	1045	1070	1045
query77	486	276	272	272
query78	10654	9685	9647	9647
query79	1979	605	624	605
query80	1397	429	429	429
query81	523	238	235	235
query82	1290	119	115	115
query83	273	147	142	142
query84	282	81	81	81
query85	1027	315	285	285
query86	423	310	291	291
query87	4449	4324	4312	4312
query88	3886	2384	2344	2344
query89	426	296	289	289
query90	1958	192	190	190
query91	185	153	146	146
query92	68	51	49	49
query93	2759	555	546	546
query94	855	304	297	297
query95	357	260	256	256
query96	683	284	285	284
query97	3345	3153	3124	3124
query98	213	198	195	195
query99	1684	1413	1409	1409
Total cold run time: 321357 ms
Total hot run time: 198491 ms

@doris-robot
Copy link

doris-robot commented Feb 11, 2025

ClickBench: Total hot run time: 33.5 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7933e5f75a23e2d63ef33eb65be932f62b41debb, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.02
query3	0.23	0.06	0.07
query4	1.64	0.11	0.10
query5	0.52	0.51	0.52
query6	1.13	0.73	0.73
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.59	0.51	0.53
query10	0.56	0.57	0.56
query11	0.14	0.11	0.11
query12	0.13	0.11	0.11
query13	0.60	0.60	0.60
query14	2.86	2.76	2.74
query15	0.90	0.82	0.82
query16	0.38	0.38	0.39
query17	1.05	1.04	1.03
query18	0.24	0.22	0.22
query19	1.95	1.86	1.97
query20	0.01	0.02	0.01
query21	15.36	0.58	0.59
query22	3.19	2.88	2.28
query23	16.82	1.07	0.86
query24	3.13	1.54	1.20
query25	0.22	0.12	0.06
query26	0.58	0.14	0.13
query27	0.04	0.05	0.05
query28	10.12	1.10	1.07
query29	12.56	3.18	3.22
query30	0.25	0.06	0.07
query31	2.85	0.39	0.37
query32	3.27	0.46	0.45
query33	2.97	2.99	3.00
query34	17.13	4.51	4.48
query35	4.66	4.53	4.66
query36	0.68	0.48	0.49
query37	0.09	0.05	0.06
query38	0.06	0.03	0.04
query39	0.04	0.02	0.03
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.02	0.02
Total cold run time: 107.43 s
Total hot run time: 33.5 s

dataroaring
dataroaring approved these changes Feb 12, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 12, 2025
@github-actions
Copy link
Contributor Author

github-actions bot commented Feb 12, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor Author

github-actions bot commented Feb 12, 2025

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit ef7a2c1 into branch-3.0 Feb 12, 2025
22 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-47097-branch-3.0 branch February 12, 2025 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@hello-stephen @doris-robot @dataroaring @hust-hhb