Skip to content

[feat](debugpoint) add a backup block debug point for ccr test #50380

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 2 commits into from
Apr 25, 2025
Merged

[feat](debugpoint) add a backup block debug point for ccr test #50380

dataroaring merged 2 commits into from
Apr 25, 2025

Conversation

@koarz
Copy link
Contributor

@koarz koarz commented Apr 24, 2025 •
edited
Loading

What problem does this PR solve?

use for selectdb/ccr-syncer#600

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Apr 24, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@koarz koarz mentioned this pull request Apr 24, 2025
Merged
@koarz
Copy link
Contributor Author

koarz commented Apr 24, 2025

run buildall

dataroaring
dataroaring previously approved these changes Apr 25, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 25, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Apr 25, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 25, 2025

PR approved by anyone and no changes requested.

@koarz
Copy link
Contributor Author

koarz commented Apr 25, 2025

run buildall

@dataroaring dataroaring added dev/2.1.x dev/3.0.x usercase Important user case type label labels Apr 25, 2025
@koarz
Copy link
Contributor Author

koarz commented Apr 25, 2025

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Apr 25, 2025
@doris-robot
Copy link

doris-robot commented Apr 25, 2025

TPC-H: Total hot run time: 34154 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 41b369a3e41346167b27c7282b494f4f6d67592d, data reload: false

------ Round 1 ----------------------------------
q1	26256	5100	5071	5071
q2	2081	275	178	178
q3	10398	1247	720	720
q4	10247	1022	525	525
q5	8135	2402	2387	2387
q6	270	164	129	129
q7	924	774	612	612
q8	9326	1310	1098	1098
q9	7001	5130	5125	5125
q10	6876	2316	1877	1877
q11	487	290	269	269
q12	338	366	209	209
q13	17773	3719	3068	3068
q14	219	225	202	202
q15	521	477	485	477
q16	461	434	396	396
q17	605	864	378	378
q18	7613	7195	7283	7195
q19	1296	940	564	564
q20	331	329	219	219
q21	4365	3463	2509	2509
q22	1006	1028	946	946
Total cold run time: 116529 ms
Total hot run time: 34154 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5131	5009	5087	5009
q2	233	330	223	223
q3	2127	2659	2260	2260
q4	1473	1830	1503	1503
q5	4513	4448	4325	4325
q6	210	167	123	123
q7	1935	1888	1693	1693
q8	2583	2568	2465	2465
q9	7137	7119	7212	7119
q10	2991	3207	2738	2738
q11	586	491	505	491
q12	678	770	605	605
q13	3519	3928	3216	3216
q14	275	316	271	271
q15	532	489	485	485
q16	460	488	463	463
q17	1154	1528	1408	1408
q18	7695	7616	7392	7392
q19	852	835	875	835
q20	1982	2004	1930	1930
q21	5363	4755	4514	4514
q22	1070	1038	990	990
Total cold run time: 52499 ms
Total hot run time: 50058 ms

@doris-robot
Copy link

doris-robot commented Apr 25, 2025

TPC-DS: Total hot run time: 185425 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 41b369a3e41346167b27c7282b494f4f6d67592d, data reload: false

query1	1015	473	478	473
query2	6549	1843	1813	1813
query3	6757	237	220	220
query4	26346	23670	23274	23274
query5	4285	622	472	472
query6	295	203	178	178
query7	4621	486	283	283
query8	302	273	246	246
query9	8626	2577	2582	2577
query10	474	311	290	290
query11	15420	15051	14803	14803
query12	159	118	110	110
query13	1667	547	407	407
query14	8796	6228	6296	6228
query15	210	189	166	166
query16	7222	658	490	490
query17	1212	753	581	581
query18	1966	412	313	313
query19	199	195	160	160
query20	121	116	128	116
query21	217	125	110	110
query22	4115	4147	3995	3995
query23	33878	32901	32848	32848
query24	8465	2373	2364	2364
query25	529	452	404	404
query26	1245	275	151	151
query27	2734	492	328	328
query28	4324	2111	2089	2089
query29	754	584	420	420
query30	285	220	185	185
query31	930	857	760	760
query32	74	69	66	66
query33	558	356	301	301
query34	788	839	500	500
query35	785	832	750	750
query36	931	1006	886	886
query37	107	98	79	79
query38	4152	4191	4120	4120
query39	1445	1395	1376	1376
query40	214	117	104	104
query41	57	59	56	56
query42	118	106	102	102
query43	472	504	473	473
query44	1277	800	790	790
query45	181	177	171	171
query46	826	1013	629	629
query47	1735	1816	1704	1704
query48	366	419	300	300
query49	776	492	428	428
query50	635	682	402	402
query51	4155	4137	4008	4008
query52	109	104	97	97
query53	238	266	187	187
query54	580	572	511	511
query55	81	84	85	84
query56	300	302	289	289
query57	1135	1122	1087	1087
query58	269	263	259	259
query59	2530	2674	2538	2538
query60	321	328	315	315
query61	147	126	126	126
query62	821	742	635	635
query63	218	182	186	182
query64	4290	986	678	678
query65	4346	4285	4276	4276
query66	1150	405	329	329
query67	15412	15267	15133	15133
query68	7961	875	566	566
query69	469	300	272	272
query70	1218	1066	1099	1066
query71	471	316	317	316
query72	5571	4769	4700	4700
query73	701	599	339	339
query74	9248	9087	8913	8913
query75	3793	3194	2697	2697
query76	3622	1174	732	732
query77	788	380	287	287
query78	9904	10145	9252	9252
query79	2174	856	579	579
query80	579	580	440	440
query81	499	257	220	220
query82	483	135	96	96
query83	255	253	232	232
query84	242	108	86	86
query85	785	351	315	315
query86	410	310	303	303
query87	4477	4385	4292	4292
query88	3726	2230	2246	2230
query89	398	325	285	285
query90	1810	219	227	219
query91	140	141	109	109
query92	75	61	56	56
query93	1627	939	590	590
query94	672	402	287	287
query95	378	295	291	291
query96	482	569	275	275
query97	3177	3233	3093	3093
query98	225	213	208	208
query99	1399	1442	1254	1254
Total cold run time: 273076 ms
Total hot run time: 185425 ms

@doris-robot
Copy link

doris-robot commented Apr 25, 2025

ClickBench: Total hot run time: 29.66 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 41b369a3e41346167b27c7282b494f4f6d67592d, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.10	0.11
query3	0.26	0.20	0.19
query4	1.59	0.19	0.19
query5	0.58	0.60	0.59
query6	1.20	0.71	0.72
query7	0.03	0.01	0.02
query8	0.04	0.03	0.04
query9	0.58	0.53	0.52
query10	0.58	0.57	0.56
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	1.19	1.21	1.18
query15	0.87	0.88	0.85
query16	0.39	0.40	0.37
query17	1.01	0.99	1.04
query18	0.21	0.20	0.19
query19	1.96	1.84	1.83
query20	0.01	0.01	0.02
query21	15.42	0.92	0.57
query22	0.76	1.34	0.74
query23	14.76	1.39	0.63
query24	7.22	1.57	0.75
query25	0.48	0.11	0.09
query26	0.68	0.17	0.14
query27	0.05	0.05	0.05
query28	9.24	0.86	0.45
query29	12.54	4.07	3.36
query30	0.25	0.10	0.06
query31	2.81	0.62	0.39
query32	3.22	0.55	0.48
query33	3.02	3.07	3.11
query34	15.76	5.04	4.41
query35	4.54	4.52	4.48
query36	0.68	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.14
query41	0.06	0.03	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.47 s
Total hot run time: 29.66 s

dataroaring
dataroaring approved these changes Apr 25, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 25, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Apr 25, 2025

PR approved by at least one committer and no changes requested.

@dataroaring dataroaring merged commit 0a05e37 into apache:master Apr 25, 2025
24 of 25 checks passed
github-actions bot pushed a commit that referenced this pull request Apr 25, 2025
[feat](debugpoint) add a backup block debug point for ccr test (#50380)
1ede119 
use for selectdb/ccr-syncer#600
@github-actions github-actions bot mentioned this pull request Apr 25, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Apr 25, 2025
[feat](debugpoint) add a backup block debug point for ccr test (#50380)
1da5da7 
use for selectdb/ccr-syncer#600
@github-actions github-actions bot mentioned this pull request Apr 25, 2025
Closed
dataroaring pushed a commit that referenced this pull request Apr 27, 2025
@github-actions
branch-3.0: [feat](debugpoint) add a backup block debug point for ccr…
f3b0232 
… test #50380 (#50429)

Cherry-picked from #50380

Co-authored-by: koarz <lihao@selectdb.com>
@koarz koarz deleted the debug_point_ccr_case branch April 28, 2025 03:42
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
[feat](debugpoint) add a backup block debug point for ccr test (apach…
4472627 
…e#50380)

use for selectdb/ccr-syncer#600
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

+1 more reviewer

@wyxxxcat wyxxxcat wyxxxcat approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.x dev/3.0.6-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@koarz @hello-stephen @doris-robot @dataroaring @wyxxxcat