[feat](nereids) support explain delete from clause #36782

englefly · 2024-06-25T02:38:08Z

Proposed changes

support explain like:
explain delete from T where A=1

Issue Number: close #xxx

doris-robot · 2024-06-25T02:38:14Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

morrySnow

delete from dup/agg table still cannot explain, and will throw wrong error message, we should handle it by throw right error message.
add explain update case to ensure all dml is ok

fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/DeleteFromCommand.java

fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java

englefly · 2024-06-25T09:47:30Z

run buildall

englefly · 2024-06-25T09:57:49Z

run buildall

doris-robot · 2024-06-25T10:24:43Z

TPC-H: Total hot run time: 39338 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7d8699e8b031a6e66534e855e2e098cf8d6ca425, data reload: false

------ Round 1 ----------------------------------
q1	17586	4307	4275	4275
q2	2020	189	180	180
q3	10465	1127	1123	1123
q4	10182	737	745	737
q5	7443	2649	2612	2612
q6	214	133	130	130
q7	939	596	594	594
q8	9254	2056	2048	2048
q9	9388	6508	6419	6419
q10	8971	3718	3679	3679
q11	463	250	234	234
q12	482	217	217	217
q13	18032	2963	3005	2963
q14	278	233	216	216
q15	510	468	476	468
q16	506	371	370	370
q17	949	716	612	612
q18	7908	7340	7315	7315
q19	6444	1372	1323	1323
q20	668	323	308	308
q21	4849	3184	3893	3184
q22	382	333	331	331
Total cold run time: 117933 ms
Total hot run time: 39338 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4434	4222	4290	4222
q2	378	265	260	260
q3	2965	2884	2858	2858
q4	1984	1719	1689	1689
q5	5599	5483	5501	5483
q6	214	125	126	125
q7	2191	1884	1832	1832
q8	3255	3386	3405	3386
q9	8688	8659	8795	8659
q10	4095	3863	3694	3694
q11	594	496	516	496
q12	830	631	656	631
q13	17270	3179	3138	3138
q14	312	282	283	282
q15	517	490	474	474
q16	468	426	425	425
q17	1809	1520	1497	1497
q18	8049	7889	7804	7804
q19	1819	1536	1522	1522
q20	3052	1904	1883	1883
q21	5015	4861	4845	4845
q22	604	548	543	543
Total cold run time: 74142 ms
Total hot run time: 55748 ms

doris-robot · 2024-06-25T10:35:41Z

TPC-DS: Total hot run time: 172807 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7d8699e8b031a6e66534e855e2e098cf8d6ca425, data reload: false

query1	929	384	369	369
query2	6440	2359	2207	2207
query3	6620	205	214	205
query4	20204	17432	17204	17204
query5	3642	467	460	460
query6	240	174	168	168
query7	4585	311	290	290
query8	313	289	301	289
query9	8444	2395	2387	2387
query10	588	286	279	279
query11	10594	9999	10222	9999
query12	115	82	81	81
query13	1646	403	377	377
query14	9494	7317	6172	6172
query15	210	180	179	179
query16	7784	276	262	262
query17	1519	520	526	520
query18	1946	278	264	264
query19	192	149	154	149
query20	84	81	77	77
query21	201	128	125	125
query22	4334	4352	4103	4103
query23	33662	33730	33705	33705
query24	11149	2863	2952	2863
query25	584	361	379	361
query26	714	156	156	156
query27	2304	329	329	329
query28	5910	2184	2144	2144
query29	873	621	620	620
query30	238	148	149	148
query31	946	778	746	746
query32	96	56	53	53
query33	759	277	278	277
query34	1061	500	477	477
query35	769	630	625	625
query36	1134	975	996	975
query37	151	75	71	71
query38	2951	2893	2879	2879
query39	871	813	850	813
query40	213	128	131	128
query41	53	54	53	53
query42	118	99	98	98
query43	574	532	531	531
query44	1176	738	730	730
query45	189	164	158	158
query46	1072	693	700	693
query47	1874	1803	1773	1773
query48	362	305	297	297
query49	821	408	404	404
query50	763	396	381	381
query51	6883	6716	6740	6716
query52	101	94	94	94
query53	354	286	282	282
query54	903	439	432	432
query55	76	74	73	73
query56	278	262	256	256
query57	1128	1066	1078	1066
query58	254	235	242	235
query59	3279	3004	3077	3004
query60	279	272	266	266
query61	91	89	89	89
query62	622	453	456	453
query63	325	291	288	288
query64	8507	2226	1741	1741
query65	3164	3117	3115	3115
query66	761	335	340	335
query67	15339	14995	14922	14922
query68	4608	543	560	543
query69	590	539	395	395
query70	1188	1135	1191	1135
query71	479	273	267	267
query72	7518	5767	5472	5472
query73	753	323	325	323
query74	5881	5520	5515	5515
query75	3454	2697	2670	2670
query76	2944	929	881	881
query77	638	293	300	293
query78	10441	9827	9677	9677
query79	2748	525	517	517
query80	2515	458	459	458
query81	577	214	219	214
query82	780	109	101	101
query83	295	175	165	165
query84	270	91	81	81
query85	1252	280	269	269
query86	462	320	330	320
query87	3268	3110	3070	3070
query88	4174	2358	2377	2358
query89	474	392	379	379
query90	1813	246	189	189
query91	126	99	95	95
query92	59	49	47	47
query93	3790	512	517	512
query94	1044	183	181	181
query95	405	310	309	309
query96	592	270	270	270
query97	3227	3052	3030	3030
query98	228	211	193	193
query99	1201	863	857	857
Total cold run time: 271649 ms
Total hot run time: 172807 ms

doris-robot · 2024-06-25T10:41:12Z

ClickBench: Total hot run time: 30.37 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7d8699e8b031a6e66534e855e2e098cf8d6ca425, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.69	0.07	0.07
query5	0.47	0.49	0.47
query6	1.13	0.72	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.05
query9	0.55	0.49	0.49
query10	0.54	0.54	0.54
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.60	0.59	0.57
query14	0.75	0.77	0.78
query15	0.84	0.81	0.81
query16	0.38	0.37	0.35
query17	1.00	1.05	1.02
query18	0.23	0.23	0.25
query19	1.77	1.74	1.83
query20	0.01	0.00	0.00
query21	15.42	0.67	0.65
query22	4.36	7.43	1.66
query23	18.22	1.49	1.25
query24	2.07	0.25	0.23
query25	0.15	0.09	0.08
query26	0.28	0.17	0.18
query27	0.08	0.08	0.08
query28	13.22	1.01	1.00
query29	12.58	3.29	3.33
query30	0.26	0.06	0.06
query31	2.83	0.39	0.39
query32	3.28	0.47	0.47
query33	2.89	2.94	2.91
query34	17.12	4.41	4.43
query35	4.49	4.49	4.50
query36	0.65	0.46	0.47
query37	0.18	0.15	0.15
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.17	0.14	0.14
query41	0.09	0.05	0.04
query42	0.05	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.29 s
Total hot run time: 30.37 s

englefly · 2024-06-27T13:26:57Z

run buildall

englefly · 2024-06-27T13:28:19Z

run buildall

doris-robot · 2024-06-27T13:48:39Z

TPC-H: Total hot run time: 39805 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c61547b6f2fc75188aa6e7676d3c57ae9e7d9266, data reload: false

------ Round 1 ----------------------------------
q1	18028	4638	4339	4339
q2	2673	232	196	196
q3	11116	1252	1095	1095
q4	11218	830	723	723
q5	7517	2710	2668	2668
q6	223	137	138	137
q7	966	606	625	606
q8	9333	2040	2067	2040
q9	8709	6450	6473	6450
q10	8954	3700	3740	3700
q11	460	243	234	234
q12	438	234	217	217
q13	18301	2976	3010	2976
q14	269	225	214	214
q15	517	470	478	470
q16	513	381	379	379
q17	961	654	662	654
q18	8048	7580	7362	7362
q19	1662	1458	1437	1437
q20	635	314	323	314
q21	4955	3255	3906	3255
q22	400	339	357	339
Total cold run time: 115896 ms
Total hot run time: 39805 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4346	4223	4180	4180
q2	380	283	282	282
q3	2951	2731	2718	2718
q4	1860	1614	1639	1614
q5	5233	5250	5269	5250
q6	215	129	128	128
q7	2054	1719	1758	1719
q8	3203	3300	3295	3295
q9	8305	8291	8327	8291
q10	3895	3700	3635	3635
q11	582	499	493	493
q12	783	578	605	578
q13	17024	2978	2996	2978
q14	283	261	261	261
q15	518	480	468	468
q16	466	403	423	403
q17	1734	1471	1460	1460
q18	7653	7400	7306	7306
q19	3948	1584	1645	1584
q20	1976	1798	1750	1750
q21	4881	4742	4632	4632
q22	648	533	543	533
Total cold run time: 72938 ms
Total hot run time: 53558 ms

doris-robot · 2024-06-27T14:00:00Z

TPC-DS: Total hot run time: 169872 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c61547b6f2fc75188aa6e7676d3c57ae9e7d9266, data reload: false

query1	908	370	377	370
query2	6468	2491	2276	2276
query3	6654	206	208	206
query4	19287	17409	17343	17343
query5	4183	471	475	471
query6	278	187	167	167
query7	4601	298	300	298
query8	327	278	304	278
query9	8622	2414	2400	2400
query10	625	307	278	278
query11	10502	10066	10013	10013
query12	135	87	80	80
query13	1641	376	376	376
query14	9335	7810	6576	6576
query15	261	181	187	181
query16	7881	281	274	274
query17	1899	555	532	532
query18	2015	281	276	276
query19	197	162	156	156
query20	89	85	84	84
query21	214	134	127	127
query22	4286	4084	4050	4050
query23	33703	33160	33116	33116
query24	11870	2783	2749	2749
query25	672	376	377	376
query26	1777	159	157	157
query27	2968	318	314	314
query28	7664	2086	2068	2068
query29	1148	628	625	625
query30	282	150	157	150
query31	952	725	762	725
query32	96	53	55	53
query33	799	297	303	297
query34	936	472	479	472
query35	752	632	616	616
query36	1078	931	953	931
query37	212	77	70	70
query38	2901	2746	2773	2746
query39	853	815	796	796
query40	286	126	129	126
query41	57	55	55	55
query42	121	96	103	96
query43	604	534	547	534
query44	1265	732	730	730
query45	196	165	165	165
query46	1064	747	705	705
query47	1877	1772	1770	1770
query48	374	308	313	308
query49	1210	435	413	413
query50	763	387	385	385
query51	6856	6787	6696	6696
query52	111	93	96	93
query53	358	290	292	290
query54	971	454	452	452
query55	77	76	75	75
query56	311	286	281	281
query57	1145	1068	1055	1055
query58	273	246	286	246
query59	3421	3144	3239	3144
query60	284	266	286	266
query61	90	89	96	89
query62	657	455	457	455
query63	315	290	283	283
query64	9850	2214	1742	1742
query65	3227	3093	3101	3093
query66	1382	338	318	318
query67	15657	14804	15115	14804
query68	8874	566	566	566
query69	696	487	397	397
query70	1429	1177	1051	1051
query71	545	270	275	270
query72	8831	5695	2732	2732
query73	2218	334	327	327
query74	6343	5533	5600	5533
query75	4858	2648	2656	2648
query76	5022	971	964	964
query77	777	297	297	297
query78	10855	9715	9738	9715
query79	7849	525	528	525
query80	977	467	462	462
query81	563	217	226	217
query82	711	109	103	103
query83	348	171	172	171
query84	274	83	86	83
query85	1369	272	262	262
query86	400	303	330	303
query87	3316	3083	3104	3083
query88	4495	2382	2407	2382
query89	515	375	378	375
query90	1999	188	189	188
query91	126	101	103	101
query92	69	49	48	48
query93	5757	528	521	521
query94	1248	190	187	187
query95	402	313	316	313
query96	629	266	267	266
query97	3212	3021	3020	3020
query98	231	210	192	192
query99	1234	834	828	828
Total cold run time: 298045 ms
Total hot run time: 169872 ms

doris-robot · 2024-06-27T14:05:32Z

ClickBench: Total hot run time: 30.12 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c61547b6f2fc75188aa6e7676d3c57ae9e7d9266, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.65	0.07	0.07
query5	0.51	0.50	0.49
query6	1.13	0.72	0.73
query7	0.02	0.01	0.01
query8	0.06	0.05	0.05
query9	0.54	0.48	0.49
query10	0.55	0.53	0.54
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.60	0.60	0.61
query14	0.76	0.82	0.77
query15	0.83	0.81	0.81
query16	0.36	0.36	0.37
query17	0.97	0.94	1.00
query18	0.21	0.21	0.26
query19	1.80	1.72	1.70
query20	0.01	0.02	0.01
query21	15.44	0.73	0.66
query22	4.39	7.83	1.44
query23	18.28	1.46	1.35
query24	2.17	0.23	0.22
query25	0.15	0.09	0.08
query26	0.27	0.18	0.18
query27	0.08	0.08	0.08
query28	13.20	1.02	1.00
query29	12.62	3.33	3.32
query30	0.26	0.07	0.06
query31	2.83	0.39	0.38
query32	3.25	0.47	0.47
query33	2.87	2.89	2.93
query34	17.29	4.35	4.39
query35	4.57	4.50	4.49
query36	0.66	0.48	0.46
query37	0.18	0.16	0.15
query38	0.15	0.14	0.14
query39	0.04	0.03	0.03
query40	0.16	0.15	0.14
query41	0.09	0.04	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.69 s
Total hot run time: 30.12 s

englefly · 2024-06-28T03:32:56Z

run p0

englefly · 2024-06-28T11:25:45Z

run p0

github-actions · 2024-07-01T03:04:27Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-07-01T03:04:29Z

PR approved by anyone and no changes requested.

## Proposed changes support explain like: explain delete from T where A=1 Issue Number: close #xxx  (cherry picked from commit dc369cd)

## Proposed changes support explain like: explain delete from T where A=1 Issue Number: close #xxx

## Proposed changes pick from #36782 support explain like: explain delete from T where A=1 Issue Number: close #xxx  (cherry picked from commit dc369cd) ## Proposed changes Issue Number: close #xxx

… in Nereids planner (#38751) ## Proposed changes 1. Fix nereids planner don't use partial update for delete statement on merge-on-write table introduced in #36782 2. add `IS_PARTIAL_UPDATE: true` in `OlapTableSink.getExplainString()` 3. don't convert the delete stmt to partial update if the table has cluster key.

… in Nereids planner (apache#38751) ## Proposed changes 1. Fix nereids planner don't use partial update for delete statement on merge-on-write table introduced in apache#36782 2. add `IS_PARTIAL_UPDATE: true` in `OlapTableSink.getExplainString()` 3. don't convert the delete stmt to partial update if the table has cluster key.

morrySnow reviewed Jun 25, 2024

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/DeleteFromCommand.java Show resolved Hide resolved

fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java Show resolved Hide resolved

englefly force-pushed the explain-delete-from branch from 6dac736 to 2f81b9b Compare June 25, 2024 09:37

englefly force-pushed the explain-delete-from branch from 2f81b9b to 7d8699e Compare June 25, 2024 09:57

morrySnow previously approved these changes Jun 27, 2024

View reviewed changes

morrySnow added the dev/2.1.x label Jun 27, 2024

englefly dismissed morrySnow’s stale review via 4333231 June 27, 2024 13:26

englefly force-pushed the explain-delete-from branch from 7d8699e to 4333231 Compare June 27, 2024 13:26

explain delete from T where ...

c61547b

englefly force-pushed the explain-delete-from branch from 4333231 to c61547b Compare June 27, 2024 13:28

morrySnow approved these changes Jul 1, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 1, 2024

github-actions bot added the reviewed label Jul 1, 2024

LiBinfeng-01 approved these changes Jul 1, 2024

View reviewed changes

englefly merged commit dc369cd into apache:master Jul 1, 2024

englefly deleted the explain-delete-from branch July 1, 2024 08:28

dataroaring pushed a commit that referenced this pull request Jul 2, 2024

[feat](nereids) support explain delete from clause (#36782)

fc78d1f

## Proposed changes support explain like: explain delete from T where A=1 Issue Number: close #xxx

englefly mentioned this pull request Jul 3, 2024

[feat](nereids) support explain delete from clause #36782 #37100

Merged

englefly added the dev/2.1.5-merged label Jul 3, 2024

wm1581066 removed the dev/2.1.x label Jul 3, 2024

bobhan1 mentioned this pull request Aug 2, 2024

[fix](delete) Fix delete stmt on MOW table doesn't use partial update in Nereids planner #38751

Merged

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

[feat](nereids) support explain delete from clause #36782

[feat](nereids) support explain delete from clause #36782

Uh oh!

Conversation

englefly commented Jun 25, 2024

Proposed changes

Uh oh!

doris-robot commented Jun 25, 2024

Uh oh!

morrySnow left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

englefly commented Jun 25, 2024

Uh oh!

englefly commented Jun 25, 2024

Uh oh!

doris-robot commented Jun 25, 2024

Uh oh!

doris-robot commented Jun 25, 2024

Uh oh!

doris-robot commented Jun 25, 2024

Uh oh!

englefly commented Jun 27, 2024

Uh oh!

englefly commented Jun 27, 2024

Uh oh!

doris-robot commented Jun 27, 2024

Uh oh!

doris-robot commented Jun 27, 2024

Uh oh!

doris-robot commented Jun 27, 2024

Uh oh!

englefly commented Jun 28, 2024

Uh oh!

englefly commented Jun 28, 2024

Uh oh!

github-actions bot commented Jul 1, 2024

Uh oh!

github-actions bot commented Jul 1, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants