[fix](polixy)support drop policy for user or role #29488

zddr · 2024-01-03T13:43:15Z

Proposed changes

Issue Number: close #xxx

support DROP ROW POLICY test_row_policy_1 on table1 [FOR user| ROLE role]

row policy name,tableName and user or role as a whole unique identifiers

change KW_SHOW KW_ROW KW_POLICY KW_FOR KW_ROLE STRING_LITERAL:role to KW_SHOW KW_ROW KW_POLICY KW_FOR KW_ROLE ident:role

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

zddr · 2024-01-03T13:43:26Z

run buildall

doris-robot · 2024-01-03T14:02:49Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 6acafd473851b9927e67a5e6858dbd3441175dbe, data reload: false

------ Round 1 ----------------------------------
q1	17645	5143	5169	5143
q2	2040	172	152	152
q3	10511	1059	1159	1059
q4	10192	815	824	815
q5	7806	2938	2746	2746
q6	216	135	138	135
q7	936	508	528	508
q8	9307	2033	2021	2021
q9	6948	6468	6385	6385
q10	8279	3078	3012	3012
q11	424	225	202	202
q12	388	229	228	228
q13	18026	3613	3625	3613
q14	252	222	214	214
q15	570	508	512	508
q16	493	413	395	395
q17	967	465	495	465
q18	7442	6793	6697	6697
q19	1575	1371	1401	1371
q20	735	339	336	336
q21	2767	2410	2417	2410
q22	374	322	311	311
Total cold run time: 107893 ms
Total hot run time: 38726 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5098	5073	5094	5073
q2	338	242	245	242
q3	3301	3279	3272	3272
q4	2124	2050	2022	2022
q5	5781	5762	5760	5760
q6	219	128	125	125
q7	2302	1928	1919	1919
q8	3372	3402	3436	3402
q9	8777	8705	8666	8666
q10	3776	3856	3827	3827
q11	575	475	473	473
q12	794	626	663	626
q13	14723	3220	3191	3191
q14	285	293	304	293
q15	567	518	509	509
q16	549	495	499	495
q17	1931	1745	1735	1735
q18	8792	8402	8298	8298
q19	1616	1600	1597	1597
q20	2179	1954	1938	1938
q21	5585	5202	5305	5202
q22	538	439	458	439
Total cold run time: 73222 ms
Total hot run time: 59104 ms

doris-robot · 2024-01-03T14:03:24Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 6acafd473851b9927e67a5e6858dbd3441175dbe, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5613	5106	5126	5106
q2	390	165	189	165
q3	1453	1187	1076	1076
q4	1089	812	771	771
q5	3128	3144	3116	3116
q6	234	141	136	136
q7	993	559	515	515
q8	2156	2303	2313	2303
q9	6692	6675	6618	6618
q10	3162	3160	3118	3118
q11	338	212	217	212
q12	374	222	236	222
q13	4428	3645	3658	3645
q14	244	213	223	213
q15	591	533	528	528
q16	451	401	396	396
q17	1049	552	571	552
q18	7083	6763	6730	6730
q19	1660	1544	1635	1544
q20	586	341	331	331
q21	2925	2450	2520	2450
q22	397	316	333	316
Total cold run time: 45036 ms
Total hot run time: 40063 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5059	5103	5086	5086
q2	344	242	229	229
q3	3353	3328	3329	3328
q4	2160	2028	2023	2023
q5	5970	5945	5952	5945
q6	225	127	131	127
q7	2369	1909	1942	1909
q8	3555	3683	3677	3677
q9	9066	9010	8977	8977
q10	3867	3901	3936	3901
q11	584	470	469	469
q12	807	620	637	620
q13	3889	3208	3188	3188
q14	307	281	276	276
q15	581	536	532	532
q16	557	512	496	496
q17	2031	1827	1826	1826
q18	8843	8432	8306	8306
q19	1751	1701	1698	1698
q20	2285	2002	1973	1973
q21	5734	5283	5346	5283
q22	543	500	470	470
Total cold run time: 63880 ms
Total hot run time: 60339 ms

zddr · 2024-01-03T14:03:41Z

run buildall

doris-robot · 2024-01-03T14:23:02Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 8fc98d57d7f06e3c8eb69c4c6c8931eabc861164, data reload: false

------ Round 1 ----------------------------------
q1	17653	5147	5087	5087
q2	2029	160	146	146
q3	10584	1052	1125	1052
q4	10182	852	813	813
q5	7779	2918	2945	2918
q6	213	140	139	139
q7	911	523	551	523
q8	9275	1989	2008	1989
q9	6962	6375	6362	6362
q10	8285	2990	2903	2903
q11	442	218	231	218
q12	393	237	233	233
q13	18007	3594	3607	3594
q14	236	224	207	207
q15	559	532	525	525
q16	502	395	399	395
q17	965	460	537	460
q18	7476	6816	6684	6684
q19	1604	1350	1330	1330
q20	679	322	363	322
q21	2802	2364	2434	2364
q22	373	313	334	313
Total cold run time: 107911 ms
Total hot run time: 38577 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5111	5099	5003	5003
q2	347	238	237	237
q3	3302	3274	3299	3274
q4	2101	2012	2015	2012
q5	5803	5791	5780	5780
q6	214	126	130	126
q7	2340	1940	1921	1921
q8	3367	3424	3463	3424
q9	8805	8759	8742	8742
q10	3783	3807	3844	3807
q11	576	489	475	475
q12	791	652	625	625
q13	13384	3140	3192	3140
q14	287	265	271	265
q15	581	515	519	515
q16	544	477	506	477
q17	1947	1785	1749	1749
q18	8816	8262	8288	8262
q19	1632	1599	1566	1566
q20	2190	1976	1937	1937
q21	5523	5245	5254	5245
q22	527	444	505	444
Total cold run time: 71971 ms
Total hot run time: 59026 ms

doris-robot · 2024-01-03T14:23:35Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 8fc98d57d7f06e3c8eb69c4c6c8931eabc861164, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5576	5114	5172	5114
q2	414	187	159	159
q3	1458	1196	1201	1196
q4	1096	764	783	764
q5	3165	3128	3116	3116
q6	231	142	137	137
q7	995	557	509	509
q8	2146	2284	2302	2284
q9	6677	6700	6663	6663
q10	3181	3077	3144	3077
q11	347	204	208	204
q12	379	234	231	231
q13	4403	3632	3670	3632
q14	256	224	208	208
q15	590	543	531	531
q16	447	396	391	391
q17	1040	542	500	500
q18	7086	6755	6764	6755
q19	1651	1541	1419	1419
q20	574	326	347	326
q21	2796	2484	2518	2484
q22	398	336	322	322
Total cold run time: 44906 ms
Total hot run time: 40022 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5082	5087	5061	5061
q2	335	233	242	233
q3	3365	3339	3310	3310
q4	2149	2003	2022	2003
q5	5944	5901	5896	5896
q6	225	128	127	127
q7	2384	1925	1918	1918
q8	3544	3633	3675	3633
q9	9098	8976	8983	8976
q10	3854	3908	3940	3908
q11	578	483	486	483
q12	799	629	636	629
q13	3873	3176	3180	3176
q14	307	269	261	261
q15	589	536	528	528
q16	550	497	479	479
q17	2011	1827	1830	1827
q18	8750	8362	8340	8340
q19	1735	1688	1675	1675
q20	2246	1968	1981	1968
q21	5765	5382	5355	5355
q22	547	479	472	472
Total cold run time: 63730 ms
Total hot run time: 60258 ms

doris-robot · 2024-01-03T14:34:27Z

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit 8fc98d57d7f06e3c8eb69c4c6c8931eabc861164, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	916	376	357	357
query2	3548	1942	2010	1942
query3	5759	208	202	202
query4	27767	22709	22629	22629
query5	3452	575	591	575
query6	260	182	185	182
query7	4161	272	271	271
query8	215	201	196	196
query9	7413	2543	2592	2543
query10	316	249	271	249
query11	16413	15572	15635	15572
query12	127	78	74	74
query13	1393	326	332	326
query14	9661	7038	7052	7038
query15	221	190	187	187
query16	4438	268	261	261
query17	1607	499	494	494
query18	1351	287	271	271
query19	192	142	136	136
query20	85	78	80	78
query21	180	104	99	99
query22	5017	4904	4695	4695
query23	32580	31776	31490	31490
query24	12395	2825	2823	2823
query25	642	335	331	331
query26	1663	141	143	141
query27	2901	282	277	277
query28	6989	1916	1900	1900
query29	2650	400	384	384
query30	295	155	157	155
query31	1026	776	771	771
query32	89	57	55	55
query33	745	277	273	273
query34	954	450	448	448
query35	914	789	760	760
query36	1399	1307	1297	1297
query37	168	66	71	66
query38	3428	3301	3259	3259
query39	1335	1310	1288	1288
query40	295	97	93	93
query41	39	37	35	35
query42	92	79	83	79
query43	545	483	483	483
query44	1117	700	714	700
query45	193	187	179	179
query46	1097	660	670	660
query47	1723	1567	1517	1517
query48	349	253	252	252
query49	1157	321	332	321
query50	807	371	340	340
query51	5491	5327	5334	5327
query52	96	90	89	89
query53	217	152	149	149
query54	1689	572	569	569
query55	99	88	78	78
query56	201	191	194	191
query57	969	947	874	874
query58	227	188	197	188
query59	2780	2652	2658	2652
query60	241	225	234	225
query61	83	82	82	82
query62	645	444	458	444
query63	169	146	145	145
query64	4990	1770	1693	1693
query65	3412	3298	3321	3298
query66	1364	339	330	330
query67	15899	15410	15428	15410
query68	12104	534	558	534
query69	485	253	258	253
query70	1719	1494	1522	1494
query71	415	218	226	218
query72	5347	3543	3586	3543
query73	2630	313	320	313
query74	6914	6356	6407	6356
query75	5303	2370	2319	2319
query76	5572	1087	1094	1087
query77	646	245	255	245
query78	9631	8879	8900	8879
query79	2301	529	510	510
query80	570	353	341	341
query81	484	221	223	221
query82	209	95	92	92
query83	180	138	135	135
query84	245	55	52	52
query85	940	270	261	261
query86	386	389	386	386
query87	3547	3389	3314	3314
query88	3123	2259	2255	2255
query89	358	265	261	261
query90	1922	193	199	193
query91	118	91	92	91
query92	62	58	53	53
query93	1713	459	444	444
query94	868	182	179	179
query95	472	425	420	420
query96	635	322	318	318
query97	4398	4184	4160	4160
query98	219	200	187	187
query99	1086	826	776	776
Total cold run time: 285253 ms
Total hot run time: 180388 ms

doris-robot · 2024-01-03T14:36:05Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.39 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.7 seconds inserted 10000000 Rows, about 361K ops/s
storage size: 17183888586 Bytes

zddr · 2024-01-04T01:58:56Z

run p0

zddr · 2024-01-04T01:59:52Z

run pipelinex_p0

zddr · 2024-01-04T10:41:48Z

run buildall

doris-robot · 2024-01-04T11:00:36Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit f05fcc458706b420348db6c7ca4aad6f1440a4e2, data reload: false

------ Round 1 ----------------------------------
q1	17696	5202	5139	5139
q2	2017	152	145	145
q3	10544	1083	1142	1083
q4	10191	794	840	794
q5	7783	2965	2922	2922
q6	214	134	139	134
q7	953	509	499	499
q8	9284	2003	2025	2003
q9	7138	6394	6363	6363
q10	8293	3032	3033	3032
q11	441	223	217	217
q12	388	228	232	228
q13	18003	3649	3625	3625
q14	249	219	211	211
q15	554	518	520	518
q16	515	410	403	403
q17	965	506	499	499
q18	7308	6692	6689	6689
q19	1586	1423	1321	1321
q20	686	346	340	340
q21	2806	2456	2359	2359
q22	372	305	328	305
Total cold run time: 107986 ms
Total hot run time: 38829 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5192	5040	5129	5040
q2	347	238	248	238
q3	3322	3299	3278	3278
q4	2134	2040	2055	2040
q5	5788	5793	5776	5776
q6	217	129	130	129
q7	2334	1918	1919	1918
q8	3372	3440	3467	3440
q9	8813	8746	8786	8746
q10	3785	3828	3833	3828
q11	580	488	491	488
q12	808	632	629	629
q13	6940	3208	3199	3199
q14	300	266	263	263
q15	577	525	511	511
q16	578	509	501	501
q17	1917	1752	1798	1752
q18	8697	8289	8267	8267
q19	1642	1563	1611	1563
q20	2174	1951	1933	1933
q21	5550	5203	5234	5203
q22	525	458	531	458
Total cold run time: 65592 ms
Total hot run time: 59200 ms

doris-robot · 2024-01-04T11:12:02Z

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit f05fcc458706b420348db6c7ca4aad6f1440a4e2, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	933	347	327	327
query2	6423	1880	1908	1880
query3	6674	207	204	204
query4	26810	22520	22309	22309
query5	3881	563	529	529
query6	294	189	192	189
query7	4598	278	261	261
query8	233	202	193	193
query9	8241	2516	2506	2506
query10	464	258	240	240
query11	16247	15509	15393	15393
query12	136	80	76	76
query13	1632	334	314	314
query14	10939	7115	7137	7115
query15	211	181	189	181
query16	6327	270	261	261
query17	1867	505	493	493
query18	1949	282	264	264
query19	193	142	137	137
query20	79	73	77	73
query21	187	103	93	93
query22	4816	4423	4670	4423
query23	32160	31047	31038	31038
query24	12106	2798	2766	2766
query25	607	348	347	347
query26	1789	146	142	142
query27	2914	270	277	270
query28	6874	1880	1854	1854
query29	2165	397	407	397
query30	295	141	149	141
query31	934	762	760	760
query32	91	59	56	56
query33	750	257	248	248
query34	902	449	462	449
query35	870	811	806	806
query36	1320	1206	1175	1175
query37	185	68	62	62
query38	3355	3296	3268	3268
query39	1340	1298	1281	1281
query40	302	86	89	86
query41	38	36	37	36
query42	86	86	80	80
query43	560	500	489	489
query44	1066	699	712	699
query45	203	182	178	178
query46	1066	640	638	638
query47	1716	1541	1490	1490
query48	341	263	258	258
query49	1227	317	328	317
query50	787	339	337	337
query51	5377	5448	5258	5258
query52	89	91	86	86
query53	220	146	146	146
query54	1415	570	580	570
query55	95	87	80	80
query56	186	191	194	191
query57	1022	907	971	907
query58	234	190	203	190
query59	2843	2680	2670	2670
query60	237	222	228	222
query61	89	91	86	86
query62	640	468	407	407
query63	167	151	146	146
query64	5904	1767	1704	1704
query65	3303	3238	3264	3238
query66	1396	355	322	322
query67	15533	14961	15497	14961
query68	11085	520	523	520
query69	540	278	268	268
query70	1736	1572	1528	1528
query71	501	201	201	201
query72	5709	3610	3656	3610
query73	2141	313	315	313
query74	6858	6436	6401	6401
query75	4796	2278	2308	2278
query76	6273	1170	1142	1142
query77	667	249	282	249
query78	9122	8637	8618	8618
query79	1936	505	507	505
query80	568	363	347	347
query81	471	205	209	205
query82	213	100	100	100
query83	172	144	140	140
query84	246	55	55	55
query85	1017	303	281	281
query86	382	396	398	396
query87	3603	3367	3413	3367
query88	2883	2243	2236	2236
query89	342	255	249	249
query90	1927	200	197	197
query91	124	97	95	95
query92	61	55	53	53
query93	1640	431	480	431
query94	878	188	181	181
query95	464	415	407	407
query96	635	317	315	315
query97	4251	4125	4148	4125
query98	225	192	192	192
query99	1127	814	842	814
Total cold run time: 290526 ms
Total hot run time: 178132 ms

doris-robot · 2024-01-04T11:14:19Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.93 seconds
stream load tsv: 573 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.1 seconds inserted 10000000 Rows, about 355K ops/s
storage size: 17183845609 Bytes

doris-robot · 2024-01-04T11:29:10Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit f05fcc458706b420348db6c7ca4aad6f1440a4e2, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5510	5158	5127	5127
q2	383	171	159	159
q3	1479	1168	1158	1158
q4	1085	809	795	795
q5	3191	3123	3114	3114
q6	219	133	133	133
q7	959	556	529	529
q8	2164	2184	2209	2184
q9	6696	6673	6663	6663
q10	3181	3205	3170	3170
q11	344	222	207	207
q12	376	235	229	229
q13	4404	3613	3659	3613
q14	251	225	219	219
q15	578	532	520	520
q16	455	398	387	387
q17	1050	540	516	516
q18	7080	6732	6794	6732
q19	1628	1506	1438	1438
q20	554	348	354	348
q21	2893	2461	2514	2461
q22	401	331	340	331
Total cold run time: 44881 ms
Total hot run time: 40033 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5122	5066	5031	5031
q2	343	235	236	235
q3	3345	3360	3311	3311
q4	2151	1979	1991	1979
q5	5952	5929	5925	5925
q6	225	124	129	124
q7	2392	1906	1918	1906
q8	3551	3693	3672	3672
q9	9035	8991	8974	8974
q10	3865	3905	3915	3905
q11	576	482	477	477
q12	816	614	629	614
q13	3841	3194	3188	3188
q14	314	269	284	269
q15	587	533	530	530
q16	560	506	533	506
q17	2034	1824	1820	1820
q18	8712	8336	8326	8326
q19	1740	1687	1698	1687
q20	2262	1969	1951	1951
q21	5693	5310	5315	5310
q22	581	526	493	493
Total cold run time: 63697 ms
Total hot run time: 60233 ms

morningman

LGTM

github-actions · 2024-01-05T10:40:28Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-01-05T10:40:31Z

PR approved by anyone and no changes requested.

…9645) bp #29488

zddr added 2 commits January 3, 2024 21:37

1

e497a45

1

6acafd4

1

8fc98d5

xiaokang requested a review from morningman January 4, 2024 10:28

Merge branch 'master' into fix_remove_policy

f05fcc4

morningman added the dev/2.0.4 label Jan 5, 2024

morningman approved these changes Jan 5, 2024

View reviewed changes

morningman added the kind/behavior-changed label Jan 5, 2024

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 5, 2024

github-actions bot added the reviewed label Jan 5, 2024

BePPPower approved these changes Jan 5, 2024

View reviewed changes

yiguolei merged commit cc7b948 into apache:master Jan 6, 2024

zddr added a commit to zddr/incubator-doris that referenced this pull request Jan 8, 2024

[fix](polixy)support drop policy for user or role (apache#29488)

36b68a7

xiaokang mentioned this pull request Jan 8, 2024

[branch-2.0](polixy)support drop policy for user or role (#29488) #29645

Merged

morningman pushed a commit that referenced this pull request Jan 11, 2024

[branch-2.0](polixy)support drop policy for user or role (#29488) (#2…

ceb4fd5

…9645) bp #29488

morningman added dev/2.0.4-merged and removed dev/2.0.4 labels Jan 11, 2024

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024

[fix](polixy)support drop policy for user or role (apache#29488)

2c2a931

xiaokang mentioned this pull request Jan 14, 2024

Release Note 2.0.4 #29906

Closed

xiaokang mentioned this pull request Feb 23, 2024

Release Note 2.0.5 #31356

Closed

zddr deleted the fix_remove_policy branch March 28, 2024 02:37

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

[fix](polixy)support drop policy for user or role #29488

[fix](polixy)support drop policy for user or role #29488

Uh oh!

Conversation

zddr commented Jan 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

zddr commented Jan 3, 2024

Uh oh!

doris-robot commented Jan 3, 2024

Uh oh!

doris-robot commented Jan 3, 2024

Uh oh!

zddr commented Jan 3, 2024

Uh oh!

doris-robot commented Jan 3, 2024

Uh oh!

doris-robot commented Jan 3, 2024

Uh oh!

doris-robot commented Jan 3, 2024

Uh oh!

doris-robot commented Jan 3, 2024

Uh oh!

zddr commented Jan 4, 2024

Uh oh!

zddr commented Jan 4, 2024

Uh oh!

zddr commented Jan 4, 2024

Uh oh!

doris-robot commented Jan 4, 2024

Uh oh!

doris-robot commented Jan 4, 2024

Uh oh!

doris-robot commented Jan 4, 2024

Uh oh!

doris-robot commented Jan 4, 2024

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 5, 2024

Uh oh!

github-actions bot commented Jan 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

zddr commented Jan 3, 2024 •

edited

Loading