Skip to content

Conversation

@Jibing-Li
Copy link
Contributor

  1. Refresh follower FE cache after alter column stats. So that follower could update the cached stats too.
  2. Support alter index column stats.

backport #31108

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

…Support alter index column stats (apache#31108)

1. Refresh follower FE cache after alter column stats. So that follower could update the cached stats too.
2. Support alter index column stats.
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@Jibing-Li Jibing-Li marked this pull request as ready for review February 20, 2024 03:56
@Jibing-Li
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added area/planner Issues or PRs related to the query planner kind/test labels Feb 20, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 50290 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 890e9da28094bb9661723511c9f22d7488821f96, data reload: false

------ Round 1 ----------------------------------
q1	17536	4371	4335	4335
q2	2024	151	139	139
q3	10274	1913	1942	1913
q4	10098	1283	1343	1283
q5	8548	3947	3978	3947
q6	232	123	124	123
q7	2042	1589	1614	1589
q8	9250	2699	2718	2699
q9	11120	11666	10814	10814
q10	8651	3494	3478	3478
q11	409	238	239	238
q12	468	296	308	296
q13	18357	3984	4009	3984
q14	353	314	328	314
q15	498	462	450	450
q16	700	592	592	592
q17	1129	955	983	955
q18	7264	6848	6944	6848
q19	1702	1596	1548	1548
q20	540	314	285	285
q21	4416	4107	4081	4081
q22	511	379	404	379
Total cold run time: 116122 ms
Total hot run time: 50290 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4325	4310	4303	4303
q2	321	223	226	223
q3	4191	4161	4101	4101
q4	2739	2768	2738	2738
q5	7324	7165	7216	7165
q6	231	116	112	112
q7	3176	2790	2859	2790
q8	4332	4445	4453	4445
q9	17650	17268	17384	17268
q10	4199	4254	4231	4231
q11	754	684	659	659
q12	1012	841	859	841
q13	5536	3760	3732	3732
q14	457	418	420	418
q15	496	458	446	446
q16	744	690	702	690
q17	3764	3793	3928	3793
q18	8805	8717	8802	8717
q19	1720	1679	1681	1679
q20	2401	2124	2111	2111
q21	8502	8432	8521	8432
q22	1043	949	952	949
Total cold run time: 83722 ms
Total hot run time: 79843 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 240397 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 890e9da28094bb9661723511c9f22d7488821f96, data reload: false

query1	902	392	385	385
query2	6513	2100	2014	2014
query3	6914	207	205	205
query4	20240	18012	17960	17960
query5	19712	6482	6508	6482
query6	291	229	230	229
query7	4154	298	291	291
query8	264	301	229	229
query9	3137	2720	2661	2661
query10	408	282	301	282
query11	11358	10709	10671	10671
query12	123	81	71	71
query13	5584	651	626	626
query14	18074	13637	13611	13611
query15	363	224	233	224
query16	6411	267	270	267
query17	1719	1457	879	879
query18	2290	411	394	394
query19	206	145	144	144
query20	81	73	76	73
query21	182	93	98	93
query22	5381	5191	5143	5143
query23	32564	32011	31956	31956
query24	7483	6439	6486	6439
query25	532	415	411	411
query26	664	164	156	156
query27	2064	291	296	291
query28	6074	2241	2205	2205
query29	2903	2707	2711	2707
query30	243	164	162	162
query31	895	738	742	738
query32	65	60	57	57
query33	391	236	267	236
query34	848	459	480	459
query35	1115	950	956	950
query36	1571	1411	1405	1405
query37	94	59	58	58
query38	3078	2907	2902	2902
query39	1357	1318	1316	1316
query40	302	94	98	94
query41	34	31	31	31
query42	90	81	83	81
query43	620	599	523	523
query44	1144	723	715	715
query45	240	233	220	220
query46	1240	981	967	967
query47	1882	1720	1653	1653
query48	976	683	678	678
query49	637	360	366	360
query50	868	602	626	602
query51	5628	5490	5432	5432
query52	82	77	69	69
query53	433	322	313	313
query54	2646	2473	2466	2466
query55	86	71	72	71
query56	210	218	202	202
query57	1230	1177	1184	1177
query58	214	203	189	189
query59	3530	3302	3110	3110
query60	203	199	202	199
query61	90	85	85	85
query62	874	436	463	436
query63	472	331	333	331
query64	2806	1625	1494	1494
query65	3640	3545	3562	3545
query66	787	364	373	364
query67	16142	15258	14759	14759
query68	8822	650	673	650
query69	558	331	348	331
query70	1864	1657	1758	1657
query71	407	300	302	300
query72	6498	3466	3470	3466
query73	752	325	320	320
query74	6341	5845	5813	5813
query75	5325	3766	3700	3700
query76	5290	1144	1147	1144
query77	828	249	252	249
query78	33879	49681	49652	49652
query79	15475	655	630	630
query80	4687	387	394	387
query81	562	229	233	229
query82	1207	104	89	89
query83	321	130	138	130
query84	254	71	68	68
query85	1928	283	273	273
query86	428	369	380	369
query87	3250	2988	3058	2988
query88	7112	2342	2327	2327
query89	445	279	285	279
query90	2436	200	208	200
query91	158	126	129	126
query92	66	48	52	48
query93	6292	571	583	571
query94	1422	203	203	203
query95	1118	1054	1037	1037
query96	657	329	324	324
query97	6557	6332	6401	6332
query98	189	170	164	164
query99	3770	908	793	793
Total cold run time: 352252 ms
Total hot run time: 240397 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.4 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 890e9da28094bb9661723511c9f22d7488821f96, data reload: false

query1	0.03	0.02	0.02
query2	0.06	0.02	0.02
query3	0.25	0.05	0.04
query4	1.82	0.07	0.07
query5	0.53	0.52	0.52
query6	1.24	0.61	0.61
query7	0.02	0.01	0.02
query8	0.04	0.02	0.02
query9	0.52	0.48	0.48
query10	0.52	0.52	0.54
query11	0.11	0.09	0.09
query12	0.11	0.09	0.09
query13	0.61	0.61	0.61
query14	0.78	0.78	0.78
query15	0.78	0.75	0.75
query16	0.37	0.36	0.38
query17	1.00	1.04	1.00
query18	0.23	0.24	0.26
query19	1.88	1.79	1.83
query20	0.01	0.01	0.01
query21	15.48	0.56	0.55
query22	2.39	1.96	1.38
query23	17.34	0.92	0.89
query24	5.11	0.80	1.32
query25	1.54	0.12	0.11
query26	0.24	0.13	0.13
query27	0.11	0.11	0.11
query28	7.80	0.72	0.69
query29	12.75	2.39	2.34
query30	0.59	0.52	0.52
query31	2.79	0.38	0.39
query32	3.42	0.49	0.49
query33	3.07	3.06	3.09
query34	15.27	4.82	4.85
query35	4.85	4.88	4.83
query36	1.06	1.01	1.01
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.01	0.02	0.02
query40	0.15	0.14	0.14
query41	0.07	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 105.09 s
Total hot run time: 30.4 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 890e9da28094bb9661723511c9f22d7488821f96 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.0 seconds inserted 10000000 Rows, about 454K ops/s

@Jibing-Li Jibing-Li merged commit d38dcd7 into apache:branch-2.0 Feb 20, 2024
@Jibing-Li Jibing-Li deleted the alter2.0 branch February 20, 2024 04:56
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
…Support alter index column stats (apache#31108) (apache#31139)

1. Refresh follower FE cache after alter column stats. So that follower could update the cached stats too.
2. Support alter index column stats.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/planner Issues or PRs related to the query planner kind/test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants