Skip to content

Conversation

@hubgeter
Copy link
Contributor

@hubgeter hubgeter commented Sep 7, 2024

bp #39239
just pick : fix event npe

Proposed changes

Issue Number: close #xxx

…using meta cache and add parameters to the Hive catalog. (apache#39239) (apache#39865)

bp apache#39239

Co-authored-by: daidai <2017501503@qq.com>
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@hubgeter
Copy link
Contributor Author

hubgeter commented Sep 7, 2024

run buildall

@hubgeter hubgeter changed the title [fix](hive)Modify the Hive notification event processing method when using meta cache and add parameters to the Hive catalog. (#39239) (#39239) [fix](hive)Modify the Hive notification event processing method when using meta cache and add parameters to the Hive catalog. (#39239) Sep 7, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 49232 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e7fa670a98f6c12c6ed61039ec9ba88dbc08aa09, data reload: false

------ Round 1 ----------------------------------
q1	17735	4448	4359	4359
q2	2036	179	141	141
q3	10251	1910	1937	1910
q4	10387	1285	1332	1285
q5	8438	3885	3946	3885
q6	234	118	123	118
q7	2010	1606	1592	1592
q8	9533	2736	2697	2697
q9	12659	9977	9735	9735
q10	8662	3520	3487	3487
q11	421	252	234	234
q12	473	299	303	299
q13	18618	3986	4040	3986
q14	356	331	335	331
q15	501	458	466	458
q16	518	469	459	459
q17	1138	993	990	990
q18	7298	6894	6990	6894
q19	1708	1585	1562	1562
q20	518	326	298	298
q21	4492	4128	4113	4113
q22	495	399	410	399
Total cold run time: 118481 ms
Total hot run time: 49232 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4414	4333	4317	4317
q2	321	230	231	230
q3	4200	4186	4180	4180
q4	2784	2762	2770	2762
q5	7193	7139	7097	7097
q6	239	120	121	120
q7	3208	2829	2848	2829
q8	4453	4650	4650	4650
q9	14092	13751	14013	13751
q10	4351	4314	4301	4301
q11	760	700	668	668
q12	1032	878	848	848
q13	7615	3757	3746	3746
q14	454	424	423	423
q15	499	468	450	450
q16	648	573	575	573
q17	3782	4025	3867	3867
q18	8815	8686	8758	8686
q19	1701	1719	1644	1644
q20	2376	2119	2115	2115
q21	8420	8410	8451	8410
q22	1013	916	894	894
Total cold run time: 82370 ms
Total hot run time: 76561 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 211817 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e7fa670a98f6c12c6ed61039ec9ba88dbc08aa09, data reload: false

query1	924	394	408	394
query2	6545	2130	2102	2102
query3	6914	207	203	203
query4	23652	21346	21674	21346
query5	19732	6462	6513	6462
query6	290	221	240	221
query7	4331	301	291	291
query8	265	248	228	228
query9	3085	2656	2573	2573
query10	469	319	318	318
query11	15878	14785	14856	14785
query12	127	73	75	73
query13	1029	444	440	440
query14	17524	13349	13166	13166
query15	374	217	230	217
query16	5669	286	255	255
query17	1730	951	915	915
query18	891	322	311	311
query19	190	152	151	151
query20	81	79	81	79
query21	189	103	95	95
query22	5241	5024	4932	4932
query23	34232	33547	33438	33438
query24	6935	6307	6276	6276
query25	527	425	428	425
query26	1184	157	159	157
query27	2435	302	292	292
query28	6057	2249	2238	2238
query29	3071	2793	2844	2793
query30	246	174	170	170
query31	982	738	743	738
query32	69	61	61	61
query33	442	253	252	252
query34	857	469	473	469
query35	1128	931	921	921
query36	1294	1332	1163	1163
query37	96	60	59	59
query38	3016	2897	2931	2897
query39	1385	1320	1329	1320
query40	248	93	93	93
query41	39	36	44	36
query42	86	83	82	82
query43	618	580	575	575
query44	1161	716	732	716
query45	246	229	230	229
query46	1221	960	968	960
query47	1770	1714	1821	1714
query48	502	408	403	403
query49	624	366	380	366
query50	865	613	597	597
query51	4751	4700	4658	4658
query52	98	88	82	82
query53	234	190	192	190
query54	2678	2506	2480	2480
query55	77	93	77	77
query56	231	199	205	199
query57	1400	1109	1057	1057
query58	227	203	214	203
query59	3320	3224	3299	3224
query60	220	207	200	200
query61	95	92	91	91
query62	842	494	448	448
query63	198	178	174	174
query64	3474	1560	1482	1482
query65	3629	3529	3522	3522
query66	811	405	431	405
query67	16303	15796	16844	15796
query68	9076	651	652	651
query69	487	246	258	246
query70	1780	1500	1399	1399
query71	396	305	308	305
query72	6837	4842	4773	4773
query73	754	320	324	320
query74	6288	5813	5813	5813
query75	4710	3720	3639	3639
query76	4869	1127	1224	1127
query77	682	253	263	253
query78	12536	11860	11509	11509
query79	8818	645	637	637
query80	1638	381	388	381
query81	496	244	234	234
query82	1684	100	94	94
query83	171	130	134	130
query84	258	68	69	68
query85	884	313	316	313
query86	341	327	286	286
query87	3237	3053	2980	2980
query88	4873	2318	2296	2296
query89	484	281	267	267
query90	1992	205	211	205
query91	163	124	131	124
query92	58	51	51	51
query93	6794	611	574	574
query94	750	207	208	207
query95	1929	1926	1948	1926
query96	644	335	320	320
query97	6448	6351	6366	6351
query98	227	207	210	207
query99	2808	846	923	846
Total cold run time: 318166 ms
Total hot run time: 211817 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.7 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e7fa670a98f6c12c6ed61039ec9ba88dbc08aa09, data reload: false

query1	0.02	0.03	0.02
query2	0.06	0.03	0.02
query3	0.24	0.06	0.05
query4	1.78	0.07	0.07
query5	0.53	0.52	0.52
query6	1.24	0.61	0.61
query7	0.01	0.01	0.02
query8	0.04	0.02	0.02
query9	0.52	0.48	0.48
query10	0.53	0.52	0.53
query11	0.12	0.09	0.09
query12	0.11	0.09	0.09
query13	0.62	0.62	0.61
query14	0.80	0.77	0.80
query15	0.78	0.76	0.75
query16	0.36	0.37	0.39
query17	1.00	0.99	1.01
query18	0.22	0.26	0.23
query19	1.95	1.84	1.81
query20	0.02	0.01	0.01
query21	15.47	0.57	0.56
query22	2.04	1.90	1.18
query23	16.85	1.14	0.94
query24	4.46	1.53	1.73
query25	0.38	0.06	0.06
query26	0.60	0.15	0.17
query27	0.05	0.05	0.03
query28	7.41	0.73	0.77
query29	12.77	2.25	2.30
query30	0.56	0.50	0.51
query31	2.82	0.39	0.36
query32	3.37	0.49	0.49
query33	3.09	3.09	3.04
query34	15.26	4.80	4.76
query35	4.86	4.82	4.86
query36	1.05	1.02	1.02
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.13	0.14
query41	0.08	0.01	0.02
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 102.39 s
Total hot run time: 30.7 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit e7fa670a98f6c12c6ed61039ec9ba88dbc08aa09 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.9 seconds inserted 10000000 Rows, about 456K ops/s

@xiaokang xiaokang merged commit ed8aaf4 into apache:branch-2.0 Sep 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants