Skip to content

Conversation

@Yulei-Yang
Copy link
Contributor

@Yulei-Yang Yulei-Yang commented Jul 31, 2024

Brought by: #38244

how to reproduce:
create a hms catalog named hive with property use_meta_cache = false, after that restart fe, and run sql switch hive; show databases;, will get error:

ERROR 1105 (HY000): NullPointerException, msg: java.lang.NullPointerException: Cannot invoke "org.apache.doris.datasource.metacache.MetaCache.listNames()" because "this.metaCache" is null

reason, property use_meta_cache of newly created hms catalog after pr #38244 was force convert to true, but field useMetaCache of class ExternalCatalog is not converted at the same time.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Yulei-Yang
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41928 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 741f85433301cd25dd70a11a47cd3b447eb44be5, data reload: false

------ Round 1 ----------------------------------
q1	17876	4138	4090	4090
q2	2010	213	215	213
q3	10806	1310	1291	1291
q4	10168	787	925	787
q5	7655	2911	2985	2911
q6	224	138	134	134
q7	1045	623	633	623
q8	9964	1941	1936	1936
q9	8541	6643	6695	6643
q10	8813	3870	3884	3870
q11	440	258	252	252
q12	419	225	223	223
q13	17881	3073	3041	3041
q14	287	255	260	255
q15	551	477	488	477
q16	548	434	411	411
q17	968	932	914	914
q18	11167	7653	7356	7356
q19	1476	1233	1225	1225
q20	566	340	351	340
q21	5286	4656	4720	4656
q22	354	280	280	280
Total cold run time: 117045 ms
Total hot run time: 41928 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4104	4058	4035	4035
q2	341	236	227	227
q3	3106	3200	3146	3146
q4	1978	2036	2017	2017
q5	5495	5387	5399	5387
q6	223	131	138	131
q7	2213	1815	1764	1764
q8	3320	3379	3324	3324
q9	8571	8715	8640	8640
q10	3974	3920	4028	3920
q11	562	464	473	464
q12	757	592	643	592
q13	13988	3107	3140	3107
q14	305	281	273	273
q15	523	501	500	500
q16	473	421	416	416
q17	1736	1698	1665	1665
q18	8356	7759	7761	7759
q19	3448	1719	1696	1696
q20	2049	1854	1842	1842
q21	5732	5260	5328	5260
q22	511	462	473	462
Total cold run time: 71765 ms
Total hot run time: 56627 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170811 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 741f85433301cd25dd70a11a47cd3b447eb44be5, data reload: false

query1	897	385	365	365
query2	6461	1686	1671	1671
query3	6642	214	220	214
query4	20339	17557	17393	17393
query5	3644	532	515	515
query6	267	162	167	162
query7	4589	293	292	292
query8	255	191	198	191
query9	8501	2376	2403	2376
query10	426	281	270	270
query11	10640	10307	10138	10138
query12	122	89	86	86
query13	1641	378	384	378
query14	8832	7855	7802	7802
query15	215	164	167	164
query16	6992	463	461	461
query17	973	570	573	570
query18	1925	287	288	287
query19	198	150	151	150
query20	97	89	88	88
query21	203	107	111	107
query22	4440	4224	4225	4224
query23	34079	33676	33459	33459
query24	10244	3182	3055	3055
query25	724	439	462	439
query26	1754	158	153	153
query27	3139	298	280	280
query28	7669	2023	2014	2014
query29	1443	441	406	406
query30	238	157	154	154
query31	953	798	783	783
query32	101	54	57	54
query33	722	325	333	325
query34	937	487	498	487
query35	882	751	759	751
query36	1025	904	862	862
query37	278	81	88	81
query38	2881	2788	2790	2788
query39	866	835	814	814
query40	283	111	110	110
query41	41	39	43	39
query42	120	100	108	100
query43	484	408	402	402
query44	1189	726	735	726
query45	205	180	178	178
query46	1086	809	789	789
query47	1795	1745	1781	1745
query48	363	288	292	288
query49	1003	424	409	409
query50	907	428	425	425
query51	6760	6583	6609	6583
query52	94	96	89	89
query53	261	180	178	178
query54	623	445	452	445
query55	76	75	80	75
query56	267	252	261	252
query57	1132	1030	1024	1024
query58	310	316	278	278
query59	2520	2563	2320	2320
query60	292	270	276	270
query61	94	92	91	91
query62	885	675	670	670
query63	208	179	181	179
query64	5623	1917	1895	1895
query65	3161	3089	3098	3089
query66	1329	329	328	328
query67	15454	14820	14837	14820
query68	4373	559	565	559
query69	720	356	332	332
query70	1120	1043	1089	1043
query71	479	283	260	260
query72	8096	2676	2484	2484
query73	803	324	327	324
query74	6198	5788	5703	5703
query75	4258	2700	2729	2700
query76	3111	1292	1395	1292
query77	723	303	308	303
query78	9479	8926	9061	8926
query79	2633	532	527	527
query80	1052	519	503	503
query81	555	222	226	222
query82	762	127	128	127
query83	347	167	167	167
query84	275	80	81	80
query85	1750	380	315	315
query86	430	275	291	275
query87	3286	3116	3092	3092
query88	3806	2380	2371	2371
query89	403	294	289	289
query90	1964	196	186	186
query91	125	98	97	97
query92	61	48	48	48
query93	2320	608	624	608
query94	936	302	290	290
query95	389	270	272	270
query96	618	281	271	271
query97	3230	3066	3073	3066
query98	220	203	199	199
query99	1665	1276	1290	1276
Total cold run time: 268986 ms
Total hot run time: 170811 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.01 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 741f85433301cd25dd70a11a47cd3b447eb44be5, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.04
query3	0.22	0.04	0.04
query4	1.69	0.07	0.07
query5	0.50	0.47	0.49
query6	1.14	0.71	0.71
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.58	0.50	0.52
query10	0.55	0.56	0.57
query11	0.15	0.11	0.12
query12	0.14	0.12	0.13
query13	0.62	0.60	0.60
query14	0.79	0.80	0.78
query15	0.90	0.86	0.86
query16	0.35	0.35	0.35
query17	0.99	1.01	1.00
query18	0.22	0.20	0.21
query19	1.89	1.73	1.70
query20	0.01	0.01	0.01
query21	15.42	0.79	0.67
query22	4.51	7.44	1.41
query23	17.93	1.31	1.26
query24	2.27	0.23	0.24
query25	0.19	0.08	0.08
query26	0.31	0.22	0.21
query27	0.45	0.24	0.24
query28	13.16	1.00	0.97
query29	12.56	3.34	3.32
query30	0.26	0.06	0.06
query31	2.88	0.40	0.39
query32	3.25	0.51	0.48
query33	2.95	2.92	2.93
query34	15.46	4.29	4.25
query35	4.29	4.26	4.30
query36	0.68	0.48	0.48
query37	0.21	0.17	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.15	0.13	0.12
query41	0.09	0.05	0.04
query42	0.05	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 108.24 s
Total hot run time: 30.01 s

@Yulei-Yang
Copy link
Contributor Author

run cloud_p0

@morningman morningman self-assigned this Aug 1, 2024
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Aug 1, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 1, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 1, 2024

PR approved by anyone and no changes requested.

@morningman morningman merged commit 4b58cee into apache:master Aug 1, 2024
morningman pushed a commit to morningman/doris that referenced this pull request Aug 1, 2024
…#38619)

Brought by: apache#38244

how to reproduce:
create a hms catalog named hive with property use_meta_cache = false,
after that restart fe, and run sql `switch hive; show databases;`, will
get error:
```sql
ERROR 1105 (HY000): NullPointerException, msg: java.lang.NullPointerException: Cannot invoke "org.apache.doris.datasource.metacache.MetaCache.listNames()" because "this.metaCache" is null
```

reason, property use_meta_cache of newly created hms catalog after pr
apache#38244 was force convert to true, but field `useMetaCache` of class
ExternalCatalog is not converted at the same time.
@Yulei-Yang Yulei-Yang deleted the fix_metacache_is_null branch August 1, 2024 06:19
morningman added a commit that referenced this pull request Aug 2, 2024
)(#38619) (#38355)

bp #38244 #38352 #38619

---------

Co-authored-by: Yulei-Yang <yulei.yang0699@gmail.com>
feiniaofeiafei pushed a commit to feiniaofeiafei/doris that referenced this pull request Aug 9, 2024
…#38619)

Brought by: apache#38244

how to reproduce:
create a hms catalog named hive with property use_meta_cache = false,
after that restart fe, and run sql `switch hive; show databases;`, will
get error:
```sql
ERROR 1105 (HY000): NullPointerException, msg: java.lang.NullPointerException: Cannot invoke "org.apache.doris.datasource.metacache.MetaCache.listNames()" because "this.metaCache" is null
```

reason, property use_meta_cache of newly created hms catalog after pr
apache#38244 was force convert to true, but field `useMetaCache` of class
ExternalCatalog is not converted at the same time.
dataroaring pushed a commit that referenced this pull request Aug 11, 2024
Brought by: #38244

how to reproduce:
create a hms catalog named hive with property use_meta_cache = false,
after that restart fe, and run sql `switch hive; show databases;`, will
get error:
```sql
ERROR 1105 (HY000): NullPointerException, msg: java.lang.NullPointerException: Cannot invoke "org.apache.doris.datasource.metacache.MetaCache.listNames()" because "this.metaCache" is null
```

reason, property use_meta_cache of newly created hms catalog after pr
#38244 was force convert to true, but field `useMetaCache` of class
ExternalCatalog is not converted at the same time.
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
Brought by: #38244

how to reproduce:
create a hms catalog named hive with property use_meta_cache = false,
after that restart fe, and run sql `switch hive; show databases;`, will
get error:
```sql
ERROR 1105 (HY000): NullPointerException, msg: java.lang.NullPointerException: Cannot invoke "org.apache.doris.datasource.metacache.MetaCache.listNames()" because "this.metaCache" is null
```

reason, property use_meta_cache of newly created hms catalog after pr
#38244 was force convert to true, but field `useMetaCache` of class
ExternalCatalog is not converted at the same time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants