[Fix](inverted index) Resolve null processing issue in arrays_overlap #41495

airborne12 · 2024-09-30T04:59:29Z

Proposed changes

Fix problem "Runtime Error: Null pointer passed to 'StringTypeInvertedIndexReader::query', which requires a non-null argument."


Stack Trace:
#0 doris::segment_v2::StringTypeInvertedIndexReader::query(...) inverted_index_reader.cpp:473
#1 doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(...) inverted_index_reader.cpp:1237
#2 doris::vectorized::FunctionArraysOverlap::evaluate_inverted_index(...) function_arrays_overlap.h:192
#3 doris::vectorized::DefaultFunction::evaluate_inverted_index(...) function.h:532
#4 doris::vectorized::VExpr::_evaluate_inverted_index(...) vexpr.cpp:708
#5 doris::vectorized::VectorizedFnCall::evaluate_inverted_index(...) vectorized_fn_call.cpp:143
#6 doris::vectorized::VExprContext::evaluate_inverted_index(...) vexpr_context.cpp:126

doris-robot · 2024-09-30T04:59:34Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

airborne12 · 2024-09-30T05:00:13Z

run buildall

doris-robot · 2024-09-30T05:34:39Z

TeamCity be ut coverage result:
Function Coverage: 37.32% (9630/25805)
Line Coverage: 28.70% (79860/278277)
Region Coverage: 28.15% (41303/146745)
Branch Coverage: 24.76% (21038/84968)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c23ba1abe59c58a131a71c835989c91e24eae59c_c23ba1abe59c58a131a71c835989c91e24eae59c/report/index.html

doris-robot · 2024-09-30T06:33:18Z

TPC-H: Total hot run time: 42598 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c23ba1abe59c58a131a71c835989c91e24eae59c, data reload: false

------ Round 1 ----------------------------------
q1	17669	8350	8030	8030
q2	2016	328	181	181
q3	10462	1231	1318	1231
q4	10417	777	899	777
q5	7760	3050	2917	2917
q6	261	156	157	156
q7	1124	629	601	601
q8	9349	2070	2000	2000
q9	6989	6861	6754	6754
q10	7018	2345	2342	2342
q11	458	247	250	247
q12	455	226	223	223
q13	17783	3003	2994	2994
q14	250	221	211	211
q15	600	523	534	523
q16	655	593	576	576
q17	1031	581	671	581
q18	7351	6684	6674	6674
q19	1370	1041	1130	1041
q20	494	210	204	204
q21	4216	3350	3519	3350
q22	1105	985	990	985
Total cold run time: 108833 ms
Total hot run time: 42598 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7953	7867	7824	7824
q2	366	241	241	241
q3	3204	3091	3054	3054
q4	2255	1890	1841	1841
q5	5843	5879	5921	5879
q6	254	145	151	145
q7	2405	1783	1848	1783
q8	3551	3782	3815	3782
q9	9046	9057	9014	9014
q10	3749	3699	3753	3699
q11	635	487	488	487
q12	829	575	632	575
q13	11412	3178	3205	3178
q14	315	294	285	285
q15	581	531	537	531
q16	707	644	657	644
q17	2052	1755	1700	1700
q18	8328	7897	7608	7608
q19	1871	1473	1645	1473
q20	2199	1871	1866	1866
q21	5821	5493	5457	5457
q22	1188	1093	1014	1014
Total cold run time: 74564 ms
Total hot run time: 62080 ms

doris-robot · 2024-09-30T06:45:21Z

TPC-DS: Total hot run time: 191992 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c23ba1abe59c58a131a71c835989c91e24eae59c, data reload: false

query1	877	409	416	409
query2	6250	2062	2027	2027
query3	8687	194	206	194
query4	33555	23523	23446	23446
query5	3994	479	481	479
query6	291	167	173	167
query7	4204	302	307	302
query8	271	211	215	211
query9	9288	2711	2712	2711
query10	471	295	293	293
query11	17828	15163	15206	15163
query12	152	100	99	99
query13	1573	487	429	429
query14	9857	7534	7495	7495
query15	265	172	176	172
query16	8062	505	486	486
query17	1640	638	586	586
query18	2172	312	323	312
query19	386	159	155	155
query20	120	113	114	113
query21	216	114	106	106
query22	4896	4543	4407	4407
query23	35038	33964	33912	33912
query24	11004	2857	2838	2838
query25	611	415	398	398
query26	1152	160	164	160
query27	2257	307	302	302
query28	7416	2442	2414	2414
query29	845	449	437	437
query30	254	152	149	149
query31	1060	812	823	812
query32	98	57	56	56
query33	774	298	310	298
query34	916	504	538	504
query35	913	725	731	725
query36	1099	931	955	931
query37	149	83	87	83
query38	4003	3995	3995	3995
query39	1455	1390	1430	1390
query40	209	98	99	98
query41	47	43	43	43
query42	113	96	100	96
query43	530	480	472	472
query44	1250	824	812	812
query45	198	164	168	164
query46	1133	714	748	714
query47	1937	1823	1837	1823
query48	429	366	348	348
query49	921	413	412	412
query50	832	409	418	409
query51	6985	6963	6882	6882
query52	103	88	91	88
query53	254	182	180	180
query54	1203	484	474	474
query55	84	78	76	76
query56	290	261	271	261
query57	1268	1156	1160	1156
query58	237	228	237	228
query59	3089	2981	3074	2981
query60	308	281	257	257
query61	111	120	107	107
query62	879	672	662	662
query63	226	183	184	183
query64	4039	650	619	619
query65	3274	3244	3253	3244
query66	847	296	309	296
query67	15783	15489	15603	15489
query68	4593	568	566	566
query69	535	334	301	301
query70	1198	1115	1127	1115
query71	361	277	278	277
query72	7175	3922	4014	3922
query73	781	345	357	345
query74	10404	9019	8945	8945
query75	3437	2667	2661	2661
query76	2974	883	920	883
query77	409	297	295	295
query78	10560	9597	9433	9433
query79	2208	600	605	600
query80	1217	463	480	463
query81	594	238	240	238
query82	694	140	138	138
query83	267	136	135	135
query84	252	72	83	72
query85	1367	302	292	292
query86	376	289	288	288
query87	4560	4384	4267	4267
query88	3975	2438	2414	2414
query89	412	288	297	288
query90	2031	191	186	186
query91	139	108	119	108
query92	66	45	49	45
query93	1586	546	542	542
query94	1101	291	300	291
query95	366	249	251	249
query96	612	283	279	279
query97	3289	3104	3137	3104
query98	211	201	198	198
query99	1542	1312	1317	1312
Total cold run time: 299970 ms
Total hot run time: 191992 ms

doris-robot · 2024-09-30T06:50:48Z

ClickBench: Total hot run time: 32.2 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c23ba1abe59c58a131a71c835989c91e24eae59c, data reload: false

query1	0.05	0.04	0.04
query2	0.06	0.03	0.02
query3	0.24	0.06	0.06
query4	1.64	0.10	0.10
query5	0.50	0.52	0.53
query6	1.12	0.72	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.57	0.52	0.50
query10	0.56	0.55	0.52
query11	0.14	0.10	0.11
query12	0.14	0.12	0.11
query13	0.61	0.60	0.59
query14	2.73	2.72	2.69
query15	0.90	0.83	0.82
query16	0.39	0.40	0.38
query17	1.06	1.03	1.07
query18	0.20	0.20	0.20
query19	1.95	1.86	1.95
query20	0.01	0.01	0.01
query21	15.48	0.63	0.60
query22	2.83	1.87	1.78
query23	17.03	0.96	0.89
query24	2.91	1.45	0.47
query25	0.23	0.10	0.04
query26	0.50	0.14	0.14
query27	0.04	0.03	0.04
query28	10.73	1.09	1.07
query29	12.56	3.30	3.26
query30	0.26	0.06	0.06
query31	2.88	0.39	0.38
query32	3.27	0.48	0.46
query33	2.97	3.01	3.01
query34	16.72	4.50	4.50
query35	4.47	4.49	4.43
query36	0.67	0.49	0.47
query37	0.08	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.02	0.03
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.04	0.02
Total cold run time: 106.94 s
Total hot run time: 32.2 s

amorynan

LGTM

github-actions · 2024-10-08T14:34:32Z

PR approved by anyone and no changes requested.

xiaokang

LGTM

github-actions · 2024-10-09T00:34:09Z

PR approved by at least one committer and no changes requested.

## Proposed changes feature for array overlaps backports: #41161 #41495 Issue Number: close #xxx

…apache#41495) ## Proposed changes Fix problem "Runtime Error: Null pointer passed to 'StringTypeInvertedIndexReader::query', which requires a non-null argument." ``` Stack Trace: #0 doris::segment_v2::StringTypeInvertedIndexReader::query(...) inverted_index_reader.cpp:473 #1 doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(...) inverted_index_reader.cpp:1237 apache#2 doris::vectorized::FunctionArraysOverlap::evaluate_inverted_index(...) function_arrays_overlap.h:192 apache#3 doris::vectorized::DefaultFunction::evaluate_inverted_index(...) function.h:532 apache#4 doris::vectorized::VExpr::_evaluate_inverted_index(...) vexpr.cpp:708 apache#5 doris::vectorized::VectorizedFnCall::evaluate_inverted_index(...) vectorized_fn_call.cpp:143 apache#6 doris::vectorized::VExprContext::evaluate_inverted_index(...) vexpr_context.cpp:126 ```

…apache#41495) ## Proposed changes Fix problem "Runtime Error: Null pointer passed to 'StringTypeInvertedIndexReader::query', which requires a non-null argument." ``` Stack Trace: #0 doris::segment_v2::StringTypeInvertedIndexReader::query(...) inverted_index_reader.cpp:473 apache#1 doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(...) inverted_index_reader.cpp:1237 apache#2 doris::vectorized::FunctionArraysOverlap::evaluate_inverted_index(...) function_arrays_overlap.h:192 apache#3 doris::vectorized::DefaultFunction::evaluate_inverted_index(...) function.h:532 apache#4 doris::vectorized::VExpr::_evaluate_inverted_index(...) vexpr.cpp:708 apache#5 doris::vectorized::VectorizedFnCall::evaluate_inverted_index(...) vectorized_fn_call.cpp:143 apache#6 doris::vectorized::VExprContext::evaluate_inverted_index(...) vexpr_context.cpp:126 ```

…apache#41495) ## Proposed changes Fix problem "Runtime Error: Null pointer passed to 'StringTypeInvertedIndexReader::query', which requires a non-null argument." ``` Stack Trace: #0 doris::segment_v2::StringTypeInvertedIndexReader::query(...) inverted_index_reader.cpp:473 #1 doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(...) inverted_index_reader.cpp:1237 apache#2 doris::vectorized::FunctionArraysOverlap::evaluate_inverted_index(...) function_arrays_overlap.h:192 apache#3 doris::vectorized::DefaultFunction::evaluate_inverted_index(...) function.h:532 apache#4 doris::vectorized::VExpr::_evaluate_inverted_index(...) vexpr.cpp:708 apache#5 doris::vectorized::VectorizedFnCall::evaluate_inverted_index(...) vectorized_fn_call.cpp:143 apache#6 doris::vectorized::VExprContext::evaluate_inverted_index(...) vexpr_context.cpp:126 ```

) ## Proposed changes backport : #41286 #41495 Issue Number: close #xxx

…e#41495) (apache#41286) (apache#3601)

[Fix](inverted index) Resolve null processing issue in arrays_overlap

c23ba1a

amorynan approved these changes Oct 8, 2024

View reviewed changes

github-actions bot added the reviewed label Oct 8, 2024

xiaokang approved these changes Oct 9, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 9, 2024

airborne12 merged commit bc9b87d into apache:master Oct 9, 2024

airborne12 deleted the fix-array branch October 9, 2024 01:28

airborne12 added dev/2.1.x dev/3.0.x and removed dev/2.1.x dev/3.0.x labels Oct 9, 2024

amorynan mentioned this pull request Oct 9, 2024

[Improve](array-overlaps) improve array overlaps #41571

Merged

dataroaring pushed a commit that referenced this pull request Oct 10, 2024

[Improve](array-overlaps) improve array overlaps (#41571)

1fedfd5

## Proposed changes feature for array overlaps backports: #41161 #41495 Issue Number: close #xxx

dataroaring added the dev/3.0.3-merged label Oct 10, 2024

amorynan mentioned this pull request Oct 18, 2024

[fix](arrays_overlap) support arrays overlap with inverted index #42090

Merged

yiguolei pushed a commit that referenced this pull request Oct 18, 2024

[fix](arrays_overlap) support arrays overlap with inverted index (#42090

5db44a1

) ## Proposed changes backport : #41286 #41495 Issue Number: close #xxx

yiguolei added the dev/2.1.7-merged label Oct 18, 2024

eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Nov 12, 2024

[improve](selectdb-cloud)array-overlaps support inverted index (apach…

8ae600d

…e#41495) (apache#41286) (apache#3601)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fix](inverted index) Resolve null processing issue in arrays_overlap #41495

[Fix](inverted index) Resolve null processing issue in arrays_overlap #41495

Uh oh!

airborne12 commented Sep 30, 2024

Uh oh!

doris-robot commented Sep 30, 2024

Uh oh!

airborne12 commented Sep 30, 2024

Uh oh!

doris-robot commented Sep 30, 2024

Uh oh!

doris-robot commented Sep 30, 2024

Uh oh!

doris-robot commented Sep 30, 2024

Uh oh!

doris-robot commented Sep 30, 2024

Uh oh!

amorynan left a comment

Uh oh!

github-actions bot commented Oct 8, 2024

Uh oh!

xiaokang left a comment

Uh oh!

github-actions bot commented Oct 9, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[Fix](inverted index) Resolve null processing issue in arrays_overlap #41495

[Fix](inverted index) Resolve null processing issue in arrays_overlap #41495

Uh oh!

Conversation

airborne12 commented Sep 30, 2024

Proposed changes

Uh oh!

doris-robot commented Sep 30, 2024

Uh oh!

airborne12 commented Sep 30, 2024

Uh oh!

doris-robot commented Sep 30, 2024

Uh oh!

doris-robot commented Sep 30, 2024

Uh oh!

doris-robot commented Sep 30, 2024

Uh oh!

doris-robot commented Sep 30, 2024

Uh oh!

amorynan left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 8, 2024

Uh oh!

xiaokang left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 9, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants