Skip to content

[fix] (inverted index) Fix match function without inverted index (#38989) #39221

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
airborne12 merged 5 commits into from
Aug 13, 2024
Merged

[fix] (inverted index) Fix match function without inverted index (#38989) #39221

airborne12 merged 5 commits into from
Aug 13, 2024

Conversation

@csun5285
Copy link
Contributor

@csun5285 csun5285 commented Aug 12, 2024

Proposed changes

pick from #38989

@doris-robot
Copy link

doris-robot commented Aug 12, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added area/nereids area/planner Issues or PRs related to the query planner kind/test labels Aug 12, 2024
@csun5285
Copy link
Contributor Author

csun5285 commented Aug 12, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 12, 2024

clang-tidy review says "All clean, LGTM! 👍"

@csun5285
Copy link
Contributor Author

csun5285 commented Aug 12, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 12, 2024

clang-tidy review says "All clean, LGTM! 👍"

@csun5285
Copy link
Contributor Author

csun5285 commented Aug 12, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 12, 2024

clang-tidy review says "All clean, LGTM! 👍"

@csun5285
Copy link
Contributor Author

csun5285 commented Aug 12, 2024

run buildall

github-actions[bot]
github-actions bot reviewed Aug 12, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@csun5285
Copy link
Contributor Author

csun5285 commented Aug 12, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 12, 2024

clang-tidy review says "All clean, LGTM! 👍"

@csun5285
Copy link
Contributor Author

csun5285 commented Aug 13, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 13, 2024

clang-tidy review says "All clean, LGTM! 👍"

airborne12
airborne12 reviewed Aug 13, 2024
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@csun5285 csun5285 force-pushed the fix_match_phrse_2.0 branch from e2d7602 to c82f379 Compare August 13, 2024 02:47
@csun5285
Copy link
Contributor Author

csun5285 commented Aug 13, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 13, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Aug 13, 2024

TPC-H: Total hot run time: 49913 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c82f379a915b70cb0d14d637dd91883fc2858030, data reload: false

------ Round 1 ----------------------------------
q1	11356	4392	4397	4392
q2	1240	189	149	149
q3	2094	1952	1959	1952
q4	1431	1334	1342	1334
q5	4031	4001	3910	3910
q6	239	129	127	127
q7	2072	1636	1587	1587
q8	2721	2774	2718	2718
q9	10802	10355	10361	10355
q10	3493	3499	3539	3499
q11	356	253	250	250
q12	452	307	304	304
q13	7677	4022	4027	4022
q14	368	321	325	321
q15	529	470	478	470
q16	648	579	580	579
q17	1110	952	906	906
q18	7317	6808	6877	6808
q19	1641	1537	1490	1490
q20	519	308	310	308
q21	4468	4085	4039	4039
q22	502	393	404	393
Total cold run time: 65066 ms
Total hot run time: 49913 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4351	4348	4294	4294
q2	327	229	233	229
q3	4188	4163	4188	4163
q4	2762	2736	2738	2736
q5	7281	7175	7148	7148
q6	241	119	123	119
q7	3285	2850	2895	2850
q8	4392	4481	4533	4481
q9	16799	16781	16894	16781
q10	4260	4270	4237	4237
q11	754	711	679	679
q12	1023	855	868	855
q13	4197	3754	3768	3754
q14	470	424	414	414
q15	505	460	462	460
q16	758	673	687	673
q17	3801	3881	3778	3778
q18	8842	8877	9089	8877
q19	1701	1686	1650	1650
q20	2432	2159	2124	2124
q21	8459	8511	8478	8478
q22	1021	906	960	906
Total cold run time: 81849 ms
Total hot run time: 79686 ms

@doris-robot
Copy link

doris-robot commented Aug 13, 2024

TPC-DS: Total hot run time: 201725 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c82f379a915b70cb0d14d637dd91883fc2858030, data reload: false

query1	943	394	420	394
query2	6527	2251	2199	2199
query3	6920	211	204	204
query4	20136	17874	18033	17874
query5	19725	6506	6495	6495
query6	277	214	229	214
query7	4168	296	315	296
query8	245	248	246	246
query9	3142	2731	2652	2652
query10	402	321	313	313
query11	11246	10776	10733	10733
query12	131	79	75	75
query13	5591	678	639	639
query14	17379	13568	13415	13415
query15	369	224	230	224
query16	6467	286	264	264
query17	1661	1453	897	897
query18	2266	416	414	414
query19	220	155	149	149
query20	80	76	81	76
query21	186	95	94	94
query22	5146	5045	4966	4966
query23	32433	32060	32056	32056
query24	7913	6502	6503	6502
query25	517	443	430	430
query26	1245	160	160	160
query27	2291	301	298	298
query28	6127	2327	2277	2277
query29	2948	2768	2662	2662
query30	242	170	170	170
query31	896	781	762	762
query32	69	63	60	60
query33	439	246	265	246
query34	854	479	486	479
query35	1122	910	922	910
query36	1404	1149	1075	1075
query37	176	61	61	61
query38	3098	2906	2936	2906
query39	1377	1328	1342	1328
query40	315	100	96	96
query41	39	36	36	36
query42	87	93	85	85
query43	602	678	584	584
query44	1125	718	721	718
query45	246	232	232	232
query46	1223	943	957	943
query47	1867	1878	1767	1767
query48	978	679	669	669
query49	643	374	374	374
query50	865	633	623	623
query51	4751	4689	4681	4681
query52	94	79	92	79
query53	444	323	342	323
query54	2679	2473	2450	2450
query55	92	83	85	83
query56	231	226	213	213
query57	1269	1136	1057	1057
query58	226	199	201	199
query59	3492	3169	3413	3169
query60	222	214	198	198
query61	101	96	98	96
query62	847	483	446	446
query63	485	340	339	339
query64	2831	1563	1473	1473
query65	3661	3579	3578	3578
query66	798	381	380	380
query67	15318	14752	16034	14752
query68	9575	654	624	624
query69	578	348	355	348
query70	1624	1492	1301	1301
query71	423	320	318	318
query72	6145	3483	3449	3449
query73	724	325	322	322
query74	6312	5876	5899	5876
query75	5414	3765	3726	3726
query76	6116	1127	1161	1127
query77	1056	267	263	263
query78	12346	13274	11830	11830
query79	12072	650	624	624
query80	724	404	407	404
query81	495	237	230	230
query82	240	97	103	97
query83	167	135	130	130
query84	256	72	70	70
query85	736	319	326	319
query86	349	312	292	292
query87	3234	3046	3047	3046
query88	5407	2297	2291	2291
query89	411	282	304	282
query90	1919	214	211	211
query91	175	125	123	123
query92	59	58	54	54
query93	3485	606	582	582
query94	709	215	213	213
query95	1136	1072	1058	1058
query96	644	329	327	327
query97	6523	6349	6374	6349
query98	184	178	164	164
query99	2848	817	850	817
Total cold run time: 313995 ms
Total hot run time: 201725 ms

@doris-robot
Copy link

doris-robot commented Aug 13, 2024

ClickBench: Total hot run time: 31.1 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c82f379a915b70cb0d14d637dd91883fc2858030, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.03	0.02
query3	0.25	0.05	0.06
query4	1.78	0.07	0.08
query5	0.53	0.52	0.53
query6	1.24	0.61	0.62
query7	0.02	0.01	0.01
query8	0.04	0.03	0.02
query9	0.53	0.50	0.48
query10	0.52	0.53	0.53
query11	0.12	0.08	0.08
query12	0.12	0.09	0.09
query13	0.62	0.61	0.61
query14	0.80	0.77	0.78
query15	0.78	0.75	0.80
query16	0.37	0.36	0.35
query17	0.99	1.02	1.02
query18	0.22	0.26	0.24
query19	1.90	1.87	1.88
query20	0.02	0.01	0.01
query21	15.45	0.57	0.54
query22	2.29	2.82	1.61
query23	16.76	1.01	1.00
query24	5.26	1.30	1.30
query25	0.40	0.08	0.05
query26	0.62	0.15	0.16
query27	0.05	0.06	0.05
query28	7.31	0.75	0.75
query29	12.65	2.34	2.27
query30	0.61	0.51	0.55
query31	2.80	0.38	0.37
query32	3.37	0.51	0.49
query33	3.06	3.01	3.06
query34	15.27	4.80	4.78
query35	4.87	4.88	4.84
query36	1.05	1.02	1.02
query37	0.06	0.04	0.05
query38	0.04	0.02	0.03
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.07	0.01	0.01
query42	0.03	0.01	0.02
query43	0.03	0.01	0.01
Total cold run time: 103.16 s
Total hot run time: 31.1 s

@doris-robot
Copy link

doris-robot commented Aug 13, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c82f379a915b70cb0d14d637dd91883fc2858030 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@doris-robot
Copy link

doris-robot commented Aug 13, 2024

TeamCity be ut coverage result:
Function Coverage: 37.72% (8105/21486)
Line Coverage: 29.35% (66384/226199)
Region Coverage: 28.86% (34254/118704)
Branch Coverage: 24.74% (17604/71170)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c82f379a915b70cb0d14d637dd91883fc2858030_c82f379a915b70cb0d14d637dd91883fc2858030/report/index.html

@airborne12 airborne12 merged commit a432408 into apache:branch-2.0 Aug 13, 2024
GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Aug 27, 2024
@csun5285
[fix] (inverted index) Fix match function without inverted index (apa…
8c2df08 
…che#38989) (apache#39221)

## Proposed changes

pick from apache#38989
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@airborne12 airborne12 airborne12 left review comments

@github-actions github-actions[bot] github-actions[bot] left review comments

Assignees

No one assigned

Labels

area/nereids area/planner Issues or PRs related to the query planner kind/test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@csun5285 @doris-robot @airborne12