Skip to content

Conversation

@zzzxl1993
Copy link
Contributor

@zzzxl1993 zzzxl1993 commented Jun 19, 2024

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zzzxl1993
Copy link
Contributor Author

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

template <typename Derived>
template <PrimitiveType T>
Status ScanLocalState<Derived>::_normalize_binary_in_compound_predicate(
Status ScanLocalState<Derived>::_normalize_binary_compound_predicate(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function '_normalize_binary_compound_predicate' has cognitive complexity of 54 (threshold 50) [readability-function-cognitive-complexity]

Status ScanLocalState<Derived>::_normalize_binary_compound_predicate(
                                ^
Additional context

be/src/pipeline/exec/scan_operator.cpp:1044: +1, including nesting penalty of 0, nesting level increased to 1

    if (TExprNodeType::BINARY_PRED == expr->node_type()) {
    ^

be/src/pipeline/exec/scan_operator.cpp:1045: nesting level increased to 2

        auto eq_checker = [](const std::string& fn_name) { return fn_name == "eq"; };
                          ^

be/src/pipeline/exec/scan_operator.cpp:1046: nesting level increased to 2

        auto ne_checker = [](const std::string& fn_name) { return fn_name == "ne"; };
                          ^

be/src/pipeline/exec/scan_operator.cpp:1047: nesting level increased to 2

        auto noneq_checker = [](const std::string& fn_name) {
                             ^

be/src/pipeline/exec/scan_operator.cpp:1048: +1

            return fn_name != "ne" && fn_name != "eq" && fn_name != "eq_for_null";
                                                      ^

be/src/pipeline/exec/scan_operator.cpp:1056: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(_should_push_down_binary_predicate(
        ^

be/src/common/status.h:626: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/pipeline/exec/scan_operator.cpp:1056: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(_should_push_down_binary_predicate(
        ^

be/src/common/status.h:628: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/pipeline/exec/scan_operator.cpp:1059: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(_should_push_down_binary_predicate(
        ^

be/src/common/status.h:626: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/pipeline/exec/scan_operator.cpp:1059: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(_should_push_down_binary_predicate(
        ^

be/src/common/status.h:628: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/pipeline/exec/scan_operator.cpp:1062: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(_should_push_down_binary_predicate(
        ^

be/src/common/status.h:626: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/pipeline/exec/scan_operator.cpp:1062: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(_should_push_down_binary_predicate(
        ^

be/src/common/status.h:628: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/pipeline/exec/scan_operator.cpp:1065: +2, including nesting penalty of 1, nesting level increased to 2

        if (eq_pdt == PushDownType::UNACCEPTABLE && ne_pdt == PushDownType::UNACCEPTABLE &&
        ^

be/src/pipeline/exec/scan_operator.cpp:1065: +1

        if (eq_pdt == PushDownType::UNACCEPTABLE && ne_pdt == PushDownType::UNACCEPTABLE &&
                                                                                         ^

be/src/pipeline/exec/scan_operator.cpp:1072: +2, including nesting penalty of 1, nesting level increased to 2

        if (eq_pdt == PushDownType::ACCEPTABLE || ne_pdt == PushDownType::ACCEPTABLE ||
        ^

be/src/pipeline/exec/scan_operator.cpp:1072: +1

        if (eq_pdt == PushDownType::ACCEPTABLE || ne_pdt == PushDownType::ACCEPTABLE ||
                                                                                     ^

be/src/pipeline/exec/scan_operator.cpp:1074: +3, including nesting penalty of 2, nesting level increased to 3

            if (value.data != nullptr) {
            ^

be/src/pipeline/exec/scan_operator.cpp:1075: +4, including nesting penalty of 3, nesting level increased to 4

                if constexpr (T == TYPE_CHAR || T == TYPE_VARCHAR || T == TYPE_STRING ||
                ^

be/src/pipeline/exec/scan_operator.cpp:1075: +1

                if constexpr (T == TYPE_CHAR || T == TYPE_VARCHAR || T == TYPE_STRING ||
                                                                                      ^

be/src/pipeline/exec/scan_operator.cpp:1078: +5, including nesting penalty of 4, nesting level increased to 5

                    RETURN_IF_ERROR(_change_value_range<false>(
                    ^

be/src/common/status.h:626: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/pipeline/exec/scan_operator.cpp:1078: +6, including nesting penalty of 5, nesting level increased to 6

                    RETURN_IF_ERROR(_change_value_range<false>(
                    ^

be/src/common/status.h:628: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/pipeline/exec/scan_operator.cpp:1082: +1, nesting level increased to 4

                } else {
                  ^

be/src/pipeline/exec/scan_operator.cpp:1083: +5, including nesting penalty of 4, nesting level increased to 5

                    RETURN_IF_ERROR(_change_value_range<false>(
                    ^

be/src/common/status.h:626: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/pipeline/exec/scan_operator.cpp:1083: +6, including nesting penalty of 5, nesting level increased to 6

                    RETURN_IF_ERROR(_change_value_range<false>(
                    ^

be/src/common/status.h:628: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

return Status::OK();
}
} else {
vectorized::VInPredicate* pred = static_cast<vectorized::VInPredicate*>(expr);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: use auto when initializing with a cast to avoid duplicating the type name [modernize-use-auto]

Suggested change
vectorized::VInPredicate* pred = static_cast<vectorized::VInPredicate*>(expr);
auto* pred = static_cast<vectorized::VInPredicate*>(expr);

} else {
vectorized::VInPredicate* pred = static_cast<vectorized::VInPredicate*>(expr);

vectorized::InState* state = reinterpret_cast<vectorized::InState*>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: use auto when initializing with a cast to avoid duplicating the type name [modernize-use-auto]

Suggested change
vectorized::InState* state = reinterpret_cast<vectorized::InState*>(
auto* state = reinterpret_cast<vectorized::InState*>(

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.47% (9001/24683)
Line Coverage: 28.01% (73790/263465)
Region Coverage: 27.50% (38330/139394)
Branch Coverage: 24.19% (19534/80750)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a0e106ab4714b528536c36c576633371f0f7a126_a0e106ab4714b528536c36c576633371f0f7a126/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 39902 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a0e106ab4714b528536c36c576633371f0f7a126, data reload: false

------ Round 1 ----------------------------------
q1	18050	4816	4400	4400
q2	2962	192	203	192
q3	11809	1135	1112	1112
q4	10386	936	864	864
q5	7695	2730	2664	2664
q6	227	142	142	142
q7	973	621	592	592
q8	9230	2072	2060	2060
q9	9080	6545	6467	6467
q10	9014	3734	3765	3734
q11	458	244	237	237
q12	405	230	230	230
q13	17783	3001	2973	2973
q14	275	215	213	213
q15	518	480	485	480
q16	528	391	371	371
q17	969	652	743	652
q18	8093	7418	7348	7348
q19	6893	1498	1468	1468
q20	649	312	331	312
q21	4868	3066	3860	3066
q22	394	334	325	325
Total cold run time: 121259 ms
Total hot run time: 39902 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4369	4228	4273	4228
q2	368	258	267	258
q3	2993	2742	2747	2742
q4	1897	1594	1627	1594
q5	5228	5278	5285	5278
q6	212	124	126	124
q7	2071	1775	1738	1738
q8	3213	3336	3309	3309
q9	8367	8399	8300	8300
q10	3861	3713	3632	3632
q11	576	492	501	492
q12	771	625	611	611
q13	17539	2982	2994	2982
q14	301	247	246	246
q15	523	477	473	473
q16	462	426	411	411
q17	1792	1479	1465	1465
q18	7663	7461	7447	7447
q19	1656	1611	1548	1548
q20	1963	1783	1757	1757
q21	4839	4831	4680	4680
q22	582	538	527	527
Total cold run time: 71246 ms
Total hot run time: 53842 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172285 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a0e106ab4714b528536c36c576633371f0f7a126, data reload: false

query1	929	396	367	367
query2	6479	2456	2333	2333
query3	6647	208	207	207
query4	18956	17390	17282	17282
query5	4182	481	459	459
query6	257	171	163	163
query7	4601	299	290	290
query8	314	275	267	267
query9	8448	2401	2372	2372
query10	611	308	283	283
query11	10578	10030	9975	9975
query12	136	88	86	86
query13	1631	355	363	355
query14	9873	6832	7556	6832
query15	232	189	182	182
query16	7812	269	258	258
query17	1922	553	515	515
query18	1954	280	284	280
query19	201	156	157	156
query20	93	86	85	85
query21	217	134	129	129
query22	4351	4221	3915	3915
query23	33626	33222	33086	33086
query24	12066	2807	2811	2807
query25	684	383	383	383
query26	1838	159	154	154
query27	2931	313	318	313
query28	7652	2030	2012	2012
query29	1232	614	593	593
query30	289	152	147	147
query31	1006	746	740	740
query32	97	54	59	54
query33	755	284	287	284
query34	931	480	457	457
query35	721	605	594	594
query36	1075	916	966	916
query37	288	75	70	70
query38	2931	2717	2753	2717
query39	846	794	807	794
query40	282	128	123	123
query41	56	51	50	50
query42	126	97	105	97
query43	600	522	558	522
query44	1266	711	716	711
query45	199	166	165	165
query46	1091	735	733	733
query47	1856	1767	1775	1767
query48	378	298	301	298
query49	1189	410	397	397
query50	763	390	389	389
query51	6813	6757	6614	6614
query52	100	94	92	92
query53	365	290	290	290
query54	931	441	459	441
query55	73	73	71	71
query56	282	260	277	260
query57	1149	1058	1047	1047
query58	246	238	243	238
query59	3404	3229	3283	3229
query60	316	278	272	272
query61	97	91	92	91
query62	643	451	446	446
query63	315	283	296	283
query64	9854	2314	1713	1713
query65	3190	3104	3159	3104
query66	1364	346	334	334
query67	15456	15231	14961	14961
query68	4680	543	535	535
query69	578	429	358	358
query70	1176	1178	1152	1152
query71	434	275	278	275
query72	7258	5425	6004	5425
query73	762	324	328	324
query74	5900	5428	5490	5428
query75	3735	2680	2642	2642
query76	3358	903	875	875
query77	640	308	309	308
query78	10231	9828	9692	9692
query79	2277	513	521	513
query80	1481	482	472	472
query81	565	222	228	222
query82	826	106	99	99
query83	290	170	167	167
query84	244	86	90	86
query85	1648	322	279	279
query86	484	329	343	329
query87	3272	3092	3064	3064
query88	3999	2347	2352	2347
query89	471	382	380	380
query90	1787	201	192	192
query91	131	100	98	98
query92	60	50	49	49
query93	2212	514	504	504
query94	1247	182	190	182
query95	404	305	317	305
query96	595	272	270	270
query97	3261	3031	3136	3031
query98	233	209	197	197
query99	1268	835	855	835
Total cold run time: 277303 ms
Total hot run time: 172285 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.72 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a0e106ab4714b528536c36c576633371f0f7a126, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.05	0.05
query4	1.67	0.08	0.08
query5	0.51	0.50	0.48
query6	1.13	0.73	0.73
query7	0.01	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.56	0.57	0.55
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.60
query14	0.79	0.76	0.80
query15	0.84	0.81	0.81
query16	0.37	0.36	0.34
query17	1.02	0.98	0.95
query18	0.22	0.26	0.23
query19	1.75	1.73	1.66
query20	0.01	0.01	0.01
query21	15.42	0.67	0.65
query22	4.82	7.07	2.14
query23	18.30	1.31	1.26
query24	2.17	0.21	0.20
query25	0.15	0.09	0.08
query26	0.26	0.17	0.18
query27	0.08	0.09	0.08
query28	13.23	1.01	1.00
query29	12.71	3.26	3.30
query30	0.27	0.06	0.06
query31	2.86	0.38	0.39
query32	3.28	0.47	0.46
query33	2.91	2.88	2.91
query34	16.96	4.48	4.48
query35	4.56	4.54	4.48
query36	0.66	0.46	0.46
query37	0.17	0.16	0.15
query38	0.15	0.15	0.16
query39	0.05	0.04	0.04
query40	0.18	0.15	0.14
query41	0.10	0.05	0.04
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.1 s
Total hot run time: 30.72 s

@xiaokang xiaokang changed the title [fix](inverted index) fixed in_list condition not indexed on new pipe… [fix](inverted index) fixed in_list condition not indexed on new pipeline Jun 19, 2024
@zzzxl1993
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.48% (9007/24691)
Line Coverage: 28.02% (73862/263578)
Region Coverage: 27.52% (38370/139450)
Branch Coverage: 24.21% (19557/80788)
Coverage Report: http://coverage.selectdb-in.cc/coverage/25e0b0a9d13c0262f5c35b9d5e8bf34b6684b208_25e0b0a9d13c0262f5c35b9d5e8bf34b6684b208/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 39739 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 25e0b0a9d13c0262f5c35b9d5e8bf34b6684b208, data reload: false

------ Round 1 ----------------------------------
q1	17640	4432	4333	4333
q2	2038	192	200	192
q3	10512	1235	1103	1103
q4	10199	800	753	753
q5	7479	2705	2665	2665
q6	220	137	136	136
q7	973	608	586	586
q8	9213	2053	2086	2053
q9	8925	6475	6519	6475
q10	8940	3703	3979	3703
q11	491	245	245	245
q12	466	236	231	231
q13	17762	2971	2965	2965
q14	261	217	221	217
q15	514	483	478	478
q16	530	384	375	375
q17	959	614	754	614
q18	8010	7510	7268	7268
q19	8799	1525	1498	1498
q20	694	316	326	316
q21	5010	3201	3244	3201
q22	392	332	335	332
Total cold run time: 120027 ms
Total hot run time: 39739 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4378	4257	4275	4257
q2	368	258	269	258
q3	3158	2890	2967	2890
q4	2012	1695	1753	1695
q5	5493	5505	5543	5505
q6	225	132	139	132
q7	2232	1834	1851	1834
q8	3313	3419	3416	3416
q9	8731	8861	8771	8771
q10	4068	3722	3992	3722
q11	602	513	491	491
q12	809	616	651	616
q13	16307	3191	3220	3191
q14	311	278	275	275
q15	538	501	497	497
q16	496	422	432	422
q17	1804	1501	1512	1501
q18	8177	7882	7814	7814
q19	1857	1627	1517	1517
q20	3087	1877	1827	1827
q21	5005	4897	4760	4760
q22	612	567	555	555
Total cold run time: 73583 ms
Total hot run time: 55946 ms

@zzzxl1993
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40533 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 25e0b0a9d13c0262f5c35b9d5e8bf34b6684b208, data reload: false

------ Round 1 ----------------------------------
q1	17608	4305	4299	4299
q2	2032	195	207	195
q3	10443	1066	1144	1066
q4	10182	738	742	738
q5	7554	2701	2660	2660
q6	220	135	131	131
q7	956	619	599	599
q8	9236	2079	2111	2079
q9	8705	6531	6499	6499
q10	9000	3754	3697	3697
q11	449	232	232	232
q12	565	228	228	228
q13	18024	2997	3005	2997
q14	268	224	222	222
q15	529	490	468	468
q16	528	396	386	386
q17	1031	680	685	680
q18	7965	7453	7422	7422
q19	2856	1403	1435	1403
q20	634	308	339	308
q21	5058	3886	3973	3886
q22	412	341	338	338
Total cold run time: 114255 ms
Total hot run time: 40533 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4424	4316	4297	4297
q2	377	281	276	276
q3	3016	2728	2993	2728
q4	1999	1714	1722	1714
q5	5665	5532	5567	5532
q6	218	130	127	127
q7	2179	1835	1848	1835
q8	3327	3473	3423	3423
q9	8775	8764	8832	8764
q10	4104	3934	3743	3743
q11	566	517	494	494
q12	811	675	632	632
q13	17119	3200	3174	3174
q14	298	265	267	265
q15	551	492	491	491
q16	511	456	427	427
q17	1835	1508	1524	1508
q18	8115	8134	7847	7847
q19	1813	1751	1698	1698
q20	2138	1885	1849	1849
q21	9463	4844	4692	4692
q22	612	573	567	567
Total cold run time: 77916 ms
Total hot run time: 56083 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169640 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 25e0b0a9d13c0262f5c35b9d5e8bf34b6684b208, data reload: false

query1	927	374	379	374
query2	6452	2429	2389	2389
query3	6636	207	214	207
query4	20387	17311	17172	17172
query5	3600	491	458	458
query6	249	160	161	160
query7	4578	295	285	285
query8	332	286	300	286
query9	8523	2416	2393	2393
query10	572	311	281	281
query11	10632	9988	9927	9927
query12	120	89	85	85
query13	1649	374	372	372
query14	10077	6323	6228	6228
query15	235	187	192	187
query16	7248	261	254	254
query17	1503	533	517	517
query18	1931	273	296	273
query19	197	152	161	152
query20	91	88	81	81
query21	215	136	127	127
query22	4315	4117	4042	4042
query23	34062	33603	33718	33603
query24	11509	2892	2815	2815
query25	613	388	381	381
query26	712	157	150	150
query27	2325	316	327	316
query28	6234	2140	2094	2094
query29	871	632	603	603
query30	255	155	159	155
query31	966	749	742	742
query32	98	54	62	54
query33	794	327	300	300
query34	1039	479	466	466
query35	754	688	646	646
query36	1133	957	959	957
query37	147	70	75	70
query38	2921	2843	2897	2843
query39	908	813	828	813
query40	211	131	131	131
query41	54	53	50	50
query42	106	107	104	104
query43	624	536	536	536
query44	1170	719	723	719
query45	197	162	165	162
query46	1071	705	731	705
query47	1897	1786	1764	1764
query48	367	289	301	289
query49	821	405	401	401
query50	794	388	391	388
query51	6826	6626	6619	6619
query52	107	88	94	88
query53	346	284	284	284
query54	870	460	447	447
query55	72	71	72	71
query56	279	252	258	252
query57	1111	1047	1047	1047
query58	268	259	244	244
query59	3537	3341	3003	3003
query60	281	267	274	267
query61	93	90	93	90
query62	623	438	458	438
query63	310	288	282	282
query64	8518	2258	1721	1721
query65	3183	3109	3093	3093
query66	799	327	324	324
query67	15233	15158	14785	14785
query68	4536	532	523	523
query69	584	416	389	389
query70	1156	1162	1162	1162
query71	396	278	270	270
query72	7400	5951	2763	2763
query73	731	321	328	321
query74	5913	5596	5500	5500
query75	3370	2652	2675	2652
query76	2518	961	920	920
query77	610	291	284	284
query78	10652	9878	9808	9808
query79	2462	496	508	496
query80	1361	480	458	458
query81	587	219	223	219
query82	1494	105	106	105
query83	195	162	163	162
query84	263	84	82	82
query85	1338	296	275	275
query86	467	314	329	314
query87	3286	3066	3060	3060
query88	3867	2348	2330	2330
query89	465	377	371	371
query90	1720	190	189	189
query91	123	98	98	98
query92	56	49	50	49
query93	2504	485	486	485
query94	1085	187	181	181
query95	401	309	307	307
query96	597	267	260	260
query97	3248	3004	3071	3004
query98	218	195	199	195
query99	1194	846	866	846
Total cold run time: 270375 ms
Total hot run time: 169640 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.47% (9006/24691)
Line Coverage: 28.02% (73881/263627)
Region Coverage: 27.51% (38380/139499)
Branch Coverage: 24.21% (19561/80804)
Coverage Report: http://coverage.selectdb-in.cc/coverage/25e0b0a9d13c0262f5c35b9d5e8bf34b6684b208_25e0b0a9d13c0262f5c35b9d5e8bf34b6684b208/report/index.html

@doris-robot
Copy link

ClickBench: Total hot run time: 30.14 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 25e0b0a9d13c0262f5c35b9d5e8bf34b6684b208, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.08
query5	0.48	0.49	0.49
query6	1.13	0.72	0.72
query7	0.02	0.02	0.01
query8	0.06	0.04	0.04
query9	0.54	0.47	0.49
query10	0.54	0.54	0.54
query11	0.16	0.12	0.11
query12	0.14	0.12	0.13
query13	0.59	0.58	0.60
query14	0.78	0.79	0.79
query15	0.83	0.83	0.82
query16	0.35	0.35	0.37
query17	0.99	0.99	1.05
query18	0.23	0.26	0.23
query19	1.79	1.70	1.70
query20	0.01	0.00	0.00
query21	15.40	0.66	0.66
query22	4.90	7.19	1.56
query23	18.30	1.39	1.25
query24	2.10	0.23	0.22
query25	0.15	0.09	0.08
query26	0.26	0.17	0.17
query27	0.08	0.09	0.08
query28	13.19	1.03	1.00
query29	12.63	3.27	3.26
query30	0.25	0.06	0.06
query31	2.87	0.40	0.39
query32	3.25	0.48	0.47
query33	2.87	2.94	2.92
query34	17.06	4.40	4.39
query35	4.47	4.44	4.44
query36	0.64	0.47	0.46
query37	0.18	0.15	0.15
query38	0.14	0.14	0.14
query39	0.04	0.03	0.04
query40	0.18	0.14	0.14
query41	0.10	0.05	0.04
query42	0.05	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.83 s
Total hot run time: 30.14 s

airborne12
airborne12 previously approved these changes Jun 20, 2024
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 20, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@zzzxl1993
Copy link
Contributor Author

run p0

qidaye
qidaye previously approved these changes Jun 20, 2024
@zzzxl1993
Copy link
Contributor Author

run p0

@zzzxl1993 zzzxl1993 dismissed stale reviews from qidaye and airborne12 via 6309a82 June 20, 2024 10:50
@zzzxl1993
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jun 20, 2024
@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.47% (9007/24694)
Line Coverage: 28.02% (73870/263660)
Region Coverage: 27.49% (38358/139520)
Branch Coverage: 24.19% (19552/80812)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6309a82c3e316b653061e9ce598283fdbbb958a9_6309a82c3e316b653061e9ce598283fdbbb958a9/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 40099 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6309a82c3e316b653061e9ce598283fdbbb958a9, data reload: false

------ Round 1 ----------------------------------
q1	17614	4362	4302	4302
q2	2032	196	197	196
q3	10477	1119	1166	1119
q4	10201	822	837	822
q5	7578	2675	2692	2675
q6	222	139	137	137
q7	973	606	606	606
q8	9229	2070	2111	2070
q9	9087	6541	6506	6506
q10	8984	3708	3781	3708
q11	446	246	232	232
q12	432	222	228	222
q13	17755	2981	2969	2969
q14	270	215	225	215
q15	518	478	483	478
q16	526	398	386	386
q17	977	670	646	646
q18	8073	7364	7372	7364
q19	6613	1514	1576	1514
q20	669	323	344	323
q21	4982	3908	3283	3283
q22	383	326	332	326
Total cold run time: 118041 ms
Total hot run time: 40099 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4395	4271	4279	4271
q2	366	260	267	260
q3	3063	2902	2951	2902
q4	2019	1713	1782	1713
q5	5564	5562	5493	5493
q6	232	132	141	132
q7	2248	1856	1836	1836
q8	3330	3475	3410	3410
q9	8659	8810	8788	8788
q10	4146	3758	3815	3758
q11	614	519	512	512
q12	801	629	651	629
q13	15940	3168	3137	3137
q14	312	280	300	280
q15	519	486	487	486
q16	485	437	426	426
q17	1849	1546	1497	1497
q18	8174	7872	7892	7872
q19	3247	1618	1739	1618
q20	2093	1919	1852	1852
q21	5389	4948	4743	4743
q22	643	554	548	548
Total cold run time: 74088 ms
Total hot run time: 56163 ms

@zzzxl1993
Copy link
Contributor Author

run performance

@doris-robot
Copy link

TPC-H: Total hot run time: 39672 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6309a82c3e316b653061e9ce598283fdbbb958a9, data reload: false

------ Round 1 ----------------------------------
q1	5790	4307	4223	4223
q2	1630	207	207	207
q3	1371	1062	1110	1062
q4	1172	744	723	723
q5	2660	2693	2603	2603
q6	224	139	138	138
q7	948	624	616	616
q8	2023	2186	2263	2186
q9	8935	6552	6467	6467
q10	8963	3744	3729	3729
q11	439	241	248	241
q12	396	246	229	229
q13	18746	2962	2998	2962
q14	274	224	229	224
q15	529	484	477	477
q16	516	390	384	384
q17	964	661	731	661
q18	8135	7534	7301	7301
q19	5062	1392	1464	1392
q20	641	320	332	320
q21	4927	3181	3287	3181
q22	411	346	348	346
Total cold run time: 74756 ms
Total hot run time: 39672 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4393	4264	4245	4245
q2	381	263	275	263
q3	2952	2710	2748	2710
q4	1831	1589	1563	1563
q5	5312	5248	5293	5248
q6	215	131	128	128
q7	2169	1774	1728	1728
q8	3178	3357	3382	3357
q9	8426	8362	8443	8362
q10	3870	3630	3710	3630
q11	577	513	505	505
q12	774	608	602	602
q13	16345	2992	2978	2978
q14	285	261	265	261
q15	532	483	487	483
q16	462	435	406	406
q17	1766	1500	1468	1468
q18	7765	7496	7532	7496
q19	1668	1612	1419	1419
q20	1991	1769	1754	1754
q21	9363	4753	4724	4724
q22	605	576	549	549
Total cold run time: 74860 ms
Total hot run time: 53879 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173831 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6309a82c3e316b653061e9ce598283fdbbb958a9, data reload: false

query1	930	389	377	377
query2	6451	2385	2219	2219
query3	6657	210	215	210
query4	19915	17433	17309	17309
query5	4174	493	507	493
query6	247	169	166	166
query7	4579	302	298	298
query8	345	300	327	300
query9	8586	2433	2414	2414
query10	610	298	280	280
query11	10447	10007	10265	10007
query12	147	91	85	85
query13	1642	361	354	354
query14	9467	7743	7725	7725
query15	237	198	197	197
query16	7830	275	259	259
query17	1878	557	532	532
query18	2001	272	274	272
query19	198	160	162	160
query20	97	85	85	85
query21	206	129	132	129
query22	4417	4203	3954	3954
query23	33743	33101	33256	33101
query24	11761	2846	2832	2832
query25	657	380	366	366
query26	1797	158	155	155
query27	2955	313	321	313
query28	7448	2078	2050	2050
query29	1086	631	616	616
query30	290	150	156	150
query31	1017	741	759	741
query32	92	57	57	57
query33	769	288	304	288
query34	945	478	459	459
query35	748	638	636	636
query36	1110	941	920	920
query37	200	77	73	73
query38	2875	2770	2745	2745
query39	899	782	780	780
query40	284	131	127	127
query41	57	54	52	52
query42	125	99	103	99
query43	591	541	547	541
query44	1196	736	741	736
query45	196	166	168	166
query46	1072	733	726	726
query47	1838	1782	1801	1782
query48	374	302	307	302
query49	1173	424	406	406
query50	781	401	401	401
query51	6868	6798	6752	6752
query52	114	97	95	95
query53	373	294	286	286
query54	976	453	447	447
query55	77	74	75	74
query56	289	268	266	266
query57	1127	1040	1104	1040
query58	249	257	277	257
query59	3440	3256	3094	3094
query60	346	282	279	279
query61	97	109	89	89
query62	656	446	449	446
query63	329	294	297	294
query64	9837	2278	1763	1763
query65	3211	3135	3131	3131
query66	1377	342	345	342
query67	15403	14842	15035	14842
query68	4621	532	550	532
query69	553	436	400	400
query70	1186	1151	1171	1151
query71	410	282	289	282
query72	7048	5524	5548	5524
query73	770	323	329	323
query74	5869	5526	5450	5450
query75	3437	2664	2673	2664
query76	2807	967	958	958
query77	477	313	316	313
query78	10373	10059	9659	9659
query79	2383	530	522	522
query80	2172	477	489	477
query81	558	225	220	220
query82	963	108	107	107
query83	306	177	172	172
query84	262	91	87	87
query85	1525	291	272	272
query86	468	346	313	313
query87	3231	3060	3123	3060
query88	4245	2346	2345	2345
query89	460	397	378	378
query90	1852	199	196	196
query91	134	104	101	101
query92	62	51	54	51
query93	2157	513	508	508
query94	1182	189	187	187
query95	418	319	320	319
query96	593	272	273	272
query97	3227	3009	3025	3009
query98	222	217	200	200
query99	1094	837	849	837
Total cold run time: 276974 ms
Total hot run time: 173831 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.57 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6309a82c3e316b653061e9ce598283fdbbb958a9, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.49	0.48	0.48
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.53	0.49	0.48
query10	0.54	0.55	0.52
query11	0.15	0.11	0.12
query12	0.15	0.13	0.12
query13	0.59	0.60	0.59
query14	0.78	0.78	0.76
query15	0.83	0.80	0.82
query16	0.36	0.37	0.38
query17	1.03	1.05	1.04
query18	0.22	0.23	0.27
query19	1.91	1.72	1.74
query20	0.01	0.01	0.01
query21	15.42	0.65	0.65
query22	3.93	8.21	1.82
query23	18.25	1.36	1.30
query24	2.16	0.26	0.22
query25	0.16	0.08	0.08
query26	0.27	0.18	0.18
query27	0.08	0.08	0.07
query28	13.20	1.01	1.00
query29	12.63	3.31	3.28
query30	0.26	0.06	0.06
query31	2.84	0.39	0.39
query32	3.28	0.48	0.47
query33	2.89	2.93	2.85
query34	17.04	4.43	4.47
query35	4.52	4.51	4.49
query36	0.64	0.47	0.49
query37	0.19	0.16	0.16
query38	0.15	0.15	0.14
query39	0.04	0.04	0.04
query40	0.18	0.14	0.14
query41	0.09	0.05	0.05
query42	0.05	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.14 s
Total hot run time: 30.57 s

@xiaokang xiaokang changed the title [fix](inverted index) fixed in_list condition not indexed on new pipeline [fix](inverted index) fixed in_list condition not indexed on pipelinex Jun 20, 2024
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants