Skip to content

Conversation

@airborne12
Copy link
Member

Proposed changes

fix problem like this:

set enable_common_expr_pushdown=false;

create table t(a int, c varchar(1024)) PROPERTIES ("replication_allocation" = "tag.location.default: 1");

insert into t values(1, "a b c d e");

SELECT count() FROM t where c MATCH_PHRASE_EDGE 's';

I20240919 09:31:40.713896 2224044 internal_service.cpp:626] Cancel query 3e48a306704f4691-840d239bf31420aa, reason: [INTERNAL_ERROR]PStatus: (127.0.0.1)[INTERNAL_ERROR]Column match_phrase_edge({"c", "String"}) in block is nullptr, in method bytes. All Columns are c String match_phrase_edge({"c", "String"})                                                                                                                                                                                     
        3#  doris::vectorized::Block::allocated_bytes() const at be/src/vec/core/block.cpp:446
        4#  doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) at be/src/vec/exec/scan/scanner_scheduler.cpp:243

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@airborne12
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41465 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b9799357012efa72a9b5a2d8b7a4537dfb435d6f, data reload: false

------ Round 1 ----------------------------------
q1	17566	7798	7299	7299
q2	2035	161	151	151
q3	10715	1091	1236	1091
q4	10266	757	845	757
q5	7749	3039	3078	3039
q6	235	154	149	149
q7	1045	621	614	614
q8	9430	2047	2052	2047
q9	6808	6383	6411	6383
q10	7002	2269	2260	2260
q11	446	243	249	243
q12	402	218	216	216
q13	17770	2965	2993	2965
q14	235	218	202	202
q15	559	505	512	505
q16	687	605	608	605
q17	981	832	805	805
q18	7275	6776	6675	6675
q19	1376	1034	1088	1034
q20	566	284	286	284
q21	3924	3147	3205	3147
q22	1123	1027	994	994
Total cold run time: 108195 ms
Total hot run time: 41465 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7325	7270	7259	7259
q2	322	225	227	225
q3	3023	2946	3003	2946
q4	2003	1778	1833	1778
q5	5641	5543	5630	5543
q6	244	150	145	145
q7	2183	1793	1786	1786
q8	3318	3476	3436	3436
q9	8778	8867	8848	8848
q10	3530	3513	3508	3508
q11	587	486	486	486
q12	827	641	667	641
q13	9427	3185	3124	3124
q14	312	279	281	279
q15	586	526	568	526
q16	712	685	692	685
q17	1814	1595	1613	1595
q18	8137	7790	7733	7733
q19	1716	1565	1444	1444
q20	2114	1894	1870	1870
q21	5599	5368	5484	5368
q22	1143	1065	1046	1046
Total cold run time: 69341 ms
Total hot run time: 60271 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.33% (9584/25675)
Line Coverage: 28.72% (79248/275958)
Region Coverage: 28.19% (41026/145533)
Branch Coverage: 24.81% (20905/84262)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b9799357012efa72a9b5a2d8b7a4537dfb435d6f_b9799357012efa72a9b5a2d8b7a4537dfb435d6f/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 199312 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b9799357012efa72a9b5a2d8b7a4537dfb435d6f, data reload: false

query1	1269	869	896	869
query2	6387	2092	2044	2044
query3	10781	4050	3889	3889
query4	61955	29570	23439	23439
query5	5128	475	464	464
query6	402	167	171	167
query7	5445	308	295	295
query8	314	228	221	221
query9	8422	2618	2615	2615
query10	450	293	268	268
query11	17444	15344	15700	15344
query12	160	102	100	100
query13	1479	446	415	415
query14	10767	7481	7591	7481
query15	210	188	173	173
query16	6771	504	476	476
query17	1170	613	608	608
query18	1554	312	317	312
query19	214	150	153	150
query20	130	113	122	113
query21	209	104	111	104
query22	4656	4383	4819	4383
query23	34804	33944	33980	33944
query24	6003	2896	2868	2868
query25	526	394	398	394
query26	642	162	156	156
query27	1775	277	279	277
query28	4324	2447	2395	2395
query29	656	419	430	419
query30	228	155	153	153
query31	944	758	787	758
query32	74	52	56	52
query33	446	300	301	300
query34	896	479	485	479
query35	832	756	729	729
query36	1068	950	973	950
query37	142	89	83	83
query38	3993	3906	3900	3900
query39	1511	1467	1408	1408
query40	207	100	95	95
query41	50	47	50	47
query42	111	93	92	92
query43	525	495	485	485
query44	1128	806	798	798
query45	194	164	162	162
query46	1129	735	778	735
query47	1930	1822	1864	1822
query48	449	365	361	361
query49	718	404	383	383
query50	847	405	405	405
query51	6937	6935	6940	6935
query52	95	85	86	85
query53	248	179	178	178
query54	558	445	447	445
query55	76	72	75	72
query56	255	250	271	250
query57	1186	1077	1088	1077
query58	234	227	232	227
query59	3175	2924	2914	2914
query60	276	274	266	266
query61	106	101	114	101
query62	768	660	686	660
query63	210	183	176	176
query64	1380	659	614	614
query65	3271	3144	3166	3144
query66	683	288	303	288
query67	16150	15496	15650	15496
query68	1260	845	854	845
query69	449	351	344	344
query70	1199	1141	1203	1141
query71	344	340	345	340
query72	5925	3477	3334	3334
query73	594	574	576	574
query74	9228	8991	8949	8949
query75	2991	2900	2890	2890
query76	1197	887	868	868
query77	466	363	361	361
query78	9523	9292	9246	9246
query79	908	878	871	871
query80	585	567	575	567
query81	481	250	260	250
query82	231	232	228	228
query83	159	157	162	157
query84	267	108	101	101
query85	660	383	370	370
query86	323	317	318	317
query87	4360	4358	4310	4310
query88	4155	4067	4056	4056
query89	379	364	357	357
query90	1755	309	305	305
query91	164	162	168	162
query92	74	76	79	76
query93	916	899	903	899
query94	586	360	387	360
query95	424	410	409	409
query96	483	484	483	483
query97	3178	3111	3113	3111
query98	231	228	243	228
query99	1451	1317	1295	1295
Total cold run time: 302023 ms
Total hot run time: 199312 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.38 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b9799357012efa72a9b5a2d8b7a4537dfb435d6f, data reload: false

query1	0.04	0.05	0.04
query2	0.06	0.03	0.02
query3	0.24	0.06	0.06
query4	1.64	0.10	0.10
query5	0.51	0.50	0.51
query6	1.15	0.73	0.72
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.57	0.50	0.50
query10	0.54	0.56	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.60	0.59
query14	2.98	2.93	3.10
query15	0.89	0.82	0.81
query16	0.38	0.41	0.39
query17	1.01	1.06	1.05
query18	0.20	0.19	0.19
query19	1.96	1.87	2.01
query20	0.01	0.01	0.01
query21	15.35	0.63	0.61
query22	3.01	1.81	1.77
query23	17.27	0.98	0.84
query24	3.16	1.47	1.53
query25	0.21	0.07	0.17
query26	0.51	0.14	0.13
query27	0.03	0.05	0.03
query28	9.79	1.10	1.07
query29	12.58	3.37	3.39
query30	0.25	0.06	0.05
query31	2.87	0.38	0.36
query32	3.30	0.47	0.45
query33	2.99	3.04	2.98
query34	17.31	4.43	4.42
query35	4.38	4.44	4.41
query36	0.68	0.47	0.48
query37	0.08	0.06	0.06
query38	0.04	0.03	0.03
query39	0.02	0.02	0.02
query40	0.16	0.13	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 107.25 s
Total hot run time: 33.38 s

Copy link
Contributor

@zhiqiang-hhhh zhiqiang-hhhh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 19, 2024
@airborne12 airborne12 merged commit 2495d12 into apache:master Sep 19, 2024
@airborne12 airborne12 deleted the fix-filter-block branch September 19, 2024 06:16
airborne12 added a commit to airborne12/apache-doris that referenced this pull request Sep 20, 2024
…che#40953)

fix problem like this:
```
set enable_common_expr_pushdown=false;

create table t(a int, c varchar(1024)) PROPERTIES ("replication_allocation" = "tag.location.default: 1");

insert into t values(1, "a b c d e");

SELECT count() FROM t where c MATCH_PHRASE_EDGE 's';

I20240919 09:31:40.713896 2224044 internal_service.cpp:626] Cancel query 3e48a306704f4691-840d239bf31420aa, reason: [INTERNAL_ERROR]PStatus: (127.0.0.1)[INTERNAL_ERROR]Column match_phrase_edge({"c", "String"}) in block is nullptr, in method bytes. All Columns are c String match_phrase_edge({"c", "String"})
        3#  doris::vectorized::Block::allocated_bytes() const at be/src/vec/core/block.cpp:446
        4#  doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) at be/src/vec/exec/scan/scanner_scheduler.cpp:243
```
airborne12 added a commit that referenced this pull request Sep 20, 2024
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
)

## Proposed changes

fix problem like this:
```
set enable_common_expr_pushdown=false;

create table t(a int, c varchar(1024)) PROPERTIES ("replication_allocation" = "tag.location.default: 1");

insert into t values(1, "a b c d e");

SELECT count() FROM t where c MATCH_PHRASE_EDGE 's';

I20240919 09:31:40.713896 2224044 internal_service.cpp:626] Cancel query 3e48a306704f4691-840d239bf31420aa, reason: [INTERNAL_ERROR]PStatus: (127.0.0.1)[INTERNAL_ERROR]Column match_phrase_edge({"c", "String"}) in block is nullptr, in method bytes. All Columns are c String match_phrase_edge({"c", "String"})                                                                                                                                                                                     
        3#  doris::vectorized::Block::allocated_bytes() const at be/src/vec/core/block.cpp:446
        4#  doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) at be/src/vec/exec/scan/scanner_scheduler.cpp:243
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.3-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants