[Fix](inverted index) fix inverted index read data opt not work on MOW #31075

airborne12 · 2024-02-18T14:33:20Z

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-02-18T14:33:25Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

airborne12 · 2024-02-18T14:33:32Z

run buildall

github-actions · 2024-02-18T14:42:30Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-02-18T15:04:38Z

TeamCity be ut coverage result:
Function Coverage: 35.77% (8559/23929)
Line Coverage: 27.73% (69423/250379)
Region Coverage: 26.86% (36031/134157)
Branch Coverage: 23.65% (18420/77874)
Coverage Report: http://coverage.selectdb-in.cc/coverage/32260c4d0860e74bb8d840dc887a8f9a93e4ea8a_32260c4d0860e74bb8d840dc887a8f9a93e4ea8a/report/index.html

xiaokang

LGTM

github-actions · 2024-02-18T15:25:38Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-02-18T15:25:39Z

PR approved by anyone and no changes requested.

airborne12 · 2024-02-19T10:23:34Z

run buildall

airborne12 · 2024-02-19T10:31:17Z

run buildall

github-actions · 2024-02-19T10:32:11Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-02-19T10:40:05Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-02-19T10:55:21Z

TeamCity be ut coverage result:
Function Coverage: 35.77% (8558/23924)
Line Coverage: 27.71% (69385/250364)
Region Coverage: 26.84% (36008/134172)
Branch Coverage: 23.65% (18421/77884)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c7cf48b2d0f9a3ae7ef01b33f89d908c876305d7_c7cf48b2d0f9a3ae7ef01b33f89d908c876305d7/report/index.html

doris-robot · 2024-02-19T10:57:20Z

TPC-H: Total hot run time: 41226 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c7cf48b2d0f9a3ae7ef01b33f89d908c876305d7, data reload: false

------ Round 1 ----------------------------------
q1	17641	5008	4872	4872
q2	2039	139	132	132
q3	10586	1027	1012	1012
q4	4706	961	968	961
q5	7717	3225	3260	3225
q6	194	136	131	131
q7	1258	762	757	757
q8	9386	2086	2056	2056
q9	7577	6675	6642	6642
q10	8307	2629	2624	2624
q11	416	203	214	203
q12	726	330	326	326
q13	17992	3659	3704	3659
q14	291	261	259	259
q15	587	510	517	510
q16	479	404	411	404
q17	925	861	842	842
q18	7360	6532	6586	6532
q19	1532	1481	1495	1481
q20	641	346	331	331
q21	6753	3931	3957	3931
q22	879	351	336	336
Total cold run time: 107992 ms
Total hot run time: 41226 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4868	4852	4866	4852
q2	293	183	184	183
q3	3621	3605	3585	3585
q4	2546	2557	2565	2557
q5	5762	5750	5745	5745
q6	208	125	126	125
q7	2246	1635	1646	1635
q8	3055	3141	3123	3123
q9	8695	8668	8691	8668
q10	6813	4261	4229	4229
q11	514	386	374	374
q12	782	552	549	549
q13	4313	3446	3410	3410
q14	253	239	237	237
q15	629	553	485	485
q16	477	435	441	435
q17	1657	1611	1603	1603
q18	8254	7686	7679	7679
q19	1626	1626	1625	1625
q20	2096	1849	1823	1823
q21	6572	6250	6219	6219
q22	581	522	530	522
Total cold run time: 65861 ms
Total hot run time: 59663 ms

doris-robot · 2024-02-19T11:08:53Z

TPC-DS: Total hot run time: 186148 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c7cf48b2d0f9a3ae7ef01b33f89d908c876305d7, data reload: false

query1	906	352	342	342
query2	6531	1781	1740	1740
query3	6688	215	204	204
query4	22946	21056	21155	21056
query5	4318	452	370	370
query6	259	169	162	162
query7	4606	296	290	290
query8	244	192	197	192
query9	8446	2794	2781	2781
query10	426	237	246	237
query11	14968	14732	14731	14731
query12	143	83	82	82
query13	1707	418	410	410
query14	9648	7628	7783	7628
query15	208	188	190	188
query16	7963	274	259	259
query17	2917	596	533	533
query18	2094	268	262	262
query19	188	151	147	147
query20	90	87	90	87
query21	181	117	115	115
query22	5039	4769	4680	4680
query23	32786	32744	32222	32222
query24	12708	3386	3336	3336
query25	639	360	352	352
query26	1935	159	156	156
query27	3055	306	313	306
query28	6596	1839	1844	1839
query29	1133	610	601	601
query30	282	136	145	136
query31	972	738	761	738
query32	93	56	56	56
query33	726	235	231	231
query34	1062	484	489	484
query35	924	809	813	809
query36	987	887	931	887
query37	267	62	63	62
query38	3270	3199	3126	3126
query39	1367	1326	1327	1326
query40	280	100	104	100
query41	35	35	34	34
query42	107	105	98	98
query43	468	470	443	443
query44	1056	696	698	696
query45	201	183	172	172
query46	1046	778	777	777
query47	1637	1570	1548	1548
query48	420	337	343	337
query49	1206	300	295	295
query50	763	366	370	366
query51	5304	5188	5133	5133
query52	108	90	93	90
query53	397	295	287	287
query54	289	215	220	215
query55	85	78	80	78
query56	219	196	192	192
query57	1066	950	971	950
query58	213	193	194	193
query59	2343	2226	2295	2226
query60	242	213	220	213
query61	81	87	81	81
query62	588	399	374	374
query63	325	279	283	279
query64	6262	3074	3131	3074
query65	3284	3249	3259	3249
query66	1330	331	320	320
query67	14604	14080	14222	14080
query68	5276	535	544	535
query69	519	353	351	351
query70	1267	1263	1214	1214
query71	448	244	280	244
query72	6228	2803	2615	2615
query73	710	305	305	305
query74	6852	6417	6450	6417
query75	3194	2551	2541	2541
query76	3342	1179	1205	1179
query77	422	234	248	234
query78	9423	8807	8833	8807
query79	967	499	507	499
query80	513	353	339	339
query81	436	203	203	203
query82	240	84	82	82
query83	138	120	116	116
query84	225	77	78	77
query85	1016	340	327	327
query86	280	307	301	301
query87	3450	3306	3246	3246
query88	2691	2269	2256	2256
query89	435	360	367	360
query90	1969	162	161	161
query91	154	127	123	123
query92	57	47	47	47
query93	995	518	480	480
query94	1252	176	180	176
query95	8859	8590	8357	8357
query96	583	268	261	261
query97	4425	4295	4312	4295
query98	214	200	194	194
query99	1068	752	743	743
Total cold run time: 280925 ms
Total hot run time: 186148 ms

doris-robot · 2024-02-19T11:14:16Z

ClickBench: Total hot run time: 30.52 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c7cf48b2d0f9a3ae7ef01b33f89d908c876305d7, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.02	0.02
query3	0.23	0.07	0.08
query4	1.65	0.08	0.08
query5	0.49	0.48	0.49
query6	1.33	0.62	0.61
query7	0.02	0.01	0.02
query8	0.03	0.02	0.02
query9	0.53	0.46	0.46
query10	0.48	0.49	0.48
query11	0.13	0.10	0.10
query12	0.11	0.10	0.10
query13	0.58	0.60	0.59
query14	0.76	0.80	0.78
query15	0.82	0.80	0.79
query16	0.34	0.32	0.33
query17	0.92	0.92	0.91
query18	0.19	0.19	0.15
query19	1.81	1.72	1.64
query20	0.01	0.01	0.02
query21	15.40	0.63	0.60
query22	3.21	3.31	2.23
query23	17.60	1.09	0.97
query24	2.18	0.52	0.19
query25	0.62	0.08	0.05
query26	0.16	0.14	0.14
query27	0.06	0.06	0.05
query28	12.07	0.84	0.80
query29	12.50	3.22	3.31
query30	0.52	0.48	0.48
query31	2.81	0.37	0.36
query32	3.35	0.48	0.48
query33	3.16	3.15	3.12
query34	15.35	4.50	4.46
query35	4.51	4.52	4.51
query36	1.06	0.95	0.95
query37	0.07	0.05	0.06
query38	0.04	0.02	0.02
query39	0.01	0.01	0.02
query40	0.16	0.14	0.15
query41	0.07	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 105.48 s
Total hot run time: 30.52 s

doris-robot · 2024-02-19T11:17:21Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c7cf48b2d0f9a3ae7ef01b33f89d908c876305d7 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.5 seconds inserted 10000000 Rows, about 740K ops/s

…on MOW #31075 (#31076)

xiaokang

LGTM

github-actions · 2024-02-19T13:20:28Z

PR approved by at least one committer and no changes requested.

#31075)

…on MOW apache#31075 (apache#31076)

[Fix](inverted index) fix inverted index read data opt not work on MOW

32260c4

airborne12 mentioned this pull request Feb 18, 2024

[Pick 2.0](inverted index) fix inverted index read data opt not work on MOW #31076

Merged

xiaokang previously approved these changes Feb 18, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 18, 2024

github-actions bot added the reviewed label Feb 18, 2024

xiaokang mentioned this pull request Feb 18, 2024

[fix](inverted index) add MOW for _need_read_key_data and _need_read_data #31078

Closed

fix delete

25e979b

airborne12 dismissed xiaokang’s stale review via 25e979b February 19, 2024 10:22

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Feb 19, 2024

fix delete

c7cf48b

xiaokang pushed a commit that referenced this pull request Feb 19, 2024

[Pick 2.0](inverted index) fix inverted index read data opt not work …

1485f37

…on MOW #31075 (#31076)

xiaokang added the dev/2.0.5-merged label Feb 19, 2024

xiaokang approved these changes Feb 19, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 19, 2024

qidaye approved these changes Feb 19, 2024

View reviewed changes

qidaye merged commit 6345985 into apache:master Feb 19, 2024

yiguolei pushed a commit that referenced this pull request Feb 20, 2024

[Fix](inverted index) fix inverted index read data opt not work on MOW (

066d674

#31075)

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024

[Pick 2.0](inverted index) fix inverted index read data opt not work …

e29e267

…on MOW apache#31075 (apache#31076)

[Fix](inverted index) fix inverted index read data opt not work on MOW #31075

[Fix](inverted index) fix inverted index read data opt not work on MOW #31075

Uh oh!

Conversation

airborne12 commented Feb 18, 2024

Proposed changes

Further comments

Uh oh!

doris-robot commented Feb 18, 2024

Uh oh!

airborne12 commented Feb 18, 2024

Uh oh!

github-actions bot commented Feb 18, 2024

Uh oh!

doris-robot commented Feb 18, 2024

Uh oh!

xiaokang left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 18, 2024

Uh oh!

github-actions bot commented Feb 18, 2024

Uh oh!

airborne12 commented Feb 19, 2024

Uh oh!

airborne12 commented Feb 19, 2024

Uh oh!

github-actions bot commented Feb 19, 2024

Uh oh!

github-actions bot commented Feb 19, 2024

Uh oh!

doris-robot commented Feb 19, 2024

Uh oh!

doris-robot commented Feb 19, 2024

Uh oh!

doris-robot commented Feb 19, 2024

Uh oh!

doris-robot commented Feb 19, 2024

Uh oh!

doris-robot commented Feb 19, 2024

Uh oh!

xiaokang left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 19, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants