Skip to content

[fix](bitmap) incorrect type of BitmapValue with fastunion (#36834) #36898

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 1 commit into from
Jul 1, 2024
Merged

[fix](bitmap) incorrect type of BitmapValue with fastunion (#36834) #36898

yiguolei merged 1 commit into from
Jul 1, 2024

Conversation

@mrhhsg
Copy link
Member

@mrhhsg mrhhsg commented Jun 27, 2024

Proposed changes

pick #36834

@doris-robot
Copy link

doris-robot commented Jun 27, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@mrhhsg
Copy link
Member Author

mrhhsg commented Jun 27, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 27, 2024

clang-tidy review says "All clean, LGTM! 👍"

HappenLee
HappenLee approved these changes Jun 27, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Jun 27, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 27, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 27, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Jun 27, 2024

TeamCity be ut coverage result:
Function Coverage: 37.88% (8113/21418)
Line Coverage: 29.55% (66443/224870)
Region Coverage: 29.01% (34241/118015)
Branch Coverage: 24.89% (17588/70656)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5a3bdc24c3041c1bf3176ab8fdeade418d2aab22_5a3bdc24c3041c1bf3176ab8fdeade418d2aab22/report/index.html

@doris-robot
Copy link

doris-robot commented Jun 27, 2024

TPC-H: Total hot run time: 50143 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5a3bdc24c3041c1bf3176ab8fdeade418d2aab22, data reload: false

------ Round 1 ----------------------------------
q1	17801	4469	4396	4396
q2	2059	161	153	153
q3	10420	1951	1947	1947
q4	10252	1228	1327	1228
q5	8790	3941	3921	3921
q6	258	130	128	128
q7	2081	1590	1594	1590
q8	9293	2754	2732	2732
q9	10714	10412	10331	10331
q10	8673	3489	3516	3489
q11	415	243	241	241
q12	479	303	294	294
q13	18385	3985	3980	3980
q14	360	335	326	326
q15	504	459	469	459
q16	670	569	574	569
q17	1144	989	985	985
q18	7387	6925	7012	6925
q19	1793	1660	1601	1601
q20	538	319	302	302
q21	4510	4152	4102	4102
q22	539	454	444	444
Total cold run time: 117065 ms
Total hot run time: 50143 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4305	4345	4303	4303
q2	320	226	234	226
q3	4183	4151	4181	4151
q4	2762	2749	2765	2749
q5	7180	7124	7141	7124
q6	238	118	120	118
q7	3250	2903	2810	2810
q8	4391	4497	4529	4497
q9	16985	16908	16878	16878
q10	4254	4270	4301	4270
q11	772	708	677	677
q12	1032	840	858	840
q13	6908	3707	3736	3707
q14	442	426	421	421
q15	510	460	458	458
q16	737	698	676	676
q17	3847	3905	3874	3874
q18	8897	8750	8722	8722
q19	1733	1732	1655	1655
q20	2366	2093	2153	2093
q21	8436	8552	8491	8491
q22	1073	1045	977	977
Total cold run time: 84621 ms
Total hot run time: 79717 ms

@doris-robot
Copy link

doris-robot commented Jun 27, 2024

TPC-DS: Total hot run time: 205071 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5a3bdc24c3041c1bf3176ab8fdeade418d2aab22, data reload: false

query1	937	422	378	378
query2	6547	2938	2553	2553
query3	6922	217	206	206
query4	21575	17990	18009	17990
query5	19745	6482	6579	6482
query6	294	220	233	220
query7	4165	295	304	295
query8	417	461	408	408
query9	3139	2701	2628	2628
query10	417	314	289	289
query11	11274	10778	10721	10721
query12	132	80	77	77
query13	5600	691	680	680
query14	18054	13245	13454	13245
query15	364	244	237	237
query16	6485	291	268	268
query17	1726	1437	888	888
query18	2323	410	413	410
query19	209	143	155	143
query20	80	80	87	80
query21	192	102	100	100
query22	5257	5087	5028	5028
query23	32606	31914	32083	31914
query24	6977	6518	6511	6511
query25	513	439	425	425
query26	531	170	163	163
query27	1846	299	303	299
query28	6105	2353	2281	2281
query29	2974	2760	2844	2760
query30	239	167	170	167
query31	919	759	751	751
query32	73	61	58	58
query33	407	259	253	253
query34	832	481	492	481
query35	1133	947	898	898
query36	1276	1168	1213	1168
query37	94	57	61	57
query38	3084	2911	2897	2897
query39	1362	1330	1315	1315
query40	201	100	94	94
query41	47	44	45	44
query42	87	79	83	79
query43	719	755	649	649
query44	1161	718	712	712
query45	250	236	236	236
query46	1221	980	968	968
query47	1952	1754	1765	1754
query48	1007	719	694	694
query49	624	377	372	372
query50	869	617	627	617
query51	4736	4713	4717	4713
query52	92	77	77	77
query53	456	328	319	319
query54	2643	2451	2478	2451
query55	92	82	92	82
query56	247	225	202	202
query57	1225	1114	1044	1044
query58	209	225	213	213
query59	3974	4287	4341	4287
query60	213	210	194	194
query61	98	93	95	93
query62	875	470	468	468
query63	484	343	361	343
query64	2608	1544	1501	1501
query65	3593	3550	3549	3549
query66	766	386	378	378
query67	17490	16561	17145	16561
query68	5759	654	653	653
query69	558	368	355	355
query70	1491	1473	1316	1316
query71	400	317	319	317
query72	6353	3484	3464	3464
query73	719	316	318	316
query74	6341	5878	5892	5878
query75	4511	3733	3680	3680
query76	3322	1147	1176	1147
query77	544	249	250	249
query78	12582	11619	11426	11426
query79	6844	646	641	641
query80	2659	420	416	416
query81	528	240	235	235
query82	1366	111	98	98
query83	191	134	129	129
query84	226	70	73	70
query85	1420	319	323	319
query86	357	294	304	294
query87	3226	3079	3082	3079
query88	4684	2317	2296	2296
query89	373	299	291	291
query90	1704	210	214	210
query91	173	140	160	140
query92	61	55	54	54
query93	2225	550	572	550
query94	921	209	209	209
query95	1125	1076	1088	1076
query96	642	325	326	325
query97	6489	6417	6343	6343
query98	192	184	173	173
query99	2992	927	904	904
Total cold run time: 305742 ms
Total hot run time: 205071 ms

@doris-robot
Copy link

doris-robot commented Jun 27, 2024

ClickBench: Total hot run time: 30.17 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5a3bdc24c3041c1bf3176ab8fdeade418d2aab22, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.24	0.05	0.05
query4	1.80	0.06	0.06
query5	0.55	0.53	0.51
query6	1.26	0.63	0.61
query7	0.01	0.01	0.02
query8	0.03	0.02	0.02
query9	0.52	0.48	0.48
query10	0.54	0.53	0.54
query11	0.12	0.08	0.08
query12	0.12	0.09	0.09
query13	0.62	0.61	0.60
query14	0.78	0.79	0.78
query15	0.78	0.75	0.77
query16	0.37	0.37	0.36
query17	1.00	1.05	1.00
query18	0.24	0.28	0.23
query19	1.89	1.79	1.85
query20	0.02	0.01	0.01
query21	15.48	0.57	0.56
query22	1.92	2.18	1.68
query23	17.29	1.10	0.87
query24	5.38	4.21	0.42
query25	0.41	0.11	0.05
query26	0.77	0.16	0.16
query27	0.04	0.04	0.04
query28	5.18	0.78	0.75
query29	12.63	2.35	2.35
query30	0.56	0.53	0.52
query31	2.81	0.40	0.37
query32	3.37	0.50	0.49
query33	3.09	3.06	3.09
query34	15.25	4.80	4.78
query35	4.88	4.81	4.85
query36	1.07	1.02	1.02
query37	0.05	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.13	0.14
query41	0.07	0.01	0.02
query42	0.03	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 101.5 s
Total hot run time: 30.17 s

@doris-robot
Copy link

doris-robot commented Jun 27, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 5a3bdc24c3041c1bf3176ab8fdeade418d2aab22 with default session variables
Stream load json:         21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@yiguolei yiguolei merged commit c44d158 into apache:branch-2.0 Jul 1, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@mrhhsg
[fix](bitmap) incorrect type of BitmapValue with fastunion (apache#36834
583794f 
) (apache#36898)

## Proposed changes

pick apache#36834

<!--Describe your changes.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HappenLee HappenLee HappenLee approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mrhhsg @doris-robot @HappenLee @yiguolei