Skip to content

[opt]function use sse to opt match_ipv6_subnet #38755

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
zhangstar333 merged 1 commit into from
Aug 19, 2024
Merged

[opt]function use sse to opt match_ipv6_subnet #38755

zhangstar333 merged 1 commit into from
Aug 19, 2024

Conversation

@Mryange
Copy link
Contributor

@Mryange Mryange commented Aug 2, 2024

Proposed changes

test

-------------------------------------------------------------------
Benchmark                         Time             CPU   Iterations
-------------------------------------------------------------------
BM_matchIPv6SubnetSSE          1.89 ns         1.89 ns   1000000000
BM_matchIPv6SubnetNative       4.99 ns         4.99 ns    561455254

@doris-robot
Copy link

doris-robot commented Aug 2, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Mryange
Copy link
Contributor Author

Mryange commented Aug 2, 2024

run buildall

@github-actions github-actions bot added the doing label Aug 2, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Aug 2, 2024

TPC-H: Total hot run time: 41688 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6a5ff5917a2d47a818e02acc44370f9571d9b8fa, data reload: false

------ Round 1 ----------------------------------
q1	17684	4129	4089	4089
q2	2024	219	199	199
q3	10469	1265	1329	1265
q4	10171	813	891	813
q5	7631	3027	2971	2971
q6	220	137	137	137
q7	1060	609	606	606
q8	9452	1889	1951	1889
q9	8459	6594	6608	6594
q10	8692	3849	3821	3821
q11	436	252	254	252
q12	409	230	229	229
q13	17761	2941	2933	2933
q14	273	244	241	241
q15	514	483	498	483
q16	527	394	386	386
q17	980	947	921	921
q18	8108	7323	7365	7323
q19	2170	1195	1225	1195
q20	596	321	334	321
q21	5345	4787	4744	4744
q22	351	276	287	276
Total cold run time: 113332 ms
Total hot run time: 41688 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4093	4018	4019	4018
q2	328	230	234	230
q3	2994	3006	3150	3006
q4	1997	2014	1986	1986
q5	5623	5514	5457	5457
q6	226	131	131	131
q7	2123	1797	1829	1797
q8	3355	3360	3360	3360
q9	8750	8657	8831	8657
q10	3906	4077	3891	3891
q11	556	463	484	463
q12	796	618	589	589
q13	16324	3111	3105	3105
q14	307	272	280	272
q15	534	493	486	486
q16	465	427	413	413
q17	1780	1774	1753	1753
q18	8295	7871	7776	7776
q19	1954	1712	1711	1711
q20	2065	1863	1847	1847
q21	5668	5452	5360	5360
q22	540	450	481	450
Total cold run time: 72679 ms
Total hot run time: 56758 ms

@doris-robot
Copy link

doris-robot commented Aug 2, 2024

TPC-DS: Total hot run time: 169050 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6a5ff5917a2d47a818e02acc44370f9571d9b8fa, data reload: false

query1	917	380	371	371
query2	6457	1690	1691	1690
query3	6655	214	227	214
query4	19316	17408	17147	17147
query5	3663	503	509	503
query6	295	173	166	166
query7	4600	296	295	295
query8	245	194	204	194
query9	8524	2368	2373	2368
query10	449	282	268	268
query11	10491	10097	10139	10097
query12	125	86	90	86
query13	1632	384	380	380
query14	9933	6834	7506	6834
query15	222	162	161	161
query16	6928	422	462	422
query17	928	557	555	555
query18	1899	271	279	271
query19	190	139	140	139
query20	88	83	88	83
query21	205	100	95	95
query22	4197	4019	3927	3927
query23	33788	33860	33228	33228
query24	9324	3111	3076	3076
query25	683	395	404	395
query26	1543	150	156	150
query27	2873	283	288	283
query28	7566	2034	2010	2010
query29	1186	447	426	426
query30	242	157	156	156
query31	956	766	771	766
query32	106	56	59	56
query33	703	331	324	324
query34	937	493	506	493
query35	929	746	753	746
query36	1048	903	881	881
query37	212	80	83	80
query38	2966	2857	2749	2749
query39	884	803	816	803
query40	258	114	110	110
query41	46	48	44	44
query42	119	103	98	98
query43	468	419	424	419
query44	1161	737	784	737
query45	213	180	180	180
query46	1080	794	770	770
query47	1814	1722	1730	1722
query48	359	296	302	296
query49	954	438	460	438
query50	894	430	431	430
query51	6789	6720	6562	6562
query52	102	89	93	89
query53	253	185	179	179
query54	640	454	465	454
query55	80	79	76	76
query56	267	257	256	256
query57	1125	1039	1047	1039
query58	275	266	301	266
query59	2491	2489	2324	2324
query60	295	276	266	266
query61	99	94	98	94
query62	872	665	672	665
query63	213	183	179	179
query64	5662	1945	1883	1883
query65	3165	3088	3082	3082
query66	1213	333	330	330
query67	15099	15018	14825	14825
query68	4340	567	577	567
query69	435	305	306	305
query70	1119	1088	1068	1068
query71	378	278	277	277
query72	7188	2753	2501	2501
query73	763	334	327	327
query74	6040	5654	5690	5654
query75	3366	2735	2750	2735
query76	2127	1217	1263	1217
query77	431	311	297	297
query78	9366	8987	8898	8898
query79	1830	530	523	523
query80	1116	554	502	502
query81	578	229	232	229
query82	1065	128	127	127
query83	236	167	167	167
query84	261	77	78	77
query85	1243	319	303	303
query86	393	301	310	301
query87	3246	3125	3112	3112
query88	3011	2525	2512	2512
query89	393	302	291	291
query90	1686	202	187	187
query91	127	100	101	100
query92	60	54	50	50
query93	1540	616	612	612
query94	817	279	296	279
query95	381	260	264	260
query96	601	284	288	284
query97	3245	3095	3101	3095
query98	215	203	258	203
query99	1659	1289	1315	1289
Total cold run time: 259167 ms
Total hot run time: 169050 ms

@doris-robot
Copy link

doris-robot commented Aug 2, 2024

ClickBench: Total hot run time: 30.59 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6a5ff5917a2d47a818e02acc44370f9571d9b8fa, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.06
query5	0.48	0.47	0.47
query6	1.14	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.59	0.50	0.51
query10	0.56	0.56	0.57
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.62	0.61	0.61
query14	0.78	0.80	0.79
query15	0.89	0.85	0.86
query16	0.34	0.35	0.36
query17	1.00	1.01	1.04
query18	0.22	0.21	0.21
query19	1.90	1.80	1.75
query20	0.01	0.01	0.01
query21	15.40	0.77	0.68
query22	3.96	7.58	1.90
query23	18.18	1.30	1.28
query24	2.15	0.24	0.22
query25	0.19	0.08	0.08
query26	0.32	0.21	0.20
query27	0.46	0.23	0.22
query28	13.15	1.00	0.96
query29	12.59	3.23	3.22
query30	0.26	0.06	0.05
query31	2.87	0.40	0.40
query32	3.25	0.51	0.48
query33	2.96	2.96	2.96
query34	15.45	4.27	4.26
query35	4.29	4.31	4.30
query36	0.68	0.48	0.48
query37	0.19	0.17	0.16
query38	0.16	0.14	0.15
query39	0.05	0.04	0.04
query40	0.15	0.14	0.14
query41	0.09	0.04	0.05
query42	0.06	0.06	0.04
query43	0.04	0.05	0.04
Total cold run time: 107.81 s
Total hot run time: 30.59 s

@Mryange Mryange closed this Aug 2, 2024
@Mryange Mryange reopened this Aug 2, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@Mryange
Copy link
Contributor Author

Mryange commented Aug 5, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Aug 5, 2024

TPC-H: Total hot run time: 41285 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 64e0e3c00790770a6e53cea9529498bd9403d746, data reload: false

------ Round 1 ----------------------------------
q1	17698	4042	4031	4031
q2	2012	199	194	194
q3	10481	1256	1377	1256
q4	10179	828	880	828
q5	7657	2990	2961	2961
q6	222	143	142	142
q7	1057	614	615	614
q8	9423	1737	1931	1737
q9	9258	6573	6584	6573
q10	8737	3815	3838	3815
q11	427	252	253	252
q12	451	231	232	231
q13	17764	2937	2935	2935
q14	270	240	240	240
q15	522	485	496	485
q16	519	397	386	386
q17	968	880	925	880
q18	7976	7276	7218	7218
q19	1432	1218	1212	1212
q20	554	316	329	316
q21	5285	4700	4763	4700
q22	352	279	285	279
Total cold run time: 113244 ms
Total hot run time: 41285 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4098	3994	4000	3994
q2	322	217	225	217
q3	2991	3005	3138	3005
q4	2007	2061	1980	1980
q5	5606	5446	5431	5431
q6	228	137	129	129
q7	2117	1799	1819	1799
q8	3290	3358	3318	3318
q9	8686	8615	8806	8615
q10	3925	4073	3904	3904
q11	573	453	481	453
q12	764	595	588	588
q13	13354	3151	3129	3129
q14	321	283	267	267
q15	530	478	490	478
q16	457	432	429	429
q17	1791	1746	1731	1731
q18	8173	7653	7614	7614
q19	2315	1716	1744	1716
q20	2097	1857	1853	1853
q21	5691	5544	5244	5244
q22	542	454	468	454
Total cold run time: 69878 ms
Total hot run time: 56348 ms

@doris-robot
Copy link

doris-robot commented Aug 5, 2024

TPC-DS: Total hot run time: 169481 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 64e0e3c00790770a6e53cea9529498bd9403d746, data reload: false

query1	905	380	368	368
query2	6475	1664	1615	1615
query3	6642	212	239	212
query4	19941	17246	17168	17168
query5	3621	527	533	527
query6	294	197	176	176
query7	4587	305	304	304
query8	263	216	189	189
query9	8495	2335	2339	2335
query10	446	297	270	270
query11	10578	9941	10067	9941
query12	120	88	91	88
query13	1621	372	373	372
query14	9622	6912	7755	6912
query15	217	164	165	164
query16	6859	429	473	429
query17	935	560	554	554
query18	1733	291	295	291
query19	202	156	153	153
query20	97	90	93	90
query21	208	101	102	101
query22	4243	3985	4064	3985
query23	33900	33659	33702	33659
query24	10269	3188	3132	3132
query25	680	433	413	413
query26	1727	151	152	151
query27	3086	291	290	290
query28	7220	2035	1968	1968
query29	1129	438	447	438
query30	237	156	155	155
query31	915	763	783	763
query32	105	56	57	56
query33	685	353	336	336
query34	918	524	512	512
query35	887	769	776	769
query36	1066	914	901	901
query37	181	85	86	85
query38	2923	2865	2763	2763
query39	880	829	812	812
query40	260	113	112	112
query41	49	43	47	43
query42	127	104	107	104
query43	443	403	413	403
query44	1174	739	779	739
query45	215	188	181	181
query46	1102	812	782	782
query47	1840	1722	1704	1704
query48	369	301	293	293
query49	903	427	426	426
query50	897	447	450	447
query51	6699	6734	6680	6680
query52	112	96	94	94
query53	254	187	184	184
query54	636	452	458	452
query55	81	73	74	73
query56	276	264	262	262
query57	1131	1045	1022	1022
query58	268	264	269	264
query59	2579	2549	2253	2253
query60	291	281	267	267
query61	95	107	96	96
query62	913	659	675	659
query63	212	193	191	191
query64	5681	1946	1870	1870
query65	3166	3132	3102	3102
query66	1309	340	344	340
query67	15287	14682	14711	14682
query68	4328	587	583	583
query69	454	315	311	311
query70	1115	1068	1054	1054
query71	384	294	282	282
query72	7191	2661	2527	2527
query73	762	337	338	337
query74	6034	5600	5645	5600
query75	3352	2697	2765	2697
query76	2091	1229	1267	1229
query77	431	311	318	311
query78	9344	9037	8812	8812
query79	2107	528	540	528
query80	1081	522	526	522
query81	543	228	230	228
query82	1046	130	132	130
query83	240	183	167	167
query84	265	79	79	79
query85	1276	311	300	300
query86	457	323	286	286
query87	3312	3101	3109	3101
query88	3016	2541	2528	2528
query89	385	298	289	289
query90	1690	192	197	192
query91	129	175	103	103
query92	61	50	50	50
query93	1969	620	615	615
query94	809	306	304	304
query95	386	283	276	276
query96	610	287	293	287
query97	3261	3082	3109	3082
query98	220	198	194	194
query99	1629	1305	1314	1305
Total cold run time: 261259 ms
Total hot run time: 169481 ms

@doris-robot
Copy link

doris-robot commented Aug 5, 2024

ClickBench: Total hot run time: 29.73 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 64e0e3c00790770a6e53cea9529498bd9403d746, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.04	0.04
query4	1.69	0.07	0.07
query5	0.49	0.57	0.48
query6	1.13	0.72	0.74
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.57	0.50	0.50
query10	0.56	0.57	0.58
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.62	0.60	0.60
query14	0.79	0.77	0.79
query15	0.91	0.87	0.86
query16	0.36	0.36	0.36
query17	0.99	0.98	0.97
query18	0.22	0.20	0.21
query19	1.90	1.75	1.72
query20	0.01	0.01	0.02
query21	15.40	0.75	0.66
query22	3.95	7.90	1.07
query23	17.92	1.29	1.24
query24	2.24	0.23	0.22
query25	0.19	0.08	0.09
query26	0.32	0.21	0.22
query27	0.46	0.23	0.23
query28	13.18	1.00	0.97
query29	12.54	3.29	3.30
query30	0.26	0.05	0.06
query31	2.87	0.41	0.41
query32	3.24	0.48	0.48
query33	2.90	2.96	2.98
query34	15.42	4.28	4.26
query35	4.32	4.29	4.32
query36	0.69	0.47	0.48
query37	0.20	0.16	0.16
query38	0.17	0.15	0.15
query39	0.04	0.04	0.04
query40	0.16	0.14	0.13
query41	0.10	0.04	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 107.61 s
Total hot run time: 29.73 s

@Mryange
Copy link
Contributor Author

Mryange commented Aug 5, 2024

run p0

HappenLee
HappenLee reviewed Aug 6, 2024
View reviewed changes
be/src/vec/runtime/ip_address_cidr.h
return (addr[IPV6_BINARY_LENGTH - 1 - offset] & cmpmask) ==
(cidr_addr[IPV6_BINARY_LENGTH - 1 - offset] & cmpmask);
} else {
// All the bytes are equal.
Copy link
Contributor

@HappenLee HappenLee Aug 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why keep the unless else

Copy link
Contributor Author

@Mryange Mryange Aug 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for the comment

HappenLee
HappenLee approved these changes Aug 16, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Aug 16, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 16, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 16, 2024

PR approved by anyone and no changes requested.

@zhangstar333 zhangstar333 merged commit 37044e1 into apache:master Aug 19, 2024
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
@Mryange @dataroaring
[opt]function use sse to opt match_ipv6_subnet (#38755)
a51b1ee 
## Proposed changes

test
```
-------------------------------------------------------------------
Benchmark                         Time             CPU   Iterations
-------------------------------------------------------------------
BM_matchIPv6SubnetSSE          1.89 ns         1.89 ns   1000000000
BM_matchIPv6SubnetNative       4.99 ns         4.99 ns    561455254
```
Mryange added a commit to Mryange/doris that referenced this pull request Nov 8, 2024
@Mryange
[opt]function use sse to opt match_ipv6_subnet (apache#38755)
389bd00 
## Proposed changes

test
```
-------------------------------------------------------------------
Benchmark                         Time             CPU   Iterations
-------------------------------------------------------------------
BM_matchIPv6SubnetSSE          1.89 ns         1.89 ns   1000000000
BM_matchIPv6SubnetNative       4.99 ns         4.99 ns    561455254
```
@Mryange Mryange mentioned this pull request Nov 8, 2024
Merged
16 tasks
yiguolei pushed a commit that referenced this pull request Nov 11, 2024
@Mryange
[opt]function use sse to opt match_ipv6_subnet (#38755) (#43513)
dc9ec5b 
## Proposed changes
#38755
test
```
-------------------------------------------------------------------
Benchmark                         Time             CPU   Iterations
-------------------------------------------------------------------
BM_matchIPv6SubnetSSE          1.89 ns         1.89 ns   1000000000
BM_matchIPv6SubnetNative       4.99 ns         4.99 ns    561455254
```
@gavinchou gavinchou mentioned this pull request Nov 26, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HappenLee HappenLee HappenLee approved these changes

@zclllyybb zclllyybb zclllyybb approved these changes

@zhangstar333 zhangstar333 zhangstar333 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@Mryange @doris-robot @HappenLee @zclllyybb @zhangstar333 @yiguolei @dataroaring