Skip to content

Conversation

@amorynan
Copy link
Contributor

@amorynan amorynan commented Jan 17, 2025

What problem does this PR solve?

Enhance ipv6 type to accept uint128 strings in network byte order.
according to function : ipv6_from_uint128_string_or_null(string)
For load we can load the uint128 number like 338288524927261089657673092949903306134 using function
ipv6_from_uint128_string_or_null
or just using function in select sql

mysql> select ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455');
+-----------------------------------------------------------------------------+
| ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455') |
+-----------------------------------------------------------------------------+
| ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff                                     |
+-----------------------------------------------------------------------------+
1 row in set (0.06 sec)

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@amorynan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32362 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 05bc29de475292ed4633c465d49ddd0f8c128be6, data reload: false

------ Round 1 ----------------------------------
q1	17566	5514	5380	5380
q2	2046	313	172	172
q3	10413	1258	749	749
q4	10240	1006	522	522
q5	8447	2368	2176	2176
q6	196	167	134	134
q7	899	764	607	607
q8	9242	1384	1184	1184
q9	5246	4830	4876	4830
q10	6816	2352	1888	1888
q11	478	269	256	256
q12	339	363	221	221
q13	17785	3724	3180	3180
q14	227	241	207	207
q15	514	475	479	475
q16	641	618	589	589
q17	577	861	325	325
q18	7237	6606	6434	6434
q19	1528	949	535	535
q20	319	321	189	189
q21	2943	2183	1990	1990
q22	373	338	319	319
Total cold run time: 104072 ms
Total hot run time: 32362 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5522	5435	5453	5435
q2	237	341	233	233
q3	2280	2688	2331	2331
q4	1438	1810	1395	1395
q5	4350	4818	4670	4670
q6	177	161	130	130
q7	2063	1970	1874	1874
q8	2648	2862	2710	2710
q9	7318	7325	7376	7325
q10	3012	3324	2700	2700
q11	561	506	491	491
q12	642	727	612	612
q13	3521	3909	3342	3342
q14	291	302	284	284
q15	521	486	481	481
q16	652	679	644	644
q17	1240	1715	1268	1268
q18	7870	7480	7475	7475
q19	812	1141	1094	1094
q20	2038	2054	1878	1878
q21	5729	5311	4973	4973
q22	650	617	577	577
Total cold run time: 53572 ms
Total hot run time: 51922 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194581 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 05bc29de475292ed4633c465d49ddd0f8c128be6, data reload: false

query1	1323	988	934	934
query2	6444	2049	1985	1985
query3	11108	4800	4625	4625
query4	32514	23379	22757	22757
query5	3662	615	479	479
query6	294	201	187	187
query7	3980	489	305	305
query8	300	268	241	241
query9	9330	2590	2570	2570
query10	471	318	261	261
query11	17689	15625	15017	15017
query12	174	114	107	107
query13	1572	536	417	417
query14	9710	6687	7720	6687
query15	270	217	197	197
query16	7816	689	526	526
query17	1656	786	614	614
query18	1986	416	303	303
query19	204	181	161	161
query20	123	122	118	118
query21	218	123	106	106
query22	4554	4561	4422	4422
query23	34598	33682	33686	33682
query24	6475	2283	2319	2283
query25	506	469	400	400
query26	790	278	154	154
query27	2025	482	323	323
query28	5175	2455	2426	2426
query29	597	575	428	428
query30	215	199	168	168
query31	974	890	783	783
query32	76	59	57	57
query33	487	349	319	319
query34	762	871	515	515
query35	798	873	768	768
query36	1026	1034	963	963
query37	115	99	83	83
query38	4282	4444	4334	4334
query39	1460	1473	1429	1429
query40	206	118	106	106
query41	58	52	52	52
query42	121	102	116	102
query43	508	537	485	485
query44	1318	827	808	808
query45	187	180	171	171
query46	895	1052	669	669
query47	1913	1925	1869	1869
query48	385	394	332	332
query49	725	509	416	416
query50	644	662	394	394
query51	6951	6901	6910	6901
query52	100	102	95	95
query53	230	265	183	183
query54	480	515	421	421
query55	81	81	79	79
query56	257	274	255	255
query57	1214	1205	1188	1188
query58	260	249	226	226
query59	3092	3281	3249	3249
query60	284	276	271	271
query61	121	115	118	115
query62	812	742	682	682
query63	219	195	192	192
query64	3150	1024	668	668
query65	3248	3164	3207	3164
query66	790	396	302	302
query67	16040	15754	15447	15447
query68	6219	811	524	524
query69	492	307	250	250
query70	1173	1122	1111	1111
query71	387	288	301	288
query72	5366	3929	3901	3901
query73	634	742	350	350
query74	10023	9112	8981	8981
query75	3177	3150	2659	2659
query76	3117	1133	770	770
query77	456	371	279	279
query78	10197	10040	9360	9360
query79	2774	808	692	692
query80	712	529	445	445
query81	511	275	243	243
query82	215	147	125	125
query83	184	166	151	151
query84	234	89	70	70
query85	786	374	301	301
query86	443	331	300	300
query87	4444	4585	4490	4490
query88	4646	2130	2094	2094
query89	396	320	301	301
query90	1874	185	193	185
query91	128	139	108	108
query92	66	60	55	55
query93	2430	894	527	527
query94	637	420	289	289
query95	340	271	248	248
query96	484	623	281	281
query97	2778	2859	2766	2766
query98	228	208	199	199
query99	1303	1352	1262	1262
Total cold run time: 286902 ms
Total hot run time: 194581 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 41.53% (10823/26060)
Line Coverage: 31.93% (91472/286462)
Region Coverage: 31.08% (46863/150797)
Branch Coverage: 27.17% (23733/87356)
Coverage Report: http://coverage.selectdb-in.cc/coverage/05bc29de475292ed4633c465d49ddd0f8c128be6_05bc29de475292ed4633c465d49ddd0f8c128be6/report/index.html

@doris-robot
Copy link

ClickBench: Total hot run time: 30.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 05bc29de475292ed4633c465d49ddd0f8c128be6, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.04
query3	0.24	0.07	0.07
query4	1.62	0.10	0.10
query5	0.43	0.42	0.42
query6	1.13	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.51	0.50
query10	0.57	0.55	0.55
query11	0.15	0.11	0.11
query12	0.14	0.10	0.11
query13	0.61	0.60	0.61
query14	2.72	2.83	2.72
query15	0.89	0.82	0.84
query16	0.38	0.39	0.38
query17	1.04	1.06	1.05
query18	0.24	0.21	0.21
query19	1.98	1.83	2.05
query20	0.01	0.01	0.02
query21	15.36	0.90	0.59
query22	0.75	0.79	0.61
query23	15.34	1.42	0.52
query24	2.95	1.72	0.43
query25	0.21	0.25	0.16
query26	0.32	0.15	0.14
query27	0.05	0.04	0.06
query28	12.88	1.06	0.43
query29	12.52	4.01	3.34
query30	0.24	0.09	0.06
query31	2.84	0.58	0.38
query32	3.23	0.55	0.47
query33	3.03	3.09	3.04
query34	16.62	5.19	4.48
query35	4.53	4.58	4.57
query36	0.68	0.52	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.13	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.93 s
Total hot run time: 30.28 s

@amorynan
Copy link
Contributor Author

run buildall

@amorynan
Copy link
Contributor Author

run buildall

@amorynan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32457 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1e4eb788266ac5cad29e1529071a897d8ca1be95, data reload: false

------ Round 1 ----------------------------------
q1	17763	5563	5738	5563
q2	2046	320	178	178
q3	10494	1326	733	733
q4	10240	982	531	531
q5	8016	2458	2212	2212
q6	201	178	137	137
q7	912	763	601	601
q8	9223	1372	1131	1131
q9	5412	4944	4880	4880
q10	6838	2345	1899	1899
q11	476	286	264	264
q12	347	362	230	230
q13	17762	3685	3052	3052
q14	231	237	218	218
q15	510	469	466	466
q16	633	647	589	589
q17	582	887	325	325
q18	7288	6468	6355	6355
q19	2617	972	533	533
q20	310	320	189	189
q21	2776	2198	2060	2060
q22	364	339	311	311
Total cold run time: 105041 ms
Total hot run time: 32457 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5627	5527	5558	5527
q2	240	332	237	237
q3	2247	2663	2260	2260
q4	1374	1848	1365	1365
q5	4360	4809	4779	4779
q6	171	167	131	131
q7	2096	2019	1996	1996
q8	2631	2874	2686	2686
q9	7219	7337	7310	7310
q10	3011	3297	2847	2847
q11	571	513	501	501
q12	678	764	609	609
q13	3473	3914	3330	3330
q14	283	319	267	267
q15	509	467	464	464
q16	654	708	643	643
q17	1254	1749	1285	1285
q18	7805	7563	7357	7357
q19	768	1005	1166	1005
q20	2046	2080	1934	1934
q21	5709	5371	4916	4916
q22	594	571	530	530
Total cold run time: 53320 ms
Total hot run time: 51979 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187445 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1e4eb788266ac5cad29e1529071a897d8ca1be95, data reload: false

query1	972	383	382	382
query2	6534	2100	2094	2094
query3	6805	224	219	219
query4	33596	23631	22894	22894
query5	4386	626	467	467
query6	290	223	187	187
query7	4618	478	294	294
query8	294	239	223	223
query9	9410	2617	2578	2578
query10	504	327	249	249
query11	18227	15299	14936	14936
query12	161	116	107	107
query13	1655	531	410	410
query14	9683	6689	6598	6598
query15	239	195	190	190
query16	8056	617	492	492
query17	1623	718	540	540
query18	2110	393	289	289
query19	215	188	150	150
query20	126	110	107	107
query21	213	122	100	100
query22	4208	4237	4191	4191
query23	34310	32979	33170	32979
query24	7057	2257	2315	2257
query25	489	455	393	393
query26	1211	268	156	156
query27	2566	452	324	324
query28	5783	2413	2412	2412
query29	724	553	419	419
query30	234	187	159	159
query31	953	881	804	804
query32	98	58	59	58
query33	559	360	306	306
query34	725	845	510	510
query35	802	844	769	769
query36	994	1040	951	951
query37	125	97	86	86
query38	4164	4217	4125	4125
query39	1451	1433	1413	1413
query40	208	114	100	100
query41	53	56	50	50
query42	126	101	101	101
query43	506	529	487	487
query44	1305	789	805	789
query45	176	166	167	166
query46	859	1065	665	665
query47	1834	1864	1763	1763
query48	388	390	311	311
query49	782	506	396	396
query50	615	677	404	404
query51	6891	6909	6846	6846
query52	104	105	92	92
query53	220	247	186	186
query54	480	508	409	409
query55	79	81	81	81
query56	263	275	243	243
query57	1200	1142	1086	1086
query58	262	238	246	238
query59	2991	3251	2962	2962
query60	271	271	246	246
query61	124	118	117	117
query62	780	707	669	669
query63	226	196	188	188
query64	4534	999	666	666
query65	3303	3150	3152	3150
query66	1087	414	313	313
query67	15841	15700	15754	15700
query68	6610	799	504	504
query69	469	299	255	255
query70	1220	1135	1138	1135
query71	368	282	253	253
query72	5759	3806	3842	3806
query73	641	753	359	359
query74	10233	9085	8918	8918
query75	3240	3199	2641	2641
query76	3212	1120	765	765
query77	456	369	277	277
query78	10037	9976	9271	9271
query79	3524	797	583	583
query80	698	528	448	448
query81	510	281	240	240
query82	543	147	125	125
query83	178	175	151	151
query84	241	93	69	69
query85	868	349	298	298
query86	408	321	312	312
query87	4358	4515	4463	4463
query88	4720	2158	2116	2116
query89	450	343	303	303
query90	1864	192	189	189
query91	135	135	110	110
query92	69	56	57	56
query93	2324	853	528	528
query94	699	402	285	285
query95	336	262	257	257
query96	486	601	280	280
query97	2781	2904	2742	2742
query98	231	200	198	198
query99	1525	1363	1266	1266
Total cold run time: 290871 ms
Total hot run time: 187445 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.91 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1e4eb788266ac5cad29e1529071a897d8ca1be95, data reload: false

query1	0.04	0.06	0.03
query2	0.06	0.04	0.03
query3	0.25	0.07	0.07
query4	1.62	0.10	0.10
query5	0.44	0.42	0.42
query6	1.13	0.65	0.66
query7	0.03	0.02	0.01
query8	0.04	0.04	0.03
query9	0.57	0.51	0.51
query10	0.56	0.58	0.55
query11	0.14	0.09	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.71	2.72	2.73
query15	0.89	0.83	0.82
query16	0.40	0.39	0.39
query17	1.07	0.96	1.06
query18	0.22	0.21	0.21
query19	1.97	2.03	1.84
query20	0.02	0.01	0.01
query21	15.36	0.91	0.57
query22	0.75	0.73	0.72
query23	15.31	1.43	0.54
query24	2.70	1.09	1.49
query25	0.14	0.09	0.15
query26	0.30	0.16	0.13
query27	0.05	0.05	0.06
query28	14.23	0.97	0.42
query29	12.56	3.91	3.30
query30	0.25	0.09	0.06
query31	2.83	0.58	0.39
query32	3.23	0.55	0.49
query33	3.00	3.13	3.05
query34	16.57	5.16	4.54
query35	4.60	4.53	4.54
query36	0.64	0.50	0.48
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.9 s
Total hot run time: 30.91 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 41.52% (10823/26067)
Line Coverage: 31.92% (91468/286523)
Region Coverage: 31.07% (46857/150825)
Branch Coverage: 27.17% (23734/87368)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1e4eb788266ac5cad29e1529071a897d8ca1be95_1e4eb788266ac5cad29e1529071a897d8ca1be95/report/index.html

@amorynan amorynan requested a review from HappenLee January 17, 2025 14:11
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 20, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit f89ab32 into apache:master Feb 5, 2025
23 of 26 checks passed
lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
…k byte order (apache#47126)

### What problem does this PR solve?
 Enhance ipv6  type to accept uint128 strings in network byte order. 
according to function : ipv6_from_uint128_string_or_null(string) 
For load we can load the uint128 number like
```338288524927261089657673092949903306134``` using function
ipv6_from_uint128_string_or_null
or just using function in select sql
  ```
mysql> select
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455');

+-----------------------------------------------------------------------------+
|
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455')
|

+-----------------------------------------------------------------------------+
| ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff |

+-----------------------------------------------------------------------------+
1 row in set (0.06 sec)
```、
amorynan pushed a commit to amorynan/doris that referenced this pull request Mar 7, 2025
…k byte order (apache#47126)

### What problem does this PR solve?
 Enhance ipv6  type to accept uint128 strings in network byte order. 
according to function : ipv6_from_uint128_string_or_null(string) 
For load we can load the uint128 number like
```338288524927261089657673092949903306134``` using function
ipv6_from_uint128_string_or_null
or just using function in select sql
  ```
mysql> select
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455');

+-----------------------------------------------------------------------------+
|
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455')
|

+-----------------------------------------------------------------------------+
| ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff |

+-----------------------------------------------------------------------------+
1 row in set (0.06 sec)
```、
@wm1581066 wm1581066 added usercase Important user case type label dev/2.1.x dev/3.0.x labels Mar 7, 2025
github-actions bot pushed a commit that referenced this pull request Mar 7, 2025
…k byte order (#47126)

### What problem does this PR solve?
 Enhance ipv6  type to accept uint128 strings in network byte order. 
according to function : ipv6_from_uint128_string_or_null(string) 
For load we can load the uint128 number like
```338288524927261089657673092949903306134``` using function
ipv6_from_uint128_string_or_null
or just using function in select sql
  ```
mysql> select
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455');

+-----------------------------------------------------------------------------+
|
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455')
|

+-----------------------------------------------------------------------------+
| ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff |

+-----------------------------------------------------------------------------+
1 row in set (0.06 sec)
```、
github-actions bot pushed a commit that referenced this pull request Mar 7, 2025
…k byte order (#47126)

### What problem does this PR solve?
 Enhance ipv6  type to accept uint128 strings in network byte order. 
according to function : ipv6_from_uint128_string_or_null(string) 
For load we can load the uint128 number like
```338288524927261089657673092949903306134``` using function
ipv6_from_uint128_string_or_null
or just using function in select sql
  ```
mysql> select
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455');

+-----------------------------------------------------------------------------+
|
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455')
|

+-----------------------------------------------------------------------------+
| ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff |

+-----------------------------------------------------------------------------+
1 row in set (0.06 sec)
```、
amorynan pushed a commit to amorynan/doris that referenced this pull request Mar 7, 2025
…k byte order (apache#47126)

### What problem does this PR solve?
 Enhance ipv6  type to accept uint128 strings in network byte order. 
according to function : ipv6_from_uint128_string_or_null(string) 
For load we can load the uint128 number like
```338288524927261089657673092949903306134``` using function
ipv6_from_uint128_string_or_null
or just using function in select sql
  ```
mysql> select
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455');

+-----------------------------------------------------------------------------+
|
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455')
|

+-----------------------------------------------------------------------------+
| ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff |

+-----------------------------------------------------------------------------+
1 row in set (0.06 sec)
```、
dataroaring pushed a commit that referenced this pull request Mar 10, 2025
…#48802)

…rk byte order (#47126)

### What problem does this PR solve?
 Enhance ipv6  type to accept uint128 strings in network byte order. 
according to function : ipv6_from_uint128_string_or_null(string) For
load we can load the uint128 number like
```338288524927261089657673092949903306134``` using function
ipv6_from_uint128_string_or_null
or just using function in select sql
  ```
mysql> select
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455');

+-----------------------------------------------------------------------------+
|
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455')
|

+-----------------------------------------------------------------------------+
| ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff |

+-----------------------------------------------------------------------------+
1 row in set (0.06 sec)
```、
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
…k byte order (apache#47126)

### What problem does this PR solve?
 Enhance ipv6  type to accept uint128 strings in network byte order. 
according to function : ipv6_from_uint128_string_or_null(string) 
For load we can load the uint128 number like
```338288524927261089657673092949903306134``` using function
ipv6_from_uint128_string_or_null
or just using function in select sql
  ```
mysql> select
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455');

+-----------------------------------------------------------------------------+
|
ipv6_from_uint128_string_or_null('340282366920938463463374607431768211455')
|

+-----------------------------------------------------------------------------+
| ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff |

+-----------------------------------------------------------------------------+
1 row in set (0.06 sec)
```、
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.9-merged dev/3.0.5-merged reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants