Skip to content

[opt](auditlog) Use varchar(1024) for column frontend_ip of audit log table #52762

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
gavinchou merged 2 commits into from
Jul 7, 2025
Merged

[opt](auditlog) Use varchar(1024) for column frontend_ip of audit log table #52762

gavinchou merged 2 commits into from
Jul 7, 2025

Conversation

@gavinchou
Copy link
Contributor

@gavinchou gavinchou commented Jul 4, 2025

length 128 of column frontend_ip of audit log table may be not large enough for k8s env.

Reason:column_name ［frontend_ip］, the length of input is too long than schema.
first 32 bytes of input str:  ［ abcedf-xxxxxx-yyyyyyy］ schema length: 128; 
actual length:159; . src line ［］;

@Thearas
Copy link
Contributor

Thearas commented Jul 4, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@gavinchou
Copy link
Contributor Author

gavinchou commented Jul 4, 2025

run buildall

morningman
morningman previously approved these changes Jul 4, 2025
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@intelligentfu8
Copy link

intelligentfu8 commented Jul 4, 2025

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 4, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 4, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 4, 2025

PR approved by anyone and no changes requested.

@intelligentfu8 intelligentfu8 self-requested a review July 4, 2025 06:44
intelligentfu8
intelligentfu8 previously approved these changes Jul 4, 2025
View reviewed changes
Copy link

@intelligentfu8 intelligentfu8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot
Copy link

doris-robot commented Jul 4, 2025

TPC-H: Total hot run time: 34141 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d7dfc3fe6ce2c25390f49f024d3c83ca270e4fd4, data reload: false

------ Round 1 ----------------------------------
q1	17602	5299	5132	5132
q2	2033	291	192	192
q3	10768	1377	738	738
q4	11028	1077	523	523
q5	7785	2441	2437	2437
q6	206	167	130	130
q7	923	755	603	603
q8	9424	1486	1120	1120
q9	6875	5198	5276	5198
q10	6945	2417	1952	1952
q11	500	294	272	272
q12	340	354	200	200
q13	17786	3728	3029	3029
q14	231	235	208	208
q15	572	477	482	477
q16	423	416	381	381
q17	616	904	356	356
q18	7494	7140	7179	7140
q19	1686	1076	580	580
q20	326	356	216	216
q21	3742	3152	2287	2287
q22	1036	985	970	970
Total cold run time: 108341 ms
Total hot run time: 34141 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5505	5371	5361	5361
q2	263	330	217	217
q3	2186	2658	2273	2273
q4	1404	1764	1327	1327
q5	4618	4469	4532	4469
q6	246	173	131	131
q7	1990	2011	1849	1849
q8	2777	2680	2683	2680
q9	7482	7427	7326	7326
q10	3155	3347	2980	2980
q11	596	521	498	498
q12	706	755	585	585
q13	3730	3945	3368	3368
q14	318	302	285	285
q15	534	482	471	471
q16	555	498	446	446
q17	1225	1718	1420	1420
q18	8019	7703	7738	7703
q19	868	871	1113	871
q20	2034	1983	1812	1812
q21	4750	4276	4229	4229
q22	1074	1024	986	986
Total cold run time: 54035 ms
Total hot run time: 51287 ms

@doris-robot
Copy link

doris-robot commented Jul 4, 2025

TPC-DS: Total hot run time: 184518 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d7dfc3fe6ce2c25390f49f024d3c83ca270e4fd4, data reload: false

query1	1011	392	381	381
query2	6535	1686	1685	1685
query3	6746	218	210	210
query4	26787	23734	23390	23390
query5	4373	568	423	423
query6	306	210	219	210
query7	4624	511	290	290
query8	283	223	213	213
query9	8599	2568	2589	2568
query10	485	333	254	254
query11	15605	15114	14787	14787
query12	153	104	105	104
query13	1646	520	422	422
query14	8494	5592	5604	5592
query15	203	193	171	171
query16	7376	627	440	440
query17	1218	764	571	571
query18	1990	390	292	292
query19	192	192	197	192
query20	121	113	110	110
query21	211	119	105	105
query22	4140	4168	4052	4052
query23	33886	32982	33025	32982
query24	8478	2338	2343	2338
query25	525	452	382	382
query26	817	270	161	161
query27	2742	508	337	337
query28	4309	2124	2089	2089
query29	681	562	467	467
query30	280	216	198	198
query31	917	844	758	758
query32	70	67	61	61
query33	545	365	297	297
query34	805	830	506	506
query35	801	815	725	725
query36	963	960	894	894
query37	114	95	74	74
query38	4056	4116	4140	4116
query39	1487	1437	1414	1414
query40	206	115	103	103
query41	59	57	55	55
query42	136	117	107	107
query43	487	508	481	481
query44	1372	829	838	829
query45	179	167	159	159
query46	849	1020	616	616
query47	1767	1832	1700	1700
query48	371	416	314	314
query49	692	469	365	365
query50	634	690	408	408
query51	4077	4130	4069	4069
query52	111	114	102	102
query53	221	253	190	190
query54	564	555	492	492
query55	77	76	82	76
query56	295	290	297	290
query57	1235	1177	1137	1137
query58	272	254	254	254
query59	2624	2668	2477	2477
query60	317	314	309	309
query61	128	121	120	120
query62	785	710	642	642
query63	222	189	190	189
query64	3488	1019	652	652
query65	4248	4179	4225	4179
query66	998	407	313	313
query67	15788	15510	15255	15255
query68	7833	877	516	516
query69	488	299	268	268
query70	1191	1114	1103	1103
query71	429	316	302	302
query72	5587	4982	4800	4800
query73	676	616	349	349
query74	9231	9095	9181	9095
query75	3531	3269	2709	2709
query76	3241	1130	714	714
query77	584	380	298	298
query78	9929	10032	9193	9193
query79	2464	818	582	582
query80	667	515	446	446
query81	485	260	224	224
query82	186	127	95	95
query83	245	246	228	228
query84	254	98	83	83
query85	758	356	358	356
query86	364	296	296	296
query87	4389	4500	4253	4253
query88	3732	2250	2252	2250
query89	385	312	287	287
query90	1992	204	198	198
query91	134	145	110	110
query92	76	63	54	54
query93	2204	960	587	587
query94	672	421	302	302
query95	387	293	331	293
query96	494	574	274	274
query97	2689	2756	2639	2639
query98	233	213	204	204
query99	1313	1381	1266	1266
Total cold run time: 271619 ms
Total hot run time: 184518 ms

@doris-robot
Copy link

doris-robot commented Jul 4, 2025

ClickBench: Total hot run time: 29.23 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d7dfc3fe6ce2c25390f49f024d3c83ca270e4fd4, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.05
query3	0.24	0.07	0.07
query4	1.63	0.11	0.11
query5	0.42	0.44	0.42
query6	1.16	0.65	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.60	0.51	0.53
query10	0.57	0.57	0.56
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.61	0.62
query14	0.82	0.80	0.80
query15	0.88	0.86	0.86
query16	0.39	0.38	0.39
query17	1.07	1.01	1.07
query18	0.22	0.22	0.21
query19	1.93	1.82	1.89
query20	0.01	0.02	0.02
query21	15.40	0.91	0.54
query22	0.76	1.15	0.61
query23	15.02	1.38	0.60
query24	7.82	0.64	0.86
query25	0.47	0.30	0.08
query26	0.64	0.17	0.14
query27	0.07	0.05	0.05
query28	9.20	0.93	0.46
query29	12.56	4.05	3.34
query30	0.25	0.09	0.06
query31	2.84	0.59	0.38
query32	3.24	0.55	0.46
query33	3.13	3.07	3.07
query34	16.06	5.32	4.76
query35	4.87	4.90	4.86
query36	0.70	0.49	0.48
query37	0.09	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.14	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 104.58 s
Total hot run time: 29.23 s

@gavinchou gavinchou dismissed stale reviews from intelligentfu8 and morningman via 18ca6e7 July 5, 2025 15:42
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 5, 2025
@gavinchou
Copy link
Contributor Author

gavinchou commented Jul 5, 2025

skip buildall

@gavinchou
Copy link
Contributor Author

gavinchou commented Jul 5, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 5, 2025

TPC-H: Total hot run time: 33546 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 18ca6e77369c6befa743bae8741859b696b2302b, data reload: false

------ Round 1 ----------------------------------
q1	17591	5207	5039	5039
q2	1923	297	195	195
q3	10302	1257	709	709
q4	10230	1007	502	502
q5	7540	2297	2319	2297
q6	188	160	130	130
q7	868	741	600	600
q8	9339	1303	1040	1040
q9	6775	5064	5067	5064
q10	6873	2378	1955	1955
q11	484	303	268	268
q12	338	342	210	210
q13	17780	3664	3053	3053
q14	232	219	211	211
q15	550	469	479	469
q16	424	415	388	388
q17	602	850	360	360
q18	7462	7145	7022	7022
q19	1207	949	533	533
q20	328	354	214	214
q21	3801	2570	2303	2303
q22	1027	1009	984	984
Total cold run time: 105864 ms
Total hot run time: 33546 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5117	5091	5045	5045
q2	249	325	230	230
q3	2145	2678	2271	2271
q4	1365	1757	1360	1360
q5	4204	4277	4519	4277
q6	219	169	125	125
q7	2047	2017	1840	1840
q8	2599	2676	2588	2588
q9	7295	7233	7306	7233
q10	3306	3339	2837	2837
q11	580	549	504	504
q12	663	793	651	651
q13	3523	3990	3339	3339
q14	274	314	298	298
q15	513	476	463	463
q16	430	488	451	451
q17	1167	1558	1384	1384
q18	8049	7549	7605	7549
q19	798	798	943	798
q20	1920	1944	1782	1782
q21	4799	4346	4355	4346
q22	1079	1047	958	958
Total cold run time: 52341 ms
Total hot run time: 50329 ms

@doris-robot
Copy link

doris-robot commented Jul 5, 2025

TPC-DS: Total hot run time: 185015 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 18ca6e77369c6befa743bae8741859b696b2302b, data reload: false

query1	1017	403	414	403
query2	6527	1674	1669	1669
query3	6733	211	207	207
query4	26344	23309	23684	23309
query5	4344	584	443	443
query6	297	217	215	215
query7	4632	492	288	288
query8	280	229	210	210
query9	8623	2599	2611	2599
query10	443	331	260	260
query11	15372	15064	14801	14801
query12	143	104	99	99
query13	1651	529	412	412
query14	8627	5747	5737	5737
query15	206	186	175	175
query16	7121	668	432	432
query17	920	680	566	566
query18	1964	391	299	299
query19	190	185	152	152
query20	116	118	112	112
query21	209	124	143	124
query22	4156	4360	4101	4101
query23	33777	33151	33026	33026
query24	8414	2362	2357	2357
query25	520	474	398	398
query26	1221	266	142	142
query27	2783	511	334	334
query28	4310	2104	2077	2077
query29	744	560	451	451
query30	282	223	195	195
query31	924	831	763	763
query32	77	68	66	66
query33	569	357	319	319
query34	788	845	527	527
query35	815	818	745	745
query36	956	968	887	887
query37	116	94	73	73
query38	4121	4138	4052	4052
query39	1453	1417	1421	1417
query40	203	113	106	106
query41	56	53	50	50
query42	120	106	103	103
query43	486	521	485	485
query44	1312	828	820	820
query45	176	174	169	169
query46	842	1004	624	624
query47	1744	1816	1725	1725
query48	394	414	303	303
query49	738	488	398	398
query50	624	682	397	397
query51	4119	4053	4077	4053
query52	108	109	105	105
query53	216	248	182	182
query54	571	561	502	502
query55	80	78	79	78
query56	284	309	298	298
query57	1160	1180	1140	1140
query58	281	254	265	254
query59	2480	2724	2465	2465
query60	312	319	298	298
query61	127	128	123	123
query62	803	719	645	645
query63	221	188	193	188
query64	4386	1106	745	745
query65	4301	4166	4137	4137
query66	1180	416	321	321
query67	15711	15634	15581	15581
query68	7833	895	534	534
query69	486	330	267	267
query70	1215	1162	1143	1143
query71	428	333	303	303
query72	5504	4740	4781	4740
query73	680	626	346	346
query74	8884	9222	8812	8812
query75	3720	3162	2654	2654
query76	3411	1136	710	710
query77	785	376	284	284
query78	10008	10318	9277	9277
query79	2695	826	590	590
query80	623	508	504	504
query81	505	252	226	226
query82	443	126	95	95
query83	242	256	248	248
query84	294	96	81	81
query85	794	349	306	306
query86	384	296	297	296
query87	4405	4492	4365	4365
query88	3643	2258	2296	2258
query89	380	318	287	287
query90	1847	206	203	203
query91	135	141	106	106
query92	74	55	58	55
query93	2228	946	586	586
query94	693	401	308	308
query95	377	285	290	285
query96	494	584	285	285
query97	2751	2759	2619	2619
query98	231	212	204	204
query99	1396	1390	1301	1301
Total cold run time: 272408 ms
Total hot run time: 185015 ms

@doris-robot
Copy link

doris-robot commented Jul 5, 2025

ClickBench: Total hot run time: 29.02 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 18ca6e77369c6befa743bae8741859b696b2302b, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.25	0.08	0.08
query4	1.62	0.11	0.11
query5	0.44	0.44	0.40
query6	1.16	0.66	0.65
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.61	0.51	0.52
query10	0.56	0.56	0.57
query11	0.15	0.11	0.11
query12	0.15	0.12	0.11
query13	0.61	0.61	0.61
query14	0.79	0.81	0.82
query15	0.91	0.88	0.87
query16	0.38	0.39	0.40
query17	1.04	1.07	1.07
query18	0.22	0.21	0.21
query19	2.03	1.82	1.80
query20	0.02	0.01	0.01
query21	15.42	0.89	0.55
query22	0.76	1.16	0.71
query23	14.91	1.35	0.64
query24	6.54	2.04	0.33
query25	0.40	0.25	0.08
query26	0.53	0.16	0.15
query27	0.07	0.05	0.05
query28	9.22	0.92	0.44
query29	12.56	3.91	3.32
query30	0.25	0.10	0.06
query31	2.84	0.59	0.38
query32	3.27	0.56	0.46
query33	3.03	3.05	3.19
query34	16.02	5.34	4.75
query35	4.79	4.90	4.83
query36	0.72	0.51	0.48
query37	0.09	0.07	0.07
query38	0.06	0.03	0.04
query39	0.03	0.03	0.02
query40	0.18	0.15	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 102.97 s
Total hot run time: 29.02 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 6, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 6, 2025

PR approved by at least one committer and no changes requested.

@gavinchou gavinchou merged commit ea62025 into apache:master Jul 7, 2025
27 of 29 checks passed
github-actions bot pushed a commit that referenced this pull request Jul 7, 2025
@gavinchou
[opt](auditlog) Use varchar(1024) for column frontend_ip of audit log…
2ae1f0c 
… table (#52762)

length 128 of column frontend_ip of audit log table may be not large
enough for k8s env.
```
Reason:column_name ［frontend_ip］, the length of input is too long than schema.
first 32 bytes of input str:  ［ abcedf-xxxxxx-yyyyyyy］ schema length: 128; 
actual length:159; . src line ［］;
```
@github-actions github-actions bot mentioned this pull request Jul 7, 2025
Merged
@yagagagaga yagagagaga mentioned this pull request Jul 8, 2025
Merged
16 tasks
hello-stephen pushed a commit that referenced this pull request Jul 8, 2025
@yagagagaga
[fix](audit) fix audit log test (#52913)
355b4b9 
### What problem does this PR solve?

Issue Number: close #xxx

Related PR:

because this PR #52762 modify the audit log schema, so the regression
test need to be modified.
morrySnow pushed a commit that referenced this pull request Jul 8, 2025
@github-actions @gavinchou
branch-3.1: [opt](auditlog) Use varchar(1024) for column frontend_ip …
e27f2d8 
…of audit log table #52762 (#52846)

Cherry-picked from #52762

Co-authored-by: Gavin Chou <gavin@selectdb.com>
gavinchou added a commit to gavinchou/doris that referenced this pull request Jul 9, 2025
@gavinchou
[opt](auditlog) Use varchar(1024) for column frontend_ip of audit log…
77ffee1 
… table apache#52762
@gavinchou gavinchou mentioned this pull request Jul 9, 2025
Merged
gavinchou added a commit to gavinchou/doris that referenced this pull request Jul 9, 2025
@gavinchou
[opt](auditlog) Use varchar(1024) for column frontend_ip of audit log…
3cd065e 
… table apache#52762
@gavinchou gavinchou mentioned this pull request Jul 9, 2025
Merged
dataroaring pushed a commit that referenced this pull request Jul 9, 2025
@gavinchou
[opt](auditlog) Use varchar(1024) for column frontend_ip of audit log…
36ea0a5 
… table #52762 (#52984)

pick #52762
yiguolei pushed a commit that referenced this pull request Jul 10, 2025
@gavinchou
[opt](auditlog) Use varchar(1024) for column frontend_ip of audit log…
6e12dc4 
… table #52762 (#52985)

pick #52762
@gavinchou gavinchou mentioned this pull request Aug 15, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@intelligentfu8 intelligentfu8 intelligentfu8 left review comments

+1 more reviewer

@koarz koarz koarz approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.11-merged dev/3.0.7-merged dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@gavinchou @Thearas @intelligentfu8 @doris-robot @morningman @koarz @yiguolei @dataroaring @morrySnow