Skip to content

[fix](kerberos)enable hadoop auto renew tgt #34439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 2 commits into from
May 6, 2024
Merged

[fix](kerberos)enable hadoop auto renew tgt #34439

yiguolei merged 2 commits into from
May 6, 2024

Conversation

@wsjz
Copy link
Contributor

@wsjz wsjz commented May 6, 2024

Proposed changes

  1. use hadoop properties key
  2. enable auto renew tgt for kerberos auth, set HADOOP_KERBEROS_KEYTAB_LOGIN_AUTORENEWAL_ENABLED

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

doris-robot commented May 6, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@wsjz wsjz mentioned this pull request May 6, 2024
Closed
@wsjz
Copy link
Contributor Author

wsjz commented May 6, 2024

run buildall

@doris-robot
Copy link

doris-robot commented May 6, 2024

TPC-H: Total hot run time: 41127 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f88bc8c0a800a42a6f136cc0a76873c64568bbcb, data reload: false

------ Round 1 ----------------------------------
q1	17804	5346	4389	4389
q2	2603	194	188	188
q3	10940	1279	1219	1219
q4	10459	779	769	769
q5	7517	2795	2794	2794
q6	221	133	134	133
q7	1037	630	589	589
q8	9405	2106	2092	2092
q9	9011	6576	6525	6525
q10	9014	3747	3762	3747
q11	448	238	232	232
q12	508	218	220	218
q13	17789	2961	2986	2961
q14	256	243	221	221
q15	506	480	466	466
q16	516	393	381	381
q17	966	681	733	681
q18	8035	7421	7395	7395
q19	1631	1566	1532	1532
q20	646	311	304	304
q21	5287	4015	4009	4009
q22	354	292	282	282
Total cold run time: 114953 ms
Total hot run time: 41127 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4338	4293	4243	4243
q2	389	271	271	271
q3	2993	2772	2737	2737
q4	1868	1588	1607	1588
q5	5299	5307	5332	5307
q6	215	125	128	125
q7	2304	1921	1916	1916
q8	3244	3375	3374	3374
q9	8478	8479	8513	8479
q10	3924	3654	3688	3654
q11	591	486	491	486
q12	776	584	617	584
q13	16800	3032	2963	2963
q14	296	279	271	271
q15	532	475	469	469
q16	488	411	416	411
q17	1762	1485	1477	1477
q18	7635	7626	7397	7397
q19	3125	1602	1553	1553
q20	1964	1786	1822	1786
q21	4970	4911	4908	4908
q22	584	481	474	474
Total cold run time: 72575 ms
Total hot run time: 54473 ms

@doris-robot
Copy link

doris-robot commented May 6, 2024

TPC-DS: Total hot run time: 185635 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f88bc8c0a800a42a6f136cc0a76873c64568bbcb, data reload: false

query1	907	368	354	354
query2	7076	2517	2447	2447
query3	6648	204	206	204
query4	23036	21290	21113	21113
query5	4149	444	432	432
query6	273	179	176	176
query7	4584	290	286	286
query8	245	204	186	186
query9	8594	2358	2340	2340
query10	447	256	265	256
query11	14690	14250	14168	14168
query12	146	93	87	87
query13	1651	380	378	378
query14	9721	7235	7533	7235
query15	232	176	173	173
query16	8066	269	264	264
query17	1847	602	549	549
query18	2063	291	276	276
query19	207	157	154	154
query20	95	99	93	93
query21	201	137	131	131
query22	5000	4795	4823	4795
query23	33832	33198	33215	33198
query24	12149	2921	2880	2880
query25	692	381	382	381
query26	1786	160	155	155
query27	3072	313	328	313
query28	7705	2056	2029	2029
query29	1077	629	613	613
query30	297	157	154	154
query31	972	729	729	729
query32	97	55	116	55
query33	746	248	247	247
query34	1099	478	486	478
query35	820	672	688	672
query36	1048	954	920	920
query37	268	65	69	65
query38	3173	2976	3019	2976
query39	1618	1519	1525	1519
query40	275	126	125	125
query41	40	53	42	42
query42	112	98	95	95
query43	578	555	537	537
query44	1218	715	730	715
query45	274	245	225	225
query46	1079	730	722	722
query47	1960	1857	1878	1857
query48	367	295	298	295
query49	1187	389	416	389
query50	792	384	394	384
query51	6780	6553	6750	6553
query52	103	96	96	96
query53	351	275	281	275
query54	315	242	242	242
query55	83	80	73	73
query56	243	225	232	225
query57	1214	1112	1169	1112
query58	223	206	206	206
query59	3470	3295	3356	3295
query60	270	236	243	236
query61	91	91	87	87
query62	663	456	440	440
query63	300	276	275	275
query64	9611	7197	7252	7197
query65	3124	3087	3050	3050
query66	1391	339	344	339
query67	15623	14859	15218	14859
query68	6871	522	534	522
query69	537	326	304	304
query70	1182	1174	1106	1106
query71	493	266	267	266
query72	7751	2583	2353	2353
query73	729	321	320	320
query74	6584	6067	5997	5997
query75	4141	2618	2664	2618
query76	4994	957	1007	957
query77	632	274	272	272
query78	10853	10386	10104	10104
query79	9977	513	520	513
query80	1522	439	444	439
query81	513	219	220	219
query82	433	99	94	94
query83	226	167	169	167
query84	269	88	88	88
query85	1011	271	260	260
query86	357	286	313	286
query87	3324	3072	3036	3036
query88	4607	2342	2322	2322
query89	526	385	383	383
query90	2114	192	185	185
query91	127	99	98	98
query92	59	51	50	50
query93	7040	526	514	514
query94	1358	186	194	186
query95	404	306	308	306
query96	643	269	267	267
query97	3139	2921	2937	2921
query98	249	221	218	218
query99	1273	889	837	837
Total cold run time: 306004 ms
Total hot run time: 185635 ms

morningman
morningman approved these changes May 6, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 6, 2024
@github-actions
Copy link
Contributor

github-actions bot commented May 6, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented May 6, 2024

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit 729a7da into apache:master May 6, 2024
yiguolei pushed a commit that referenced this pull request May 6, 2024
@wsjz @Doris-Extras
[fix](kerberos)enable hadoop auto renew tgt (#34439)
2d4da7d
ByteYue pushed a commit to ByteYue/doris that referenced this pull request May 15, 2024
@wsjz @ByteYue
[fix](kerberos)enable hadoop auto renew tgt (apache#34439)
5cc2eca
@wsjz wsjz mentioned this pull request May 28, 2024
Merged
morningman pushed a commit that referenced this pull request May 28, 2024
@wsjz
[fix](kerberos)enable hadoop auto renew tgt (#35522)
2762f14 
bp #34439
@wsjz wsjz deleted the renew_tgt_enable branch July 4, 2024 08:39
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@wsjz
[fix](kerberos)enable hadoop auto renew tgt (apache#35522)
4560421 
bp apache#34439
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@yiguolei yiguolei yiguolei approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.11-merged dev/2.1.3-merged dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@wsjz @doris-robot @morningman @yiguolei @dataroaring