Skip to content

branch-3.0: [fix](iceberg)Fix the failure of creating a table in COS #49885 #49901

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Apr 11, 2025
Merged

branch-3.0: [fix](iceberg)Fix the failure of creating a table in COS #49885 #49901

dataroaring merged 1 commit into from
Apr 11, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Apr 9, 2025

Cherry-picked from #49885

@wuwenchi
[fix](iceberg)Fix the failure of creating a table in COS (#49885)
e7ecc8b 
### What problem does this PR solve?

Problem Summary:

When use JDK17 with cos, if create an iceberg table, an error will
occur:
```
java.lang.IllegalAccessError: class org.apache.hadoop.fs.cosn.buffer.CosNMappedBuffer (in unnamed module @0x4bb33f74) cannot access class sun.nio.ch.FileChannelImpl (in module java.base) because module java.base does not export
sun.nio.ch to unnamed module @0x4bb33f74
```

so, we should add JVM config: `--add-opens
java.base/sun.nio.ch=ALL-UNNAMED`.
@github-actions github-actions bot requested a review from dataroaring as a code owner April 9, 2025 06:44
@Thearas
Copy link
Contributor

Thearas commented Apr 9, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Apr 9, 2025
@dataroaring dataroaring reopened this Apr 9, 2025
@Thearas
Copy link
Contributor

Thearas commented Apr 9, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Apr 9, 2025

TPC-H: Total hot run time: 39924 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e7ecc8bcfcadb610066031f0116e8bc097d5ea56, data reload: false

------ Round 1 ----------------------------------
q1	17609	6905	6751	6751
q2	2079	184	185	184
q3	11217	1061	1189	1061
q4	10544	738	733	733
q5	7750	2865	2749	2749
q6	218	133	132	132
q7	983	615	603	603
q8	9366	1966	1989	1966
q9	6561	6348	6403	6348
q10	6978	2278	2270	2270
q11	470	277	273	273
q12	401	206	210	206
q13	17761	2967	2975	2967
q14	237	205	217	205
q15	501	446	461	446
q16	656	592	581	581
q17	971	588	553	553
q18	7262	6756	6560	6560
q19	1413	1116	1136	1116
q20	495	204	207	204
q21	4001	3036	3129	3036
q22	1086	980	982	980
Total cold run time: 108559 ms
Total hot run time: 39924 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6599	6543	6594	6543
q2	332	236	231	231
q3	3106	2739	2945	2739
q4	2003	1782	1800	1782
q5	5830	5716	5743	5716
q6	207	126	128	126
q7	2227	1825	1801	1801
q8	3381	3528	3486	3486
q9	8844	8878	8903	8878
q10	3547	3480	3521	3480
q11	586	501	489	489
q12	792	591	585	585
q13	8116	3131	3221	3131
q14	299	268	284	268
q15	510	462	466	462
q16	682	652	664	652
q17	1837	1622	1606	1606
q18	8340	7887	7805	7805
q19	1644	1548	1586	1548
q20	2068	1861	1888	1861
q21	5635	5309	5330	5309
q22	1124	1026	1037	1026
Total cold run time: 67709 ms
Total hot run time: 59524 ms

@doris-robot
Copy link

doris-robot commented Apr 9, 2025

TPC-DS: Total hot run time: 198182 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e7ecc8bcfcadb610066031f0116e8bc097d5ea56, data reload: false

query1	1314	918	904	904
query2	6274	2041	2044	2041
query3	10824	4339	4218	4218
query4	60489	28926	23354	23354
query5	5211	472	440	440
query6	390	180	174	174
query7	5431	316	315	315
query8	300	232	229	229
query9	8639	2634	2601	2601
query10	449	275	267	267
query11	17912	15258	15706	15258
query12	163	102	103	102
query13	1457	450	429	429
query14	10935	7793	7811	7793
query15	202	177	186	177
query16	7236	507	506	506
query17	1153	614	634	614
query18	1897	329	329	329
query19	225	167	164	164
query20	124	116	109	109
query21	209	101	106	101
query22	4825	4625	4480	4480
query23	34219	33838	34208	33838
query24	6265	2988	3012	2988
query25	547	429	431	429
query26	667	180	174	174
query27	1839	363	366	363
query28	4271	2486	2435	2435
query29	727	482	475	475
query30	287	167	161	161
query31	1014	804	818	804
query32	66	55	57	55
query33	444	295	324	295
query34	927	507	517	507
query35	898	730	708	708
query36	1079	993	976	976
query37	118	74	73	73
query38	4263	4032	3991	3991
query39	1534	1472	1470	1470
query40	201	102	101	101
query41	49	47	47	47
query42	119	103	105	103
query43	528	502	497	497
query44	1183	830	832	830
query45	184	177	165	165
query46	1160	757	726	726
query47	2001	1932	1955	1932
query48	486	374	407	374
query49	719	401	385	385
query50	861	442	428	428
query51	7486	7272	7178	7178
query52	104	90	91	90
query53	271	194	190	190
query54	579	476	469	469
query55	82	78	80	78
query56	285	253	246	246
query57	1275	1168	1169	1168
query58	228	203	211	203
query59	3245	2947	2947	2947
query60	279	251	249	249
query61	109	108	106	106
query62	775	671	675	671
query63	215	185	188	185
query64	1392	668	632	632
query65	3261	3191	3216	3191
query66	719	296	339	296
query67	16387	15627	15567	15567
query68	4171	555	573	555
query69	443	268	270	268
query70	1120	1118	1133	1118
query71	342	261	248	248
query72	6368	4116	4061	4061
query73	756	354	353	353
query74	10194	9139	9201	9139
query75	3366	2669	2657	2657
query76	2147	1069	1012	1012
query77	494	286	268	268
query78	10727	9694	9568	9568
query79	1154	602	599	599
query80	824	439	431	431
query81	506	240	242	240
query82	1299	90	97	90
query83	162	152	152	152
query84	283	84	88	84
query85	846	309	295	295
query86	339	299	296	296
query87	4432	4403	4267	4267
query88	3536	2401	2397	2397
query89	421	291	291	291
query90	2001	186	191	186
query91	186	148	150	148
query92	64	52	53	52
query93	1147	552	531	531
query94	741	287	296	287
query95	349	255	258	255
query96	618	283	279	279
query97	3359	3207	3169	3169
query98	210	200	213	200
query99	1587	1285	1369	1285
Total cold run time: 313199 ms
Total hot run time: 198182 ms

@doris-robot
Copy link

doris-robot commented Apr 9, 2025

ClickBench: Total hot run time: 33.35 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e7ecc8bcfcadb610066031f0116e8bc097d5ea56, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.09
query5	0.50	0.53	0.53
query6	1.15	0.73	0.74
query7	0.02	0.01	0.01
query8	0.06	0.02	0.03
query9	0.59	0.49	0.50
query10	0.55	0.56	0.55
query11	0.14	0.10	0.11
query12	0.14	0.12	0.11
query13	0.61	0.61	0.60
query14	2.87	2.87	2.75
query15	0.88	0.83	0.83
query16	0.40	0.38	0.37
query17	1.06	1.07	1.07
query18	0.23	0.22	0.22
query19	1.99	1.82	1.98
query20	0.02	0.01	0.01
query21	15.41	0.57	0.57
query22	2.49	2.30	2.52
query23	16.98	1.07	0.91
query24	3.28	1.43	1.48
query25	0.16	0.17	0.15
query26	0.58	0.14	0.14
query27	0.05	0.05	0.04
query28	9.51	0.57	0.48
query29	12.61	3.27	3.32
query30	0.24	0.06	0.06
query31	2.87	0.39	0.39
query32	3.24	0.46	0.47
query33	3.00	3.03	3.06
query34	17.03	4.45	4.45
query35	4.53	4.48	4.52
query36	0.67	0.48	0.47
query37	0.09	0.05	0.06
query38	0.04	0.03	0.04
query39	0.04	0.03	0.02
query40	0.16	0.12	0.13
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.29 s
Total hot run time: 33.35 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Apr 10, 2025

run compile

dataroaring
dataroaring approved these changes Apr 11, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit c75b7e7 into branch-3.0 Apr 11, 2025
22 of 23 checks passed
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 11, 2025
@github-actions
Copy link
Contributor Author

github-actions bot commented Apr 11, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor Author

github-actions bot commented Apr 11, 2025

PR approved by anyone and no changes requested.

@github-actions github-actions bot deleted the auto-pick-49885-branch-3.0 branch April 11, 2025 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Thearas @doris-robot @hello-stephen @dataroaring @wuwenchi