Skip to content

[fix](iceberg)Table operations are not supported for catalogs of the dlf type. #50696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 1 commit into from
May 15, 2025
Merged

[fix](iceberg)Table operations are not supported for catalogs of the dlf type. #50696

morningman merged 1 commit into from
May 15, 2025

Conversation

@wuwenchi
Copy link
Contributor

@wuwenchi wuwenchi commented May 8, 2025

What problem does this PR solve?

Table operations are not supported for catalogs of the dlf type.

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • [] Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented May 8, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@wuwenchi
Copy link
Contributor Author

wuwenchi commented May 8, 2025

run buildall

@doris-robot
Copy link

doris-robot commented May 8, 2025

TPC-H: Total hot run time: 33909 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d3534065dd2f4b85d65d6ec1b131644a644d9687, data reload: false

------ Round 1 ----------------------------------
q1	27747	5030	5074	5030
q2	2077	287	190	190
q3	10381	1233	708	708
q4	10229	1013	518	518
q5	7512	2371	2368	2368
q6	184	173	141	141
q7	920	742	614	614
q8	9326	1286	1124	1124
q9	6907	5292	5156	5156
q10	6875	2333	1892	1892
q11	487	288	270	270
q12	351	355	211	211
q13	17783	3672	3071	3071
q14	236	228	217	217
q15	548	496	493	493
q16	421	428	372	372
q17	595	860	358	358
q18	7662	7116	7016	7016
q19	1633	951	575	575
q20	339	338	231	231
q21	3907	3418	2381	2381
q22	1058	1004	973	973
Total cold run time: 117178 ms
Total hot run time: 33909 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5125	5102	5034	5034
q2	230	321	228	228
q3	2147	2670	2304	2304
q4	1380	1806	1440	1440
q5	4526	4441	4401	4401
q6	219	178	132	132
q7	2029	1927	1787	1787
q8	2611	2668	2543	2543
q9	7282	7047	7110	7047
q10	2992	3209	2764	2764
q11	571	499	476	476
q12	684	755	622	622
q13	3500	3899	3383	3383
q14	263	289	281	281
q15	522	469	457	457
q16	448	479	457	457
q17	1135	1570	1374	1374
q18	7658	7655	7477	7477
q19	832	802	856	802
q20	1917	1991	1847	1847
q21	5091	4878	4691	4691
q22	1106	1054	1005	1005
Total cold run time: 52268 ms
Total hot run time: 50552 ms

@doris-robot
Copy link

doris-robot commented May 8, 2025

TPC-DS: Total hot run time: 192713 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d3534065dd2f4b85d65d6ec1b131644a644d9687, data reload: false

query1	1422	1091	1063	1063
query2	6185	1782	1774	1774
query3	11044	4384	4525	4384
query4	53456	26566	23561	23561
query5	4860	574	453	453
query6	332	208	199	199
query7	4881	505	289	289
query8	320	250	244	244
query9	5312	2545	2566	2545
query10	446	346	264	264
query11	15085	15008	14942	14942
query12	168	108	104	104
query13	1028	517	407	407
query14	10217	6465	6477	6465
query15	214	196	178	178
query16	7139	643	539	539
query17	1100	779	599	599
query18	1605	423	344	344
query19	211	211	171	171
query20	125	167	115	115
query21	201	122	109	109
query22	4375	4438	4342	4342
query23	34193	33594	33647	33594
query24	6554	2462	2445	2445
query25	456	482	400	400
query26	702	268	163	163
query27	2472	504	335	335
query28	3023	2102	2118	2102
query29	579	595	419	419
query30	277	220	191	191
query31	881	876	786	786
query32	72	65	61	61
query33	445	355	319	319
query34	771	859	516	516
query35	798	850	745	745
query36	933	1018	896	896
query37	113	103	89	89
query38	4232	4245	4128	4128
query39	1483	1470	1408	1408
query40	211	120	115	115
query41	57	53	54	53
query42	119	105	107	105
query43	498	506	470	470
query44	1377	839	827	827
query45	188	175	167	167
query46	874	1073	631	631
query47	1841	1871	1787	1787
query48	381	417	319	319
query49	708	548	406	406
query50	669	682	402	402
query51	4216	4238	4175	4175
query52	112	109	99	99
query53	231	272	185	185
query54	579	596	504	504
query55	84	83	83	83
query56	299	299	302	299
query57	1169	1175	1108	1108
query58	267	254	263	254
query59	2674	2775	2659	2659
query60	325	333	302	302
query61	137	131	128	128
query62	762	724	676	676
query63	218	193	186	186
query64	1830	1072	720	720
query65	4472	4252	4273	4252
query66	709	457	312	312
query67	15821	15866	15358	15358
query68	6849	871	504	504
query69	521	302	284	284
query70	1189	1097	1088	1088
query71	497	324	287	287
query72	5779	4799	4786	4786
query73	1457	596	339	339
query74	9434	9178	8733	8733
query75	3931	3186	2693	2693
query76	4226	1188	750	750
query77	701	371	292	292
query78	10061	10091	9142	9142
query79	2495	808	560	560
query80	657	536	446	446
query81	480	250	218	218
query82	446	124	93	93
query83	307	301	226	226
query84	291	109	83	83
query85	781	346	319	319
query86	387	293	298	293
query87	4433	4355	4337	4337
query88	3237	2219	2197	2197
query89	416	310	290	290
query90	1896	214	208	208
query91	156	144	113	113
query92	77	60	56	56
query93	1651	929	582	582
query94	655	400	313	313
query95	376	295	295	295
query96	499	561	273	273
query97	3155	3230	3134	3134
query98	223	211	199	199
query99	1462	1408	1310	1310
Total cold run time: 297263 ms
Total hot run time: 192713 ms

@doris-robot
Copy link

doris-robot commented May 8, 2025

ClickBench: Total hot run time: 29.29 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d3534065dd2f4b85d65d6ec1b131644a644d9687, data reload: false

query1	0.04	0.04	0.03
query2	0.11	0.11	0.11
query3	0.26	0.19	0.19
query4	1.59	0.19	0.19
query5	0.58	0.57	0.58
query6	1.18	0.71	0.73
query7	0.02	0.01	0.02
query8	0.03	0.04	0.03
query9	0.57	0.54	0.51
query10	0.58	0.58	0.56
query11	0.15	0.10	0.11
query12	0.15	0.11	0.12
query13	0.61	0.59	0.59
query14	0.78	0.81	0.80
query15	0.86	0.86	0.85
query16	0.38	0.37	0.39
query17	0.99	1.02	1.01
query18	0.21	0.19	0.20
query19	1.90	1.83	1.79
query20	0.01	0.01	0.01
query21	15.39	0.90	0.55
query22	0.74	1.23	1.03
query23	14.69	1.38	0.60
query24	7.06	1.93	0.69
query25	0.56	0.26	0.07
query26	0.53	0.18	0.13
query27	0.05	0.05	0.05
query28	8.92	0.87	0.44
query29	12.53	3.95	3.28
query30	0.26	0.09	0.07
query31	2.82	0.60	0.38
query32	3.23	0.54	0.46
query33	3.04	2.98	3.04
query34	15.75	5.12	4.55
query35	4.52	4.48	4.54
query36	0.67	0.50	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.14	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 102.23 s
Total hot run time: 29.29 s

morningman
morningman approved these changes May 9, 2025
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman force-pushed the fix-dlf-notsupport branch from d353406 to fafaf62 Compare May 9, 2025 03:29
@morningman
Copy link
Contributor

morningman commented May 9, 2025

run buildall

@doris-robot
Copy link

doris-robot commented May 9, 2025

TPC-H: Total hot run time: 34515 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fafaf62f865ea6358066b35bce6fa6c6d82fb85c, data reload: false

------ Round 1 ----------------------------------
q1	25728	5063	5035	5035
q2	2063	294	184	184
q3	10383	1220	689	689
q4	10219	971	515	515
q5	8012	2382	2318	2318
q6	178	171	135	135
q7	933	737	605	605
q8	9323	1274	1086	1086
q9	6759	5083	5122	5083
q10	6810	2291	1860	1860
q11	482	289	259	259
q12	363	361	218	218
q13	17757	3616	3079	3079
q14	236	222	223	222
q15	529	480	488	480
q16	417	430	371	371
q17	591	862	368	368
q18	7606	7069	7177	7069
q19	1216	942	587	587
q20	357	349	218	218
q21	4046	3341	3149	3149
q22	1050	1018	985	985
Total cold run time: 115058 ms
Total hot run time: 34515 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5154	5145	5179	5145
q2	246	323	227	227
q3	2137	2640	2287	2287
q4	1364	1790	1367	1367
q5	4470	4402	4382	4382
q6	209	171	139	139
q7	2062	1975	1797	1797
q8	2671	2660	2631	2631
q9	7382	7251	7137	7137
q10	3024	3187	2755	2755
q11	579	502	499	499
q12	667	761	627	627
q13	3511	3894	3302	3302
q14	293	317	285	285
q15	532	471	467	467
q16	444	483	433	433
q17	1157	1519	1414	1414
q18	7612	7577	7372	7372
q19	869	879	990	879
q20	1958	2032	1836	1836
q21	5168	4881	4771	4771
q22	1106	1105	1040	1040
Total cold run time: 52615 ms
Total hot run time: 50792 ms

@doris-robot
Copy link

doris-robot commented May 9, 2025

TPC-DS: Total hot run time: 191714 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fafaf62f865ea6358066b35bce6fa6c6d82fb85c, data reload: false

query1	1430	1092	1051	1051
query2	6344	1792	1761	1761
query3	11182	4458	4582	4458
query4	52083	25977	23391	23391
query5	5215	587	433	433
query6	326	196	208	196
query7	4881	530	295	295
query8	318	258	235	235
query9	5555	2571	2559	2559
query10	449	314	260	260
query11	14983	14988	14726	14726
query12	163	113	104	104
query13	1081	512	406	406
query14	10122	6264	6381	6264
query15	215	187	180	180
query16	7105	680	533	533
query17	1088	761	601	601
query18	1579	411	322	322
query19	199	189	208	189
query20	129	127	115	115
query21	208	128	105	105
query22	4286	4290	4322	4290
query23	34255	33414	33257	33257
query24	6695	2397	2370	2370
query25	455	466	403	403
query26	739	278	150	150
query27	2414	507	341	341
query28	2948	2133	2110	2110
query29	597	540	433	433
query30	267	214	194	194
query31	890	866	785	785
query32	70	63	66	63
query33	460	365	304	304
query34	771	850	524	524
query35	785	839	804	804
query36	921	984	909	909
query37	116	105	77	77
query38	4303	4345	4143	4143
query39	1480	1457	1493	1457
query40	205	122	109	109
query41	54	61	54	54
query42	134	114	107	107
query43	521	484	481	481
query44	1344	847	839	839
query45	195	177	167	167
query46	831	1007	628	628
query47	1840	1866	1824	1824
query48	379	410	310	310
query49	665	549	448	448
query50	661	696	429	429
query51	4191	4312	4145	4145
query52	111	107	101	101
query53	242	263	187	187
query54	600	570	512	512
query55	81	87	79	79
query56	313	296	298	296
query57	1146	1198	1127	1127
query58	263	254	263	254
query59	2662	2761	2647	2647
query60	326	314	296	296
query61	128	135	131	131
query62	727	739	658	658
query63	222	184	185	184
query64	1873	1046	696	696
query65	4414	4190	4170	4170
query66	717	414	293	293
query67	15730	15389	15315	15315
query68	7466	870	509	509
query69	538	293	257	257
query70	1163	1088	1132	1088
query71	496	347	283	283
query72	5750	4779	4779	4779
query73	1455	634	340	340
query74	9061	9156	8862	8862
query75	4148	3202	2752	2752
query76	4283	1184	751	751
query77	767	383	290	290
query78	9903	10041	9260	9260
query79	2399	808	551	551
query80	641	502	429	429
query81	480	240	214	214
query82	441	127	95	95
query83	282	243	225	225
query84	288	101	81	81
query85	784	398	322	322
query86	340	309	288	288
query87	4431	4415	4216	4216
query88	2943	2173	2167	2167
query89	397	312	282	282
query90	1982	203	204	203
query91	140	134	111	111
query92	78	55	53	53
query93	1179	925	578	578
query94	671	386	301	301
query95	362	282	290	282
query96	490	545	268	268
query97	3177	3176	3145	3145
query98	221	221	204	204
query99	1447	1382	1243	1243
Total cold run time: 295885 ms
Total hot run time: 191714 ms

@doris-robot
Copy link

doris-robot commented May 9, 2025

ClickBench: Total hot run time: 28.7 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fafaf62f865ea6358066b35bce6fa6c6d82fb85c, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.10	0.11
query3	0.25	0.19	0.18
query4	1.59	0.19	0.19
query5	0.60	0.57	0.58
query6	1.18	0.72	0.72
query7	0.02	0.02	0.02
query8	0.04	0.04	0.04
query9	0.56	0.52	0.50
query10	0.57	0.57	0.56
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.61	0.60	0.60
query14	0.78	0.78	0.78
query15	0.85	0.84	0.85
query16	0.37	0.38	0.39
query17	1.03	1.06	1.04
query18	0.21	0.21	0.20
query19	1.82	1.78	1.73
query20	0.01	0.02	0.01
query21	15.41	0.90	0.56
query22	0.76	1.18	0.64
query23	14.92	1.34	0.64
query24	6.81	1.30	0.45
query25	0.54	0.23	0.06
query26	0.66	0.16	0.13
query27	0.05	0.05	0.04
query28	9.53	0.86	0.42
query29	12.54	3.92	3.34
query30	0.25	0.09	0.07
query31	2.82	0.57	0.37
query32	3.23	0.55	0.45
query33	3.05	2.98	3.05
query34	15.87	5.17	4.51
query35	4.57	4.54	4.55
query36	0.67	0.50	0.48
query37	0.08	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.13 s
Total hot run time: 28.7 s

morningman
morningman approved these changes May 13, 2025
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented May 13, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 13, 2025
@github-actions
Copy link
Contributor

github-actions bot commented May 13, 2025

PR approved by anyone and no changes requested.

@morningman morningman merged commit ae48cbe into apache:master May 15, 2025
26 of 28 checks passed
github-actions bot pushed a commit that referenced this pull request May 15, 2025
@wuwenchi
[fix](iceberg)Table operations are not supported for catalogs of the …
1d18c53 
…dlf type. (#50696)

### What problem does this PR solve?

Table operations are not supported for catalogs of the dlf type.
@github-actions github-actions bot mentioned this pull request May 15, 2025
Closed
github-actions bot pushed a commit that referenced this pull request May 15, 2025
@wuwenchi
[fix](iceberg)Table operations are not supported for catalogs of the …
05363ed 
…dlf type. (#50696)

### What problem does this PR solve?

Table operations are not supported for catalogs of the dlf type.
@github-actions github-actions bot mentioned this pull request May 15, 2025
Closed
@liutang123 liutang123 mentioned this pull request May 15, 2025
Closed
15 tasks
@wuwenchi wuwenchi mentioned this pull request May 15, 2025
Merged
16 tasks
hello-stephen pushed a commit that referenced this pull request May 15, 2025
@wuwenchi
[fix](iceberg)Table operations are not supported for catalogs of the …
345cbd2 
…dlf type for nereids (#50936)

### What problem does this PR solve?

followUp: #50696

Problem Summary:

conflict with #50196

Table operations are not supported for catalogs of the dlf type for
nereids.
wuwenchi added a commit to wuwenchi/doris that referenced this pull request May 15, 2025
@wuwenchi
[fix](iceberg)Table operations are not supported for catalogs of the …
c76d959 
…dlf type. (apache#50696)

### What problem does this PR solve?

Table operations are not supported for catalogs of the dlf type.
This was referenced May 15, 2025
Merged
Merged
yiguolei pushed a commit that referenced this pull request May 23, 2025
@wuwenchi
[fix](iceberg)Table operations are not supported for catalogs of the …
1652101 
…dlf type for 2.1 #50696 (#51112)

bp: #50696
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@wuwenchi @koarz
[fix](iceberg)Table operations are not supported for catalogs of the …
ae97dec 
…dlf type. (apache#50696)

### What problem does this PR solve?

Table operations are not supported for catalogs of the dlf type.
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@wuwenchi @koarz
[fix](iceberg)Table operations are not supported for catalogs of the …
140caef 
…dlf type for nereids (apache#50936)

### What problem does this PR solve?

followUp: apache#50696

Problem Summary:

conflict with apache#50196

Table operations are not supported for catalogs of the dlf type for
nereids.
dataroaring pushed a commit that referenced this pull request Jun 24, 2025
@wuwenchi
[fix](iceberg)Table operations are not supported for catalogs of the …
868cc91 
…dlf type for 3.0 #50696 (#50956)

bp: #50696
github-actions bot pushed a commit that referenced this pull request Jun 24, 2025
@wuwenchi
[fix](iceberg)Table operations are not supported for catalogs of the …
08390e1 
…dlf type. (#50696)

### What problem does this PR solve?

Table operations are not supported for catalogs of the dlf type.
@github-actions github-actions bot mentioned this pull request Jun 24, 2025
Closed
morningman pushed a commit to morningman/doris that referenced this pull request Jul 1, 2025
@wuwenchi @morningman
[fix](iceberg)Table operations are not supported for catalogs of the …
5a90353 
…dlf type. (apache#50696)

### What problem does this PR solve?

Table operations are not supported for catalogs of the dlf type.
morningman pushed a commit to morningman/doris that referenced this pull request Jul 1, 2025
@wuwenchi @morningman
[fix](iceberg)Table operations are not supported for catalogs of the …
2f68bc4 
…dlf type for nereids (apache#50936)

followUp: apache#50696

Problem Summary:

conflict with apache#50196

Table operations are not supported for catalogs of the dlf type for
nereids.
@morningman morningman mentioned this pull request Jul 1, 2025
Merged
morningman pushed a commit to morningman/doris that referenced this pull request Jul 1, 2025
@wuwenchi @morningman
[fix](iceberg)Table operations are not supported for catalogs of the …
2d11e21 
…dlf type for nereids (apache#50936)

followUp: apache#50696

Problem Summary:

conflict with apache#50196

Table operations are not supported for catalogs of the dlf type for
nereids.
morningman pushed a commit to morningman/doris that referenced this pull request Jul 3, 2025
@wuwenchi @morningman
[fix](iceberg)Table operations are not supported for catalogs of the …
9bde7ec 
…dlf type. (apache#50696)

### What problem does this PR solve?

Table operations are not supported for catalogs of the dlf type.
morningman pushed a commit to morningman/doris that referenced this pull request Jul 3, 2025
@wuwenchi @morningman
[fix](iceberg)Table operations are not supported for catalogs of the …
b027049 
…dlf type for nereids (apache#50936)

followUp: apache#50696

Problem Summary:

conflict with apache#50196

Table operations are not supported for catalogs of the dlf type for
nereids.
morningman pushed a commit to morningman/doris that referenced this pull request Jul 3, 2025
@wuwenchi @morningman
[fix](iceberg)Table operations are not supported for catalogs of the …
003f436 
…dlf type. (apache#50696)

Table operations are not supported for catalogs of the dlf type.
morrySnow pushed a commit that referenced this pull request Jul 3, 2025
@morningman @wuwenchi
branch-3.1: [fix](iceberg)Table operations are not supported for cata…
7582dbe 
…logs of the dlf type. #50696 (#52571)

bp #50696

Co-authored-by: wuwenchi <wuwenchi@selectdb.com>
etah000 pushed a commit to etah000/doris that referenced this pull request Jul 7, 2025
@morningman @wuwenchi @etah000
branch-3.1: [fix](iceberg)Table operations are not supported for cata…
64bdad9 
…logs of the dlf type. apache#50696 (apache#52571)

bp apache#50696

Co-authored-by: wuwenchi <wuwenchi@selectdb.com>
@gavinchou gavinchou mentioned this pull request Aug 15, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

+2 more reviewers

@suxiaogang223 suxiaogang223 suxiaogang223 approved these changes

@hubgeter hubgeter hubgeter approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.11-merged dev/3.0.7-merged dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@wuwenchi @hello-stephen @doris-robot @morningman @suxiaogang223 @hubgeter @yiguolei @dataroaring @morrySnow