branch-3.0: [improve](functon) improve json_object with complex type #47627 #47779

github-actions · 2025-02-11T13:10:45Z

Cherry-picked from #47627

This PR enhances the value parameter of the json_object function as a complex type which refer to postgresql https://www.postgresql.org/docs/current/functions-json.html ``` json_object ( [ { key_expression { VALUE | ':' } value_expression [ FORMAT JSON [ ENCODING UTF8 ] ] }[, ...] ] [ { NULL | ABSENT } ON NULL ] [ { WITH | WITHOUT } UNIQUE [ KEYS ] ] [ RETURNING data_type [ FORMAT JSON [ ENCODING UTF8 ] ] ]) Constructs a JSON object of all the key/value pairs given, or an empty object if none are given. key_expression is a scalar expression defining the JSON key, which is converted to the text type. It cannot be NULL nor can it belong to a type that has a cast to the json type. If WITH UNIQUE KEYS is specified, there must not be any duplicate key_expression. Any pair for which the value_expression evaluates to NULL is omitted from the output if ABSENT ON NULL is specified; if NULL ON NULL is specified or the clause omitted, the key is included with value NULL. ```

Thearas · 2025-02-11T13:10:51Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Thearas · 2025-02-11T13:10:55Z

run buildall

doris-robot · 2025-02-11T13:50:25Z

TPC-H: Total hot run time: 40608 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ef3726c713b3408d26c03722156242eef7332964, data reload: false

------ Round 1 ----------------------------------
q1	17582	7342	7197	7197
q2	2059	172	164	164
q3	10571	1057	1137	1057
q4	10580	725	723	723
q5	7726	2858	2825	2825
q6	235	145	143	143
q7	976	612	596	596
q8	9351	1900	2004	1900
q9	6580	6415	6370	6370
q10	7019	2325	2307	2307
q11	464	265	263	263
q12	401	212	201	201
q13	17795	2985	3038	2985
q14	254	221	222	221
q15	573	525	522	522
q16	661	597	567	567
q17	978	630	625	625
q18	7168	6643	6614	6614
q19	1390	998	1034	998
q20	460	210	196	196
q21	3945	3210	3148	3148
q22	1076	1006	986	986
Total cold run time: 107844 ms
Total hot run time: 40608 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7149	7204	7141	7141
q2	333	232	228	228
q3	2888	2879	2875	2875
q4	2040	1786	1820	1786
q5	5711	5713	5821	5713
q6	227	138	142	138
q7	2214	1839	1846	1839
q8	3355	3543	3518	3518
q9	8779	8805	8892	8805
q10	3621	3540	3588	3540
q11	603	499	510	499
q12	814	606	583	583
q13	10846	3160	3175	3160
q14	300	265	280	265
q15	595	538	508	508
q16	703	661	648	648
q17	1846	1638	1579	1579
q18	8073	7763	7739	7739
q19	1656	1440	1584	1440
q20	2043	1829	1860	1829
q21	5430	5427	5384	5384
q22	1099	1052	1016	1016
Total cold run time: 70325 ms
Total hot run time: 60233 ms

doris-robot · 2025-02-11T14:03:02Z

TPC-DS: Total hot run time: 197155 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ef3726c713b3408d26c03722156242eef7332964, data reload: false

query1	1285	900	918	900
query2	6235	2133	2074	2074
query3	10858	4318	4085	4085
query4	61830	29415	23711	23711
query5	5235	456	471	456
query6	403	177	168	168
query7	5468	310	322	310
query8	297	218	219	218
query9	8372	2686	2676	2676
query10	468	272	259	259
query11	17615	15341	15779	15341
query12	150	103	106	103
query13	1442	445	439	439
query14	10479	6572	7392	6572
query15	193	182	181	181
query16	7094	517	472	472
query17	1188	583	593	583
query18	1820	343	315	315
query19	210	164	157	157
query20	118	122	117	117
query21	209	103	109	103
query22	4708	4350	4497	4350
query23	34271	34006	34369	34006
query24	6153	2891	2964	2891
query25	534	421	426	421
query26	655	167	171	167
query27	1753	365	362	362
query28	4017	2512	2432	2432
query29	732	470	468	468
query30	233	164	163	163
query31	1004	837	855	837
query32	72	54	55	54
query33	413	285	277	277
query34	939	504	524	504
query35	853	740	716	716
query36	1095	967	954	954
query37	121	73	72	72
query38	4031	3939	3990	3939
query39	1506	1466	1484	1466
query40	209	100	97	97
query41	51	50	52	50
query42	109	98	99	98
query43	552	501	511	501
query44	1223	839	820	820
query45	184	167	165	165
query46	1146	739	722	722
query47	2073	1959	1978	1959
query48	471	394	379	379
query49	721	393	378	378
query50	825	422	428	422
query51	7243	7271	7190	7190
query52	101	93	90	90
query53	279	184	186	184
query54	572	463	447	447
query55	80	79	74	74
query56	261	241	248	241
query57	1338	1180	1155	1155
query58	215	205	223	205
query59	3422	2977	2876	2876
query60	276	244	242	242
query61	117	107	109	107
query62	838	741	731	731
query63	226	194	189	189
query64	1366	689	662	662
query65	3275	3174	3194	3174
query66	705	321	305	305
query67	16144	15708	15849	15708
query68	4017	585	574	574
query69	448	267	256	256
query70	1194	1146	1120	1120
query71	354	273	256	256
query72	6350	4027	4006	4006
query73	756	349	362	349
query74	10421	9020	8959	8959
query75	3368	2635	2612	2612
query76	1975	1046	1065	1046
query77	483	270	264	264
query78	10619	9616	9529	9529
query79	2055	593	594	593
query80	1357	438	423	423
query81	518	241	233	233
query82	1265	118	118	118
query83	164	146	139	139
query84	282	73	77	73
query85	996	309	283	283
query86	384	298	298	298
query87	4602	4334	4350	4334
query88	3973	2369	2363	2363
query89	420	286	293	286
query90	1988	182	181	181
query91	181	151	145	145
query92	64	51	58	51
query93	3087	552	551	551
query94	778	292	292	292
query95	351	253	249	249
query96	627	283	272	272
query97	3325	3207	3204	3204
query98	211	201	213	201
query99	1738	1424	1405	1405
Total cold run time: 316432 ms
Total hot run time: 197155 ms

doris-robot · 2025-02-11T14:08:32Z

ClickBench: Total hot run time: 33.33 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ef3726c713b3408d26c03722156242eef7332964, data reload: false

query1	0.03	0.04	0.02
query2	0.08	0.03	0.03
query3	0.23	0.06	0.06
query4	1.63	0.10	0.10
query5	0.51	0.52	0.51
query6	1.12	0.73	0.74
query7	0.02	0.01	0.02
query8	0.04	0.03	0.04
query9	0.55	0.50	0.52
query10	0.56	0.56	0.55
query11	0.13	0.10	0.10
query12	0.13	0.11	0.11
query13	0.60	0.59	0.61
query14	2.76	2.75	2.83
query15	0.89	0.82	0.82
query16	0.38	0.39	0.41
query17	0.98	1.00	1.01
query18	0.24	0.21	0.22
query19	1.93	1.84	2.04
query20	0.01	0.01	0.01
query21	15.36	0.58	0.59
query22	2.97	2.03	2.08
query23	16.94	1.03	0.87
query24	2.82	1.28	2.56
query25	0.34	0.15	0.14
query26	0.43	0.13	0.13
query27	0.04	0.04	0.04
query28	9.15	1.10	1.06
query29	12.59	3.22	3.24
query30	0.24	0.06	0.05
query31	2.87	0.39	0.38
query32	3.26	0.46	0.45
query33	3.01	3.01	3.01
query34	17.11	4.50	4.47
query35	4.54	4.48	4.54
query36	0.65	0.47	0.48
query37	0.10	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.03	0.03
query40	0.15	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 105.62 s
Total hot run time: 33.33 s

shuke987 · 2025-02-12T06:29:25Z

run buildall

shuke987 · 2025-02-12T06:29:42Z

run buildall

doris-robot · 2025-02-12T07:17:49Z

TPC-H: Total hot run time: 40538 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ef3726c713b3408d26c03722156242eef7332964, data reload: false

------ Round 1 ----------------------------------
q1	17578	7333	7191	7191
q2	2073	176	177	176
q3	10534	1074	1141	1074
q4	10534	744	708	708
q5	7739	2840	2861	2840
q6	241	147	142	142
q7	967	618	622	618
q8	9377	1958	2032	1958
q9	6577	6370	6382	6370
q10	7043	2324	2334	2324
q11	469	260	274	260
q12	392	216	205	205
q13	17771	2998	3038	2998
q14	244	210	211	210
q15	566	518	512	512
q16	695	587	596	587
q17	961	500	576	500
q18	7262	6684	6632	6632
q19	1431	1074	1038	1038
q20	471	198	190	190
q21	4049	3202	3028	3028
q22	1105	1013	977	977
Total cold run time: 108079 ms
Total hot run time: 40538 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7179	7235	7179	7179
q2	326	234	224	224
q3	2906	2880	2886	2880
q4	2063	1847	1838	1838
q5	5703	5730	5690	5690
q6	225	139	139	139
q7	2196	1780	1785	1780
q8	3389	3493	3537	3493
q9	8865	8832	8869	8832
q10	3605	3560	3559	3559
q11	597	503	502	502
q12	790	571	584	571
q13	10675	3237	3162	3162
q14	302	269	289	269
q15	582	531	530	530
q16	680	668	646	646
q17	1817	1590	1607	1590
q18	8171	7698	7785	7698
q19	1653	1535	1573	1535
q20	2108	1867	1885	1867
q21	5479	5371	5378	5371
q22	1139	1052	1067	1052
Total cold run time: 70450 ms
Total hot run time: 60407 ms

doris-robot · 2025-02-12T07:30:28Z

TPC-DS: Total hot run time: 199443 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ef3726c713b3408d26c03722156242eef7332964, data reload: false

query1	1268	889	942	889
query2	6253	2087	2135	2087
query3	10931	4409	4464	4409
query4	62508	28955	23697	23697
query5	5089	476	461	461
query6	397	182	180	180
query7	5453	309	313	309
query8	313	230	219	219
query9	8262	2702	2679	2679
query10	443	267	268	267
query11	17395	15217	15603	15217
query12	170	107	106	106
query13	1407	455	420	420
query14	9980	7747	7525	7525
query15	195	178	177	177
query16	7156	521	442	442
query17	1170	599	613	599
query18	1877	353	321	321
query19	212	164	158	158
query20	121	113	124	113
query21	215	104	107	104
query22	4742	4583	4827	4583
query23	34792	34405	34115	34115
query24	6172	3004	2981	2981
query25	570	430	433	430
query26	659	180	173	173
query27	1801	351	369	351
query28	4037	2494	2455	2455
query29	686	446	436	436
query30	246	172	162	162
query31	1000	848	810	810
query32	66	53	56	53
query33	422	283	279	279
query34	903	519	530	519
query35	854	738	732	732
query36	1096	961	995	961
query37	124	76	73	73
query38	4189	3983	3977	3977
query39	1495	1460	1486	1460
query40	200	100	100	100
query41	50	48	51	48
query42	110	102	103	102
query43	563	512	521	512
query44	1151	829	843	829
query45	187	173	172	172
query46	1150	727	736	727
query47	2028	1941	1979	1941
query48	478	382	402	382
query49	710	403	400	400
query50	844	431	425	425
query51	7407	7250	7286	7250
query52	100	94	92	92
query53	262	191	203	191
query54	583	467	479	467
query55	82	79	83	79
query56	274	267	258	258
query57	1311	1198	1128	1128
query58	214	215	218	215
query59	3368	3152	3121	3121
query60	277	255	253	253
query61	109	134	124	124
query62	825	767	732	732
query63	216	187	191	187
query64	1418	692	687	687
query65	3245	3189	3233	3189
query66	695	310	316	310
query67	16032	15560	15774	15560
query68	4127	580	568	568
query69	431	264	271	264
query70	1130	1127	1105	1105
query71	360	286	258	258
query72	6376	4101	4116	4101
query73	761	351	360	351
query74	10510	9040	9005	9005
query75	3391	2628	2673	2628
query76	1988	1104	1119	1104
query77	484	273	289	273
query78	10505	9646	9671	9646
query79	1548	599	599	599
query80	896	431	419	419
query81	511	235	240	235
query82	1287	115	111	111
query83	254	149	148	148
query84	289	81	80	80
query85	904	301	299	299
query86	341	274	311	274
query87	4459	4340	4357	4340
query88	3793	2400	2397	2397
query89	420	296	298	296
query90	2041	193	204	193
query91	186	152	149	149
query92	67	48	52	48
query93	1969	544	550	544
query94	758	299	301	299
query95	352	267	259	259
query96	620	279	283	279
query97	3390	3122	3189	3122
query98	214	198	208	198
query99	1695	1459	1441	1441
Total cold run time: 314615 ms
Total hot run time: 199443 ms

doris-robot · 2025-02-12T07:35:57Z

ClickBench: Total hot run time: 32.8 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ef3726c713b3408d26c03722156242eef7332964, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.24	0.07	0.07
query4	1.63	0.11	0.11
query5	0.54	0.50	0.50
query6	1.13	0.72	0.73
query7	0.02	0.01	0.02
query8	0.04	0.04	0.03
query9	0.55	0.51	0.50
query10	0.56	0.56	0.54
query11	0.14	0.10	0.10
query12	0.15	0.11	0.10
query13	0.60	0.60	0.60
query14	2.81	2.83	2.73
query15	0.89	0.84	0.82
query16	0.38	0.38	0.37
query17	0.98	1.03	0.96
query18	0.22	0.21	0.22
query19	1.98	1.89	2.00
query20	0.02	0.01	0.01
query21	15.36	0.58	0.58
query22	2.55	2.10	2.02
query23	16.97	1.08	0.76
query24	3.19	0.88	0.99
query25	0.29	0.07	0.19
query26	0.38	0.13	0.14
query27	0.05	0.04	0.04
query28	10.95	1.09	1.06
query29	12.60	3.20	3.20
query30	0.24	0.06	0.06
query31	2.86	0.38	0.39
query32	3.26	0.46	0.45
query33	3.01	3.03	3.00
query34	16.96	4.53	4.52
query35	4.64	4.53	4.54
query36	0.69	0.49	0.52
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.04	0.03	0.02
query40	0.17	0.12	0.12
query41	0.07	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.44 s
Total hot run time: 32.8 s

dataroaring

LGTM

github-actions bot requested a review from dataroaring as a code owner February 11, 2025 13:10

dataroaring closed this Feb 11, 2025

dataroaring reopened this Feb 11, 2025

dataroaring approved these changes Feb 21, 2025

View reviewed changes

dataroaring merged commit 9ded8ff into branch-3.0 Feb 21, 2025
22 of 24 checks passed

github-actions bot deleted the auto-pick-47627-branch-3.0 branch February 21, 2025 07:23

gavinchou mentioned this pull request Apr 23, 2025

Release Notes 3.0.5 #50333

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

branch-3.0: [improve](functon) improve json_object with complex type #47627 #47779

branch-3.0: [improve](functon) improve json_object with complex type #47627 #47779

Uh oh!

github-actions bot commented Feb 11, 2025

Uh oh!

Thearas commented Feb 11, 2025

Uh oh!

Thearas commented Feb 11, 2025

Uh oh!

doris-robot commented Feb 11, 2025

Uh oh!

doris-robot commented Feb 11, 2025

Uh oh!

doris-robot commented Feb 11, 2025

Uh oh!

shuke987 commented Feb 12, 2025

Uh oh!

shuke987 commented Feb 12, 2025

Uh oh!

doris-robot commented Feb 12, 2025

Uh oh!

doris-robot commented Feb 12, 2025

Uh oh!

doris-robot commented Feb 12, 2025

Uh oh!

dataroaring left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

branch-3.0: [improve](functon) improve json_object with complex type #47627 #47779

branch-3.0: [improve](functon) improve json_object with complex type #47627 #47779

Uh oh!

Conversation

github-actions bot commented Feb 11, 2025

Uh oh!

Thearas commented Feb 11, 2025

Uh oh!

Thearas commented Feb 11, 2025

Uh oh!

doris-robot commented Feb 11, 2025

Uh oh!

doris-robot commented Feb 11, 2025

Uh oh!

doris-robot commented Feb 11, 2025

Uh oh!

shuke987 commented Feb 12, 2025

Uh oh!

shuke987 commented Feb 12, 2025

Uh oh!

doris-robot commented Feb 12, 2025

Uh oh!

doris-robot commented Feb 12, 2025

Uh oh!

doris-robot commented Feb 12, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants