Skip to content

[Improve](variant) optimize cast performance when variant only contains null values #57003

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
eldenmoon merged 1 commit into from
Oct 16, 2025
Merged

[Improve](variant) optimize cast performance when variant only contains null values #57003

eldenmoon merged 1 commit into from
Oct 16, 2025

Conversation

@csun5285
Copy link
Contributor

@csun5285 csun5285 commented Oct 15, 2025 •
edited by eldenmoon
Loading

What problem does this PR solve?

6.39s->0.5s

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 15, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@csun5285
Copy link
Contributor Author

csun5285 commented Oct 15, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Oct 15, 2025

TPC-DS: Total hot run time: 190848 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 32008860b3859cce9a7f30fa625e235b917131b9, data reload: false

query1	1102	420	406	406
query2	6575	1735	1640	1640
query3	6755	238	229	229
query4	26444	23710	23292	23292
query5	5296	645	510	510
query6	333	256	227	227
query7	4656	493	314	314
query8	317	261	258	258
query9	8701	2594	2593	2593
query10	536	349	315	315
query11	15266	15137	14997	14997
query12	190	119	114	114
query13	1711	577	439	439
query14	11681	9246	9255	9246
query15	223	191	185	185
query16	7681	700	555	555
query17	1604	780	617	617
query18	2109	520	373	373
query19	243	219	190	190
query20	146	139	141	139
query21	227	143	132	132
query22	4807	4667	4544	4544
query23	34884	33757	34056	33757
query24	8927	2444	2491	2444
query25	586	570	457	457
query26	1314	280	161	161
query27	2924	538	406	406
query28	4453	2272	2219	2219
query29	786	690	503	503
query30	313	225	207	207
query31	951	864	792	792
query32	92	90	77	77
query33	617	415	370	370
query34	849	876	550	550
query35	880	862	751	751
query36	1019	1070	981	981
query37	134	114	92	92
query38	3688	3709	3560	3560
query39	1456	1412	1591	1412
query40	223	126	118	118
query41	65	59	60	59
query42	133	124	115	115
query43	487	511	481	481
query44	1346	845	845	845
query45	193	185	183	183
query46	834	997	652	652
query47	1749	1759	1725	1725
query48	421	456	323	323
query49	782	509	426	426
query50	648	694	414	414
query51	3913	3934	3919	3919
query52	117	115	107	107
query53	248	273	211	211
query54	625	615	533	533
query55	95	87	90	87
query56	342	332	320	320
query57	1255	1181	1129	1129
query58	293	278	278	278
query59	2507	2684	2481	2481
query60	348	348	320	320
query61	158	163	149	149
query62	795	732	695	695
query63	232	201	193	193
query64	4439	1180	875	875
query65	4102	3988	4071	3988
query66	1072	477	329	329
query67	15650	15206	15056	15056
query68	9133	934	597	597
query69	502	328	286	286
query70	1386	1378	1250	1250
query71	500	339	309	309
query72	5967	5002	4904	4904
query73	718	575	362	362
query74	9455	8977	8951	8951
query75	4389	3343	2906	2906
query76	3765	1157	740	740
query77	854	421	324	324
query78	9724	9751	8934	8934
query79	2606	849	602	602
query80	682	564	514	514
query81	485	266	238	238
query82	484	156	135	135
query83	299	272	254	254
query84	299	116	93	93
query85	893	473	433	433
query86	376	329	312	312
query87	3788	3744	3679	3679
query88	3465	2253	2336	2253
query89	426	328	294	294
query90	2059	219	216	216
query91	165	165	138	138
query92	82	67	61	61
query93	1955	992	638	638
query94	687	440	353	353
query95	396	321	311	311
query96	493	585	287	287
query97	2938	2973	2901	2901
query98	259	217	215	215
query99	1470	1398	1293	1293
Total cold run time: 284248 ms
Total hot run time: 190848 ms

@doris-robot
Copy link

doris-robot commented Oct 15, 2025

ClickBench: Total hot run time: 30.1 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 32008860b3859cce9a7f30fa625e235b917131b9, data reload: false

query1	0.05	0.06	0.05
query2	0.10	0.06	0.07
query3	0.26	0.08	0.08
query4	1.61	0.12	0.12
query5	0.28	0.26	0.25
query6	1.15	0.66	0.65
query7	0.04	0.03	0.02
query8	0.06	0.05	0.05
query9	0.64	0.53	0.52
query10	0.58	0.58	0.58
query11	0.17	0.11	0.12
query12	0.16	0.12	0.13
query13	0.64	0.62	0.61
query14	1.03	1.03	1.02
query15	0.89	0.87	0.86
query16	0.40	0.40	0.38
query17	1.10	1.08	1.02
query18	0.22	0.21	0.20
query19	2.00	1.84	1.87
query20	0.01	0.01	0.02
query21	15.44	0.95	0.56
query22	0.77	1.21	0.65
query23	14.97	1.41	0.65
query24	7.47	1.38	0.51
query25	0.48	0.18	0.08
query26	0.76	0.16	0.15
query27	0.07	0.06	0.06
query28	8.71	1.37	0.92
query29	12.58	3.94	3.27
query30	0.29	0.15	0.11
query31	2.84	0.59	0.40
query32	3.27	0.57	0.49
query33	3.07	3.07	3.12
query34	16.27	5.49	4.84
query35	4.93	4.93	4.90
query36	0.72	0.51	0.51
query37	0.11	0.08	0.08
query38	0.08	0.05	0.04
query39	0.03	0.03	0.03
query40	0.18	0.15	0.14
query41	0.09	0.03	0.04
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 104.6 s
Total hot run time: 30.1 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 15, 2025

BE UT Coverage Report

Increment line coverage 100.00% (8/8) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.57% (17788/33834)
Line Coverage 37.77% (161671/428018)
Region Coverage 32.20% (123239/382683)
Branch Coverage 33.63% (54148/160993)

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 15, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.24% (23626/33166)
Line Coverage 57.66% (246640/427761)
Region Coverage 52.73% (204391/387630)
Branch Coverage 54.64% (88447/161882)

eldenmoon
eldenmoon approved these changes Oct 16, 2025
View reviewed changes
Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eldenmoon eldenmoon changed the title [fix](variant) fix variant cast [Improve](variant) optimize cast performance when variant only contains null values Oct 16, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 16, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Oct 16, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 16, 2025

PR approved by anyone and no changes requested.

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 16, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.24% (23626/33166)
Line Coverage 57.66% (246640/427761)
Region Coverage 52.73% (204391/387630)
Branch Coverage 54.64% (88447/161882)

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 16, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.20% (23615/33166)
Line Coverage 57.62% (246474/427761)
Region Coverage 52.71% (204338/387630)
Branch Coverage 54.60% (88383/161882)

1 similar comment
@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 16, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (8/8) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.20% (23615/33166)
Line Coverage 57.62% (246474/427761)
Region Coverage 52.71% (204338/387630)
Branch Coverage 54.60% (88383/161882)

airborne12
airborne12 approved these changes Oct 16, 2025
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eldenmoon eldenmoon merged commit 02cd275 into apache:master Oct 16, 2025
31 of 33 checks passed
github-actions bot pushed a commit that referenced this pull request Oct 16, 2025
@csun5285
[Improve](variant) optimize cast performance when variant only contai…
964b90e 
…ns null values (#57003)

6.39s->0.5s
@github-actions github-actions bot mentioned this pull request Oct 16, 2025
Merged
yiguolei pushed a commit that referenced this pull request Oct 17, 2025
@github-actions @csun5285
branch-4.0: [Improve](variant) optimize cast performance when variant…
ee45f98 
… only contains null values #57003 (#57054)

Cherry-picked from #57003

Co-authored-by: Sun Chenyang <sunchenyang@selectdb.com>
csun5285 added a commit to csun5285/doris that referenced this pull request Oct 20, 2025
@csun5285
[Improve](variant) optimize cast performance when variant only contai…
f9e85ff 
…ns null values (apache#57003)

6.39s->0.5s
@csun5285 csun5285 mentioned this pull request Oct 20, 2025
Merged
morrySnow pushed a commit that referenced this pull request Oct 28, 2025
@csun5285
branch-3.1: [Improve](variant) optimize cast performance when variant…
e00bb5e 
… only contains null values #57003 (#57161)

pick from master #57003
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@airborne12 airborne12 airborne12 approved these changes

@eldenmoon eldenmoon eldenmoon approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.3-merged dev/4.0.1-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@csun5285 @hello-stephen @doris-robot @airborne12 @eldenmoon @yiguolei @morrySnow