[feature](cmd) add UNSET_VARIABLE statement to set back variables #27552

Yulei-Yang · 2023-11-24T10:14:40Z

Proposed changes

Implement issue Number: close #20643
furthermore, implement global scope.

UNSET [SESSION|GLOBAL] VARIABLE ${variable_name};
UNSET [SESSION|GLOBAL] VARIABLE ALL;

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

Yulei-Yang · 2023-11-24T10:20:04Z

run buildall

Yulei-Yang · 2023-11-24T11:23:36Z

run p0

doris-robot · 2023-11-24T11:39:51Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 034f0c5eeb6af1ffc4e6677b0933aa6a8cbf816b, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4932	4668	4655	4655
q2	358	170	158	158
q3	2044	1937	1927	1927
q4	1401	1255	1228	1228
q5	3972	3993	4064	3993
q6	248	130	130	130
q7	1428	876	884	876
q8	2797	2801	2781	2781
q9	9753	9564	9699	9564
q10	3436	3512	3522	3512
q11	384	253	250	250
q12	434	289	291	289
q13	4565	3856	3794	3794
q14	324	282	285	282
q15	585	520	522	520
q16	671	581	594	581
q17	1134	989	941	941
q18	7881	7500	7481	7481
q19	1665	1680	1651	1651
q20	537	315	291	291
q21	4409	4008	3999	3999
q22	485	379	380	379
Total cold run time: 53443 ms
Total hot run time: 49282 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4595	4585	4567	4567
q2	340	250	269	250
q3	4032	4010	4013	4010
q4	2724	2701	2701	2701
q5	9755	9705	9730	9705
q6	251	123	125	123
q7	3001	2483	2420	2420
q8	4479	4486	4470	4470
q9	12885	12734	12767	12734
q10	4076	4138	4197	4138
q11	824	709	655	655
q12	986	801	818	801
q13	4333	3567	3564	3564
q14	372	347	351	347
q15	573	536	522	522
q16	751	654	691	654
q17	3875	3936	3805	3805
q18	9774	9182	9104	9104
q19	1809	1765	1777	1765
q20	2391	2068	2033	2033
q21	8822	8488	8705	8488
q22	881	822	795	795
Total cold run time: 81529 ms
Total hot run time: 77651 ms

doris-robot · 2023-11-24T11:50:52Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.37 seconds
stream load tsv: 568 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17098797333 Bytes

Yulei-Yang · 2023-11-26T11:11:06Z

run buildall

Yulei-Yang · 2023-11-26T12:30:35Z

run buildall

Yulei-Yang · 2023-11-26T13:30:15Z

run p0

Yulei-Yang · 2023-11-26T13:52:36Z

run pipelinex_p0

doris-robot · 2023-11-26T14:23:13Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit a504ddf9add4c3e7f7041411b889e601a23d2495, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4915	4663	4629	4629
q2	356	165	159	159
q3	2046	1915	1864	1864
q4	1376	1246	1267	1246
q5	3937	3943	4046	3943
q6	249	124	131	124
q7	1468	898	873	873
q8	2774	2804	2787	2787
q9	9581	9535	9348	9348
q10	3457	3528	3522	3522
q11	382	244	244	244
q12	435	290	288	288
q13	4572	3798	3807	3798
q14	317	300	277	277
q15	591	536	527	527
q16	664	584	594	584
q17	1143	991	941	941
q18	7741	7369	7290	7290
q19	1717	1678	1687	1678
q20	550	314	320	314
q21	4399	3929	3976	3929
q22	480	386	379	379
Total cold run time: 53150 ms
Total hot run time: 48744 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4585	4592	4608	4592
q2	350	249	279	249
q3	4025	3994	4004	3994
q4	2699	2678	2680	2678
q5	9689	9651	9685	9651
q6	242	125	125	125
q7	3016	2450	2463	2450
q8	4444	4452	4438	4438
q9	12875	12843	12804	12804
q10	4074	4162	4171	4162
q11	769	640	689	640
q12	973	790	809	790
q13	4293	3582	3549	3549
q14	379	338	352	338
q15	580	535	530	530
q16	736	676	673	673
q17	3902	3895	3875	3875
q18	9330	8893	8946	8893
q19	1846	1784	1785	1784
q20	2425	2090	2046	2046
q21	8836	8639	8516	8516
q22	906	766	799	766
Total cold run time: 80974 ms
Total hot run time: 77543 ms

doris-robot · 2023-11-26T14:34:39Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.25 seconds
stream load tsv: 563 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 30 seconds loaded 2358488459 Bytes, about 74 MB/s
stream load orc: 71 seconds loaded 1101869774 Bytes, about 14 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17100361619 Bytes

nextdreamblue · 2023-11-27T02:14:31Z

LGTM

github-actions · 2023-11-27T02:15:28Z

PR approved by at least one committer and no changes requested.

github-actions · 2023-11-27T02:15:30Z

PR approved by anyone and no changes requested.

yuxuan-luo

LGTM

…ache#27552)

* Problem introduced by apache#27552

…31080) * Problem introduced by #27552

[feature](cmd) add UNSET_VARIABLE statement

034f0c5

update

3459e26

update

a504ddf

nextdreamblue approved these changes Nov 27, 2023

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 27, 2023

github-actions bot added the reviewed label Nov 27, 2023

yuxuan-luo approved these changes Nov 27, 2023

View reviewed changes

lide-reed merged commit 3d7d166 into apache:master Nov 27, 2023

seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 28, 2023

[feature](cmd) add UNSET_VARIABLE statement to set back variables (ap…

b9dfbc3

…ache#27552)

Yulei-Yang mentioned this pull request Dec 5, 2023

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a batch of tablet-ids #27994

Merged

XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023

[feature](cmd) add UNSET_VARIABLE statement to set back variables (ap…

597b07d

…ache#27552)

swjtu-zhanglei added a commit to swjtu-zhanglei/incubator-doris that referenced this pull request Feb 19, 2024

[fix](fe) Fix UnsetVariableStmt write editlog in non master node

5731d82

* Problem introduced by apache#27552

swjtu-zhanglei mentioned this pull request Feb 19, 2024

[fix](fe) Fix UnsetVariableStmt write editlog in non master node #31080

Merged

morningman pushed a commit that referenced this pull request Feb 19, 2024

[fix](fe) Fix UnsetVariableStmt write editlog in non master node (#…

efd9a0b

…31080) * Problem introduced by #27552

yiguolei pushed a commit that referenced this pull request Feb 20, 2024

[fix](fe) Fix UnsetVariableStmt write editlog in non master node (#…

76767f2

…31080) * Problem introduced by #27552

Yulei-Yang deleted the reset_variable branch April 3, 2024 05:24

Yulei-Yang mentioned this pull request May 16, 2024

[feature](cmd) add UNSET_VARIABLE statement to set back variables #34949

Merged

[feature](cmd) add UNSET_VARIABLE statement to set back variables #27552

[feature](cmd) add UNSET_VARIABLE statement to set back variables #27552

Uh oh!

Conversation

Yulei-Yang commented Nov 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

Yulei-Yang commented Nov 24, 2023

Uh oh!

Yulei-Yang commented Nov 24, 2023

Uh oh!

doris-robot commented Nov 24, 2023

Uh oh!

doris-robot commented Nov 24, 2023

Uh oh!

Yulei-Yang commented Nov 26, 2023

Uh oh!

Yulei-Yang commented Nov 26, 2023

Uh oh!

Yulei-Yang commented Nov 26, 2023

Uh oh!

Yulei-Yang commented Nov 26, 2023

Uh oh!

doris-robot commented Nov 26, 2023

Uh oh!

doris-robot commented Nov 26, 2023

Uh oh!

nextdreamblue commented Nov 27, 2023

Uh oh!

github-actions bot commented Nov 27, 2023

Uh oh!

github-actions bot commented Nov 27, 2023

Uh oh!

yuxuan-luo left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Yulei-Yang commented Nov 24, 2023 •

edited

Loading