Skip to content

branch-3.0: [Fix](bug) Percentile* func core when percent args is negative number #47068 #47219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
HappenLee merged 1 commit into from
Feb 7, 2025
Merged

branch-3.0: [Fix](bug) Percentile* func core when percent args is negative number #47068 #47219

HappenLee merged 1 commit into from
Feb 7, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Jan 20, 2025

Cherry-picked from #47068

@HappenLee
[Fix](bug) Percentile* func core when percent args is negative number (…
6e19e99 
…#47068)

### What problem does this PR solve?

Fix Percentile* func core when percent args is negative number

After the pr, can check the percent args is in [0,1], else throw
exception

```
start BE in local mode
*** Query id: cdf31a0f13d04a55-8b84a72628d5a8a0 ***
*** is nereids: 1 ***
*** tablet id: 0 ***
*** Aborted at 1736941916 (unix time) try "date -d @1736941916" if you are using GNU date ***
*** Current BE git commitID: 77031ac ***
*** SIGSEGV address not mapped to object (@0x7fba49150840) received by PID 7647 (TID 10021 OR 0x7fb14659f640) from PID 1226115136; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 3# 0x00007FB26EB4E520 in /lib/x86_64-linux-gnu/libc.so.6
 4# doris::Counts::terminate(double) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/counts.h:207
 5# doris::vectorized::AggregateFunctionPercentileArray::insert_result_into(char const*, doris::vectorized::IColumn&) const at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/aggregate_functions/aggregate_function_percentile.h:528
 6# doris::pipeline::AggLocalState::_get_without_key_result(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/aggregation_source_operator.cpp:385
 7# std::_Function_handler, std::_Placeholder<2>, std::_Placeholder<3>))(doris::RuntimeState*, doris::vectorized::Block*, bool*)> >::_M_invoke(std::_Any_data const&, doris::RuntimeState*&&, doris::vectorized::Block*&&, bool*&&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
 8# doris::pipeline::AggSourceOperatorX::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/aggregation_source_operator.cpp:438
 9# doris::pipeline::OperatorXBase::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) in /mnt/hdd01/ci/doris-deploy-branch-3.0-local/be/lib/doris_be
10# doris::pipeline::PipelineTask::execute(bool*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/pipeline_task.cpp:376
11# doris::pipeline::TaskScheduler::_do_work(unsigned long) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/task_scheduler.cpp:138
12# doris::ThreadPool::dispatch_thread() in /mnt/hdd01/ci/doris-deploy-branch-3.0-local/be/lib/doris_be
13# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:499
14# start_thread at ./nptl/pthread_create.c:442
15# 0x00007FB26EC32850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83
172.20.48.199 last coredump sql: 2025-01-15 19:52:25,410 [query] Query cdf31a0f13d04a55-8b84a72628d5a8a0 1 times with new query id: 171586b2ada34e68-b90ec1fa65013f9f
2025-01-15 19:52:57,126 [query] Query cdf31a0f13d04a55-8b84a72628d5a8a0 2 times with new query id: a44ad498175f4b2d-b3eb8478dc70fa97 
```
@github-actions github-actions bot requested a review from dataroaring as a code owner January 20, 2025 03:26
@Thearas
Copy link
Contributor

Thearas commented Jan 20, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Jan 20, 2025
@Thearas
Copy link
Contributor

Thearas commented Jan 20, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jan 20, 2025

TPC-H: Total hot run time: 41211 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6e19e99c590e0b98e0b34914e64d5b3a4a4bcc13, data reload: false

------ Round 1 ----------------------------------
q1	17652	7495	7277	7277
q2	2675	165	161	161
q3	11163	1156	1143	1143
q4	10559	794	761	761
q5	7772	2941	2931	2931
q6	238	151	160	151
q7	989	624	620	620
q8	9712	1969	2084	1969
q9	6844	6525	6534	6525
q10	7403	2259	2321	2259
q11	476	268	258	258
q12	399	210	213	210
q13	17786	2998	3021	2998
q14	239	211	213	211
q15	576	529	530	529
q16	702	618	596	596
q17	995	560	588	560
q18	7345	6674	6647	6647
q19	1414	1069	1262	1069
q20	469	206	200	200
q21	4033	3203	3167	3167
q22	1109	969	977	969
Total cold run time: 110550 ms
Total hot run time: 41211 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7297	7216	7278	7216
q2	341	245	239	239
q3	3129	3009	3005	3005
q4	2099	1865	1812	1812
q5	5693	5935	5786	5786
q6	224	137	143	137
q7	2284	1820	1786	1786
q8	3341	3510	3526	3510
q9	8896	8887	8865	8865
q10	3584	3559	3565	3559
q11	600	501	502	501
q12	821	606	590	590
q13	6225	3104	3125	3104
q14	315	265	277	265
q15	584	527	517	517
q16	715	667	662	662
q17	1851	1633	1599	1599
q18	8306	7789	7607	7607
q19	1671	1584	1556	1556
q20	2044	1810	1790	1790
q21	5263	5149	5132	5132
q22	1104	1046	986	986
Total cold run time: 66387 ms
Total hot run time: 60224 ms

@doris-robot
Copy link

doris-robot commented Jan 20, 2025

TPC-DS: Total hot run time: 191508 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6e19e99c590e0b98e0b34914e64d5b3a4a4bcc13, data reload: false

query1	989	369	363	363
query2	6553	2172	2122	2122
query3	6708	220	219	219
query4	33934	23503	23554	23503
query5	4318	461	438	438
query6	291	183	179	179
query7	4640	315	316	315
query8	285	233	214	214
query9	9713	2662	2661	2661
query10	485	263	259	259
query11	18150	15329	15218	15218
query12	155	107	101	101
query13	1653	424	413	413
query14	10312	6842	6843	6842
query15	259	175	174	174
query16	8122	462	486	462
query17	1622	584	566	566
query18	2168	309	323	309
query19	288	174	158	158
query20	115	105	107	105
query21	206	102	103	102
query22	4355	4296	4231	4231
query23	34626	33815	33749	33749
query24	11609	2874	2833	2833
query25	683	416	425	416
query26	1777	169	170	169
query27	2929	344	347	344
query28	8099	2455	2433	2433
query29	1048	455	441	441
query30	339	171	179	171
query31	1008	794	821	794
query32	100	58	61	58
query33	802	307	299	299
query34	943	491	532	491
query35	865	723	746	723
query36	1069	963	963	963
query37	151	80	79	79
query38	3981	3950	3830	3830
query39	1463	1423	1469	1423
query40	288	104	105	104
query41	57	52	55	52
query42	116	99	102	99
query43	564	495	517	495
query44	1283	810	803	803
query45	183	169	172	169
query46	1143	710	692	692
query47	1933	1817	1882	1817
query48	468	386	397	386
query49	1257	426	417	417
query50	802	416	407	407
query51	7290	7060	7281	7060
query52	105	90	92	90
query53	258	186	188	186
query54	1312	475	467	467
query55	81	80	79	79
query56	281	283	241	241
query57	1201	1119	1096	1096
query58	238	213	211	211
query59	3374	3195	2974	2974
query60	287	256	270	256
query61	115	121	108	108
query62	859	674	644	644
query63	216	189	188	188
query64	5165	663	651	651
query65	3313	3198	3249	3198
query66	1434	317	319	317
query67	16029	15542	15564	15542
query68	4994	573	568	568
query69	452	279	265	265
query70	1172	1141	1141	1141
query71	459	260	255	255
query72	6391	4040	4063	4040
query73	770	349	363	349
query74	10592	8952	9026	8952
query75	3360	2614	2636	2614
query76	2958	1114	986	986
query77	452	281	272	272
query78	10376	9667	9591	9591
query79	1188	585	579	579
query80	941	448	426	426
query81	552	234	241	234
query82	1169	126	112	112
query83	223	151	141	141
query84	237	88	87	87
query85	1302	318	294	294
query86	371	279	303	279
query87	4356	4299	4343	4299
query88	3668	2414	2366	2366
query89	401	288	298	288
query90	1950	190	184	184
query91	182	149	151	149
query92	62	53	60	53
query93	1078	554	548	548
query94	918	301	291	291
query95	355	263	255	255
query96	607	292	284	284
query97	3351	3222	3201	3201
query98	216	212	192	192
query99	1519	1315	1294	1294
Total cold run time: 303087 ms
Total hot run time: 191508 ms

@doris-robot
Copy link

doris-robot commented Jan 20, 2025

ClickBench: Total hot run time: 33.2 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6e19e99c590e0b98e0b34914e64d5b3a4a4bcc13, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.06	0.06
query4	1.62	0.10	0.10
query5	0.53	0.51	0.52
query6	1.14	0.73	0.72
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.54	0.50	0.49
query10	0.55	0.56	0.56
query11	0.15	0.11	0.10
query12	0.15	0.12	0.12
query13	0.60	0.60	0.60
query14	2.97	2.91	2.94
query15	0.91	0.83	0.83
query16	0.39	0.38	0.38
query17	1.06	1.06	1.00
query18	0.23	0.22	0.22
query19	1.97	1.97	1.89
query20	0.01	0.01	0.01
query21	15.35	0.61	0.58
query22	2.50	2.39	2.66
query23	17.09	0.99	0.85
query24	3.34	0.32	1.39
query25	0.22	0.06	0.16
query26	0.39	0.14	0.14
query27	0.04	0.04	0.04
query28	10.62	1.11	1.06
query29	12.54	3.36	3.38
query30	0.24	0.06	0.07
query31	2.86	0.40	0.39
query32	3.24	0.46	0.46
query33	3.02	3.02	3.05
query34	17.03	4.50	4.59
query35	4.60	4.55	4.54
query36	0.66	0.50	0.50
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.46 s
Total hot run time: 33.2 s

@HappenLee HappenLee merged commit 2c155a4 into branch-3.0 Feb 7, 2025
21 of 22 checks passed
@github-actions github-actions bot deleted the auto-pick-47068-branch-3.0 branch February 7, 2025 04:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring Awaiting requested review from dataroaring

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Thearas @doris-robot @HappenLee @dataroaring