[fix](array/map) Fix BE crash in lambda functions #49140

Gabriel39 · 2025-03-17T07:12:40Z

What problem does this PR solve?

Nullable is not processed correctly.

*** Query id: 8e1a9a13cd314e6d-8770b871278e3646 ***
*** is nereids: 1 ***
*** tablet id: 0 ***
*** Aborted at 1741754420 (unix time) try "date -d @1741754420" if you are using GNU date ***
*** Current BE git commitID: 52396fb ***
*** SIGSEGV address not mapped to object (@0xffff0101) received by PID 2499400 (TID 2502138 OR 0x7fdf5cbe3640) from PID 18446744073709486337; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_release/doris/be/src/common/signal_handler.h:421
1# os::Linux::chained_handler(int, siginfo*, void*) in /opt/openjdk-8u352-b08/jdk8u352-b08/jre/lib/amd64/server/libjvm.so
2# JVM_handle_linux_signal in /opt/openjdk-8u352-b08/jdk8u352-b08/jre/lib/amd64/server/libjvm.so
3# signalHandler(int, siginfo*, void*) in /opt/openjdk-8u352-b08/jdk8u352-b08/jre/lib/amd64/server/libjvm.so
4# 0x00007FEBC11FA090 in /lib64/libc.so.6
5# doris::vectorized::is_column_nullable(doris::vectorized::IColumn const&) at /home/zcp/repo_center/doris_release/doris/be/src/vec/columns/column.cpp:83
6# doris::vectorized::FunctionArrayElement::_execute_nullable(std::vector<doris::vectorized::ColumnWithTypeAndName, std::allocatordoris::vectorized::ColumnWithTypeAndName > const&, unsigned long, unsigned char const*, unsigned char*) const in /opt/apache-doris-2.1.8-bin-x64/be/lib/doris_be
7# doris::vectorized::FunctionArrayElement::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator > const&, unsigned long, unsigned long) const in /opt/apache-doris-2.1.8-bin-x64/be/lib/doris_be
8# doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator > const&, unsigned long, unsigned long) const at /home/zcp/repo_center/doris_release/doris/be/src/vec/functions/function.h:462
9# doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator > const&, unsigned long, unsigned long, bool) const at /home/zcp/repo_center/doris_release/doris/be/src/vec/functions/function.cpp:244
10# doris::vectorized::PreparedFunctionImpl::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator > const&, unsigned long, unsigned long, bool) const at /home/zcp/repo_center/doris_release/doris/be/src/vec/functions/function.cpp:250
11# doris::vectorized::IFunctionBase::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator > const&, unsigned long, unsigned long, bool) const at /home/zcp/repo_center/doris_release/doris/be/src/vec/functions/function.h:195
12# doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector<unsigned long, std::allocator >&) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exprs/vectorized_fn_call.cpp:182
13# doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exprs/vectorized_fn_call.cpp:198
14# doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector<unsigned long, std::allocator >&) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exprs/vectorized_fn_call.cpp:173
15# doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exprs/vectorized_fn_call.cpp:198
16# doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector<unsigned long, std::allocator >&) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exprs/vectorized_fn_call.cpp:173
17# doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exprs/vectorized_fn_call.cpp:198
18# doris::vectorized::VCaseExpr::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exprs/vcase_expr.cpp:111
19# doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exprs/vexpr_context.cpp:54
20# doris::vectorized::VScanner::_do_projections(doris::vectorized::Block*, doris::vectorized::Block*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:210
21# doris::vectorized::VScanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:100
22# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:280
23# std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
24# doris::ThreadPool::dispatch_thread() in /opt/apache-doris-2.1.8-bin-x64/be/lib/doris_be
25# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_release/doris/be/src/util/thread.cpp:499
26# start_thread in /lib64/libpthread.so.0
27# __clone in /lib64/libc.so.6

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-03-17T07:12:45Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Gabriel39 · 2025-03-17T07:15:27Z

run buildall

regression-test/conf/regression-conf.groovy

Gabriel39 · 2025-03-17T08:00:53Z

run buildall

be/src/vec/exprs/lambda_function/varray_map_function.cpp

Gabriel39 · 2025-03-17T08:19:34Z

run buildall

yiguolei

LGTM

github-actions · 2025-03-17T08:32:09Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-03-17T08:32:11Z

PR approved by anyone and no changes requested.

doris-robot · 2025-03-17T09:02:38Z

TPC-H: Total hot run time: 32553 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 101965913ccd90bbf80edf33a557476bbc8bac79, data reload: false

------ Round 1 ----------------------------------
q1	24318	5739	5083	5083
q2	2062	320	185	185
q3	10338	1272	709	709
q4	10219	990	540	540
q5	7517	2362	2321	2321
q6	189	164	137	137
q7	925	730	634	634
q8	9299	1299	1128	1128
q9	4985	4853	4717	4717
q10	6809	2305	1905	1905
q11	480	278	263	263
q12	373	372	218	218
q13	17767	3719	3094	3094
q14	244	229	211	211
q15	532	472	485	472
q16	640	615	586	586
q17	598	875	345	345
q18	6707	6491	6336	6336
q19	1207	970	561	561
q20	333	351	203	203
q21	2939	2184	1936	1936
q22	1022	1034	969	969
Total cold run time: 109503 ms
Total hot run time: 32553 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5254	5217	5200	5200
q2	241	329	238	238
q3	2160	2649	2291	2291
q4	1429	1798	1383	1383
q5	4271	4115	4394	4115
q6	221	171	131	131
q7	1982	1909	1806	1806
q8	2653	2617	2567	2567
q9	7131	7179	7132	7132
q10	3002	3171	2666	2666
q11	566	501	514	501
q12	719	798	629	629
q13	3606	3903	3296	3296
q14	285	308	281	281
q15	527	477	497	477
q16	664	679	660	660
q17	1169	1604	1371	1371
q18	7725	7570	7309	7309
q19	904	888	1000	888
q20	1942	2064	1849	1849
q21	5623	4745	4808	4745
q22	1112	1065	1031	1031
Total cold run time: 53186 ms
Total hot run time: 50566 ms

doris-robot · 2025-03-17T09:14:30Z

TPC-DS: Total hot run time: 191841 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 101965913ccd90bbf80edf33a557476bbc8bac79, data reload: false

query1	1393	1081	1055	1055
query2	6256	1942	1937	1937
query3	11032	4416	4637	4416
query4	26066	23844	22992	22992
query5	4483	687	477	477
query6	295	187	192	187
query7	3978	486	288	288
query8	290	233	214	214
query9	8500	2484	2505	2484
query10	471	304	245	245
query11	15512	15225	14840	14840
query12	158	107	106	106
query13	1556	535	373	373
query14	9204	6494	6392	6392
query15	217	211	161	161
query16	7242	612	466	466
query17	1181	722	576	576
query18	1946	410	318	318
query19	194	196	157	157
query20	131	126	121	121
query21	203	123	106	106
query22	4484	4653	4327	4327
query23	34165	33878	33622	33622
query24	7782	2485	2420	2420
query25	556	515	435	435
query26	1313	269	162	162
query27	2890	482	338	338
query28	4348	2405	2399	2399
query29	748	605	472	472
query30	282	225	200	200
query31	945	865	829	829
query32	72	71	65	65
query33	552	394	316	316
query34	826	879	514	514
query35	825	868	757	757
query36	975	1024	912	912
query37	121	106	77	77
query38	4176	4204	4152	4152
query39	1549	1499	1466	1466
query40	223	115	101	101
query41	52	50	50	50
query42	123	105	103	103
query43	528	524	508	508
query44	1336	801	799	799
query45	192	175	171	171
query46	899	1023	655	655
query47	1799	1854	1770	1770
query48	395	423	303	303
query49	765	508	451	451
query50	695	749	434	434
query51	4345	4328	4311	4311
query52	110	104	103	103
query53	233	268	202	202
query54	492	508	421	421
query55	84	81	84	81
query56	282	265	279	265
query57	1229	1159	1111	1111
query58	254	238	236	236
query59	2860	2870	2699	2699
query60	289	278	262	262
query61	123	123	119	119
query62	776	732	683	683
query63	235	192	197	192
query64	4165	1055	722	722
query65	4508	4468	4423	4423
query66	1133	400	292	292
query67	16320	15625	15361	15361
query68	8961	885	511	511
query69	493	305	280	280
query70	1249	1093	1139	1093
query71	469	296	276	276
query72	5374	3605	3749	3605
query73	772	717	346	346
query74	8919	9088	8896	8896
query75	3861	3156	2646	2646
query76	3665	1185	763	763
query77	783	396	272	272
query78	10079	10275	9216	9216
query79	1557	832	591	591
query80	673	541	439	439
query81	493	255	221	221
query82	219	127	93	93
query83	213	177	154	154
query84	278	84	72	72
query85	758	356	320	320
query86	333	305	279	279
query87	4478	4575	4463	4463
query88	2924	2258	2226	2226
query89	402	311	285	285
query90	2022	217	216	216
query91	138	139	116	116
query92	73	58	57	57
query93	1207	1041	580	580
query94	662	399	300	300
query95	352	268	282	268
query96	491	567	275	275
query97	3309	3399	3295	3295
query98	216	210	202	202
query99	1426	1388	1283	1283
Total cold run time: 278211 ms
Total hot run time: 191841 ms

doris-robot · 2025-03-17T09:19:48Z

ClickBench: Total hot run time: 31.76 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 101965913ccd90bbf80edf33a557476bbc8bac79, data reload: false

query1	0.04	0.04	0.04
query2	0.12	0.10	0.10
query3	0.25	0.19	0.19
query4	1.59	0.19	0.19
query5	0.59	0.60	0.58
query6	1.18	0.72	0.71
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.58	0.50	0.52
query10	0.57	0.58	0.56
query11	0.16	0.11	0.12
query12	0.14	0.12	0.12
query13	0.62	0.60	0.60
query14	2.66	2.73	2.73
query15	0.92	0.85	0.86
query16	0.38	0.37	0.38
query17	0.99	1.03	1.05
query18	0.21	0.20	0.19
query19	1.91	1.90	1.80
query20	0.01	0.01	0.01
query21	15.36	0.92	0.53
query22	0.77	1.21	0.66
query23	14.87	1.39	0.62
query24	6.84	1.46	1.43
query25	0.56	0.37	0.07
query26	0.54	0.16	0.14
query27	0.05	0.05	0.05
query28	9.70	0.82	0.43
query29	12.56	4.03	3.36
query30	0.25	0.09	0.06
query31	2.82	0.58	0.38
query32	3.22	0.54	0.46
query33	3.07	3.03	3.06
query34	15.81	5.15	4.52
query35	4.54	4.49	4.56
query36	0.66	0.50	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.12
query41	0.07	0.03	0.02
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.06 s
Total hot run time: 31.76 s

doris-robot · 2025-03-17T10:47:43Z

BE UT Coverage Report

Increment line coverage 44.44% (4/9) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	48.47% (12982/26782)
Line Coverage	37.93% (111296/293448)
Region Coverage	36.89% (56815/154011)
Branch Coverage	32.05% (28577/89156)

…49168) Cherry-picked from #49140 Co-authored-by: Gabriel <liwenqiang@selectdb.com>

…che#3832)

[fix](array/map) Fix BE crash in lambda functions

7c3c61a

Gabriel39 mentioned this pull request Mar 17, 2025

[fix](array/map) Fix BE crash in lambda functions #49139

Merged

16 tasks

Gabriel39 added p0_c dev/2.1.x dev/3.0.x labels Mar 17, 2025

yiguolei added the usercase Important user case type label label Mar 17, 2025

yiguolei reviewed Mar 17, 2025

View reviewed changes

regression-test/conf/regression-conf.groovy Outdated Show resolved Hide resolved

udpate

f45abe8

yiguolei reviewed Mar 17, 2025

View reviewed changes

be/src/vec/exprs/lambda_function/varray_map_function.cpp Show resolved Hide resolved

udpate

1019659

yiguolei approved these changes Mar 17, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 17, 2025

github-actions bot added the reviewed label Mar 17, 2025

BiteTheDDDDt approved these changes Mar 17, 2025

View reviewed changes

yiguolei merged commit 171053f into apache:master Mar 17, 2025
23 of 26 checks passed

github-actions bot pushed a commit that referenced this pull request Mar 17, 2025

[fix](array/map) Fix BE crash in lambda functions (#49140)

e0e3b26

github-actions bot mentioned this pull request Mar 17, 2025

branch-3.0: [fix](array/map) Fix BE crash in lambda functions #49140 #49168

Merged

github-actions bot pushed a commit that referenced this pull request Mar 17, 2025

[fix](array/map) Fix BE crash in lambda functions (#49140)

f8fd471

github-actions bot mentioned this pull request Mar 17, 2025

branch-2.1: [fix](array/map) Fix BE crash in lambda functions #49140 #49169

Closed

yiguolei mentioned this pull request Mar 18, 2025

[Bug] Simple query leads to BE crash with high probability #48951

Closed

3 tasks

yiguolei added dev/2.1.9-merged and removed dev/2.1.x labels Mar 18, 2025

dataroaring pushed a commit that referenced this pull request Mar 19, 2025

branch-3.0: [fix](array/map) Fix BE crash in lambda functions #49140 (#…

0a599fa

…49168) Cherry-picked from #49140 Co-authored-by: Gabriel <liwenqiang@selectdb.com>

dataroaring added dev/3.0.5-merged and removed dev/3.0.x labels Mar 19, 2025

gavinchou mentioned this pull request Apr 23, 2025

Release Notes 3.0.5 #50333

Closed

starocean999 pushed a commit to starocean999/incubator-doris that referenced this pull request Apr 27, 2025

[fix](array/map) Fix BE crash in lambda functions (apache#49140) (apa…

5c0ff38

…che#3832)

koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025

[fix](array/map) Fix BE crash in lambda functions (apache#49140)

0b612d8

[fix](array/map) Fix BE crash in lambda functions #49140

[fix](array/map) Fix BE crash in lambda functions #49140

Uh oh!

Conversation

Gabriel39 commented Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Mar 17, 2025

Uh oh!

Gabriel39 commented Mar 17, 2025

Uh oh!

Uh oh!

Gabriel39 commented Mar 17, 2025

Uh oh!

Uh oh!

Gabriel39 commented Mar 17, 2025

Uh oh!

yiguolei left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 17, 2025

Uh oh!

github-actions bot commented Mar 17, 2025

Uh oh!

doris-robot commented Mar 17, 2025

Uh oh!

doris-robot commented Mar 17, 2025

Uh oh!

doris-robot commented Mar 17, 2025

Uh oh!

doris-robot commented Mar 17, 2025

BE UT Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Gabriel39 commented Mar 17, 2025 •

edited

Loading