Skip to content

接口压测时 curl -i ip:port/rpcz 耗时 30s,gstack 查看 brpc 调用栈异常 #2189

@Flynn-Zh

Description

@Flynn-Zh

Describe the bug (描述bug)
使用 jmeter 压测 brpc server 的某个接口(200并发),持续10分钟,过程中使用中间使用 curl -i ip:port/rpcz 查看接口执行情况,curl 约 30s 返回,接口普遍延时很高(正常时是100ms级,此时是10s级),此时通过 gstack 查看 brpc server 堆栈如下:
......
......
#16 0x00007f1d026d6803 in brpc::policy::ProcessHttpRequest(brpc::InputMessageBase*) () from ./libbrpc.so
#17 0x00007f1d0269a4da in brpc::ProcessInputMessage(void*) () from ./libbrpc.so
#18 0x00007f1d0269b283 in brpc::InputMessenger::OnNewMessages(brpc::Socket*) () from ./libbrpc.so
#19 0x00007f1d0274f57d in brpc::Socket::ProcessEvent(void*) () from ./libbrpc.so
#20 0x00007f1d025f381a in bthread::TaskGroup::task_runner(long) () from ./libbrpc.so
#21 0x00007f1d025dcf11 in bthread_make_fcontext () from ./libbrpc.so
#22 0x2be01d9a22db1800 in ?? ()
#23 0x2be01d9a4605fa49 in ?? ()
#24 0x025a3f014605fa49 in ?? ()
#25 0xe88ac60800000000 in ?? ()
#26 0x10019a8ef885b4cf in ?? ()
......
......
#264433 0x016300002c2d0000 in ?? ()
#264434 0x87192214f6000000 in ?? ()
#264435 0x0180aa94080180aa in ?? ()
#264436 0x00000000000063ce in ?? ()
#264437 0x0000000000000000 in ?? ()
Thread 57 (Thread 0x7f1cd79fe640 (LWP 65)):

To Reproduce (复现方法)
使用 jmeter 压测 brpc server 的某个接口(200并发)

Expected behavior (期望行为)
请求正常响应

Versions (各种版本)
OS: centos 7.6 编译,ubuntu22.04
Compiler: gcc 4.8.5
brpc: 1.1.0
protobuf:3.19.1

Additional context/screenshots (更多上下文/截图)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions