WebSocketClient::open 偶发宕机

[cloud-yd]

再做性能测试的时候，做了100并发的websocket连接，稳定跑了半个多小时之后，在ws新建链接的时候宕机了，看了代码只调用了WebSocketClient::open
堆栈如下：
(gdb) bt
#0 0x00007ff64d833d9c in iowatcher_add_event () from /home/hytxccp/unimrcpserver/lib/libhv.so
#1 0x00007ff64d83872f in hio_add () from /home/hytxccp/unimrcpserver/lib/libhv.so
#2 0x00007ff64d832ab5 in hio_connect () from /home/hytxccp/unimrcpserver/lib/libhv.so
#3 0x00007ff64d87f639 in hv::TcpClientEventLoopTmplhv::WebSocketChannel::startConnect() () from /home/hytxccp/unimrcpserver/lib/libhv.so
#4 0x00007ff64dabc3c7 in operator() (__args#0=0x7ff648017948, this=0x7ff648017988) at /usr/include/c++/4.8.2/functional:2471
#5 hv::EventLoop::onCustomEvent (hev=) at ../../plugins/common/include/hv/EventLoop.h:224
#6 0x00007ff64d835b27 in eventfd_read_cb () from /home/hytxccp/unimrcpserver/lib/libhv.so
#7 0x00007ff64d834b6d in hio_read_cb () from /home/hytxccp/unimrcpserver/lib/libhv.so
#8 0x00007ff64d835567 in hio_handle_read () from /home/hytxccp/unimrcpserver/lib/libhv.so
#9 0x00007ff64d83363b in hio_handle_events () from /home/hytxccp/unimrcpserver/lib/libhv.so
#10 0x00007ff64d837517 in hloop_process_events () from /home/hytxccp/unimrcpserver/lib/libhv.so
#11 0x00007ff64d838c6b in hloop_run () from /home/hytxccp/unimrcpserver/lib/libhv.so
#12 0x00007ff64dabd310 in run (this=0x7ff648144028) at ../../plugins/common/include/hv/EventLoop.h:54
#13 hv::EventLoopThread::loop_thread(std::function<int ()> const&, std::function<int ()> const&) (this=0x7ff64801fc88, pre=..., post=...) at ../../plugins/common/include/hv/EventLoopThread.h:95
#14 0x00007ff64d5a6330 in ?? () from /lib64/libstdc++.so.6
#15 0x00007ff64fe00ea5 in start_thread () from /lib64/libpthread.so.0
#16 0x00007ff64f82796d in clone () from /lib64/libc.so.6

[cloud-yd]

头文件

#ifndef _PM_WEBSOCKET_CLIENT_H_
#define _PM_WEBSOCKET_CLIENT_H_

#include "hv/WebSocketClient.h"
#include "hv/EventLoopThread.h"
#include <string>
#include <functional>
#include <atomic>
#include <memory>

namespace pmp {

    using MessageCallback = std::function<void(const std::string &)>;
    using ConnectionCallback = std::function<void(bool)>;

    class PMWebSocketClient
    {
    public:
        PMWebSocketClient(int max_retry_cnt = 5);
        ~PMWebSocketClient();

        // 连接服务器
        bool connect(const std::string &url, const std::string &_user_data);
        // 断开连接
        void disconnect();
        // 发送消息
        // bool send(const std::string&& message);
        bool send(const std::string &&msg, enum ws_opcode opcode = WS_OPCODE_BINARY);
        // 设置接收消息回调
        void setMessageCallback(const MessageCallback &cb) { messageCallback = cb; }
        // 设置连接状态回调
        void setConnectionCallback(const ConnectionCallback &cb) { connectionCallback = cb; }
        // 是否已连接
        bool isConnected() const { return connected; }

    private:
        void init_ws(int max_retry_cnt);

    private:
        std::shared_ptr<hv::EventLoopThread> loop_thread;
        std::shared_ptr<hv::WebSocketClient> ws;
        reconn_setting_t reconn;

        std::string server_url;
        std::string user_data;
        MessageCallback messageCallback;
        ConnectionCallback connectionCallback;

        std::atomic<bool> connected{false};
};

} // namespace pmp 

#endif

[cloud-yd]

就是在调用PMWebSocketClient::connect的时候发生了宕机

#include "websocket_client.h"

namespace pmp {

PMWebSocketClient::PMWebSocketClient(int max_retry_cnt) {
    init_ws(max_retry_cnt);
}

PMWebSocketClient::~PMWebSocketClient() {
    if (connected){
        disconnect();
    }
    // std::cout << ">>> PMWebSocketClient destorying ..." << std::endl;
}

void PMWebSocketClient::init_ws(int max_retry_cnt) {
    loop_thread = std::make_shared<hv::EventLoopThread>();
    loop_thread->start();

    ws = std::make_shared<hv::WebSocketClient>(loop_thread->loop());
    
    // 设置websocket事件回调
    ws->onopen = [this]() {
        connected = true;
        if (connectionCallback) {
            connectionCallback(true);
        }
        // std::cout << ">>>  WebSocket connected to: " << std::endl;
    };

    ws->onclose = [this]() {
        connected = false;
        if (connectionCallback) {
            connectionCallback(false);
        }
        // std::cout << ">>>  WebSocket disconnected from: " << std::endl;
    };

    ws->onmessage = [this](const std::string& msg) {
        if (messageCallback) {
            messageCallback(msg);
        }
        // std::cout << ">>>  WebSocket received message: " << std::endl;
    };

    reconn.min_delay = 1000;
    reconn.max_delay = 3000;
    reconn.delay_policy = 1;
    reconn.max_retry_cnt = max_retry_cnt;
    ws->setReconnect(&reconn);

    //设置心跳间隔
    ws->setPingInterval(2000);
}

bool PMWebSocketClient::connect(const std::string& url, const std::string& _user_data) {
    if (connected) {
        return true;
    }

    if (!url.empty())
        server_url = url;

    user_data = _user_data;
    
    int ret = ws->open(url.c_str());
    if (ret != 0) {
        // std::cout << ">>>  Failed to connect to: " << std::endl;
        return false;
    }
    return true;
}

void PMWebSocketClient::disconnect() {
    connected = false;
    if (ws){
        ws->close();
        // std::cout << ">>>  ws->close " << std::endl;
    }
    loop_thread->stop();
    loop_thread->join();
}

bool PMWebSocketClient::send(const std::string &&msg, enum ws_opcode opcode)
{
    if (!connected || !ws) {
        return false;
    }
    return ws->send(msg.c_str(), msg.length(), opcode) > 0;
}

} 

[ithewei]

看你的实现，一个loop_thread只用于一个client，那就没必要自己创建loop_thread了，如果想多个client共用一个loop_thread，可以参考

libhv/examples/websocket_client_test.cpp

Line 65 in 23ff591

int TestMultiClientsRunInOneEventLoop(const char* url, int nclients) {

[cloud-yd]

我在外部启动了一个loopthread，后续所有ws客户端都使用这个loopthread
hv::EventLoopThread* loop_thread = new hv::EventLoopThread();
loop_thread->start();
ws = new hv::WebSocketClient(loop_thread->loop());
然后客户端主动断开连接时，也是在onClose的回调中使用deleteInLoop删除ws对象，但是还是会发生宕机，原因就是onclose的时候，可能onMessage回调也会触发。
后面没办法了用boost.beast重新开发了一遍，发现了一个不一样的地方，beast在客户端主动结束时，可以使用ws_.next_layer().cancel();主动结束所有挂起的回调事件，并返回beast::error_code，然后再调用async_close去断开连接，此时只会触发on_close。
而且beast的异步事件都是使用shared_from_this()的，所以对象的生命周期会因为有异步事件存在而延长，直到事件结束或用户主动断开连接。

[ithewei]

虽然不知道你崩溃具体原因是什么，但是onClose后，不可能还有onMessage触发的。
你可以基于这个

libhv/examples/websocket_client_test.cpp

Line 65 in 23ff591

int TestMultiClientsRunInOneEventLoop(const char* url, int nclients) {

做测试