服务中可能会使什么原因导致CLOSE_WAIT出现

[fixee]

目前服务运行过程中会产生少许CLOSE_WAIT， 然后导致程序关闭时卡在 Server.Join() 这一步。 请问以下这种情况可能是什么原因导致的呢？

[fixee]

检查了一下， 各个分支都调用了done->Run() 所以应该排除 未调用done->Run() 这一条原因

[jamesge]

这些现象说明大概率还是done->Run()漏了，建议再检查一下

[fixee]

worker线程：
#0 0x00007f60a1032bf9 in syscall () from /lib64/libc.so.6
#1 0x00007f60a3d1c382 in construct<butil::ResourcePoolFreeChunk<bthread::Id, 0ul>, butil::ResourcePoolFreeChunk<bthread::Id, 0ul> const&> (__p=0xfffffffffffffe00, this=0x7f5747a0d050) at /usr/include/c++/4.8.2/ext/new_allocator.h:120
#2 _S_construct<butil::ResourcePoolFreeChunk<bthread::Id, 0ul>, butil::ResourcePoolFreeChunk<bthread::Id, 0ul> const&> (__p=0xfffffffffffffe00, __a=...) at /usr/include/c++/4.8.2/bits/alloc_traits.h:254
#3 construct<butil::ResourcePoolFreeChunk<bthread::Id, 0ul>, butil::ResourcePoolFreeChunk<bthread::Id, 0ul> const&> (__p=0xfffffffffffffe00, __a=...) at /usr/include/c++/4.8.2/bits/alloc_traits.h:393
#4 push_back (__x=@0x7f57491fe6a0: 0x0, this=0x7f5747a0d050) at /usr/include/c++/4.8.2/bits/stl_vector.h:905
#5 push_free_chunk (c=..., this=0x7f5747a0d050) at /home/jenkins/workspace/brpc/src/butil/resource_pool_inl.h:534
#6 return_resource (id=..., this=0x7f5747a0d000) at /home/jenkins/workspace/brpc/src/butil/resource_pool_inl.h:231
#7 return_resource (id=..., this=) at /home/jenkins/workspace/brpc/src/butil/resource_pool_inl.h:306
#8 return_resourcebthread::Id (id=...) at /home/jenkins/workspace/brpc/src/butil/resource_pool.h:115
#9 bthread_id_unlock_and_destroy (id=...) at /home/jenkins/workspace/brpc/src/bthread/id.cpp:634
#10 0x00007f60a3d1e9bb in set_data (data=, key=..., this=) at /home/jenkins/workspace/brpc/src/bthread/key.cpp:194
#11 bthread_keytable_pool_reserve (pool=0x7f5fb5777580, nfree=, key=..., ctor=, ctor_args=) at /home/jenkins/workspace/brpc/src/bthread/key.cpp:362
#12 0x00007f60a3d0a2e6 in ~lock_guard (this=, __in_chrg=) at /home/jenkins/workspace/brpc/src/butil/scoped_lock.h:172
#13 get_or_new_local_pool (this=) at /home/jenkins/workspace/brpc/src/butil/object_pool_inl.h:392
#14 return_object (ptr=, this=) at /home/jenkins/workspace/brpc/src/butil/object_pool_inl.h:250
#15 return_object<bthread::StackFactorybthread::LargeStackClass::Wrapper> (ptr=) at /home/jenkins/workspace/brpc/src/butil/object_pool.h:93
#16 return_stack (sc=) at /home/jenkins/workspace/brpc/src/bthread/stack_inl.h:72
#17 bthread::return_stack (s=) at /home/jenkins/workspace/brpc/src/bthread/stack_inl.h:122
#18 0x00007f60a1309dc5 in start_thread () from /lib64/libpthread.so.0
#19 0x00007f60a103873d in clone () from /lib64/libc.so.6

main 线程：
#0 0x00007f60a130d6d5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00007f60a3db31f8 in brpc::Socket::Write (this=, msg=..., options_in=) at /home/jenkins/workspace/brpc/src/brpc/socket.cpp:1535
#2 0x00007f60a3dc9121 in http_request (this=0x7ffdd9d26f40) at /home/jenkins/workspace/brpc/src/brpc/controller.h:211
#3 brpc::VarsService::default_method (this=, cntl_base=0x7ffdd9d26f40, done=0x1) at /home/jenkins/workspace/brpc/src/brpc/builtin/vars_service.cpp:341
#4 0x0000000000437434 in util::Util::RunUntilSendQuit(brpc::Server&, std::function<void ()>, std::function<void ()>) (server=..., before=..., after=...) at /home/work/buildspace/aa2f735022/src/util.cpp:64
#5 0x000000000042ed66 in main (argc=1, argv=0x7ffdd9d27690) at /home/work/buildspace/aa2f735022/src/server.cpp:191

上面贴了pstack产出的两个线程的调用栈， @jamesge 大神能帮忙看看吗？

[wwbmmm]

We close this issue because it is irreproducible and inactive for a long time.
If you can reproduce this issue with the latest version of bRPC, please reopen this issue and tell us how to reproduce.

[MapleLove2014]

目前服务运行过程中会产生少许CLOSE_WAIT， 然后导致程序关闭时卡在 Server.Join() 这一步。 请问以下这种情况可能是什么原因导致的呢？

我们这边使用brpc0.9版本作为thrift server，当client的请求体的thrift定义和server的定义不一致时，复现了这个必超时+CLOSE_WAIT的问题。
比如client端请求体定义：
struct Request{
1: required i32 id;
}
但是server请求体的定义是：
struct Request{
1: optional i32 id;
}

这种情况下如果发起请求就会直接超时+CLOSE_WAIT，改成相同的定义之后，就没这个问题了。

但我的这种情况和你的应该不同，我这个是所有的请求都必会超时，CLOSE_WAIT特别多