Skip to content

Fix Acceptor::BeforeRecycle during Accept::StartAccept dead lock #1791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wwbmmm wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fix Acceptor::BeforeRecycle during Accept::StartAccept dead lock #1791

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1a9bcde
Fix Acceptor::BeforeRecycle during Accept::StartAccept dead lock
wwbmmm Jun 10, 2022
a9b3aca
Don't call _user->BeforeRecycle on Socket::Create
wwbmmm Jun 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions src/brpc/socket.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -618,8 +618,7 @@ int Socket::Create(const SocketOptions& options, SocketId* id) {
m->_auth_flag_error.store(0, butil::memory_order_relaxed);
const int rc2 = bthread_id_create(&m->_auth_id, NULL, NULL);
if (rc2) {
LOG(ERROR) << "Fail to create auth_id: " << berror(rc2);
m->SetFailed(rc2, "Fail to create auth_id: %s", berror(rc2));
m->SetFailedOnCreate(rc2, "Fail to create auth_id");
return -1;
}
// Disable SSL check if there is no SSL context
Expand All @@ -640,8 +639,7 @@ int Socket::Create(const SocketOptions& options, SocketId* id) {
// NOTE: last two params are useless in bthread > r32787
const int rc = bthread_id_list_init(&m->_id_wait_list, 512, 512);
if (rc) {
LOG(ERROR) << "Fail to init _id_wait_list: " << berror(rc);
m->SetFailed(rc, "Fail to init _id_wait_list: %s", berror(rc));
m->SetFailedOnCreate(rc, "Fail to init _id_wait_list");
return -1;
}
m->_last_writetime_us.store(cpuwide_now, butil::memory_order_relaxed);
Expand All @@ -650,10 +648,7 @@ int Socket::Create(const SocketOptions& options, SocketId* id) {
// Must be last one! Internal fields of this Socket may be access
// just after calling ResetFileDescriptor.
if (m->ResetFileDescriptor(options.fd) != 0) {
const int saved_errno = errno;
PLOG(ERROR) << "Fail to ResetFileDescriptor";
m->SetFailed(saved_errno, "Fail to ResetFileDescriptor: %s",
berror(saved_errno));
m->SetFailedOnCreate(errno, "Fail to ResetFileDescriptor");
return -1;
}
*id = m->_this_id;
Expand Down Expand Up @@ -875,6 +870,12 @@ int Socket::SetFailed() {
return SetFailed(EFAILEDSOCKET, NULL);
}

int Socket::SetFailedOnCreate(int error_code, const char* error_text) {
_user = NULL; // Avoid calling _user->BeforeRecycle() which may cause dead lock or double free
Copy link
Contributor

@jamesge jamesge Jun 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是不是可以反过来在BeforeRecycle前判一下rpc failed?

Copy link
Contributor Author

@wwbmmm wwbmmm Jun 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里主要是为了区分是在Create Socket过程中出错，还是使用Socket过程中出错
回调BeforeRecycle的时候，Socket已经是Failed的状态了，似乎难以区分是不是Create Socket的过程

LOG(ERROR) << error_text << ": " << berror(error_code);
return SetFailed(error_code, "%s: %s", error_text, berror(error_code));
}

void Socket::FeedbackCircuitBreaker(int error_code, int64_t latency_us) {
if (!GetOrNewSharedPart()->circuit_breaker.OnCallEnd(error_code, latency_us)) {
if (SetFailed(main_socket_id()) == 0) {
Expand Down
3 changes: 3 additions & 0 deletions src/brpc/socket.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -586,6 +586,9 @@ friend void DereferenceSocket(Socket*);

int ResetFileDescriptor(int fd);

// Handle failure on Socket::Create
int SetFailedOnCreate(int error_code, const char* error_text);

// Wait until nref hits `expected_nref' and reset some internal resources.
int WaitAndReset(int32_t expected_nref);

Expand Down