Processes piling up in fabric_doc_attachments:receive_unchunked_attachment/2

[davisp]

Working ops I noticed that a particular cluster had a bunch of processes stuck in this function at [1]. Something somewhere must be orphaning processes here that then leaves them forever in the VM to accumulate until they slowly use up all available RAM. The particular cluster I was looking at took about 45 days to use up about 7G of RAM so its not a super pressing issue I don't think.

I figured I'd open a ticket as a note that I'd seen this.

[eiri]

I assume [1] should point to a code line?

[davisp]

[1]

couchdb/src/fabric/src/fabric_doc_attachments.erl

Line 93 in 02c9429

receive {MiddleMan, go} ->

[nickva]

Maybe the easiest fix is that middleman process in after Timeout kills the receiver process:

    after Timeout ->
        exit(Receiver, kill),
        ok

Also timeout is the [fabric] request_timeout = 60000 maybe we really want that to be [fabric] attachments_timeout = 600000 since it involves attachment uploading

[nickva]

And maybe we don't want the attachment timeout as that would be a total timeout for whole attachment upload but here it is really per-chunk so 60 sec seems about right

[davisp]

+1 after swapping that timeout config as you suggest