rpc: free buffer after client disconnect

[chraac]

In PR #6829, @rgerganov add support to rpc backend, after using it for several days, I have noticed an issue:

• When the client closes the connection, the server does not free the memory it has allocated.

Upon investigating the source code, I discovered that instead of releasing the memory, we simply exit the inner loop and immediately wait for a new connection (ggml-rpc.cpp#L1027).

So here I create this PR, which monitor the ALLOC_BUFFER and FREE_BUFFER command, maintaining a list of allocated buffers, then free the remind buffer after client disconnect.

[chraac]

add the allocated buffer into list.

[chraac]

remove the freed buffer from list

[chraac]

free the reminding buffers.

[rgerganov]

I think a better approach would be to track allocated buffers with std::unordered_set, so we can also perform input validation in SET_TENSOR, `GET_TENSOR, etc. I will submit a patch shortly and add you as a reviewer.

[chraac]

I think a better approach would be to track allocated buffers with std::unordered_set, so we can also perform input validation in SET_TENSOR, `GET_TENSOR, etc. I will submit a patch shortly and add you as a reviewer.

emm, just forgot the SET_TENSOR and GET_TENSOR case, see there's also a tensor allocation in there.

also, for the std::unordered_set, thought the std::list will be okay here, since we won't have too many tensor, so this will no impact the free operation's performance.

but agree that std::unordered_set is a better alternative, let use it instead.

[chraac]

close this PR and wait for @rgerganov 's fix, related discussion: #7407