rpc : track allocated buffers#7411
Conversation
|
|
||
|
|
||
| ggml_backend_t backend; | ||
| std::unordered_set<ggml_backend_buffer_t> buffers; |
There was a problem hiding this comment.
nice work! just some thought:
- should we also handle the life span of
client_socketandgml_backend_tin this class?
There was a problem hiding this comment.
As I noted in PR #7407, we serve only one client at a time and I don't see reasons why we should add support for multiple clients.
There was a problem hiding this comment.
As I noted in PR #7407, we serve only one client at a time and I don't see reasons why we should add support for multiple clients.
okay, then we can do it later, when we wanna introduce multi-client supports.
| } | ||
| #endif | ||
| GGML_PRINT_DEBUG("Connecting to %s\n", endpoint); | ||
| fprintf(stderr, "Connecting to %s\n", endpoint); |
There was a problem hiding this comment.
nit: should it be better to use the GGML_PRINT_DEBUG instead of stdio directly?
There was a problem hiding this comment.
I think this message is helpful for troubleshooting connection problems, especially when using multiple rpc-servers, so I think it's fine to always print this
mofosyne
added
Review Complexity : Medium
|
Test on my machine, the memory was freed correctly after client disconnected:
|
* rpc : track allocated buffers ref: ggml-org#7407 * rpc : pack rpc_tensor tightly
* rpc : track allocated buffers ref: ggml-org#7407 * rpc : pack rpc_tensor tightly
5 participants
ref: #7407