[Feature] ProxyService avoid keeping ownership of the RegisteredMemory

[liangyuRain]

Hi, the current implementation of proxy service keeps ownership of RegisteredMemory:

mscclpp/src/port_channel.cc

Line 41 in c184485

memories_.push_back(memory);

Thus, any isCuMemMapAlloc memory added to proxy service won't be able to be released unless proxy service is destructed, which may cause trouble for a long running program. Can we avoid doing this? Maybe we can add a mechanism to detect if user issues operations on the memory id of a released memory or simply make the user responsible for not doing so.

[Binyang2014]

How about add a function to delete registered memory from proxyService. It's user responsible to delete it from proxyService.

[liangyuRain]

Adding a delete function sounds good. Do we also want the registered memory to be automatically removed from proxy service if it is being released? Right now there is a discrepancy: for isCuMemMapAlloc memory, it needs to be removed from proxy service to be released due to the shareable handle; for non isCuMemMapAlloc memory, the memory can be released without removal from proxy service.

[Binyang2014]

Oh, I don't notice that. Is this due to fileDesc? Which used to export isCuMemMapAlloc memory. I think this handle can be free after the memory is imported by peers (same behavior as NCCL). But we need an async way to handle this.

[liangyuRain]

Sorry for the confusion. Yes it is the fileDesc preventing the gpu memory to be released. From cuda driver api link:

The memory allocation will be freed when all outstanding mappings to the memory are unmapped and when all outstanding references to the handle (including it's shareable counterparts) are also released.

The problem is that the fileDesc is only closed in RegisteredMemory::Impl::~Impl(), but the proxy service always keeps the RegisteredMemory in ProxyService.memories_, preventing the Impl from being destructed.

[Binyang2014]

Okay, I think it's tricky to track the memory reference. I prefer to add a function ProxyService::removeMemory(MemoryId memoryId) to remove registered memory from proxyService. Pls refer this PR: #534