[Disco] Explicitly set the session on DPackedFunc and DModule #15996
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR changes how to get the
SessionfromDPackedFuncorDModule. Without this, there will be data corruption in the message channel, if Python gc happens in a different thread than the thread that owns the Disco session, which is typical in a multi-thread environment like LLM inference server.An explanation on how such data corruption could occur:
tvm/python/tvm/runtime/disco/session.py
Line 42 in ebbe38f
tvm/python/tvm/runtime/disco/session.py
Line 97 in ebbe38f
tvm/python/tvm/runtime/disco/session.py
Line 107 in ebbe38f
tvm/python/tvm/_ffi/_ctypes/packed_func.py
Line 239 in ebbe38f
tvm/python/tvm/_ffi/base.py
Line 373 in ebbe38f
tvm/include/tvm/runtime/disco/session.h
Line 301 in ebbe38f
This PR fixes (1) and (3), so that the data corruption will not occur anymore.
cc @junrushao