fix: add FlexKVConnectorV1 to PdConnector allowed first connector types#8787
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughAdds optional FlexKV support to Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
/ok to test 5e40097 |
|
/ok to test c23b9b7 |
|
/ok to test c23b9b7 |
Add FlexKVConnectorV1 to PdConnector's allowed first connector types, following the same optional-import pattern as LMCacheConnectorV1. This fixes the TypeError when launching disagg_flexkv.sh. Fixes: NVBug 6088159 Signed-off-by: linhu-nv <linhu@nvidia.com>
…agg connector config - Add FlexKV installation steps (git clone + ./build.sh) to prerequisites - Clarify that liburing-dev/libxxhash-dev are only needed for SSD offloading - Fix disaggregated serving example to use correct PdConnector configuration with [FlexKVConnectorV1, NixlConnector] sub-connectors - Add connector configuration table and experimental status note - Reference the disagg_flexkv.sh launch script Fixes: NVBug 6088159 Signed-off-by: linhu-nv <linhu@nvidia.com>
c23b9b7 to
f5fa63c
Compare
|
/ok to test f5fa63c |
…es (#8787) Signed-off-by: linhu-nv <linhu@nvidia.com>
Fix
PdConnectorTypeError when launching disaggregated FlexKV serving (disagg_flexkv.sh).PdConnector.__init__only allowedDynamoConnectorandLMCacheConnectorV1as the first connector. WhenFlexKVConnectorV1is configured as the first connector (KV offloadingrole), it raises:
TypeError: Expected first connector to be DynamoConnector or LMCacheConnectorV1, got FlexKVConnectorV1
This PR adds
FlexKVConnectorV1to the allowed first connector types using the same optional-import pattern established forLMCacheConnectorV1in PR #4319.Fixes: NVBug 6088159
Changes
FlexKVConnectorV1fromvllm.distributed.kv_transfer.kv_connector.v1.flexkv_connector(soft dependency, no error if FlexKV is not installed)allowed_first_typesand error message to includeFlexKVConnectorV1Test plan
examples/backends/vllm/launch/disagg_flexkv.sh— confirm prefill worker starts without TypeErrordisagg_kvbm.sh/disagg_lmcache.sh— confirm no regression for DynamoConnector and LMCacheConnectorV1 pathsPdConnectorstill works with DynamoConnector/LMCacheConnectorV1Summary by CodeRabbit
New Features
Documentation