Skip to content

fix: add FlexKVConnectorV1 to PdConnector allowed first connector types#8787

Merged
dagil-nvidia merged 2 commits into
ai-dynamo:mainfrom
linhu-nv:linhu-nv/fix-pdconnector-flexkv
Apr 29, 2026
Merged

fix: add FlexKVConnectorV1 to PdConnector allowed first connector types#8787
dagil-nvidia merged 2 commits into
ai-dynamo:mainfrom
linhu-nv:linhu-nv/fix-pdconnector-flexkv

Conversation

@linhu-nv
Copy link
Copy Markdown
Contributor

@linhu-nv linhu-nv commented Apr 28, 2026

Fix PdConnector TypeError when launching disaggregated FlexKV serving (disagg_flexkv.sh).

PdConnector.__init__ only allowed DynamoConnector and LMCacheConnectorV1 as the first connector. When FlexKVConnectorV1 is configured as the first connector (KV offloading
role), it raises:

TypeError: Expected first connector to be DynamoConnector or LMCacheConnectorV1, got FlexKVConnectorV1

This PR adds FlexKVConnectorV1 to the allowed first connector types using the same optional-import pattern established for LMCacheConnectorV1 in PR #4319.

Fixes: NVBug 6088159

Changes

  • Add optional import for FlexKVConnectorV1 from vllm.distributed.kv_transfer.kv_connector.v1.flexkv_connector (soft dependency, no error if FlexKV is not installed)
  • Extend allowed_first_types and error message to include FlexKVConnectorV1

Test plan

  • Launch examples/backends/vllm/launch/disagg_flexkv.sh — confirm prefill worker starts without TypeError
  • Launch existing disagg_kvbm.sh / disagg_lmcache.sh — confirm no regression for DynamoConnector and LMCacheConnectorV1 paths
  • Verify FlexKV-not-installed case: import falls back gracefully, PdConnector still works with DynamoConnector/LMCacheConnectorV1

Open in Devin Review

Summary by CodeRabbit

  • New Features

    • Added FlexKV as an alternative KV offloading connector option.
  • Documentation

    • Updated documentation to reflect the new connector option availability.

@linhu-nv linhu-nv requested review from a team as code owners April 28, 2026 04:38
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Apr 28, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 28, 2026

@github-actions github-actions Bot added external-contribution Pull request is from an external contributor fix labels Apr 28, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 28, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b58a33b6-1650-4f19-a701-acf7aa2bb29d

📥 Commits

Reviewing files that changed from the base of the PR and between ee3389e and 5e40097.

📒 Files selected for processing (1)
  • lib/bindings/kvbm/python/kvbm/vllm_integration/connector/pd_connector.py

Walkthrough

Adds optional FlexKV support to PdConnector as an alternative first KV offloading connector. Introduces guarded import for FlexKVConnectorV1, conditionally updates connector type validation, and updates class documentation to reflect the additional option.

Changes

Cohort / File(s) Summary
FlexKV Integration
lib/bindings/kvbm/python/kvbm/vllm_integration/connector/pd_connector.py
Adds optional FlexKV support with guarded import for FlexKVConnectorV1. Conditionally extends the type validation set for the first connector to accept FlexKV alongside KVBM and LMCache. Updates error message and class documentation to reflect the new option.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The pull request title accurately and concisely summarizes the main change: adding FlexKVConnectorV1 to the allowed first connector types in PdConnector.
Description check ✅ Passed The pull request description is comprehensive and well-structured, covering the problem statement, solution approach, specific changes made, and a detailed test plan with checkboxes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@dagil-nvidia
Copy link
Copy Markdown
Collaborator

/ok to test 5e40097

@pull-request-size pull-request-size Bot added size/M and removed size/S labels Apr 29, 2026
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Apr 29, 2026
@dagil-nvidia
Copy link
Copy Markdown
Collaborator

/ok to test c23b9b7

@dagil-nvidia
Copy link
Copy Markdown
Collaborator

/ok to test c23b9b7

Add FlexKVConnectorV1 to PdConnector's allowed first connector types,
following the same optional-import pattern as LMCacheConnectorV1.
This fixes the TypeError when launching disagg_flexkv.sh.

Fixes: NVBug 6088159
Signed-off-by: linhu-nv <linhu@nvidia.com>
…agg connector config

- Add FlexKV installation steps (git clone + ./build.sh) to prerequisites
- Clarify that liburing-dev/libxxhash-dev are only needed for SSD offloading
- Fix disaggregated serving example to use correct PdConnector configuration
  with [FlexKVConnectorV1, NixlConnector] sub-connectors
- Add connector configuration table and experimental status note
- Reference the disagg_flexkv.sh launch script

Fixes: NVBug 6088159
Signed-off-by: linhu-nv <linhu@nvidia.com>
@dagil-nvidia dagil-nvidia force-pushed the linhu-nv/fix-pdconnector-flexkv branch from c23b9b7 to f5fa63c Compare April 29, 2026 22:43
@dagil-nvidia
Copy link
Copy Markdown
Collaborator

/ok to test f5fa63c

@dagil-nvidia dagil-nvidia enabled auto-merge (squash) April 29, 2026 22:59
@dagil-nvidia dagil-nvidia disabled auto-merge April 29, 2026 23:38
@dagil-nvidia dagil-nvidia merged commit cfafea6 into ai-dynamo:main Apr 29, 2026
86 of 87 checks passed
pvijayakrish added a commit that referenced this pull request Apr 30, 2026
…es (#8787) (#8872)

Signed-off-by: linhu-nv <linhu@nvidia.com>
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: linhu-nv <141609318+linhu-nv@users.noreply.github.com>
Co-authored-by: Pavithra Vijayakrishnan <160681768+pvijayakrish@users.noreply.github.com>
furionw pushed a commit that referenced this pull request May 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation external-contribution Pull request is from an external contributor fix size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants