fix: llava EPD deployment. add llava coverage by GuanLuo · Pull Request #8330 · ai-dynamo/dynamo

GuanLuo · 2026-04-17T21:53:41Z

Signed-off-by: Guan Luo 41310872+GuanLuo@users.noreply.github.com

Overview:

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

closes GitHub issue: #xxx

Summary by CodeRabbit

Tests
- Added test support for a new multimodal model profile (llava-1.5-7b) with configurations for multiple topology variants and image-based request payloads.

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

devin-ai-integration

Devin Review found 1 potential issue.

View 1 additional finding in Devin Review.

coderabbitai · 2026-04-17T21:57:45Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d8946614-84a6-428e-80f8-63c3a7b8bce9

📥 Commits

Reviewing files that changed from the base of the PR and between c2ec335 and 6e8c8d5.

📒 Files selected for processing (1)

tests/serve/multimodal_profiles/vllm.py

Walkthrough

Added a new MultimodalModelProfile entry for the llava-hf/llava-1.5-7b-hf model to the vLLM multimodal profiles configuration. The profile includes topologies for both e_pd and epd modes with specified timeout and GPU configuration settings.

Changes

Cohort / File(s)	Summary
vLLM Multimodal Profile `tests/serve/multimodal_profiles/vllm.py`	Added new `MultimodalModelProfile` entry for `llava-hf/llava-1.5-7b-hf` with two topologies (`e_pd` and `epd`), each configured with `TopologyConfig` including `pytest.mark.pre_merge`, timeout values (340s and 300s), `single_gpu=True`, and image payload requests.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description follows the required template structure but all main sections (Overview, Details, Where to review) are empty with only placeholder comments, making it incomplete.	Fill in the Overview, Details, and Where should the reviewer start sections with actual information about the Llava profile addition and EPD deployment fixes.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title directly addresses the main change: adding Llava model coverage to multimodal profiles and fixing EPD deployment configuration.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

devin-ai-integration

Devin Review found 4 new potential issues.

View 3 additional findings in Devin Review.

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

… generate one response) Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

furionw

Thank you

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com> Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

test: add llava coverage

6e8c8d5

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

GuanLuo requested review from a team as code owners April 17, 2026 21:53

pull-request-size Bot added the size/S label Apr 17, 2026

github-actions Bot added the fix label Apr 17, 2026

devin-ai-integration Bot reviewed Apr 17, 2026

View reviewed changes

Comment thread tests/serve/multimodal_profiles/vllm.py

furionw reviewed Apr 17, 2026

View reviewed changes

Comment thread tests/serve/multimodal_profiles/vllm.py

chore: use gemma-3-4b

a79db84

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

copy-pr-bot Bot temporarily deployed to GITLAB April 17, 2026 23:35 Inactive

copy-pr-bot Bot temporarily deployed to GITLAB April 17, 2026 23:43 Inactive

chore: remove gated

2144f62

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

copy-pr-bot Bot temporarily deployed to GITLAB April 18, 2026 00:19 Inactive

copy-pr-bot Bot temporarily deployed to GITLAB April 18, 2026 00:28 Inactive

chore: update test

f132264

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

copy-pr-bot Bot temporarily deployed to GITLAB April 19, 2026 22:08 Inactive

fix: fix marker. fix up

3c2a73f

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

copy-pr-bot Bot temporarily deployed to GITLAB April 19, 2026 23:47 Inactive

github-actions Bot added backend::vllm Relates to the vllm backend multimodal labels Apr 19, 2026

copy-pr-bot Bot temporarily deployed to GITLAB April 19, 2026 23:47 Inactive

fix: fix up

3312e2b

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

copy-pr-bot Bot temporarily deployed to GITLAB April 20, 2026 02:53 Inactive

fix: fix up

2c9ac29

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

GuanLuo requested review from a team as code owners April 20, 2026 08:53

pull-request-size Bot added size/M and removed size/S labels Apr 20, 2026

copy-pr-bot Bot temporarily deployed to GITLAB April 20, 2026 08:53 Inactive

GuanLuo requested a review from furionw April 20, 2026 17:09

devin-ai-integration Bot reviewed Apr 20, 2026

View reviewed changes

Comment thread components/src/dynamo/vllm/handlers.py

Comment thread examples/backends/vllm/launch/disagg_multimodal_e_pd.sh

Comment thread components/src/dynamo/vllm/handlers.py Outdated

Comment thread components/src/dynamo/vllm/handlers.py Outdated

Update components/src/dynamo/vllm/handlers.py

9ae6e27

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>

copy-pr-bot Bot temporarily deployed to GITLAB April 20, 2026 17:38 Inactive

chore: address comment

6ce8211

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

copy-pr-bot Bot temporarily deployed to GITLAB April 20, 2026 18:00 Inactive

chore: avoid repeatly build embedding params (in reality prefill only…

e2b076a

… generate one response) Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

copy-pr-bot Bot temporarily deployed to GITLAB April 20, 2026 18:05 Inactive

chore: revert

2432a94

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

copy-pr-bot Bot temporarily deployed to GITLAB April 20, 2026 18:22 Inactive

fix: fix test

0ec1630

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>

copy-pr-bot Bot temporarily deployed to GITLAB April 20, 2026 19:02 Inactive

furionw reviewed Apr 20, 2026

View reviewed changes

Comment thread examples/backends/vllm/launch/disagg_multimodal_e_pd.sh

Comment thread tests/serve/multimodal_profiles/vllm.py

Comment thread components/src/dynamo/vllm/multimodal_utils/hash_utils.py

furionw approved these changes Apr 20, 2026

View reviewed changes

GuanLuo merged commit e71f1d2 into main Apr 20, 2026
85 checks passed

GuanLuo deleted the gluo/dyn-2723-dynamorelease110vllm-upstream-disagg_multimodal_epdsh branch April 20, 2026 20:17

copy-pr-bot Bot had a problem deploying to GITLAB April 20, 2026 20:33 Failure

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: llava EPD deployment. add llava coverage#8330

fix: llava EPD deployment. add llava coverage#8330
GuanLuo merged 12 commits into
mainfrom
gluo/dyn-2723-dynamorelease110vllm-upstream-disagg_multimodal_epdsh

GuanLuo commented Apr 17, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

Uh oh!

coderabbitai Bot commented Apr 17, 2026

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

furionw left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

GuanLuo commented Apr 17, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview:

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

Summary by CodeRabbit

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coderabbitai Bot commented Apr 17, 2026

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

furionw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

GuanLuo commented Apr 17, 2026 •

edited by coderabbitai Bot

Loading