Skip to content

Conversation

@rainyfly
Copy link
Collaborator

  1. Support mixed deployment with yiyan adapter

@paddle-bot
Copy link

paddle-bot bot commented Aug 22, 2025

Thanks for your contribution!

@Jiang-Jia-Jun Jiang-Jia-Jun requested a review from Copilot August 22, 2025 06:48

This comment was marked as outdated.

@rainyfly rainyfly requested a review from Copilot August 22, 2025 08:26
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for mixed deployment with yiyan adapter by implementing an internal adapter mechanism that enables communication with external modules via TCP-based ZMQ connections.

  • Introduces an internal adapter system that allows external modules to communicate with the LLM engine through TCP connections when FD_ENABLE_INTERNAL_ADAPTER=1
  • Refactors ZMQ communication to support both IPC (existing) and TCP (new) modes with a clean abstraction layer
  • Adds control command functionality for external modules to query engine status and metrics

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
fastdeploy/splitwise/internal_adapter_utils.py New internal adapter implementation with TCP-based control command handling
fastdeploy/scheduler/local_scheduler.py Adds method to get count of unhandled requests
fastdeploy/inter_communicator/zmq_server.py New ZMQ server implementation supporting both IPC and TCP modes
fastdeploy/inter_communicator/zmq_client.py Refactored to use base class pattern and support different client types
fastdeploy/inter_communicator/init.py Updated imports for new ZMQ classes
fastdeploy/envs.py Added environment variables for internal adapter configuration
fastdeploy/entrypoints/engine_client.py Updated to use new ZmqIpcClient class
fastdeploy/engine/engine.py Integrated internal adapter and updated to use new server classes

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@Jiang-Jia-Jun Jiang-Jia-Jun merged commit e8af92a into PaddlePaddle:release/2.1 Aug 23, 2025
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants