-
Notifications
You must be signed in to change notification settings - Fork 690
[Feature] Support mixed deployment with yiyan adapter #3533
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Support mixed deployment with yiyan adapter #3533
Conversation
rainyfly
commented
Aug 22, 2025
- Support mixed deployment with yiyan adapter
|
Thanks for your contribution! |
…/github.com/rainyfly/FastDeploy into support_mixed_deployment_with_yiyan_adapter
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds support for mixed deployment with yiyan adapter by implementing an internal adapter mechanism that enables communication with external modules via TCP-based ZMQ connections.
- Introduces an internal adapter system that allows external modules to communicate with the LLM engine through TCP connections when
FD_ENABLE_INTERNAL_ADAPTER=1 - Refactors ZMQ communication to support both IPC (existing) and TCP (new) modes with a clean abstraction layer
- Adds control command functionality for external modules to query engine status and metrics
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| fastdeploy/splitwise/internal_adapter_utils.py | New internal adapter implementation with TCP-based control command handling |
| fastdeploy/scheduler/local_scheduler.py | Adds method to get count of unhandled requests |
| fastdeploy/inter_communicator/zmq_server.py | New ZMQ server implementation supporting both IPC and TCP modes |
| fastdeploy/inter_communicator/zmq_client.py | Refactored to use base class pattern and support different client types |
| fastdeploy/inter_communicator/init.py | Updated imports for new ZMQ classes |
| fastdeploy/envs.py | Added environment variables for internal adapter configuration |
| fastdeploy/entrypoints/engine_client.py | Updated to use new ZmqIpcClient class |
| fastdeploy/engine/engine.py | Integrated internal adapter and updated to use new server classes |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
e8af92a
into
PaddlePaddle:release/2.1