Skip to content

Conversation

@qwes5s5
Copy link
Collaborator

@qwes5s5 qwes5s5 commented Dec 17, 2025

Motivation

💡 If this PR is a Cherry Pick, the PR title needs to follow the format by adding the [Cherry-Pick] label at the very beginning and appending the original PR ID at the end. For example, [Cherry-Pick][CI] Add check trigger and logic(#5191)

💡 如若此PR是Cherry Pick,PR标题需遵循格式,在最开始加上[Cherry-Pick]标签,以及最后面加上原PR ID,例如[Cherry-Pick][CI] Add check trigger and logic(#5191)

Currently,PR(#5481 ) has been introduces a new FMQ (Fast Message Queue) module built on top of ZeroMQ, providing an efficient IPC/TCP messaging layer for FastDeploy.
However, the communication link used for request distribution from the API layer to the Engine layer is still based on the old ZeroMQ (push/pull) pattern. This leads to fragmented interfaces and poor performance for large data transfers. Therefore, it is necessary to use the newly developed FMQ to upgrade the existing communication link.

Modifications

API Layer

  • Replace the zmq_client used for sending requests in the API layer with fmq_producer.
  • Change the specific implementation of the API layer's request sending function, send_task, making it an asynchronous function and synchronously modifying its calling logic.
  • Add the original ZeroMQ monitoring metrics to FMQ.

Engine Layer

  • Replace the zmq_server used for receiving requests in the Engine layer with fmq_producer.
  • Modify the original request receiving logic while retaining the ADAPTER receiving logic.
  • Add the FMQ socket closing logic to the self-service exit method.

Usage or Command

Accuracy Tests

Checklist

  • Add at least a tag in the PR title.
    • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
    • You can add new tags based on the PR content, but the semantics must be clear.
  • Format your code, run pre-commit before commit.
  • Add unit tests. Please write the reason in this PR if no unit tests.
  • Provide accuracy results.
  • If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

@paddle-bot
Copy link

paddle-bot bot commented Dec 17, 2025

Thanks for your contribution!

@CLAassistant
Copy link

CLAassistant commented Dec 17, 2025

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


root seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@qwes5s5 qwes5s5 changed the title [Feature]Upgrade the communication link from the API layer to the Engine layer using FMQ [APIServer][Engine]Upgrade the communication link from the API layer to the Engine layer using FMQ Dec 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants