[Feature] mm prefix cache

[kevincheng2]

Motivation

mm prefix cache

启动时需要增加参数：

  python -m fastdeploy.entrypoints.openai.api_server \
       ...
       --enable-prefix-caching \
       --disable-chunked-mm-input

Modifications

Usage or Command

Accuracy Tests

Checklist

• Add at least a tag in the PR title.
  • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
  • You can add new tags based on the PR content, but the semantics must be clear.
• Format your code, run pre-commit before commit.
• Add unit tests. Please write the reason in this PR if no unit tests.
• Provide accuracy results.
• If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

[paddle-bot]

Thanks for your contribution!

[Copilot]

Pull Request Overview

This pull request implements multimodal prefix caching functionality, enabling efficient cache reuse for requests containing images or other multimodal inputs. The key changes involve refactoring the prefix cache manager to handle multimodal data with proper hashing and block management.

• Adds multimodal-aware prefix caching with image hash tracking
• Introduces disable_chunked_mm_input flag to prevent splitting multimodal inputs across cache blocks
• Updates cache hit tracking to use token counts instead of block counts for more accurate metrics

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
fastdeploy/cache_manager/prefix_cache_manager.py	Implements core multimodal prefix caching logic including mm_match_block, mm_build_path, and hash computation with image keys
fastdeploy/engine/sched/resource_manager_v1.py	Updates cache hit metrics to use token-level granularity instead of block-level
fastdeploy/engine/common_engine.py	Simplifies available blocks calculation by removing multimodal-specific logic
fastdeploy/engine/args_utils.py	Adds disable_chunked_mm_input configuration option and removes restriction preventing prefix caching with multimodal models
fastdeploy/config.py	Adds disable_chunked_mm_input field to CacheConfig
tests/v1/test_prefix_cache.py	Adds comprehensive tests for multimodal prefix caching functionality
tests/v1/test_revert_blocks.py	Adds tests for block reversion logic when multimodal inputs are chunked