Skip to content

[BugFix] dynamic cache kv block_wise_fp8 not need create layer.cache_k_scale#5362

Merged
yuanlehome merged 1 commit intoPaddlePaddle:developfrom
yuanlehome:fix_blockwise_fp8_scale
Dec 3, 2025
Merged

[BugFix] dynamic cache kv block_wise_fp8 not need create layer.cache_k_scale#5362
yuanlehome merged 1 commit intoPaddlePaddle:developfrom
yuanlehome:fix_blockwise_fp8_scale

Conversation

@yuanlehome
Copy link
Collaborator

Motivation

💡 If this PR is a Cherry Pick, the PR title needs to follow the format by adding the [Cherry-Pick] label at the very beginning and appending the original PR ID at the end. For example, [Cherry-Pick][CI] Add check trigger and logic(#5191)

💡 如若此PR是Cherry Pick,PR标题需遵循格式,在最开始加上[Cherry-Pick]标签,以及最后面加上原PR ID,例如[Cherry-Pick][CI] Add check trigger and logic(#5191)

动态cache kv的block_wise_fp8量化,不需要在layer里创建scale parameter

Modifications

Usage or Command

Accuracy Tests

Checklist

  • Add at least a tag in the PR title.
    • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
    • You can add new tags based on the PR content, but the semantics must be clear.
  • Format your code, run pre-commit before commit.
  • Add unit tests. Please write the reason in this PR if no unit tests.
  • Provide accuracy results.
  • If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

Copilot AI review requested due to automatic review settings December 3, 2025 11:34
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

此 PR 修复了动态缓存 KV 使用 block_wise_fp8 量化时不必要地创建 scale 参数的 bug。当使用 block_wise_fp8 量化类型时,无需在 layer 中创建 cache_k_scalecache_v_scalecache_k_out_scalecache_v_out_scale 以及零点参数。

  • 将 scale 和 zero-point 参数创建逻辑包裹在条件判断中,仅在非 block_wise 量化类型时创建
  • 保持与 process_loaded_weights 方法中的条件判断逻辑一致

@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 92.30769% with 1 line in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@4e8096b). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...loy/model_executor/layers/quantization/kv_cache.py 92.30% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #5362   +/-   ##
==========================================
  Coverage           ?   59.33%           
==========================================
  Files              ?      324           
  Lines              ?    40057           
  Branches           ?     6052           
==========================================
  Hits               ?    23769           
  Misses             ?    14401           
  Partials           ?     1887           
Flag Coverage Δ
GPU 59.33% <92.30%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yuanlehome yuanlehome merged commit be0c960 into PaddlePaddle:develop Dec 3, 2025
19 of 22 checks passed
@paddle-bot
Copy link

paddle-bot bot commented Dec 4, 2025

Thanks for your contribution!

liyonghua0910 pushed a commit to liyonghua0910/FastDeploy that referenced this pull request Dec 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants