Skip to content

Conversation

@qwes5s5
Copy link
Collaborator

@qwes5s5 qwes5s5 commented Sep 11, 2025

更新metrics指标说明文件,更改个别metrics类型
添加以下内容
| fastdeploy:prefix_cache_token_num | Counter | 前缀缓存token总数 | 个 |
| fastdeploy:prefix_gpu_cache_token_num | Counter | 位于GPU上的前缀缓存token总数 | 个 |
| fastdeploy:prefix_cpu_cache_token_num | Counter | 位于GPU上的前缀缓存token总数 | 个 |
| fastdeploy:batch_size | Gauge | 推理时的真实批处理大小 | 个 |
| fastdeploy:max_batch_size | Gauge | 服务启动时确定的最大批处理大小 | 个 |
| fastdeploy:available_gpu_block_num | Gauge | 缓存中可用的GPU块数量(包含尚未正式释放的前缀缓存块)| 个 |
| fastdeploy:free_gpu_block_num | Gauge | 缓存中的可用块数 | 个 |
| fastdeploy:max_gpu_block_num | Gauge | 服务启动时确定的总块数 | 个 |
| available_gpu_resource | Gauge | 可用块占比,即可用GPU块数量 / 最大GPU块数量| 个 |
| fastdeploy:requests_number | Counter | 已接收的请求总数 | 个 |
| fastdeploy:send_cache_failed_num | Counter | 发送缓存失败的总次数 | 个 |
| fastdeploy:first_token_latency | Gauge | 最近一次生成首token耗时 | 秒 |
| fastdeploy:infer_latency | Gauge | 最近一次生成单个token的耗时 | 秒 |

| fastdeploy:prefix_cache_token_num | Counter | Total number of cached tokens | Count |
| fastdeploy:prefix_gpu_cache_token_num | Counter | Total number of cached tokens on GPU | Count |
| fastdeploy:prefix_cpu_cache_token_num | Counter | Total number of cached tokens on CPU | Count |
| fastdeploy:batch_size | Gauge | Real batch size during inference | Count |
| fastdeploy:max_batch_size | Gauge | Maximum batch size determined when service started | Count |
| fastdeploy:available_gpu_block_num | Gauge | Number of available gpu blocks in cache, including prefix caching blocks that are not officially released | Count |
| fastdeploy:free_gpu_block_num | Gauge | Number of free blocks in cache | Count |
| fastdeploy:max_gpu_block_num | Gauge | Number of total blocks determined when service started| Count |
| available_gpu_resource | Gauge | Available blocks percentage, i.e. available_gpu_block_num / max_gpu_block_num | Count |
| fastdeploy:requests_number | Counter | Total number of requests received | Count |
| fastdeploy:send_cache_failed_num | Counter | Total number of failures of sending cache | Count |
| fastdeploy:first_token_latency | Gauge | Latest time to generate first token in seconds | Seconds |
| fastdeploy:infer_latency | Gauge | Latest time to generate one token in seconds | Seconds |

@paddle-bot
Copy link

paddle-bot bot commented Sep 11, 2025

Thanks for your contribution!

@paddle-bot paddle-bot bot added the contributor External developers label Sep 11, 2025
@CLAassistant
Copy link

CLAassistant commented Sep 11, 2025

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ qwes5s5
❌ K11OntheBoat


K11OntheBoat seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@qwes5s5 qwes5s5 changed the title adjust md update metrics markdown file Sep 11, 2025
@Jiang-Jia-Jun Jiang-Jia-Jun changed the title update metrics markdown file [metrics] update metrics markdown file Sep 12, 2025
@Jiang-Jia-Jun Jiang-Jia-Jun merged commit 58e0785 into PaddlePaddle:develop Sep 12, 2025
24 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants