Skip to content

Add Model column to collect-results summary table#195

Merged
functionstackx merged 2 commits intomainfrom
copilot/add-model-name-to-results-table-again
Nov 6, 2025
Merged

Add Model column to collect-results summary table#195
functionstackx merged 2 commits intomainfrom
copilot/add-model-name-to-results-table-again

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Nov 6, 2025

When collecting results across all experiments (not filtered by experiment name), the summary table lacked model identification, making it impossible to distinguish which model produced each result.

Changes

  • Added Model column as the first column in utils/summarize.py summary table
  • Updated sort order to group by model first: model → hardware → framework → precision → TP → EP → concurrency
  • Handles missing data gracefully with 'unknown' default for backward compatibility

Example Output

| Model | Hardware | Framework | Precision | TP | EP | DP Attention | Conc | TTFT (ms) | ... |
| :-: | :-: | :-: | :-: | :-: | :-: | :-: | :-: | :-: | ... |
| deepseek-ai/DeepSeek-R1-Distill-Llama-70B | H200 | SGLANG | FP8 | 8 | 8 | false | 4 | 13.9000 | ... |
| meta-llama/Llama-3.3-70b-Instruct | B200 | TRT | FP4 | 8 | 1 | false | 4 | 11.5000 | ... |
| meta-llama/Llama-3.3-70b-Instruct | H200 | VLLM | FP8 | 8 | 1 | false | 4 | 30.5000 | ... |

Model data already captured by process_result.py from benchmark results.

Original prompt

This section details on the original issue you should resolve

<issue_title>Add model name in some capacity to summarize results table</issue_title>
<issue_description>In InferenceMAX/InferenceMAX#167, we realized that the changes made to collect results warrant a need to display model in the table in the case that we are not collecting by experiment name but rather all results at once.

Image</issue_description>

Comments on the Issue (you are @copilot in this section)

  • Fixes InferenceMAX/InferenceMAX#193

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com>
Copilot AI changed the title [WIP] Add model name to summarize results table Add Model column to collect-results summary table Nov 6, 2025
Copilot AI requested a review from cquil11 November 6, 2025 21:23
@functionstackx functionstackx marked this pull request as ready for review November 6, 2025 21:44
@functionstackx functionstackx requested a review from a team as a code owner November 6, 2025 21:44
Copilot AI review requested due to automatic review settings November 6, 2025 21:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a Model column to the results summary table in utils/summarize.py to distinguish which model produced each result when collecting results across all experiments. Previously, when not filtering by experiment name, users couldn't identify the model associated with each benchmark result.

Key changes:

  • Added Model as the first column in the summary table
  • Updated sort order to group results by model first, then hardware, framework, precision, TP, EP, and concurrency
  • Handles missing model data with 'unknown' default for backward compatibility

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown
Contributor

@functionstackx functionstackx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@functionstackx functionstackx merged commit 9765f60 into main Nov 6, 2025
15 of 16 checks passed
@functionstackx functionstackx deleted the copilot/add-model-name-to-results-table-again branch November 6, 2025 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add model name in some capacity to summarize results table

4 participants