Add docker model bench command for benchmarking model inference

[Copilot]

Adds docker model bench MODEL to measure tokens per second across concurrent request levels (1, 2, 4, 8 by default), providing hyperfine-like output with timing statistics.

Features

• Concurrent request benchmarking with configurable parallelism levels
• Token throughput measurement (tokens/sec) from OpenAI-compatible streaming responses
• Statistical output: mean ± stddev, min/max range per concurrency level
• Summary table with optimal concurrency recommendation
• Warm-up run before measurements

Usage

# Default benchmark
docker model bench llama3.2

# Custom prompt and concurrency
docker model bench llama3.2 --prompt "Explain quantum computing" -c 1,4,8

# More samples per level
docker model bench llama3.2 -n 5

Changes

• cmd/cli/commands/bench.go - Command implementation with concurrent request handling and stats aggregation
• cmd/cli/commands/bench_test.go - Unit tests for stats calculation and formatting
• cmd/cli/commands/root.go - Register command

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• cloud.google.com
  • Triggering command: /update-job-proxy /update-job-proxy -ifaceassert -nilfunc ux_amd64/asm -ato�� -bool thored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com> p/bin/as -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.24.10//tmp/go-link-965375800/a.out -ato�� in.so /lto-wrapper nfig/composer/vendor/bin/git -errorsas 09ad6033 -nilfunc /opt/hostedtoolcache/go/1.24.10/x64/pkg/tool/lin-f (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-a81794aec9f8 -j DROP cbb10d46c06b9a96--stdin log e/git-remote-htt--alternate-refs -n1 --format=format:cat-file (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT fetch e/git-remote-https --depth=1 origin cfc5c702:refs/dublob e/git-remote-httcb61b32ac6fe84d34b81730175f91965e43d0f90:go.mod ls-r�� om/yuin/goldmark REDACTED /bin/git 3b08be06 om/docker/go-win--wait e/git git (dns block)
• dario.cat
  • Triggering command: /update-job-proxy /update-job-proxy -ifaceassert -nilfunc ux_amd64/asm -ato�� -bool thored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com> p/bin/as -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.24.10//tmp/go-link-965375800/a.out -ato�� in.so /lto-wrapper nfig/composer/vendor/bin/git -errorsas 09ad6033 -nilfunc /opt/hostedtoolcache/go/1.24.10/x64/pkg/tool/lin-f (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-a81794aec9f8 -j DROP cbb10d46c06b9a96--stdin log e/git-remote-htt--alternate-refs -n1 --format=format:cat-file (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT fetch e/git-remote-https --depth=1 origin cfc5c702:refs/dublob e/git-remote-httcb61b32ac6fe84d34b81730175f91965e43d0f90:go.mod ls-r�� om/yuin/goldmark REDACTED /bin/git 3b08be06 om/docker/go-win--wait e/git git (dns block)
• go.googlesource.com
  • Triggering command: /update-job-proxy /update-job-proxy -ifaceassert -nilfunc ux_amd64/asm -ato�� -bool thored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com> p/bin/as -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.24.10//tmp/go-link-965375800/a.out -ato�� in.so /lto-wrapper nfig/composer/vendor/bin/git -errorsas 09ad6033 -nilfunc /opt/hostedtoolcache/go/1.24.10/x64/pkg/tool/lin-f (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-a81794aec9f8 -j DROP cbb10d46c06b9a96--stdin log e/git-remote-htt--alternate-refs -n1 --format=format:cat-file (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT fetch e/git-remote-https --depth=1 origin cfc5c702:refs/dublob e/git-remote-httcb61b32ac6fe84d34b81730175f91965e43d0f90:go.mod ls-r�� om/yuin/goldmark REDACTED /bin/git 3b08be06 om/docker/go-win--wait e/git git (dns block)
• go.opentelemetry.io
  • Triggering command: /update-job-proxy /update-job-proxy -ifaceassert -nilfunc ux_amd64/asm -ato�� -bool thored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com> p/bin/as -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.24.10//tmp/go-link-965375800/a.out -ato�� in.so /lto-wrapper nfig/composer/vendor/bin/git -errorsas 09ad6033 -nilfunc /opt/hostedtoolcache/go/1.24.10/x64/pkg/tool/lin-f (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-a81794aec9f8 -j DROP cbb10d46c06b9a96--stdin log e/git-remote-htt--alternate-refs -n1 --format=format:cat-file (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT fetch e/git-remote-https --depth=1 origin cfc5c702:refs/dublob e/git-remote-httcb61b32ac6fe84d34b81730175f91965e43d0f90:go.mod ls-r�� om/yuin/goldmark REDACTED /bin/git 3b08be06 om/docker/go-win--wait e/git git (dns block)
• go.uber.org
  • Triggering command: /update-job-proxy /update-job-proxy -ifaceassert -nilfunc ux_amd64/asm -ato�� -bool thored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com> p/bin/as -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.24.10//tmp/go-link-965375800/a.out -ato�� in.so /lto-wrapper nfig/composer/vendor/bin/git -errorsas 09ad6033 -nilfunc /opt/hostedtoolcache/go/1.24.10/x64/pkg/tool/lin-f (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-a81794aec9f8 -j DROP cbb10d46c06b9a96--stdin log e/git-remote-htt--alternate-refs -n1 --format=format:cat-file (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT fetch e/git-remote-https --depth=1 origin cfc5c702:refs/dublob e/git-remote-httcb61b32ac6fe84d34b81730175f91965e43d0f90:go.mod ls-r�� om/yuin/goldmark REDACTED /bin/git 3b08be06 om/docker/go-win--wait e/git git (dns block)
• go.yaml.in
  • Triggering command: /update-job-proxy /update-job-proxy -ifaceassert -nilfunc ux_amd64/asm -ato�� -bool thored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com> p/bin/as -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.24.10//tmp/go-link-965375800/a.out -ato�� in.so /lto-wrapper nfig/composer/vendor/bin/git -errorsas 09ad6033 -nilfunc /opt/hostedtoolcache/go/1.24.10/x64/pkg/tool/lin-f (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-a81794aec9f8 -j DROP cbb10d46c06b9a96--stdin log e/git-remote-htt--alternate-refs -n1 --format=format:cat-file (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT fetch e/git-remote-https --depth=1 origin cfc5c702:refs/dublob e/git-remote-httcb61b32ac6fe84d34b81730175f91965e43d0f90:go.mod ls-r�� om/yuin/goldmark REDACTED /bin/git 3b08be06 om/docker/go-win--wait e/git git (dns block)
• gonum.org
  • Triggering command: /update-job-proxy /update-job-proxy -ifaceassert -nilfunc ux_amd64/asm -ato�� -bool thored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com> p/bin/as -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.24.10//tmp/go-link-965375800/a.out -ato�� in.so /lto-wrapper nfig/composer/vendor/bin/git -errorsas 09ad6033 -nilfunc /opt/hostedtoolcache/go/1.24.10/x64/pkg/tool/lin-f (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-a81794aec9f8 -j DROP cbb10d46c06b9a96--stdin log e/git-remote-htt--alternate-refs -n1 --format=format:cat-file (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT fetch e/git-remote-https --depth=1 origin cfc5c702:refs/dublob e/git-remote-httcb61b32ac6fe84d34b81730175f91965e43d0f90:go.mod ls-r�� om/yuin/goldmark REDACTED /bin/git 3b08be06 om/docker/go-win--wait e/git git (dns block)
• google.golang.org
  • Triggering command: /update-job-proxy /update-job-proxy -ifaceassert -nilfunc ux_amd64/asm -ato�� -bool thored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com> p/bin/as -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.24.10//tmp/go-link-965375800/a.out -ato�� in.so /lto-wrapper nfig/composer/vendor/bin/git -errorsas 09ad6033 -nilfunc /opt/hostedtoolcache/go/1.24.10/x64/pkg/tool/lin-f (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-a81794aec9f8 -j DROP cbb10d46c06b9a96--stdin log e/git-remote-htt--alternate-refs -n1 --format=format:cat-file (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT fetch e/git-remote-https --depth=1 origin cfc5c702:refs/dublob e/git-remote-httcb61b32ac6fe84d34b81730175f91965e43d0f90:go.mod ls-r�� om/yuin/goldmark REDACTED /bin/git 3b08be06 om/docker/go-win--wait e/git git (dns block)
• gopkg.in
  • Triggering command: /update-job-proxy /update-job-proxy -ifaceassert -nilfunc ux_amd64/asm -ato�� -bool thored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com> p/bin/as -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.24.10//tmp/go-link-965375800/a.out -ato�� in.so /lto-wrapper nfig/composer/vendor/bin/git -errorsas 09ad6033 -nilfunc /opt/hostedtoolcache/go/1.24.10/x64/pkg/tool/lin-f (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-a81794aec9f8 -j DROP cbb10d46c06b9a96--stdin log e/git-remote-htt--alternate-refs -n1 --format=format:cat-file (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT fetch e/git-remote-https --depth=1 origin cfc5c702:refs/dublob e/git-remote-httcb61b32ac6fe84d34b81730175f91965e43d0f90:go.mod ls-r�� om/yuin/goldmark REDACTED /bin/git 3b08be06 om/docker/go-win--wait e/git git (dns block)
• gotest.tools
  • Triggering command: /update-job-proxy /update-job-proxy -ifaceassert -nilfunc ux_amd64/asm -ato�� -bool thored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com> p/bin/as -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.24.10//tmp/go-link-965375800/a.out -ato�� in.so /lto-wrapper nfig/composer/vendor/bin/git -errorsas 09ad6033 -nilfunc /opt/hostedtoolcache/go/1.24.10/x64/pkg/tool/lin-f (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-a81794aec9f8 -j DROP cbb10d46c06b9a96--stdin log e/git-remote-htt--alternate-refs -n1 --format=format:cat-file (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT fetch e/git-remote-https --depth=1 origin cfc5c702:refs/dublob e/git-remote-httcb61b32ac6fe84d34b81730175f91965e43d0f90:go.mod ls-r�� om/yuin/goldmark REDACTED /bin/git 3b08be06 om/docker/go-win--wait e/git git (dns block)
• howett.net
  • Triggering command: /update-job-proxy /update-job-proxy -ifaceassert -nilfunc ux_amd64/asm -ato�� -bool thored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com> p/bin/as -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.24.10//tmp/go-link-965375800/a.out -ato�� in.so /lto-wrapper nfig/composer/vendor/bin/git -errorsas 09ad6033 -nilfunc /opt/hostedtoolcache/go/1.24.10/x64/pkg/tool/lin-f (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy -o br-a81794aec9f8 -j DROP cbb10d46c06b9a96--stdin log e/git-remote-htt--alternate-refs -n1 --format=format:cat-file (dns block)
  • Triggering command: /update-job-proxy /update-job-proxy conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT fetch e/git-remote-https --depth=1 origin cfc5c702:refs/dublob e/git-remote-httcb61b32ac6fe84d34b81730175f91965e43d0f90:go.mod ls-r�� om/yuin/goldmark REDACTED /bin/git 3b08be06 om/docker/go-win--wait e/git git (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Create "docker model bench MODEL"</issue_title>
<issue_description>It should be able to output the Tokens per Second of any give model with 1, 2, 4 and 8 concurrent requests. Provide a hyperfine-like experience.</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Create "docker model bench MODEL" docker/model-runner#480

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.