Skip to content

cuda : display total and free VRAM capacity during device initialization#20185

Merged
am17an merged 1 commit intoggml-org:masterfrom
tehsiuhuang:feat/cuda-vram-log
Mar 9, 2026
Merged

cuda : display total and free VRAM capacity during device initialization#20185
am17an merged 1 commit intoggml-org:masterfrom
tehsiuhuang:feat/cuda-vram-log

Conversation

@tehsiuhuang
Copy link
Copy Markdown
Contributor

@tehsiuhuang tehsiuhuang commented Mar 7, 2026

  • While running the benchark, realized that no VRAM info and had to check in another console
  • Helps users verify hardware constraints directly from the log output even when running the benchmark
  • Get total VRAM via cudaGetDeviceProperties and free memory via cudaMemGetInfo.
  • Tested on Alienware m16 R2 (RTX 4070 Laptop GPU).

Test Result

llama-bench & test-backend-ops
image

image image

ctest
1 failure (# 14) not related to my change
image

image

@github-actions github-actions Bot added Nvidia GPU Issues specific to Nvidia GPUs ggml changes relating to the ggml tensor library for machine learning labels Mar 7, 2026
@am17an am17an merged commit 5f4cdac into ggml-org:master Mar 9, 2026
78 checks passed
@tehsiuhuang
Copy link
Copy Markdown
Contributor Author

@am17an Thanks for your review!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning Nvidia GPU Issues specific to Nvidia GPUs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants