1.99.1 Cannot Detect VRAM Properly with AMD Integrated GPU on Vulkan

[lovenemesis]

Describe the Issue
Upon upgraded to 1.99.1 release, koboldcpp cannot reliably detect the shared VRAM via UMA with Vulkan backend, with --gpulayers -1 argument

It was alright on 1.98.1 release.

Additional Information:

With 1.99.1 release:

Welcome to KoboldCpp - Version 1.99.1
Loading Chat Completions Adapter: /tmp/_MEIam6bjU/kcpp_adapters/AutoGuess.json
Chat Completions Adapter Loaded
Detected AMD GPU VRAM from rocminfo: [('AMD Radeon 780M Graphics', '23983')] MB
Unable to detect VRAM, please set layers manually.
System: Linux #1 SMP PREEMPT_DYNAMIC Thu Sep 11 17:46:54 UTC 2025 x86_64 
Detected Available GPU Memory: 0 MB
Detected Available RAM: 43351 MB
Initializing dynamic library: koboldcpp_vulkan.so
...
load_tensors: loading model tensors, this can take a while... (mmap = false)
load_tensors: relocated tensors: 1 of 399
load_tensors: offloading 0 repeating layers to GPU
load_tensors: offloaded 0/37 layers to GPU
load_tensors:  Vulkan_Host model buffer size =  2750.40 MiB
load_tensors:          CPU model buffer size =   304.28 MiB

With 1.98.1 release:

Welcome to KoboldCpp - Version 1.98.1
Loading Chat Completions Adapter: /tmp/_MEId9PzAz/kcpp_adapters/AutoGuess.json
Chat Completions Adapter Loaded
Auto Recommended GPU Layers: 39
System: Linux #1 SMP PREEMPT_DYNAMIC Thu Sep 11 17:46:54 UTC 2025 x86_64 
Detected Available GPU Memory: 16384 MB
Unable to determine available RAM
Initializing dynamic library: koboldcpp_vulkan.so
...
load_tensors: loading model tensors, this can take a while... (mmap = false)
load_tensors: relocated tensors: 1 of 399
load_tensors: offloading 36 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 37/37 layers to GPU
load_tensors:      Vulkan0 model buffer size =  2749.97 MiB
load_tensors:          CPU model buffer size =   304.28 MiB

Note the differences in launching and finally offloaded GPU layers.

Yes, I could manually specify the number of layers ( in this case,`--gpulayers 37) to offload with 1.99.1, and it works just as before.
But it renders the purpose of autodetection a bit off.

[LostRuins]

Try 1.99.2, should be fixed

[lovenemesis]

Confirmed as fixed in 1.99.3 release.

[DerRehberg]

I got the problem with newest version 1.100 on arch with vulkan-radeon set gpu layers to -1 30 and 31 too and it didn't work, worked before tho. Is the problem the two new options from wbruna? RX 580 here

[wbruna]

I got the problem with newest version 1.100 on arch with vulkan-radeon set gpu layers to -1 30 and 31 too and it didn't work, worked before tho. Is the problem the two new options from wbruna? RX 580 here

Hmm, I don't remember changing stuff that would interfere directly with memory detection (although some sd options could change the amount of estimated memory usage). Could you give us more details about your issue? (and maybe open a separate one, since the RX 580 is a discrete GPU).

[LostRuins]

If the VRAM is not detected, please run rocminfo and check if the output is correct

[DerRehberg]

@LostRuins @wbruna using vulkan and doesn't detect at all

[DerRehberg]

@wbruna I know someone patched this, you guys are just mad because I use open radeon driver on linux because there's no official support anymore because you guys hate me for it because I don't wanna buy an nvidia card

[lovenemesis]

For the record, I am having Zero issue with VRAM detection with 1.99.3, 1.99.4 and 1.100.1 releases on Ryzen 5700U, 6850HS, 7840U, AI Max 370 APUs with iGPU respectively, Win11 and Fedora Linux.

If I may say, @LostRuins and other contributors to Koboldcpp would be the best folks taking care of the needs of local LLM for non-nvidia users.

[LostRuins]

@DerRehberg the detected VRAM is entirely based on what your system reports in rocminfo. You can see that from the code. Can you please try running the command and letting us know what it reports?

[DerRehberg]

@LostRuins That's retarded, i use vulkan backend, i don't use rocm and it worked before

[DerRehberg]

When I say I use an rx580 with vulkan-radeon i maybe should look out for an other gui when someone says i should look into rocminfo

[DerRehberg]

I also remember mentioning how one can fix out of memory error probably with card with more than 4gb vram on vulkan but someone wrote one file especially retarded and said he doesn't know what i want when someone even applied such a patch which i linked.

[DerRehberg]

Maybe this is more serious even then koboldcpp now because there's some wording inside the link which i feel when getting the answer i got on my problem while also stating once someone fixed the above 4gb vram not used like problem stable-diffusion.fsociety.hu/

[DerRehberg]

Next, why don't you write koboldcpp that it can only use cuda and rocm with avx2 and ddr5 memory while just using the graphics card instead of ram with spamming the stable diffusion cpp devs that they are retarded because in ur eyes vulkan backend is rocm and then spam amd that you want rocm2 with 50gbs of size