Llama.cpp 30B runs with only 6GB of RAM now - update ?

[magicmars35]

Llama.cpp 30B runs with only 6GB of RAM now :

ggml-org#613

[anotherwanwanyi]

It appears that this was just a misreading of how memory usage was being reported.

Take a look at jart's reply.
You still need 20GB of RAM.

https://news.ycombinator.com/item?id=35400066

[Openaicn]

please make clear 20GB RAM from GPU or motherboard?

[anotherwanwanyi]

@Openaicn Llama.cpp runs on the CPU.