llama-fit-params: keep explicit --ctx-size 0#19070
llama-fit-params: keep explicit --ctx-size 0#19070JohannesGaessler merged 1 commit intoggml-org:masterfrom
Conversation
ggerganov
left a comment
There was a problem hiding this comment.
@JohannesGaessler On a related note: for LlamaBarn we need a mechanism to query how much memory would be needed for a context of size -c N. Would you accept if we extend llama-fit-params to provide such information and if yes, can you recommend a way to implement it in terms of llama-fit-params UX and libllama API modifications (if needed)?
|
As of right now |
Fixes #18376 , alternative to #18567 . If the user explicitly sets
--ctx-size 0, set the minimum context size toUINT32_MAXto preventllama_params_fitfrom reducing it.