spec : save the dynamic/static ngram cache file

[petersid2022]

Overview

• When we select the COMMON_SPECULATIVE_TYPE_NGRAM_CACHE speculative implementation we create a new common_speculative_state_ngram_cache state using create_state_ngram_cache, where we instantiate the new state by specifying various parameters (e.g, n_draft, save_static and save_dynamic) by hardcoding them.

• Instead we extend common_params_speculative to include those options as well.

• An attempt was also made to implement the save_static / save_dynamic behavior by calling common_ngram_cache_save on object destruction.

Additional information

Add self‑speculative decoding (no draft model required)#18471

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: NO

[petersid2022]

This is probably the wrong way of going about this, but I am curious if the same concept of m-gram speculative tokens can be applied in the ngram-cache implemetantion

[ggerganov]

The new parameters are never populated. Did you test this change? What is the goal of this PR?

[petersid2022]

first of all, thanks for taking the time to review my PR!

TBH, my initial scope for this PR (after coming across the TODO on line 930 of common/speculative.cpp) was to move the save_[state,dynamic] bools from inside create_state_ngram_cache to the more centralized common.h, as the TODO instructed. That would allow for those options to be configurable from the user. Though, after thinking about it some more I decided to drop the booleans and instead rely on the existence or not of path_[static,dynamic] parameter.

The way I went about testing my changes was using the below command:

./build/bin/llama-server --port 1234 -m ~/models/Qwen3.5-9B-Q8_0.gguf --spec-type ngram-cache --lookup-cache-static ~/static.bin --lookup-cache-dynamic ~/dynamic.bin

P.S: In the common_speculative_state_ngram_cache constructor, I noticed that if the common_ngram_cache_load call fails on either one of them we abort execution. Could we instead call common_ngram_cache_save() so that we create the file?