Eval bug: Qwen3.5-35b-a3b unloads automatically with no errors

[anubhavgupta]

Name and Version

version: 8179 (ecbcb7e)

Operating systems

Windows

GGML backends

CUDA

Hardware

5090 Mobile (24GB Vram), cuda 13.1.

Models

unsloth's Qwen3.5-35B-A3B-UD-Q4_K_M.gguf

Problem description & steps to reproduce

args that I use to launch llama.cpp:
-m C:\Users\anubh.lmstudio\models\lmstudio-community\Qwen3.5-35b-a3b-GGUF\Qwen3.5-35B-A3B-UD-Q4_K_M.gguf -ngl 99 -t 23 --temp 0.6 --top-k 20 --top-p 0.95 --parallel 1 --mlock --swa-full -c 200000 -ctk q8_0 -ctv q8_0 -fa on --jinja --reasoning-budget 0 --host 0.0.0.0 -fit off

The server stop suddenly without any error logs, but the last line is always " load: - found better prompt with f_keep".

This happens very frequently when it is used with Claude Code and less frequently with Opencode.
Repo Step:
1 Load the model
2 Configure Claude code to run it with the model locally.
3 go inside any codebase and delete any existing CLAUED.md file
4 run /init inside claude code.
5. delete any generated CLAUED.md file
6. repeat #4 and #5 2-3 times

Expected:

• The model should keep running

Actual

• The model unloads with the last line always like " found better prompt with f_keep..."

Please check logs sections for attached logs.

---

[Not sure if it would help in debugging] Additionally, when I use a UI wrapper(https://github.com/anubhavgupta/llama-cpp-manager) to launch the app then I get an exit code as well:

srv   prompt_save:  - saving prompt with length 4207, total state size = 106.513 MiB

srv          load:  - looking for better prompt, base f_keep = 0.001, sim = 0.000
srv          load:  - found better prompt with f_keep = 0.543, sim = 0.994

Server process exited with code 3221225477 // <----------------THIS

a quick chatGPT search suggested it to be: 0xC0000005 → STATUS_ACCESS_VIOLATION

First Bad Commit

Getting it since the launch day of Qwen3.5 35b a3b.

Relevant log output

Logs

`
slot print_timing: id  0 | task 15417 |
prompt eval time =     333.66 ms /   208 tokens (    1.60 ms per token,   623.39 tokens per second)
       eval time =   29940.07 ms /  4000 tokens (    7.49 ms per token,   133.60 tokens per second)
      total time =   30273.74 ms /  4208 tokens
slot      release: id  0 | task 15417 | stop processing: n_tokens = 4207, truncated = 0
srv  update_slots: all slots are idle

srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200

srv  params_from_: Chat format: peg-constructed

slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = 837581953
srv  get_availabl: updating prompt cache

srv   prompt_save:  - saving prompt with length 4207, total state size = 106.513 MiB

srv          load:  - looking for better prompt, base f_keep = 0.001, sim = 0.000
srv          load:  - found better prompt with f_keep = 0.543, sim = 0.994

[xantrk]

Does this happen following compaction? #19945 (comment)
I think it's the same issue as above. Only happens for qwen 3.5 for me as well. I found it is triggerable with llama-bench tests when there's at least ~10k context is initialized with -d argument, it fails to handle loading and unloading context and throws an error in Windows event logger.

[anubhavgupta]

No, compaction is not triggered. usually happens after 2-3 prompts.

eg:

1. open claude code
2. type "hello"
3. run "/init"
4. close claude code
5. repeat steps 2 and 3. (it will crash after step 2)

Full logs

PS C:\projects\bin-llama-cpp\llama-b8179-bin-win-cuda-13.1-x64> ./llama-server.exe -m C:\Users\anubh\.lmstudio\models\lmstudio-community\Qwen3.5-35b-a3b-GGUF\Qwen3.5-35B-A3B-UD-Q4_K_M.gguf -ngl 99 -t 23 --temp 0.6 --top-k 20 --top-p 0.95 --parallel 1 --mlock -c 200000 -ctk q8_0 -ctv q8_0 -fa on --jinja --reasoning-budget 0 --host 0.0.0.0 -fit off
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 5090 Laptop GPU, compute capability 12.0, VMM: yes
load_backend: loaded CUDA backend from C:\projects\bin-llama-cpp\llama-b8179-bin-win-cuda-13.1-x64\ggml-cuda.dll
load_backend: loaded RPC backend from C:\projects\bin-llama-cpp\llama-b8179-bin-win-cuda-13.1-x64\ggml-rpc.dll
load_backend: loaded CPU backend from C:\projects\bin-llama-cpp\llama-b8179-bin-win-cuda-13.1-x64\ggml-cpu-alderlake.dll
build: 8179 (ecbcb7ea9) with Clang 19.1.5 for Windows x86_64
system info: n_threads = 23, n_threads_batch = 23, total_threads = 24

system_info: n_threads = 23 (n_threads_batch = 23) / 24 | CUDA : ARCHS = 750,800,860,890,1200,1210 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | BLACKWELL_NATIVE_FP4 = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |

Running without SSL
init: using 23 threads for HTTP server
start: binding port with default address family
main: loading model
srv    load_model: loading model 'C:\Users\anubh\.lmstudio\models\lmstudio-community\Qwen3.5-35b-a3b-GGUF\Qwen3.5-35B-A3B-UD-Q4_K_M.gguf'
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GeForce RTX 5090 Laptop GPU) (0000:02:00.0) - 23119 MiB free
llama_model_loader: loaded meta data with 52 key-value pairs and 733 tensors from C:\Users\anubh\.lmstudio\models\lmstudio-community\Qwen3.5-35b-a3b-GGUF\Qwen3.5-35B-A3B-UD-Q4_K_M.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = qwen35moe
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 20
llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.950000
llama_model_loader: - kv   4:                      general.sampling.temp f32              = 1.000000
llama_model_loader: - kv   5:                               general.name str              = Qwen3.5-35B-A3B
llama_model_loader: - kv   6:                           general.basename str              = Qwen3.5-35B-A3B
llama_model_loader: - kv   7:                       general.quantized_by str              = Unsloth
llama_model_loader: - kv   8:                         general.size_label str              = 35B-A3B
llama_model_loader: - kv   9:                            general.license str              = apache-2.0
llama_model_loader: - kv  10:                       general.license.link str              = https://huggingface.co/Qwen/Qwen3.5-3...
llama_model_loader: - kv  11:                           general.repo_url str              = https://huggingface.co/unsloth
llama_model_loader: - kv  12:                   general.base_model.count u32              = 1
llama_model_loader: - kv  13:                  general.base_model.0.name str              = Qwen3.5 35B A3B
llama_model_loader: - kv  14:          general.base_model.0.organization str              = Qwen
llama_model_loader: - kv  15:              general.base_model.0.repo_url str              = https://huggingface.co/Qwen/Qwen3.5-3...
llama_model_loader: - kv  16:                               general.tags arr[str,2]       = ["unsloth", "image-text-to-text"]
llama_model_loader: - kv  17:                      qwen35moe.block_count u32              = 40
llama_model_loader: - kv  18:                   qwen35moe.context_length u32              = 262144
llama_model_loader: - kv  19:                 qwen35moe.embedding_length u32              = 2048
llama_model_loader: - kv  20:             qwen35moe.attention.head_count u32              = 16
llama_model_loader: - kv  21:          qwen35moe.attention.head_count_kv u32              = 2
llama_model_loader: - kv  22:          qwen35moe.rope.dimension_sections arr[i32,4]       = [11, 11, 10, 0]
llama_model_loader: - kv  23:                   qwen35moe.rope.freq_base f32              = 10000000.000000
llama_model_loader: - kv  24: qwen35moe.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  25:                     qwen35moe.expert_count u32              = 256
llama_model_loader: - kv  26:                qwen35moe.expert_used_count u32              = 8
llama_model_loader: - kv  27:             qwen35moe.attention.key_length u32              = 256
llama_model_loader: - kv  28:           qwen35moe.attention.value_length u32              = 256
llama_model_loader: - kv  29:       qwen35moe.expert_feed_forward_length u32              = 512
llama_model_loader: - kv  30: qwen35moe.expert_shared_feed_forward_length u32              = 512
llama_model_loader: - kv  31:                  qwen35moe.ssm.conv_kernel u32              = 4
llama_model_loader: - kv  32:                   qwen35moe.ssm.state_size u32              = 128
llama_model_loader: - kv  33:                  qwen35moe.ssm.group_count u32              = 16
llama_model_loader: - kv  34:               qwen35moe.ssm.time_step_rank u32              = 32
llama_model_loader: - kv  35:                   qwen35moe.ssm.inner_size u32              = 4096
llama_model_loader: - kv  36:          qwen35moe.full_attention_interval u32              = 4
llama_model_loader: - kv  37:             qwen35moe.rope.dimension_count u32              = 64
llama_model_loader: - kv  38:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  39:                         tokenizer.ggml.pre str              = qwen35
llama_model_loader: - kv  40:                      tokenizer.ggml.tokens arr[str,248320]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  41:                  tokenizer.ggml.token_type arr[i32,248320]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  42:                      tokenizer.ggml.merges arr[str,247587]  = ["─á ─á", "─á─á ─á─á", "i n", "─á t",...
llama_model_loader: - kv  43:                tokenizer.ggml.eos_token_id u32              = 248046
llama_model_loader: - kv  44:            tokenizer.ggml.padding_token_id u32              = 248055
llama_model_loader: - kv  45:                    tokenizer.chat_template str              = {%- set image_count = namespace(value...
llama_model_loader: - kv  46:               general.quantization_version u32              = 2
llama_model_loader: - kv  47:                          general.file_type u32              = 7
llama_model_loader: - kv  48:                      quantize.imatrix.file str              = Qwen3.5-35B-A3B-GGUF/Qwen_Qwen3.5-35B...
llama_model_loader: - kv  49:                   quantize.imatrix.dataset str              = /training_dir/calibration_datav5.txt
llama_model_loader: - kv  50:             quantize.imatrix.entries_count u32              = 510
llama_model_loader: - kv  51:              quantize.imatrix.chunks_count u32              = 802
llama_model_loader: - type  f32:  301 tensors
llama_model_loader: - type q4_K:  190 tensors
llama_model_loader: - type q5_K:   82 tensors
llama_model_loader: - type q6_K:   80 tensors
llama_model_loader: - type bf16:   80 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q8_0
print_info: file size   = 18.48 GiB (4.58 BPW)
load: 0 unused tokens
load: printing all EOG tokens:
load:   - 248044 ('<|endoftext|>')
load:   - 248046 ('<|im_end|>')
load:   - 248063 ('<|fim_pad|>')
load:   - 248064 ('<|repo_name|>')
load:   - 248065 ('<|file_sep|>')
load: special tokens cache size = 33
load: token to piece cache size = 1.7581 MB
print_info: arch                  = qwen35moe
print_info: vocab_only            = 0
print_info: no_alloc              = 0
print_info: n_ctx_train           = 262144
print_info: n_embd                = 2048
print_info: n_embd_inp            = 2048
print_info: n_layer               = 40
print_info: n_head                = 16
print_info: n_head_kv             = 2
print_info: n_rot                 = 64
print_info: n_swa                 = 0
print_info: is_swa_any            = 0
print_info: n_embd_head_k         = 256
print_info: n_embd_head_v         = 256
print_info: n_gqa                 = 8
print_info: n_embd_k_gqa          = 512
print_info: n_embd_v_gqa          = 512
print_info: f_norm_eps            = 0.0e+00
print_info: f_norm_rms_eps        = 1.0e-06
print_info: f_clamp_kqv           = 0.0e+00
print_info: f_max_alibi_bias      = 0.0e+00
print_info: f_logit_scale         = 0.0e+00
print_info: f_attn_scale          = 0.0e+00
print_info: n_ff                  = 0
print_info: n_expert              = 256
print_info: n_expert_used         = 8
print_info: n_expert_groups       = 0
print_info: n_group_used          = 0
print_info: causal attn           = 1
print_info: pooling type          = 0
print_info: rope type             = 40
print_info: rope scaling          = linear
print_info: freq_base_train       = 10000000.0
print_info: freq_scale_train      = 1
print_info: n_ctx_orig_yarn       = 262144
print_info: rope_yarn_log_mul     = 0.0000
print_info: rope_finetuned        = unknown
print_info: mrope sections        = [11, 11, 10, 0]
print_info: ssm_d_conv            = 4
print_info: ssm_d_inner           = 4096
print_info: ssm_d_state           = 128
print_info: ssm_dt_rank           = 32
print_info: ssm_n_group           = 16
print_info: ssm_dt_b_c_rms        = 0
print_info: model type            = ?B
print_info: model params          = 34.66 B
print_info: general.name          = Qwen3.5-35B-A3B
print_info: vocab type            = BPE
print_info: n_vocab               = 248320
print_info: n_merges              = 247587
print_info: BOS token             = 11 ','
print_info: EOS token             = 248046 '<|im_end|>'
print_info: EOT token             = 248046 '<|im_end|>'
print_info: PAD token             = 248055 '<|vision_pad|>'
print_info: LF token              = 198 '─è'
print_info: FIM PRE token         = 248060 '<|fim_prefix|>'
print_info: FIM SUF token         = 248062 '<|fim_suffix|>'
print_info: FIM MID token         = 248061 '<|fim_middle|>'
print_info: FIM PAD token         = 248063 '<|fim_pad|>'
print_info: FIM REP token         = 248064 '<|repo_name|>'
print_info: FIM SEP token         = 248065 '<|file_sep|>'
print_info: EOG token             = 248044 '<|endoftext|>'
print_info: EOG token             = 248046 '<|im_end|>'
print_info: EOG token             = 248063 '<|fim_pad|>'
print_info: EOG token             = 248064 '<|repo_name|>'
print_info: EOG token             = 248065 '<|file_sep|>'
print_info: max token length      = 256
load_tensors: loading model tensors, this can take a while... (mmap = true, direct_io = false)
load_tensors: offloading output layer to GPU
load_tensors: offloading 39 repeating layers to GPU
load_tensors: offloaded 41/41 layers to GPU
load_tensors:   CPU_Mapped model buffer size =   333.44 MiB
load_tensors:        CUDA0 model buffer size = 18590.99 MiB
...................................................................................................
common_init_result: added <|endoftext|> logit bias = -inf
common_init_result: added <|im_end|> logit bias = -inf
common_init_result: added <|fim_pad|> logit bias = -inf
common_init_result: added <|repo_name|> logit bias = -inf
common_init_result: added <|file_sep|> logit bias = -inf
llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 200192
llama_context: n_ctx_seq     = 200192
llama_context: n_batch       = 2048
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = enabled
llama_context: kv_unified    = false
llama_context: freq_base     = 10000000.0
llama_context: freq_scale    = 1
llama_context: n_ctx_seq (200192) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
llama_context:  CUDA_Host  output buffer size =     0.95 MiB
llama_kv_cache:      CUDA0 KV buffer size =  2077.19 MiB
llama_kv_cache: size = 2077.19 MiB (200192 cells,  10 layers,  1/1 seqs), K (q8_0): 1038.59 MiB, V (q8_0): 1038.59 MiB
llama_memory_recurrent:      CUDA0 RS buffer size =    62.81 MiB
llama_memory_recurrent: size =   62.81 MiB (     1 cells,  40 layers,  1 seqs), R (f32):    2.81 MiB, S (f32):   60.00 MiB
sched_reserve: reserving ...
sched_reserve:      CUDA0 compute buffer size =   622.52 MiB
sched_reserve:  CUDA_Host compute buffer size =   399.02 MiB
sched_reserve: graph nodes  = 9399 (with bs=512), 4389 (with bs=1)
sched_reserve: graph splits = 2
sched_reserve: reserve took 56.14 ms, sched copies = 1
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
srv    load_model: initializing slots, n_slots = 1
common_speculative_is_compat: the target context does not support partial sequence removal
srv    load_model: speculative decoding not supported by this context
slot   load_model: id  0 | task -1 | new slot, n_ctx = 200192
srv    load_model: prompt cache is enabled, size limit: 8192 MiB
srv    load_model: use `--cache-ram 0` to disable the prompt cache
srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>
'
srv          init: init: chat template, thinking = 0
main: model loaded
main: server is listening on http://0.0.0.0:8080
main: starting the main loop...
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> ?temp-ext -> dist
slot launch_slot_: id  0 | task 0 | processing task, is_child = 0
slot update_slots: id  0 | task 0 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 12
slot update_slots: id  0 | task 0 | n_tokens = 0, memory_seq_rm [0, end)
slot init_sampler: id  0 | task 0 | init sampler, took 0.00 ms, tokens: text = 12, total = 12
slot update_slots: id  0 | task 0 | prompt processing done, n_tokens = 12, batch.n_tokens = 12
slot print_timing: id  0 | task 0 |
prompt eval time =     179.91 ms /    12 tokens (   14.99 ms per token,    66.70 tokens per second)
       eval time =       0.00 ms /     1 tokens (    0.00 ms per token, 1000000.00 tokens per second)
      total time =     179.91 ms /    13 tokens
slot      release: id  0 | task 0 | stop processing: n_tokens = 12, truncated = 0
srv  update_slots: all slots are idle
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = 14700290
srv  get_availabl: updating prompt cache
srv   prompt_save:  - saving prompt with length 12, total state size = 62.938 MiB
srv          load:  - looking for better prompt, base f_keep = 0.083, sim = 0.000
srv        update:  - cache state: 1 prompts, 62.938 MiB (limits: 8192.000 MiB, 200192 tokens, 200192 est)
srv        update:    - prompt 000001EB0D113270:      12 tokens, checkpoints:  0,    62.938 MiB
srv  get_availabl: prompt cache update took 36.64 ms
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 2 | processing task, is_child = 0
slot update_slots: id  0 | task 2 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 18818
slot update_slots: id  0 | task 2 | n_past = 1, slot.prompt.tokens.size() = 12, seq_id = 0, pos_min = 11, n_swa = 1
slot update_slots: id  0 | task 2 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
slot update_slots: id  0 | task 2 | n_tokens = 0, memory_seq_rm [0, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_tokens = 2048, batch.n_tokens = 2048, progress = 0.108832
slot update_slots: id  0 | task 2 | n_tokens = 2048, memory_seq_rm [2048, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_tokens = 4096, batch.n_tokens = 2048, progress = 0.217664
slot update_slots: id  0 | task 2 | n_tokens = 4096, memory_seq_rm [4096, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_tokens = 6144, batch.n_tokens = 2048, progress = 0.326496
slot update_slots: id  0 | task 2 | n_tokens = 6144, memory_seq_rm [6144, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_tokens = 8192, batch.n_tokens = 2048, progress = 0.435328
slot update_slots: id  0 | task 2 | n_tokens = 8192, memory_seq_rm [8192, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_tokens = 10240, batch.n_tokens = 2048, progress = 0.544160
slot update_slots: id  0 | task 2 | n_tokens = 10240, memory_seq_rm [10240, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_tokens = 12288, batch.n_tokens = 2048, progress = 0.652992
slot update_slots: id  0 | task 2 | n_tokens = 12288, memory_seq_rm [12288, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_tokens = 14336, batch.n_tokens = 2048, progress = 0.761824
slot update_slots: id  0 | task 2 | n_tokens = 14336, memory_seq_rm [14336, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_tokens = 16384, batch.n_tokens = 2048, progress = 0.870656
slot update_slots: id  0 | task 2 | n_tokens = 16384, memory_seq_rm [16384, end)
slot update_slots: id  0 | task 2 | prompt processing progress, n_tokens = 18306, batch.n_tokens = 1922, progress = 0.972792
slot update_slots: id  0 | task 2 | n_tokens = 18306, memory_seq_rm [18306, end)
slot init_sampler: id  0 | task 2 | init sampler, took 1.38 ms, tokens: text = 18818, total = 18818
slot update_slots: id  0 | task 2 | created context checkpoint 1 of 8 (pos_min = 18305, pos_max = 18305, n_tokens = 18306, size = 62.813 MiB)
slot update_slots: id  0 | task 2 | prompt processing done, n_tokens = 18818, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 2 |
prompt eval time =    6506.86 ms / 18818 tokens (    0.35 ms per token,  2892.02 tokens per second)
       eval time =     211.67 ms /    24 tokens (    8.82 ms per token,   113.38 tokens per second)
      total time =    6718.54 ms / 18842 tokens
slot      release: id  0 | task 2 | stop processing: n_tokens = 18841, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = 23509030
srv  get_availabl: updating prompt cache
srv   prompt_save:  - saving prompt with length 18841, total state size = 258.523 MiB
srv          load:  - looking for better prompt, base f_keep = 0.000, sim = 0.083
srv          load:  - found better prompt with f_keep = 1.000, sim = 1.000
srv        update:  - cache state: 1 prompts, 321.336 MiB (limits: 8192.000 MiB, 200192 tokens, 480324 est)
srv        update:    - prompt 000001F738380710:   18841 tokens, checkpoints:  1,   321.336 MiB
srv  get_availabl: prompt cache update took 98.90 ms
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> ?temp-ext -> dist
slot launch_slot_: id  0 | task 36 | processing task, is_child = 0
slot update_slots: id  0 | task 36 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 12
slot update_slots: id  0 | task 36 | need to evaluate at least 1 token for each active slot (n_past = 12, task.n_tokens() = 12)
slot update_slots: id  0 | task 36 | n_past was set to 11
slot update_slots: id  0 | task 36 | n_tokens = 11, memory_seq_rm [11, end)
slot update_slots: id  0 | task 36 | failed to truncate tokens with position >= 11 - clearing the memory
slot prompt_clear: id  0 | task 36 | clearing prompt with 11 tokens
slot init_sampler: id  0 | task 36 | init sampler, took 0.00 ms, tokens: text = 12, total = 12
slot update_slots: id  0 | task 36 | prompt processing done, n_tokens = 12, batch.n_tokens = 12
slot print_timing: id  0 | task 36 |
prompt eval time =      48.30 ms /    12 tokens (    4.03 ms per token,   248.42 tokens per second)
       eval time =       0.00 ms /     1 tokens (    0.00 ms per token, 1000000.00 tokens per second)
      total time =      48.31 ms /    13 tokens
slot      release: id  0 | task 36 | stop processing: n_tokens = 12, truncated = 0
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = 30083314
srv  get_availabl: updating prompt cache
srv   prompt_save:  - saving prompt with length 12, total state size = 62.938 MiB
srv          load:  - looking for better prompt, base f_keep = 0.083, sim = 0.000
srv          load:  - found better prompt with f_keep = 0.999, sim = 1.000
srv        update:  - cache state: 1 prompts, 62.938 MiB (limits: 8192.000 MiB, 200192 tokens, 200192 est)
srv        update:    - prompt 000001F738380780:      12 tokens, checkpoints:  0,    62.938 MiB
srv  get_availabl: prompt cache update took 86.62 ms
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 38 | processing task, is_child = 0
slot update_slots: id  0 | task 38 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 18818
slot update_slots: id  0 | task 38 | n_past = 18818, slot.prompt.tokens.size() = 18841, seq_id = 0, pos_min = 18840, n_swa = 1
slot update_slots: id  0 | task 38 | restored context checkpoint (pos_min = 18305, pos_max = 18305, n_tokens = 18306, size = 62.813 MiB)
slot update_slots: id  0 | task 38 | n_tokens = 18306, memory_seq_rm [18306, end)
slot init_sampler: id  0 | task 38 | init sampler, took 1.40 ms, tokens: text = 18818, total = 18818
slot update_slots: id  0 | task 38 | prompt processing done, n_tokens = 18818, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 38 |
prompt eval time =     331.95 ms /   512 tokens (    0.65 ms per token,  1542.40 tokens per second)
       eval time =     318.16 ms /    38 tokens (    8.37 ms per token,   119.44 tokens per second)
      total time =     650.11 ms /   550 tokens
slot      release: id  0 | task 38 | stop processing: n_tokens = 18855, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.978 (> 0.100 thold), f_keep = 0.998
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 77 | processing task, is_child = 0
slot update_slots: id  0 | task 77 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 19235
slot update_slots: id  0 | task 77 | n_past = 18815, slot.prompt.tokens.size() = 18855, seq_id = 0, pos_min = 18854, n_swa = 1
slot update_slots: id  0 | task 77 | restored context checkpoint (pos_min = 18305, pos_max = 18305, n_tokens = 18306, size = 62.813 MiB)
slot update_slots: id  0 | task 77 | n_tokens = 18306, memory_seq_rm [18306, end)
slot update_slots: id  0 | task 77 | prompt processing progress, n_tokens = 18723, batch.n_tokens = 417, progress = 0.973382
slot update_slots: id  0 | task 77 | n_tokens = 18723, memory_seq_rm [18723, end)
slot init_sampler: id  0 | task 77 | init sampler, took 1.45 ms, tokens: text = 19235, total = 19235
slot update_slots: id  0 | task 77 | created context checkpoint 2 of 8 (pos_min = 18722, pos_max = 18722, n_tokens = 18723, size = 62.813 MiB)
slot update_slots: id  0 | task 77 | prompt processing done, n_tokens = 19235, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 77 |
prompt eval time =     348.54 ms /   929 tokens (    0.38 ms per token,  2665.43 tokens per second)
       eval time =     521.41 ms /    59 tokens (    8.84 ms per token,   113.15 tokens per second)
      total time =     869.95 ms /   988 tokens
slot      release: id  0 | task 77 | stop processing: n_tokens = 19293, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.892 (> 0.100 thold), f_keep = 0.997
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 138 | processing task, is_child = 0
slot update_slots: id  0 | task 138 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 21559
slot update_slots: id  0 | task 138 | n_past = 19233, slot.prompt.tokens.size() = 19293, seq_id = 0, pos_min = 19292, n_swa = 1
slot update_slots: id  0 | task 138 | restored context checkpoint (pos_min = 18722, pos_max = 18722, n_tokens = 18723, size = 62.813 MiB)
slot update_slots: id  0 | task 138 | n_tokens = 18723, memory_seq_rm [18723, end)
slot update_slots: id  0 | task 138 | prompt processing progress, n_tokens = 20771, batch.n_tokens = 2048, progress = 0.963449
slot update_slots: id  0 | task 138 | n_tokens = 20771, memory_seq_rm [20771, end)
slot update_slots: id  0 | task 138 | prompt processing progress, n_tokens = 21047, batch.n_tokens = 276, progress = 0.976251
slot update_slots: id  0 | task 138 | n_tokens = 21047, memory_seq_rm [21047, end)
slot init_sampler: id  0 | task 138 | init sampler, took 1.92 ms, tokens: text = 21559, total = 21559
slot update_slots: id  0 | task 138 | created context checkpoint 3 of 8 (pos_min = 21046, pos_max = 21046, n_tokens = 21047, size = 62.813 MiB)
slot update_slots: id  0 | task 138 | prompt processing done, n_tokens = 21559, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 138 |
prompt eval time =    1013.76 ms /  2836 tokens (    0.36 ms per token,  2797.49 tokens per second)
       eval time =     461.36 ms /    51 tokens (    9.05 ms per token,   110.54 tokens per second)
      total time =    1475.12 ms /  2887 tokens
slot      release: id  0 | task 138 | stop processing: n_tokens = 21609, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.903 (> 0.100 thold), f_keep = 0.998
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 192 | processing task, is_child = 0
slot update_slots: id  0 | task 192 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 23873
slot update_slots: id  0 | task 192 | n_past = 21557, slot.prompt.tokens.size() = 21609, seq_id = 0, pos_min = 21608, n_swa = 1
slot update_slots: id  0 | task 192 | restored context checkpoint (pos_min = 21046, pos_max = 21046, n_tokens = 21047, size = 62.813 MiB)
slot update_slots: id  0 | task 192 | n_tokens = 21047, memory_seq_rm [21047, end)
slot update_slots: id  0 | task 192 | prompt processing progress, n_tokens = 23095, batch.n_tokens = 2048, progress = 0.967411
slot update_slots: id  0 | task 192 | n_tokens = 23095, memory_seq_rm [23095, end)
slot update_slots: id  0 | task 192 | prompt processing progress, n_tokens = 23361, batch.n_tokens = 266, progress = 0.978553
slot update_slots: id  0 | task 192 | n_tokens = 23361, memory_seq_rm [23361, end)
slot init_sampler: id  0 | task 192 | init sampler, took 1.74 ms, tokens: text = 23873, total = 23873
slot update_slots: id  0 | task 192 | created context checkpoint 4 of 8 (pos_min = 23360, pos_max = 23360, n_tokens = 23361, size = 62.813 MiB)
slot update_slots: id  0 | task 192 | prompt processing done, n_tokens = 23873, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 192 |
prompt eval time =    1032.17 ms /  2826 tokens (    0.37 ms per token,  2737.93 tokens per second)
       eval time =     276.56 ms /    29 tokens (    9.54 ms per token,   104.86 tokens per second)
      total time =    1308.73 ms /  2855 tokens
slot      release: id  0 | task 192 | stop processing: n_tokens = 23901, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.913 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 224 | processing task, is_child = 0
slot update_slots: id  0 | task 224 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 26154
slot update_slots: id  0 | task 224 | n_past = 23871, slot.prompt.tokens.size() = 23901, seq_id = 0, pos_min = 23900, n_swa = 1
slot update_slots: id  0 | task 224 | restored context checkpoint (pos_min = 23360, pos_max = 23360, n_tokens = 23361, size = 62.813 MiB)
slot update_slots: id  0 | task 224 | n_tokens = 23361, memory_seq_rm [23361, end)
slot update_slots: id  0 | task 224 | prompt processing progress, n_tokens = 25409, batch.n_tokens = 2048, progress = 0.971515
slot update_slots: id  0 | task 224 | n_tokens = 25409, memory_seq_rm [25409, end)
slot update_slots: id  0 | task 224 | prompt processing progress, n_tokens = 25642, batch.n_tokens = 233, progress = 0.980424
slot update_slots: id  0 | task 224 | n_tokens = 25642, memory_seq_rm [25642, end)
slot init_sampler: id  0 | task 224 | init sampler, took 1.92 ms, tokens: text = 26154, total = 26154
slot update_slots: id  0 | task 224 | created context checkpoint 5 of 8 (pos_min = 25641, pos_max = 25641, n_tokens = 25642, size = 62.813 MiB)
slot update_slots: id  0 | task 224 | prompt processing done, n_tokens = 26154, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 224 |
prompt eval time =    1071.94 ms /  2793 tokens (    0.38 ms per token,  2605.56 tokens per second)
       eval time =     351.94 ms /    40 tokens (    8.80 ms per token,   113.66 tokens per second)
      total time =    1423.87 ms /  2833 tokens
slot      release: id  0 | task 224 | stop processing: n_tokens = 26193, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.967 (> 0.100 thold), f_keep = 0.998
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 267 | processing task, is_child = 0
slot update_slots: id  0 | task 267 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 27048
slot update_slots: id  0 | task 267 | n_past = 26152, slot.prompt.tokens.size() = 26193, seq_id = 0, pos_min = 26192, n_swa = 1
slot update_slots: id  0 | task 267 | restored context checkpoint (pos_min = 25641, pos_max = 25641, n_tokens = 25642, size = 62.813 MiB)
slot update_slots: id  0 | task 267 | n_tokens = 25642, memory_seq_rm [25642, end)
slot update_slots: id  0 | task 267 | prompt processing progress, n_tokens = 26536, batch.n_tokens = 894, progress = 0.981071
slot update_slots: id  0 | task 267 | n_tokens = 26536, memory_seq_rm [26536, end)
slot init_sampler: id  0 | task 267 | init sampler, took 1.99 ms, tokens: text = 27048, total = 27048
slot update_slots: id  0 | task 267 | created context checkpoint 6 of 8 (pos_min = 26535, pos_max = 26535, n_tokens = 26536, size = 62.813 MiB)
slot update_slots: id  0 | task 267 | prompt processing done, n_tokens = 27048, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 267 |
prompt eval time =     550.51 ms /  1406 tokens (    0.39 ms per token,  2553.97 tokens per second)
       eval time =     246.75 ms /    28 tokens (    8.81 ms per token,   113.48 tokens per second)
      total time =     797.26 ms /  1434 tokens
slot      release: id  0 | task 267 | stop processing: n_tokens = 27075, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.921 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 297 | processing task, is_child = 0
slot update_slots: id  0 | task 297 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 29364
slot update_slots: id  0 | task 297 | n_past = 27046, slot.prompt.tokens.size() = 27075, seq_id = 0, pos_min = 27074, n_swa = 1
slot update_slots: id  0 | task 297 | restored context checkpoint (pos_min = 26535, pos_max = 26535, n_tokens = 26536, size = 62.813 MiB)
slot update_slots: id  0 | task 297 | n_tokens = 26536, memory_seq_rm [26536, end)
slot update_slots: id  0 | task 297 | prompt processing progress, n_tokens = 28584, batch.n_tokens = 2048, progress = 0.973437
slot update_slots: id  0 | task 297 | n_tokens = 28584, memory_seq_rm [28584, end)
slot update_slots: id  0 | task 297 | prompt processing progress, n_tokens = 28852, batch.n_tokens = 268, progress = 0.982564
slot update_slots: id  0 | task 297 | n_tokens = 28852, memory_seq_rm [28852, end)
slot init_sampler: id  0 | task 297 | init sampler, took 2.15 ms, tokens: text = 29364, total = 29364
slot update_slots: id  0 | task 297 | created context checkpoint 7 of 8 (pos_min = 28851, pos_max = 28851, n_tokens = 28852, size = 62.813 MiB)
slot update_slots: id  0 | task 297 | prompt processing done, n_tokens = 29364, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 297 |
prompt eval time =    1072.54 ms /  2828 tokens (    0.38 ms per token,  2636.73 tokens per second)
       eval time =     357.83 ms /    40 tokens (    8.95 ms per token,   111.78 tokens per second)
      total time =    1430.38 ms /  2868 tokens
slot      release: id  0 | task 297 | stop processing: n_tokens = 29403, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.978 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 340 | processing task, is_child = 0
slot update_slots: id  0 | task 340 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 30009
slot update_slots: id  0 | task 340 | n_past = 29362, slot.prompt.tokens.size() = 29403, seq_id = 0, pos_min = 29402, n_swa = 1
slot update_slots: id  0 | task 340 | restored context checkpoint (pos_min = 28851, pos_max = 28851, n_tokens = 28852, size = 62.813 MiB)
slot update_slots: id  0 | task 340 | n_tokens = 28852, memory_seq_rm [28852, end)
slot update_slots: id  0 | task 340 | prompt processing progress, n_tokens = 29497, batch.n_tokens = 645, progress = 0.982938
slot update_slots: id  0 | task 340 | n_tokens = 29497, memory_seq_rm [29497, end)
slot init_sampler: id  0 | task 340 | init sampler, took 2.33 ms, tokens: text = 30009, total = 30009
slot update_slots: id  0 | task 340 | created context checkpoint 8 of 8 (pos_min = 29496, pos_max = 29496, n_tokens = 29497, size = 62.813 MiB)
slot update_slots: id  0 | task 340 | prompt processing done, n_tokens = 30009, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 340 |
prompt eval time =     498.86 ms /  1157 tokens (    0.43 ms per token,  2319.30 tokens per second)
       eval time =     302.40 ms /    34 tokens (    8.89 ms per token,   112.43 tokens per second)
      total time =     801.26 ms /  1191 tokens
slot      release: id  0 | task 340 | stop processing: n_tokens = 30042, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.996 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 376 | processing task, is_child = 0
slot update_slots: id  0 | task 376 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 30142
slot update_slots: id  0 | task 376 | n_past = 30007, slot.prompt.tokens.size() = 30042, seq_id = 0, pos_min = 30041, n_swa = 1
slot update_slots: id  0 | task 376 | restored context checkpoint (pos_min = 29496, pos_max = 29496, n_tokens = 29497, size = 62.813 MiB)
slot update_slots: id  0 | task 376 | n_tokens = 29497, memory_seq_rm [29497, end)
slot update_slots: id  0 | task 376 | prompt processing progress, n_tokens = 29630, batch.n_tokens = 133, progress = 0.983014
slot update_slots: id  0 | task 376 | n_tokens = 29630, memory_seq_rm [29630, end)
slot init_sampler: id  0 | task 376 | init sampler, took 2.71 ms, tokens: text = 30142, total = 30142
slot update_slots: id  0 | task 376 | erasing old context checkpoint (pos_min = 18305, pos_max = 18305, n_tokens = 18306, size = 62.813 MiB)
slot update_slots: id  0 | task 376 | created context checkpoint 8 of 8 (pos_min = 29629, pos_max = 29629, n_tokens = 29630, size = 62.813 MiB)
slot update_slots: id  0 | task 376 | prompt processing done, n_tokens = 30142, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 376 |
prompt eval time =     306.92 ms /   645 tokens (    0.48 ms per token,  2101.54 tokens per second)
       eval time =     384.03 ms /    42 tokens (    9.14 ms per token,   109.37 tokens per second)
      total time =     690.95 ms /   687 tokens
slot      release: id  0 | task 376 | stop processing: n_tokens = 30183, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.797 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 420 | processing task, is_child = 0
slot update_slots: id  0 | task 420 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 37827
slot update_slots: id  0 | task 420 | n_past = 30140, slot.prompt.tokens.size() = 30183, seq_id = 0, pos_min = 30182, n_swa = 1
slot update_slots: id  0 | task 420 | restored context checkpoint (pos_min = 29629, pos_max = 29629, n_tokens = 29630, size = 62.813 MiB)
slot update_slots: id  0 | task 420 | n_tokens = 29630, memory_seq_rm [29630, end)
slot update_slots: id  0 | task 420 | prompt processing progress, n_tokens = 31678, batch.n_tokens = 2048, progress = 0.837444
slot update_slots: id  0 | task 420 | n_tokens = 31678, memory_seq_rm [31678, end)
slot update_slots: id  0 | task 420 | prompt processing progress, n_tokens = 33726, batch.n_tokens = 2048, progress = 0.891585
slot update_slots: id  0 | task 420 | n_tokens = 33726, memory_seq_rm [33726, end)
slot update_slots: id  0 | task 420 | prompt processing progress, n_tokens = 35774, batch.n_tokens = 2048, progress = 0.945727
slot update_slots: id  0 | task 420 | n_tokens = 35774, memory_seq_rm [35774, end)
slot update_slots: id  0 | task 420 | prompt processing progress, n_tokens = 37315, batch.n_tokens = 1541, progress = 0.986465
slot update_slots: id  0 | task 420 | n_tokens = 37315, memory_seq_rm [37315, end)
slot init_sampler: id  0 | task 420 | init sampler, took 2.76 ms, tokens: text = 37827, total = 37827
slot update_slots: id  0 | task 420 | erasing old context checkpoint (pos_min = 18722, pos_max = 18722, n_tokens = 18723, size = 62.813 MiB)
slot update_slots: id  0 | task 420 | created context checkpoint 8 of 8 (pos_min = 37314, pos_max = 37314, n_tokens = 37315, size = 62.813 MiB)
slot update_slots: id  0 | task 420 | prompt processing done, n_tokens = 37827, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 420 |
prompt eval time =    3204.15 ms /  8197 tokens (    0.39 ms per token,  2558.25 tokens per second)
       eval time =     389.88 ms /    42 tokens (    9.28 ms per token,   107.73 tokens per second)
      total time =    3594.02 ms /  8239 tokens
slot      release: id  0 | task 420 | stop processing: n_tokens = 37868, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.957 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 467 | processing task, is_child = 0
slot update_slots: id  0 | task 467 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 39521
slot update_slots: id  0 | task 467 | n_past = 37825, slot.prompt.tokens.size() = 37868, seq_id = 0, pos_min = 37867, n_swa = 1
slot update_slots: id  0 | task 467 | restored context checkpoint (pos_min = 37314, pos_max = 37314, n_tokens = 37315, size = 62.813 MiB)
slot update_slots: id  0 | task 467 | n_tokens = 37315, memory_seq_rm [37315, end)
slot update_slots: id  0 | task 467 | prompt processing progress, n_tokens = 39009, batch.n_tokens = 1694, progress = 0.987045
slot update_slots: id  0 | task 467 | n_tokens = 39009, memory_seq_rm [39009, end)
slot init_sampler: id  0 | task 467 | init sampler, took 2.88 ms, tokens: text = 39521, total = 39521
slot update_slots: id  0 | task 467 | erasing old context checkpoint (pos_min = 21046, pos_max = 21046, n_tokens = 21047, size = 62.813 MiB)
slot update_slots: id  0 | task 467 | created context checkpoint 8 of 8 (pos_min = 39008, pos_max = 39008, n_tokens = 39009, size = 62.813 MiB)
slot update_slots: id  0 | task 467 | prompt processing done, n_tokens = 39521, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 467 |
prompt eval time =     931.92 ms /  2206 tokens (    0.42 ms per token,  2367.16 tokens per second)
       eval time =     385.32 ms /    41 tokens (    9.40 ms per token,   106.40 tokens per second)
      total time =    1317.24 ms /  2247 tokens
slot      release: id  0 | task 467 | stop processing: n_tokens = 39561, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.968 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 510 | processing task, is_child = 0
slot update_slots: id  0 | task 510 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 40838
slot update_slots: id  0 | task 510 | n_past = 39519, slot.prompt.tokens.size() = 39561, seq_id = 0, pos_min = 39560, n_swa = 1
slot update_slots: id  0 | task 510 | restored context checkpoint (pos_min = 39008, pos_max = 39008, n_tokens = 39009, size = 62.813 MiB)
slot update_slots: id  0 | task 510 | n_tokens = 39009, memory_seq_rm [39009, end)
slot update_slots: id  0 | task 510 | prompt processing progress, n_tokens = 40326, batch.n_tokens = 1317, progress = 0.987463
slot update_slots: id  0 | task 510 | n_tokens = 40326, memory_seq_rm [40326, end)
slot init_sampler: id  0 | task 510 | init sampler, took 3.12 ms, tokens: text = 40838, total = 40838
slot update_slots: id  0 | task 510 | erasing old context checkpoint (pos_min = 23360, pos_max = 23360, n_tokens = 23361, size = 62.813 MiB)
slot update_slots: id  0 | task 510 | created context checkpoint 8 of 8 (pos_min = 40325, pos_max = 40325, n_tokens = 40326, size = 62.813 MiB)
slot update_slots: id  0 | task 510 | prompt processing done, n_tokens = 40838, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 510 |
prompt eval time =     780.89 ms /  1829 tokens (    0.43 ms per token,  2342.20 tokens per second)
       eval time =     394.43 ms /    42 tokens (    9.39 ms per token,   106.48 tokens per second)
      total time =    1175.32 ms /  1871 tokens
slot      release: id  0 | task 510 | stop processing: n_tokens = 40879, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.941 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 554 | processing task, is_child = 0
slot update_slots: id  0 | task 554 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 43403
slot update_slots: id  0 | task 554 | n_past = 40836, slot.prompt.tokens.size() = 40879, seq_id = 0, pos_min = 40878, n_swa = 1
slot update_slots: id  0 | task 554 | restored context checkpoint (pos_min = 40325, pos_max = 40325, n_tokens = 40326, size = 62.813 MiB)
slot update_slots: id  0 | task 554 | n_tokens = 40326, memory_seq_rm [40326, end)
slot update_slots: id  0 | task 554 | prompt processing progress, n_tokens = 42374, batch.n_tokens = 2048, progress = 0.976292
slot update_slots: id  0 | task 554 | n_tokens = 42374, memory_seq_rm [42374, end)
slot update_slots: id  0 | task 554 | prompt processing progress, n_tokens = 42891, batch.n_tokens = 517, progress = 0.988204
slot update_slots: id  0 | task 554 | n_tokens = 42891, memory_seq_rm [42891, end)
slot init_sampler: id  0 | task 554 | init sampler, took 3.17 ms, tokens: text = 43403, total = 43403
slot update_slots: id  0 | task 554 | erasing old context checkpoint (pos_min = 25641, pos_max = 25641, n_tokens = 25642, size = 62.813 MiB)
slot update_slots: id  0 | task 554 | created context checkpoint 8 of 8 (pos_min = 42890, pos_max = 42890, n_tokens = 42891, size = 62.813 MiB)
slot update_slots: id  0 | task 554 | prompt processing done, n_tokens = 43403, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 554 |
prompt eval time =    1306.25 ms /  3077 tokens (    0.42 ms per token,  2355.59 tokens per second)
       eval time =     398.90 ms /    42 tokens (    9.50 ms per token,   105.29 tokens per second)
      total time =    1705.15 ms /  3119 tokens
slot      release: id  0 | task 554 | stop processing: n_tokens = 43444, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.976 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 599 | processing task, is_child = 0
slot update_slots: id  0 | task 599 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 44462
slot update_slots: id  0 | task 599 | n_past = 43401, slot.prompt.tokens.size() = 43444, seq_id = 0, pos_min = 43443, n_swa = 1
slot update_slots: id  0 | task 599 | restored context checkpoint (pos_min = 42890, pos_max = 42890, n_tokens = 42891, size = 62.813 MiB)
slot update_slots: id  0 | task 599 | n_tokens = 42891, memory_seq_rm [42891, end)
slot update_slots: id  0 | task 599 | prompt processing progress, n_tokens = 43950, batch.n_tokens = 1059, progress = 0.988485
slot update_slots: id  0 | task 599 | n_tokens = 43950, memory_seq_rm [43950, end)
slot init_sampler: id  0 | task 599 | init sampler, took 3.24 ms, tokens: text = 44462, total = 44462
slot update_slots: id  0 | task 599 | erasing old context checkpoint (pos_min = 26535, pos_max = 26535, n_tokens = 26536, size = 62.813 MiB)
slot update_slots: id  0 | task 599 | created context checkpoint 8 of 8 (pos_min = 43949, pos_max = 43949, n_tokens = 43950, size = 62.813 MiB)
slot update_slots: id  0 | task 599 | prompt processing done, n_tokens = 44462, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 599 |
prompt eval time =     711.35 ms /  1571 tokens (    0.45 ms per token,  2208.46 tokens per second)
       eval time =     411.17 ms /    41 tokens (   10.03 ms per token,    99.71 tokens per second)
      total time =    1122.53 ms /  1612 tokens
slot      release: id  0 | task 599 | stop processing: n_tokens = 44502, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.991 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 642 | processing task, is_child = 0
slot update_slots: id  0 | task 642 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 44860
slot update_slots: id  0 | task 642 | n_past = 44460, slot.prompt.tokens.size() = 44502, seq_id = 0, pos_min = 44501, n_swa = 1
slot update_slots: id  0 | task 642 | restored context checkpoint (pos_min = 43949, pos_max = 43949, n_tokens = 43950, size = 62.813 MiB)
slot update_slots: id  0 | task 642 | n_tokens = 43950, memory_seq_rm [43950, end)
slot update_slots: id  0 | task 642 | prompt processing progress, n_tokens = 44348, batch.n_tokens = 398, progress = 0.988587
slot update_slots: id  0 | task 642 | n_tokens = 44348, memory_seq_rm [44348, end)
slot init_sampler: id  0 | task 642 | init sampler, took 3.27 ms, tokens: text = 44860, total = 44860
slot update_slots: id  0 | task 642 | erasing old context checkpoint (pos_min = 28851, pos_max = 28851, n_tokens = 28852, size = 62.813 MiB)
slot update_slots: id  0 | task 642 | created context checkpoint 8 of 8 (pos_min = 44347, pos_max = 44347, n_tokens = 44348, size = 62.813 MiB)
slot update_slots: id  0 | task 642 | prompt processing done, n_tokens = 44860, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 642 |
prompt eval time =     409.19 ms /   910 tokens (    0.45 ms per token,  2223.88 tokens per second)
       eval time =     411.12 ms /    41 tokens (   10.03 ms per token,    99.73 tokens per second)
      total time =     820.31 ms /   951 tokens
slot      release: id  0 | task 642 | stop processing: n_tokens = 44900, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.990 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 685 | processing task, is_child = 0
slot update_slots: id  0 | task 685 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 45314
slot update_slots: id  0 | task 685 | n_past = 44858, slot.prompt.tokens.size() = 44900, seq_id = 0, pos_min = 44899, n_swa = 1
slot update_slots: id  0 | task 685 | restored context checkpoint (pos_min = 44347, pos_max = 44347, n_tokens = 44348, size = 62.813 MiB)
slot update_slots: id  0 | task 685 | n_tokens = 44348, memory_seq_rm [44348, end)
slot update_slots: id  0 | task 685 | prompt processing progress, n_tokens = 44802, batch.n_tokens = 454, progress = 0.988701
slot update_slots: id  0 | task 685 | n_tokens = 44802, memory_seq_rm [44802, end)
slot init_sampler: id  0 | task 685 | init sampler, took 3.32 ms, tokens: text = 45314, total = 45314
slot update_slots: id  0 | task 685 | erasing old context checkpoint (pos_min = 29496, pos_max = 29496, n_tokens = 29497, size = 62.813 MiB)
slot update_slots: id  0 | task 685 | created context checkpoint 8 of 8 (pos_min = 44801, pos_max = 44801, n_tokens = 44802, size = 62.813 MiB)
slot update_slots: id  0 | task 685 | prompt processing done, n_tokens = 45314, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 685 |
prompt eval time =     430.27 ms /   966 tokens (    0.45 ms per token,  2245.12 tokens per second)
       eval time =     316.69 ms /    30 tokens (   10.56 ms per token,    94.73 tokens per second)
      total time =     746.96 ms /   996 tokens
slot      release: id  0 | task 685 | stop processing: n_tokens = 45343, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.999 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 717 | processing task, is_child = 0
slot update_slots: id  0 | task 717 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 45362
slot update_slots: id  0 | task 717 | n_past = 45312, slot.prompt.tokens.size() = 45343, seq_id = 0, pos_min = 45342, n_swa = 1
slot update_slots: id  0 | task 717 | restored context checkpoint (pos_min = 44801, pos_max = 44801, n_tokens = 44802, size = 62.813 MiB)
slot update_slots: id  0 | task 717 | n_tokens = 44802, memory_seq_rm [44802, end)
slot update_slots: id  0 | task 717 | prompt processing progress, n_tokens = 44850, batch.n_tokens = 48, progress = 0.988713
slot update_slots: id  0 | task 717 | n_tokens = 44850, memory_seq_rm [44850, end)
slot init_sampler: id  0 | task 717 | init sampler, took 4.95 ms, tokens: text = 45362, total = 45362
slot update_slots: id  0 | task 717 | prompt processing done, n_tokens = 45362, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 717 |
prompt eval time =     294.44 ms /   560 tokens (    0.53 ms per token,  1901.92 tokens per second)
       eval time =     276.22 ms /    28 tokens (    9.86 ms per token,   101.37 tokens per second)
      total time =     570.66 ms /   588 tokens
slot      release: id  0 | task 717 | stop processing: n_tokens = 45389, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.999 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 747 | processing task, is_child = 0
slot update_slots: id  0 | task 747 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 45409
slot update_slots: id  0 | task 747 | n_past = 45360, slot.prompt.tokens.size() = 45389, seq_id = 0, pos_min = 45388, n_swa = 1
slot update_slots: id  0 | task 747 | restored context checkpoint (pos_min = 44801, pos_max = 44801, n_tokens = 44802, size = 62.813 MiB)
slot update_slots: id  0 | task 747 | n_tokens = 44802, memory_seq_rm [44802, end)
slot update_slots: id  0 | task 747 | prompt processing progress, n_tokens = 44897, batch.n_tokens = 95, progress = 0.988725
slot update_slots: id  0 | task 747 | n_tokens = 44897, memory_seq_rm [44897, end)
slot init_sampler: id  0 | task 747 | init sampler, took 3.31 ms, tokens: text = 45409, total = 45409
slot update_slots: id  0 | task 747 | erasing old context checkpoint (pos_min = 29629, pos_max = 29629, n_tokens = 29630, size = 62.813 MiB)
slot update_slots: id  0 | task 747 | created context checkpoint 8 of 8 (pos_min = 44896, pos_max = 44896, n_tokens = 44897, size = 62.813 MiB)
slot update_slots: id  0 | task 747 | prompt processing done, n_tokens = 45409, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 747 |
prompt eval time =     328.20 ms /   607 tokens (    0.54 ms per token,  1849.50 tokens per second)
       eval time =     446.61 ms /    45 tokens (    9.92 ms per token,   100.76 tokens per second)
      total time =     774.80 ms /   652 tokens
slot      release: id  0 | task 747 | stop processing: n_tokens = 45453, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.998 (> 0.100 thold), f_keep = 0.999
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 794 | processing task, is_child = 0
slot update_slots: id  0 | task 794 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 45494
slot update_slots: id  0 | task 794 | n_past = 45407, slot.prompt.tokens.size() = 45453, seq_id = 0, pos_min = 45452, n_swa = 1
slot update_slots: id  0 | task 794 | restored context checkpoint (pos_min = 44896, pos_max = 44896, n_tokens = 44897, size = 62.813 MiB)
slot update_slots: id  0 | task 794 | n_tokens = 44897, memory_seq_rm [44897, end)
slot update_slots: id  0 | task 794 | prompt processing progress, n_tokens = 44982, batch.n_tokens = 85, progress = 0.988746
slot update_slots: id  0 | task 794 | n_tokens = 44982, memory_seq_rm [44982, end)
slot init_sampler: id  0 | task 794 | init sampler, took 4.85 ms, tokens: text = 45494, total = 45494
slot update_slots: id  0 | task 794 | erasing old context checkpoint (pos_min = 37314, pos_max = 37314, n_tokens = 37315, size = 62.813 MiB)
slot update_slots: id  0 | task 794 | created context checkpoint 8 of 8 (pos_min = 44981, pos_max = 44981, n_tokens = 44982, size = 62.813 MiB)
slot update_slots: id  0 | task 794 | prompt processing done, n_tokens = 45494, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 794 |
prompt eval time =     330.10 ms /   597 tokens (    0.55 ms per token,  1808.56 tokens per second)
       eval time =    9823.73 ms /  1011 tokens (    9.72 ms per token,   102.91 tokens per second)
      total time =   10153.82 ms /  1608 tokens
slot      release: id  0 | task 794 | stop processing: n_tokens = 46504, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.975 (> 0.100 thold), f_keep = 0.978
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 1807 | processing task, is_child = 0
slot update_slots: id  0 | task 1807 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 46650
slot update_slots: id  0 | task 1807 | n_past = 45492, slot.prompt.tokens.size() = 46504, seq_id = 0, pos_min = 46503, n_swa = 1
slot update_slots: id  0 | task 1807 | restored context checkpoint (pos_min = 44981, pos_max = 44981, n_tokens = 44982, size = 62.813 MiB)
slot update_slots: id  0 | task 1807 | n_tokens = 44982, memory_seq_rm [44982, end)
slot update_slots: id  0 | task 1807 | prompt processing progress, n_tokens = 46138, batch.n_tokens = 1156, progress = 0.989025
slot update_slots: id  0 | task 1807 | n_tokens = 46138, memory_seq_rm [46138, end)
slot init_sampler: id  0 | task 1807 | init sampler, took 4.14 ms, tokens: text = 46650, total = 46650
slot update_slots: id  0 | task 1807 | erasing old context checkpoint (pos_min = 39008, pos_max = 39008, n_tokens = 39009, size = 62.813 MiB)
slot update_slots: id  0 | task 1807 | created context checkpoint 8 of 8 (pos_min = 46137, pos_max = 46137, n_tokens = 46138, size = 62.813 MiB)
slot update_slots: id  0 | task 1807 | prompt processing done, n_tokens = 46650, batch.n_tokens = 512
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot print_timing: id  0 | task 1807 |
prompt eval time =     752.07 ms /  1668 tokens (    0.45 ms per token,  2217.87 tokens per second)
       eval time =    1738.55 ms /   178 tokens (    9.77 ms per token,   102.38 tokens per second)
      total time =    2490.63 ms /  1846 tokens
slot      release: id  0 | task 1807 | stop processing: n_tokens = 46827, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.409 (> 0.100 thold), f_keep = 0.411
srv  get_availabl: updating prompt cache
srv   prompt_save:  - saving prompt with length 46827, total state size = 549.225 MiB
srv          load:  - looking for better prompt, base f_keep = 0.411, sim = 0.409
srv        update:  - cache state: 2 prompts, 1114.669 MiB (limits: 8192.000 MiB, 200192 tokens, 344232 est)
srv        update:    - prompt 000001F738380780:      12 tokens, checkpoints:  0,    62.938 MiB
srv        update:    - prompt 000001F738AF6DA0:   46827 tokens, checkpoints:  8,  1051.731 MiB
srv  get_availabl: prompt cache update took 281.50 ms
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 1987 | processing task, is_child = 0
slot update_slots: id  0 | task 1987 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 47046
slot update_slots: id  0 | task 1987 | n_past = 19232, slot.prompt.tokens.size() = 46827, seq_id = 0, pos_min = 46826, n_swa = 1
slot update_slots: id  0 | task 1987 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
slot update_slots: id  0 | task 1987 | erased invalidated context checkpoint (pos_min = 40325, pos_max = 40325, n_tokens = 40326, n_swa = 1, size = 62.813 MiB)
slot update_slots: id  0 | task 1987 | erased invalidated context checkpoint (pos_min = 42890, pos_max = 42890, n_tokens = 42891, n_swa = 1, size = 62.813 MiB)
slot update_slots: id  0 | task 1987 | erased invalidated context checkpoint (pos_min = 43949, pos_max = 43949, n_tokens = 43950, n_swa = 1, size = 62.813 MiB)
slot update_slots: id  0 | task 1987 | erased invalidated context checkpoint (pos_min = 44347, pos_max = 44347, n_tokens = 44348, n_swa = 1, size = 62.813 MiB)
slot update_slots: id  0 | task 1987 | erased invalidated context checkpoint (pos_min = 44801, pos_max = 44801, n_tokens = 44802, n_swa = 1, size = 62.813 MiB)
slot update_slots: id  0 | task 1987 | erased invalidated context checkpoint (pos_min = 44896, pos_max = 44896, n_tokens = 44897, n_swa = 1, size = 62.813 MiB)
slot update_slots: id  0 | task 1987 | erased invalidated context checkpoint (pos_min = 44981, pos_max = 44981, n_tokens = 44982, n_swa = 1, size = 62.813 MiB)
slot update_slots: id  0 | task 1987 | erased invalidated context checkpoint (pos_min = 46137, pos_max = 46137, n_tokens = 46138, n_swa = 1, size = 62.813 MiB)
slot update_slots: id  0 | task 1987 | n_tokens = 0, memory_seq_rm [0, end)
slot update_slots: id  0 | task 1987 | prompt processing progress, n_tokens = 2048, batch.n_tokens = 2048, progress = 0.043532
slot update_slots: id  0 | task 1987 | n_tokens = 2048, memory_seq_rm [2048, end)
slot update_slots: id  0 | task 1987 | prompt processing progress, n_tokens = 4096, batch.n_tokens = 2048, progress = 0.087064
slot update_slots: id  0 | task 1987 | n_tokens = 4096, memory_seq_rm [4096, end)
slot update_slots: id  0 | task 1987 | prompt processing progress, n_tokens = 6144, batch.n_tokens = 2048, progress = 0.130596
srv          stop: cancel task, id_task = 1987
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
slot      release: id  0 | task 1987 | stop processing: n_tokens = 6144, truncated = 0
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = 91580131
srv  get_availabl: updating prompt cache
srv   prompt_save:  - saving prompt with length 6144, total state size = 126.634 MiB
srv         alloc:  - prompt is already in the cache, skipping
srv          load:  - looking for better prompt, base f_keep = 0.000, sim = 0.083
srv          load:  - found better prompt with f_keep = 1.000, sim = 1.000
srv        update:  - cache state: 1 prompts, 1051.731 MiB (limits: 8192.000 MiB, 200192 tokens, 364738 est)
srv        update:    - prompt 000001F738AF6DA0:   46827 tokens, checkpoints:  8,  1051.731 MiB
srv  get_availabl: prompt cache update took 12.03 ms
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> ?temp-ext -> dist
slot launch_slot_: id  0 | task 1992 | processing task, is_child = 0
slot update_slots: id  0 | task 1992 | new prompt, n_ctx_slot = 200192, n_keep = 0, task.n_tokens = 12
slot update_slots: id  0 | task 1992 | need to evaluate at least 1 token for each active slot (n_past = 12, task.n_tokens() = 12)
slot update_slots: id  0 | task 1992 | n_past was set to 11
slot update_slots: id  0 | task 1992 | n_tokens = 11, memory_seq_rm [11, end)
slot update_slots: id  0 | task 1992 | failed to truncate tokens with position >= 11 - clearing the memory
slot prompt_clear: id  0 | task 1992 | clearing prompt with 11 tokens
slot init_sampler: id  0 | task 1992 | init sampler, took 0.00 ms, tokens: text = 12, total = 12
slot update_slots: id  0 | task 1992 | prompt processing done, n_tokens = 12, batch.n_tokens = 12
slot print_timing: id  0 | task 1992 |
prompt eval time =      46.96 ms /    12 tokens (    3.91 ms per token,   255.52 tokens per second)
       eval time =       0.00 ms /     1 tokens (    0.00 ms per token, 1000000.00 tokens per second)
      total time =      46.96 ms /    13 tokens
slot      release: id  0 | task 1992 | stop processing: n_tokens = 12, truncated = 0
srv  log_server_r: done request: POST /v1/messages 127.0.0.1 200
srv  update_slots: all slots are idle
srv  params_from_: Chat format: peg-constructed
slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = 97482731
srv  get_availabl: updating prompt cache
srv   prompt_save:  - saving prompt with length 12, total state size = 62.938 MiB
srv          load:  - looking for better prompt, base f_keep = 0.083, sim = 0.000
srv          load:  - found better prompt with f_keep = 0.402, sim = 1.000
PS C:\projects\bin-llama-cpp\llama-b8179-bin-win-cuda-13.1-x64>

[anubhavgupta]

Few more interesting things I have noticed are:

• There is no involvement of --swa-full arg.
• I am able to quickly repro this issue when i use --parallel 1, with the above shared steps. The issues doesn't seems to be happening without this arg(in my limited testing).

[naxan6]

I'm seeing the same exact behavior when using qwen3.5-35b-a3b with agentzero after a few calls.
"found better prompt with f_keep..." is also the last entry I see.
the crash is visible in windows event log
--parallel -1 does not solve it for me (I also had it set to 1).

[-](https://github.com/ggml-org/llama.cpp/issues/20002#) <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
[-](https://github.com/ggml-org/llama.cpp/issues/20002#) <System>
  <Provider Name="Application Error" Guid="{a0e9b465-b939-57d7-b27d-95d8e925ff57}" /> 
  <EventID>1000</EventID> 
  <Version>0</Version> 
  <Level>2</Level> 
  <Task>100</Task> 
  <Opcode>0</Opcode> 
  <Keywords>0x8000000000000000</Keywords> 
  <TimeCreated SystemTime="2026-03-01T15:59:27.4419747Z" /> 
  <EventRecordID>64521</EventRecordID> 
  <Correlation /> 
  <Execution ProcessID="37708" ThreadID="12468" /> 
  <Channel>Application</Channel> 
  <Computer>xxxx</Computer> 
  <Security UserID="S[-](https://github.com/ggml-org/llama.cpp/issues/20002#)1-5-21-3210314402-1913893739-824284856-1001" /> 
  </System>
- <EventData>
  <Data Name="AppName">llama-server.exe</Data> 
  <Data Name="AppVersion">0.0.0.0</Data> 
  <Data Name="AppTimeStamp">69a3d56d</Data> 
  <Data Name="ModuleName">llama.dll</Data> 
  <Data Name="ModuleVersion">0.0.0.0</Data> 
  <Data Name="ModuleTimeStamp">69a3d569</Data> 
  <Data Name="ExceptionCode">c0000005</Data> 
  <Data Name="FaultingOffset">000000000003a220</Data> 
  <Data Name="ProcessId">0xa40c</Data> 
  <Data Name="ProcessCreationTime">0x1dca9945473c780</Data> 
  <Data Name="AppPath">C:\git\ki\runtimes\llama-b8183-bin-win-cuda-13.1-x64.zip\llama-server.exe</Data> 
  <Data Name="ModulePath">C:\git\ki\runtimes\llama-b8183-bin-win-cuda-13.1-x64.zip\llama.dll</Data> 
  <Data Name="IntegratorReportId">bd20ac32-a4f6-4dfb-8692-75217fde63ab</Data> 
  <Data Name="PackageFullName" /> 
  <Data Name="PackageRelativeAppId" /> 
  </EventData>
  </Event>

[am17an]

Please provide a repro with llama.cpp tools if possible (llama-completion or llama-server perferably)

[anubhavgupta]

@am17an, thanks for your amazing work!.

Here is repro with llama-server. I had to intercept the Cladue code requests and capture those to a file to share the repro.

repro.zip

Steps to run the repro:

1. unzip the attached file (-> repro.log contains serialized request to locally running (port 8080) llama-server and run-repro-requests.js fires these requests one by one)
2. run the run-repro-requests.js file via node run-repro-requests.js (requests are fired to http://localhost:8080/v1/messages endpoint)
3. You might need to run Step 2 multiple times. It usually repros the issue by the second run(~99% of the cases).

Once the issue is reprod, you will see following logs from this script:

and follwing logs from the llama-server. The server crashes with last log line as "found better prompt with f_keep"

Args used to launch the server:

 -m C:\Users\anubh\.lmstudio\models\qwen3.5-35b-a3b\Qwen3.5-35B-A3B-UD-Q4_K_XL.gguf -ngl 99 -t 23 --temp 0.6 --top-k 20 --top-p 0.95 --parallel 1 --mlock -c 200000 -ctk q8_0 -ctv q8_0 -fa on --jinja --reasoning-budget 0 --host 0.0.0.0 -fit off --mmproj C:\Users\anubh\.lmstudio\models\qwen3.5-35b-a3b\mmproj-BF16.gguf

[SurealCereal]

I am seeing this issue with b8192 and unsloth/Qwen3.5-122B-A10B-GGUF / MXFP4_MOE
Frequent re-processsing of the context with "forcing full prompt re-processing due to lack of cache data".

Part of a log:

llama  | [38185] srv  params_from_: Chat format: peg-constructed
llama  | [38185] slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.995 (> 0.100 thold), f_keep = 1.000
llama  | [38185] slot launch_slot_: id  3 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> min-p -> ?xtc -> temp-ext -> dist 
llama  | [38185] slot launch_slot_: id  3 | task 32830 | processing task, is_child = 0
llama  | [38185] slot update_slots: id  3 | task 32830 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 57677
llama  | [38185] slot update_slots: id  3 | task 32830 | n_tokens = 57369, memory_seq_rm [57369, end)
llama  | [38185] slot init_sampler: id  3 | task 32830 | init sampler, took 5.90 ms, tokens: text = 57677, total = 57677
llama  | [38185] slot update_slots: id  3 | task 32830 | erasing old context checkpoint (pos_min = 54626, pos_max = 54626, n_tokens = 54627, size = 149.063 MiB)
llama  | [38185] slot update_slots: id  3 | task 32830 | created context checkpoint 8 of 8 (pos_min = 57368, pos_max = 57368, n_tokens = 57369, size = 149.063 MiB)
llama  | [38185] slot update_slots: id  3 | task 32830 | prompt processing done, n_tokens = 57677, batch.n_tokens = 308
llama  | [38185] srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
llama  | srv  log_server_r: done request: POST /v1/chat/completions 172.18.0.13 200
llama  | [38185] slot print_timing: id  3 | task 32830 | 
llama  | [38185] prompt eval time =     319.61 ms /   308 tokens (    1.04 ms per token,   963.69 tokens per second)
llama  | [38185]        eval time =    2512.16 ms /   110 tokens (   22.84 ms per token,    43.79 tokens per second)
llama  | [38185]       total time =    2831.77 ms /   418 tokens
llama  | [38185] slot      release: id  3 | task 32830 | stop processing: n_tokens = 57786, truncated = 0
llama  | [38185] srv  update_slots: all slots are idle
llama  | srv  proxy_reques: proxying request to model Qwen3.5_122B_A10B_MXFP4_MOE on port 38185
llama  | [38185] srv  params_from_: Chat format: peg-constructed
llama  | [38185] slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.907 (> 0.100 thold), f_keep = 0.901
llama  | [38185] slot launch_slot_: id  3 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> min-p -> ?xtc -> temp-ext -> dist 
llama  | [38185] slot launch_slot_: id  3 | task 32941 | processing task, is_child = 0
llama  | [38185] slot update_slots: id  3 | task 32941 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 57404
llama  | [38185] slot update_slots: id  3 | task 32941 | n_past = 52086, slot.prompt.tokens.size() = 57786, seq_id = 3, pos_min = 57785, n_swa = 1
llama  | [38185] slot update_slots: id  3 | task 32941 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
llama  | [38185] slot update_slots: id  3 | task 32941 | erased invalidated context checkpoint (pos_min = 55016, pos_max = 55016, n_tokens = 55017, n_swa = 1, size = 149.063 MiB)
llama  | [38185] slot update_slots: id  3 | task 32941 | erased invalidated context checkpoint (pos_min = 55618, pos_max = 55618, n_tokens = 55619, n_swa = 1, size = 149.063 MiB)
llama  | [38185] slot update_slots: id  3 | task 32941 | erased invalidated context checkpoint (pos_min = 55868, pos_max = 55868, n_tokens = 55869, n_swa = 1, size = 149.063 MiB)
llama  | [38185] slot update_slots: id  3 | task 32941 | erased invalidated context checkpoint (pos_min = 56763, pos_max = 56763, n_tokens = 56764, n_swa = 1, size = 149.063 MiB)
llama  | [38185] slot update_slots: id  3 | task 32941 | erased invalidated context checkpoint (pos_min = 56895, pos_max = 56895, n_tokens = 56896, n_swa = 1, size = 149.063 MiB)
llama  | [38185] slot update_slots: id  3 | task 32941 | erased invalidated context checkpoint (pos_min = 56984, pos_max = 56984, n_tokens = 56985, n_swa = 1, size = 149.063 MiB)
llama  | [38185] slot update_slots: id  3 | task 32941 | erased invalidated context checkpoint (pos_min = 57099, pos_max = 57099, n_tokens = 57100, n_swa = 1, size = 149.063 MiB)
llama  | [38185] slot update_slots: id  3 | task 32941 | erased invalidated context checkpoint (pos_min = 57368, pos_max = 57368, n_tokens = 57369, n_swa = 1, size = 149.063 MiB)
llama  | [38185] slot update_slots: id  3 | task 32941 | n_tokens = 0, memory_seq_rm [0, end)
llama  | [38185] slot update_slots: id  3 | task 32941 | prompt processing progress, n_tokens = 2048, batch.n_tokens = 2048, progress = 0.035677
llama  | [38185] slot update_slots: id  3 | task 32941 | n_tokens = 2048, memory_seq_rm [2048, end)
llama  | [38185] slot update_slots: id  3 | task 32941 | prompt processing progress, n_tokens = 4096, batch.n_tokens = 2048, progress = 0.071354
llama  | [38185] slot update_slots: id  3 | task 32941 | n_tokens = 4096, memory_seq_rm [4096, end)
llama  | [38185] slot update_slots: id  3 | task 32941 | prompt processing progress, n_tokens = 6144, batch.n_tokens = 2048, progress = 0.107031

[kexul]

Any update with this issue?

[am17an]

@anubhavgupta thanks for the repro. Unfortunately I can't reproduce this on my linux machine, will try to get a windows box to test

[ggerganov]

@anubhavgupta Most likely you run out of memory. By default, llama-server allocates up to 8GB extra RAM for hots-memory prompt caches. You can adjust this with the --cache-ram N CLI argument

@SurealCereal This is a different issue. Use larger --ctx-checkpoints N to fix this.

[anubhavgupta]

@am17an , did you tried running the script 2-3 times ? For me when I run the script back to back twice (without unloading the model), then it happens.

@ggerganov, let me try it with a higher cram value, but this issue doesn't happen much when I remove the --parallel 1 arg.

[kexul]

@am17an , did you tried running the script 2-3 times ? For me when I run the script back to back twice (without unloading the model), then it happens.

@ggerganov, let me try it with a higher cram value, but this issue doesn't happen much when I remove the --parallel 1 arg.

I've noticed the same thing, if --parallel 1 flag is removed, it doesn't happen much. But the generation speed drop to ~60 token/s in my 4090, it's ~100 token/s when using -np 1.