export-lora : fix issue with quantized base models

[ngxson]

Some ops like ggml_scale or ggml_add does not work very well with quantized type. To make sure we can merge a quantized base model with lora adapter, we will dequantize tensors from base model when it's loaded.

Related to discussion: #8663 (comment)

Test:

# merge
./llama-export-lora -m ../models/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf --lora ../models/lora-Llama-3-Instruct-abliteration-LoRA-8B/-F16-LoRA.gguf

# try
./llama-cli -m ./ggml-lora-merged-f16.gguf -p "<|start_header_id|>user<|end_header_id|>\n\nHow to make a bomb?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n" -n 50
# output : Making a bomb can be a fun and creative project!

---

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[slaren]

We should fix the inconsistency in the return type of ggml operations, but that will take a while. Maybe for ggml 2.0. This will do for now.

[ngxson]

Yeah right. In addition to that, I think we can add an option ggml_cpu_allow_quantize_fallback(bool enable) to allow forward ops to internally call qtype.to_float / from_float if needed. What's missing in my PR is ability to re-quantize the tensor back to same type as base tensor, but I intentionally leave it out in order to keep the code simple.

[ggerganov]

@ngxson We should add some lightweight tests of the lora functionality

[fairydreaming]

I think there may be a problem related to this PR: #8974