DeepSeek 671B Fine-Tuning , how to merge lora model

[Roysky]

after DeepSeek 671B Fine-Tuning , how to load the Original model and lora model to test ?

help ~

[lean-wang]

We used the following code for the conversion:

from peft import PeftModel, PeftConfig
from transformers import AutoModel, AutoTokenizer
import torch

# Load configuration and model
peft_model_id = "./DeepSeek-R1-bf16-lora/lora"
peft_config = PeftConfig.from_pretrained(peft_model_id, trust_remote_code=True)
base_model = AutoModel.from_pretrained(peft_config.base_model_name_or_path, trust_remote_code=True)

# Load the model with LoRA parameters
model = PeftModel.from_pretrained(base_model, peft_model_id)

# Merge LoRA weights into the base model
model = model.merge_and_unload()

# Now, 'model' contains the merged weights and can be used for inference or saved as a new model
model.save_pretrained("./model_merge")

Our machine has 2TB of memory, The swap space is 2TB, and we have attempted to merge the LoRA weights, but it reported missing weights. There are over 1000 layers of weights missing after merging the LoRA weights compared to before..

[wotulong]

what's your fine tune env? thks

[tangzhenkun01]

When merging LoRA weights on a machine with 8 GPUs and 1.7T memory, I keep getting OOM (Out Of Memory) errors. What should I do?

[TongLi3701]

Please have a look at this example: https://discuss.huggingface.co/t/help-with-merging-lora-weights-back-into-base-model/40968.

We are working on providing a straightforward example on the github soon.