可以支持mistral模型吗

[cy565025164]

Describe the feature

You are using a model of type mistral to instantiate a model of type llama. This is not supported for all configurations of models and can yield errors

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Title: Can the mistral model be supported?

[flybird11111]

hi，mistral模型已经支持了，可以关注一下这个pr：#5103

[cy565025164]

hi，mistral模型已经支持了，可以关注一下这个pr：#5103

我的代码如下：
torchrun --standalone --nproc_per_node=8 train_sft.py
--pretrain ./zephyr-7b-beta
--tokenizer ./zephyr-7b-beta
--model 'llama'
--strategy colossalai_zero2_cpu
--save_path zephyr-7b-beta-sft
--dataset train_sft.json
--batch_size 1
--accumulation_steps 8
--lr 2e-5
--max_len 4096
--max_epochs 3
--grad_checkpoint

You are using a model of type mistral to instantiate a model of type llama. This is not supported for all configurations of models and can yield errors

[flybird11111]

抱歉，shardformer已经支持了mistral，coati还未支持，我跟相关同事反馈一下。

[imgaojun]

@flybird11111

抱歉，shardformer已经支持了mistral，coati还未支持，我跟相关同事反馈一下。

shardformer何时支持下QWEN？

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

@flybird11111

Sorry, shardformer already supports mistral, but coati does not yet support it. I will give feedback to relevant colleagues.

When will shardformer support QWEN?

[flybird11111]

@flybird11111

抱歉，shardformer已经支持了mistral，coati还未支持，我跟相关同事反馈一下。

shardformer何时支持下QWEN？

hi，qianwen会尽快支持的。chat也会尽快支持mistral