[BUG]: "OverflowError: int too big to convert" during SFT training using Bloom model

[chengeharrison]

🐛 Describe the bug

torchrun --standalone --nproc_per_node=2 train_sft.py \
    --pretrain "bigscience/bloom-560m" \
    --model 'bloom' \
    --strategy ddp \
    --log_interval 10 \
    --save_path  "./output/" \
    --dataset "/data/users/lcxyc/instinwild_ch.json" \
    --batch_size 2 \
    --accimulation_steps 8 \
    --lr 2e-5 \
    --max_datasets_size 512 \
    --max_epochs 1 \

When building the SFT dataset, the error occurs.

Environment

No response

[GXKIM]

I also get an error when running this file, and I can't import the package after downloading the transformers

[chengeharrison]

I also get an error when running this file, and I can't import the package after downloading the transformers

Hi, could you please tell me which package you have problem with?

For the Overflow error, a quick fix can be adding to run your code, and we will later fix it in the main branch.
The quick fix from line 59 in train_sft.py:

elif args.model == 'bloom':
        tokenizer = BloomTokenizerFast.from_pretrained(args.pretrain, use_fast=False)
        tokenizer.pad_token = tokenizer.eos_token
        tokenizer.model_max_length = 2048 # add this line to specify the length because the preset length is 1000000000000000019884624838656 which is too large

[GXKIM]

train_sft.py => from coati.models.llama import LlamaLM line11

[chengeharrison]

train_sft.py => from coati.models.llama import LlamaLM line11

It's so weird because your from coati.models.bloom import BLOOMLM can pass but for llama couldn't. Maybe you should check your coati module and reinstall it.

[GXKIM]

thank you. I will try it

[GXKIM]

I installed transformers according to ColossalAI/applications)/Chat/README.md
#git clone https://github.com/hpcaitech/transformers
#cd transformers
#pip install . (this step will download coati module)
train_sft.py => from coati.models.llama import LlamaLM line11
I always can't import module