result of the code

[skylie-xtj]

Hello, I used two A100s to follow your steps for post-training tests and found that the BLEU result of test_2017_flickr is only around 27. Is the code correct or missing?

When I used the model_best_test model you provided, some errors occur:
Error(s) in loading state_dict for CLIPTrans:
Missing key(s) in state_dict: "adapter.hidden.0.weight", "adapter.hidden.0.bias", "adapter.hidden.1.weight", "mbart.final_logits_bias", "mbart.model.shared.weight", "mbart.model.encoder.embed_tokens.weight", "mbart.model.encoder.embed_positions.weight", "mbart.model.encoder.layers.0.self_attn.k_proj.weight",

[devaansh100]

Can you give me some more details:

1. Did you retrain a model?
2. Did you perform the evaluation despite these errors?

The code is correct, and should give the reported numbers from the paper

[skylie-xtj]

1. Yes, I used the following commands to retrain the model on multi30k.en-de dataset (the last one for evaluation):

python -m torch.distributed.run --master_port=29501 --nproc_per_node 2 src/main.py --num_gpus 2 --mn multi30k --prefix_length 10 --bs 32 --update_count 4 --lr 1e-5 --stage caption --tgt_lang de

python -m torch.distributed.run --master_port=29501 --nproc_per_node 2 src/main.py --num_gpus 2 --mn multi30k --prefix_length 10 --bs 32 --update_count 4 --lr 1e-5 --test_ds 2016 val --stage translate --tgt_lang de --lm model_pretrained.pth

python -m torch.distributed.run --master_port=29501 --nproc_per_node 2 src/main.py --num_gpus 2 --mn multi30k --src_lang en --tgt_lang de --prefix_length 10 --bs 32 --stage translate --test --test_ds 2016 flickr --lm model_best_test.pth

2. I modified the error and when I used the model_best_test_2.pt provided, I found the targeted sentences were empty.