Visualize caption predictions for my image file

[JoonseoKang]

Hello, thank you for your work!

I want to get a caption for the image I have.
So I created a folder 'vis_image' put the image and ran the following code. (also created 'vis_image/val2014' and put images both directory)

python eval.py --dump_images 1 --num_images -1 --model log_relation_transformer_bu_rl_pretrain_beam5/model-best.pth --infos_path log_relation_transformer_bu_rl_pretrain_beam5/infos_relation_transformer_bu-best.pkl --image_root vis_image --input_json data/cocotalk.json --input_label_h5 data/cocotalk_label.h5 --input_fc_dir data/cocobu_fc --input_att_dir data/cocobu_att --input_box_dir data/cocobu_box --input_rel_box_dir data/cocobu_box_relative --beam_size 5 --batch_size 700

As a result, the code below was printed out.

DataLoader loading json file: data/cocotalk.json
vocab size is 9487
DataLoader loading h5 file: data/cocobu_fc data/cocobu_att data/cocobu_box data/cocotalk_label.h5
max sequence length in data is 16
read 123287 image features
assigned 113287 images to split train
assigned 5000 images to split val
assigned 5000 images to split test

cp "vis_image/val2014/COCO_val2014_000000369771.jpg" vis/imgs/369771.jpg
cp: 'vis_image/val2014/COCO_val2014_000000369771.jpg' cannot be described: No such file or directory
image 369771: two plastic containers of food on a table
...

And when I looked at the result with json file in vis and localhost:8000, I saw the caption for test 5000 images.
I just wanted to check the caption of 3 sample images I have.

Could you explain in more detail how i can get a caption for a image i have?

[litingzhou1]

Hi, I met the same problem. It seems all the loaded images are default and I cant use it to do prediction using my images. Could you please help to give more detailed explanation? Thanks a lot!

[batooooo]

Hi, i met the same problem too. Could you solve it?

[KhanhThiVo]

Hello,
i tried to put my images in a folder, added them with --image_folder to run eval.py but i got a KeyError: 'att_masks'. Could anyone please share if you also faced the same problem? Many thanks!

[yoojin9649]

Hello
i got a KeyError: 'att_masks' problem too. How could i solve this problem?

---

KeyError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_16492/3942058931.py in
1 import eval_utils
----> 2 loss, split_predictions, lang_stats = eval_utils.eval_split(model, crit, loader, vars(opt))

D:\imagecaptioning\object_relation_transformer_yahoo\object_relation_transformer-master_yahoo\eval_utils.py in eval_split(model, crit, loader, eval_kwargs)
120 data['att_feats'][np.arange(loader.batch_size) * loader.seq_per_img],
121 data['att_masks'][np.arange(loader.batch_size) * loader.seq_per_img] if data['att_masks'] is not None else None]
122

KeyError: 'att_masks'