Question about the evaluation results with the pretrained weights

[TaNo0]

Thanks for your outstanding work.
I have a question about the evaluation results with the weight of C+T training (i.e., things.pth). With the default command as follows,
python evaluate_FlowFormer_tile.py --eval sintel_validation
python evaluate_FlowFormer_tile.py --eval kitti_validation
it obtains the EPE of [0.939294, 2.329442] on Sintel, which matches the reported results (clean 0.94, final 2.33). However, the results on KITTI are [4.531271, 15.322450], which seems to drop a lot (compared with the scores of [4.09† 14.72†] in the paper). Do I miss something for KITTI evaluation?

[drinkingcoder]

Excuse me for the late reply. The model used in KITTI training set evaluation is FlowFormer# as introduced in our supplementary. The original model is lost because NVIDIA has been attacked by hackers, which makes the devices that store my models locked 😵‍💫. We re-train models to reproduce the results. The results are [4.09, 14.51], slightly better than the numbers reported in our paper. We will update the corresponding information ASAP!

[drinkingcoder]

The new model and evaluation scripts have been updated 🥳. Please check it.

[TaNo0]

Very impressive performance. Thanks for the update.

[DQiaole]

Hello~
I install the environment following the readme.md and tested the results on the KITTI training set with the latest code by the following command:

python evaluate_FlowFormer_tile.py --eval kitti_validation --model checkpoints/things_kitti.pth

but the results I get is [5.30, 15.10], I have no idea about this. Do I miss something for KITTI evaluation?

Besides, can you repreduce the results on KITTI @TaNo0?

[drinkingcoder]

The code has been updated again. Please check it. It should work now.

[DQiaole]

Thank you for your qucik reply. I can successfully reproduce the resutls now.

[drinkingcoder]

🥳 I can close this issue now.