Training on C+T

[DQiaole]

Thanks for your impressive works.
I use your codes and run command

./run_train.sh

When it comes to train model on flyingthings, I find the performance on Sintel(train) is much better than your results provided in paper's Table 1. The only difference is that I change the batch size to 4 due to the memory limit. The logs are shown in below:

The experiment name of the blue curve is: things/latentcostformer/cost_heads_num[1]vert_c_dim[64]cnet[twins]pretrain[True]add_flow_token[True]encoder_depth[3]gma[GMA]cost_encoder_res[True] (08_19_21_07), and the orange curve is the logs of training on flyingchairs: chairs/latentcostformer/cost_heads_num[1]vert_c_dim[64]cnet[twins]pretrain[True]add_flow_token[True]encoder_depth[3]gma[True]cost_encoder_res[True]arxiv2(08_17_16_47).
So, I want to know why the performance is different from the one reported in the paper, is there any difference of codes?
Looking forward to your kind reply!

[drinkingcoder]

Thanks. We directly inherent hyper parameters from GMA. We will try to train our FlowFormer with batchsize 4 and reply to you when getting the results.

[DQiaole]

Thank you for your quick reply.
When training on FlyingChairs, I use the default setting and batchsize 8. The batchsize of FlyingThings is 4, I think just train the model on FlyingThings, initialized from FlyingChairs' model, for several steps can tell the difference.
Looking forward to your early response，thank you again.

[drinkingcoder]

We find that the issue comes from the tile technique. Directly estimating flow instead of tiling the flows largely improves the flow accuracy. Using the FlowFormer trained on C+T:

	Sintel training (clean)	Sintel training (final)
GMA	1.30	2.74
FlowFormer with tile	0.95	2.35
FlowFormer w/o tile	0.64	1.50

The numbers reported in the paper are estimated with tiling. The error reduction compared with GMA comes to 51% and 45%:yum:. Many thanks:star_struck::star_struck:! We will update the paper and the repository ASAP.

[MiaoJieF]

What is tile technique?

[drinkingcoder]

We refer to the tile technique introduced by PerceiverIO. The details are presented in our supplementary.

[drinkingcoder]

I will close this issue now. Feel free to re-open it if you still have questions.

[drinkingcoder]

I'm sorry to say that there is a bug in validation without tiling as pointed out by Dr. Sun 🤕. The new numbers have been updated in README.