Training on a custom dataset

[owen24819]

Hi @drinkingcoder,

Really nice work! Really cool to see the use of transformers in optical flow. I have more of a question than an issue. I am interested in training FlowFormer on my own dataset. Is it possible create your own dataset? If so, can you give any recommendations / tools to help curate your own dataset? For context, I know very little about optical flow; I just learned about the field last month.

Thanks,
Owen

[drinkingcoder]

To the best of my knowledge, there may be two solutions to create datasets for optical flow: 1) Rendering synthetic images and obtaining GT flows via the renderer. (FlyingChairs, Things, and Sintel) 2) Transferring depth to flow with camera poses. (KITTI and MegaDepth). However, this strategy requires the scene is almost rigid and dense depth is not cheap. Directly annotating pixel-wise flow via humans is also too expensive. Our arxiv pre-print LIFE is devoted to partially relieve learning flow from such expensive training data, i.e., weakly supervised flow learning. Hope this paper can help you.

[owen24819]

I appreicate the response. I will check out your other paper!