CUDA out of memory error

[Shibrel]

I get the error "CUDA out of memory", no matter the dataset when using my 4GB GPU. Can you reduce the Batch size or is there another solution you know?

[ifei17-hfut]

Your problem can be solved by reducing batch_size in configs/default.py

[Shibrel]

Thanks for the comment. Unfortunately this leads to the same error. Only the missing memory is different. Non proportional to the batch size.

[MiaoJieF]

Your GPU memory is too small. 4GB of GPU memory can only be used for inference, not training.

[drinkingcoder]

As discussed above, I also believe this is caused by the small GPU memory. We train our model with V100x4 on FlyingChairs. You can try to change the configs to reduce the model size. For example, switching twins' of fnet&cnet to CNN', setting `encoder_depth' to 1, etc.

[drinkingcoder]

We now provide a small version of FlowFormer, which requires a smaller memory. The corresponding config has also been updated. Please refer to README for details.

[Ouya-Bytes]

I use V100x2 with 32GB/1GPU, and "small_things_eval.py" config files ， the model training params only can set batch_size=2, more than 2 will cause "Out of memory" error.
In addition, the output “nan” in some loss items. Can you provide some solutions？ Thanks.

eg：
INFO | core.utils.logger:_print_training_status:19 - [ 200, [8.977348434377084e-06]] nan, 0.8528, 0.9766, 0.9255, nan, 0.9431, 2.466