[BUG]: backend for supporting distributed training

[haofanwang]

🐛 Describe the bug

Describe the bug
I'm training with the dreambooth example with RDMA. But current backend (gloo) doesn't support it. It leads to following error.

RuntimeError: [../third_party/gloo/gloo/transport/tcp/pair.cc:799] connect 
[127.0.1.1]:29591: Connection refused

I check the corresponding function, and I guess this line is a typo?

old:
cpu_group = dist.new_group(ranks, backend='gloo') if dist.get_backend() != 'gloo' else None
new:
cpu_group = dist.new_group(ranks, backend='gloo') if dist.get_backend() == 'gloo' else None

Environment

No response

[haofanwang]

I find that the typo above existed in many other files, will add another PR soon.

[FrankLeeeee]

Great, thanks!

[haofanwang]

@FrankLeeeee Could you re-check this PR?

[FrankLeeeee]

Okay sure.

[kurisusnowdeng]

The problem is mostly caused by the firewall since GLOO uses random ports to initialize groups

[binmakeswell]

We have updated a lot. This issue was closed due to inactivity. Thanks.