Enhance MemoryEfficientSwish with optional import of SiLU #2812
Description
From Pytorch 1.7, SiLU is implemented, it is the optimized version of Swish.
In addition Mish needs to support the parameter inplace when using torch's native version.