In my case, the following code seems to consume heavy cpu usage during the backward pass in FunLSQ. (>1000% cpu usage).
indicate_middle = torch.ones(indicate_small.shape).to(indicate_small.device) - indicate_small - indicate_big
When I turn it to
indicate_middle = 1.0 - indicate_small - indicate_big
, the problem gets solved.
I wonder whether this is also a case for other models in your case. If yes, this could be a reason to explain why method 2 is slower than method 1.
In my case, the following code seems to consume heavy cpu usage during the backward pass in FunLSQ. (>1000% cpu usage).
When I turn it to
, the problem gets solved.
I wonder whether this is also a case for other models in your case. If yes, this could be a reason to explain why method 2 is slower than method 1.