[Relay][Pytorch] Add aten::new_ones, aten::new_full, aten::fill_, aten::pad, aten::reshape_as and atem::empty_like

[shingjan]

This PR intends to add the following five ops for the pytorch frontend:

• aten::new_ones
• aten::new_full
• aten::fill_
  - aten::pad (This op is relocated to [PyTorch] [Relay] Add aten::pad #11922 )
• aten::reshape_as
• aten::empty_like

Note that arguments to the above ops will be changed by relay at runtime so verify_model won't work on them.

cc: @masahi @junrushao1994 @zxybazh

[masahi]

Please add tests. If verify_model cannot be used, we can always manually compile, set input, and run.

[shingjan]

@masahi Tests are added. Should be good to take another look.

[shingjan]

@tvm-bot rerun

[shingjan]

@masahi It is possible that tests for empty and empty_like are flaky. It failed on CI once but by reading the outputs it shouldn't fail. Just to give a heads-up.

[masahi]

Did they ever fail in your local testing?

[shingjan]

@masahi Local testing no. But there is one CI failure:
https://ci.tlcpack.ai/blue/rest/organizations/jenkins/pipelines/tvm/branches/PR-11896/runs/9/nodes/383/steps/1174/log/?start=0

This is the stacktrace:

[2022-06-29T08:34:53.751Z] _______________________________ test_empty_like ________________________________
[2022-06-29T08:34:53.751Z] 
[2022-06-29T08:34:53.751Z]     def test_empty_like():
[2022-06-29T08:34:53.751Z]         def test_func(data):
[2022-06-29T08:34:53.751Z]             return torch.empty_like(data)
[2022-06-29T08:34:53.751Z]     
[2022-06-29T08:34:53.751Z] >       verify_model_with_input(test_func, [torch.rand([1, 3, 10, 10]).float()])
[2022-06-29T08:34:53.751Z] 
[2022-06-29T08:34:53.751Z] tests/python/frontend/pytorch/test_forward.py:3827: 
[2022-06-29T08:34:53.751Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2022-06-29T08:34:53.751Z] tests/python/frontend/pytorch/test_forward.py:221: in verify_model_with_input
[2022-06-29T08:34:53.751Z]     tvm.testing.assert_allclose(baseline_outputs, compiled_output, rtol=1e-5, atol=1e-5)
[2022-06-29T08:34:53.751Z] python/tvm/testing/utils.py:113: in assert_allclose
[2022-06-29T08:34:53.751Z]     np.testing.assert_allclose(actual, desired, rtol=rtol, atol=atol, verbose=True)
[2022-06-29T08:34:53.751Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2022-06-29T08:34:53.751Z] 
[2022-06-29T08:34:53.751Z] x = array([[[[ 7.7170313e-17,  0.0000000e+00,  5.3485164e-26,
[2022-06-29T08:34:53.751Z]            0.0000000e+00,  0.0000000e+00,  0.0000000e+00,
[2022-06-29T08:34:53.751Z]   ...  0.0000000e+00,
[2022-06-29T08:34:53.751Z]            0.0000000e+00,  0.0000000e+00,  0.0000000e+00,
[2022-06-29T08:34:53.751Z]            0.0000000e+00]]]], dtype=float32)
[2022-06-29T08:34:53.751Z] y = array([[[[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[2022-06-29T08:34:53.751Z]          [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[2022-06-29T08:34:53.751Z]          [0., 0....         [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[2022-06-29T08:34:53.751Z]          [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]]]], dtype=float32)
[2022-06-29T08:34:53.751Z] func = <ufunc 'isnan'>, hasval = 'nan'
[2022-06-29T08:34:53.751Z] 

looks like those two tensors should pass the allclose test but they failed on CI anyway

[driazati]

it's since failed a few times on main, I filed #11968 to disable it until there is a fix