[RELAY][RUNTIME] Add compute and schedule attributes for all ops in relay/op/tensor.py

[jroesch]

This PR ports over schedule and compute attributes for the operators in tensor.py. I hope this will serve as an example of adding this behavior for operators so others in the community can help continue the work of making Relay operator complete.

I simply copied over the same scheduling primitives from NNVM, and am using out of the box compute functions. Feedback much appreciated.

For details on helping with the rest of the operators please read here #2051.

cc @tqchen @MarisaKirisame @slyubomirsky @zhiics @siju-samuel

There are two missing cases (concatenate and copy both of which require a little more work).

[MarisaKirisame]

this code seems way too highly repetitive to me.
this is a typical snippet of _tensor.py, with every other line being virtually like this.

def sqrt_compute(attrs, inputs, output_type, target):
    assert len(inputs) == 1
    return [topi.sqrt(inputs[0])]
register_compute("sqrt", sqrt_compute)
register_schedule("sqrt", schedule_broadcast)

there are 3 things:
define, which assert len and use topi
register_compute
register_schedule

register_compute, register_schedule could be merged into one single function
sqrt_compute can be made into one line by writing a hof that take two argument (the topi func and num of input)

If I were to write this, I will declare a table that store the name, and the num of input, traverse it, and define function with metaprogramming, to minimize the chance of error, and to maximize reuse(other code that build ontop of relay can simply read the table).
It will be

registry = {}
registry['log'] = 1
registry['add'] = 2
...

Of course, it might be going too far for tvm coding style, but fusing register_compute/register_schedule is certainly doable, and so is using a hof.

[tqchen]

As per https://docs.tvm.ai/contribute/code_review.html#ensure-test-coverage please add test case for each of the operator added

reference to https://github.com/dmlc/tvm/blob/master/nnvm/tests/python/compiler/test_top_level1.py#L167

[tqchen]

Please document the public facing functions

[tqchen]

All the registry functions should goto op.registry.py

[jroesch]

@MarisaKirisame the argument against that style (which is roughly) what NNVM has done is that it becomes much harder for users to read and modify. Most code is effectively read-only, and especially code like this will not change often. I think it is better to err on repetitive and readable than inscrutable.

[tqchen]

the output data type can likely be wrong here

[tqchen]

All the registry functions should goto op.registry.py

[tqchen]

As per nnvm tradition, we should move most of the compute function to c++, and only leave a few(e.g. conv2d) in python

[jroesch]

Our current plan is to do this in a second pass.

[zhiics]

It seems that we have several classes named as Module including the ones from runtime. I am totally okay with them, but would it be a little confusing for TVM beginners? But again, I like the renaming from Environment to Module as it is more friendly to those who are familiar with LLVM.

[tqchen]

Merge this in and will update the follow-up on #2059 to depend on this