[FRONTEND][TENSORFLOW] Enhancements.

[srkreddy1238]

* Generalize the shape with explicite argument.
* Supported entire range of mobilenet_v2 models.
* Cast op updated to latest tensorflow.
* Documentation updates.
* CheckNumerics op handling without exception.
* API to fetch test data from tensorflow official releases.

Thanks for contributing to TVM! Please refer to guideline https://docs.tvm.ai/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from others in the community.

[srkreddy1238]

cc @masahi @sgrechanik-h @Huyuwei @FrozenGene welcome to review

[sgrechanik-h]

shape={} will behave the same way as shape=None. Is it intended? I'm thinking about some hypothetical corner cases like when there are no inputs.

[srkreddy1238]

It's an optional arg to override any other way of getting output shapes. Leaving { } will result in inferring wrong shapes for a graph with inputs.
I feel model with no inputs is unrealistic and can be ignored here.

[srkreddy1238]

@sgrechanik-h please have a look.
I have tested all models (MobilenetV1/V2, Resnet, Inception, VGG) from research/slim.

[srkreddy1238]

@tqchen
I have made some utils and documentation related to generating model protobuf files from tensorflow official checkpoints at https://github.com/srkreddy1238/dmlc_data/tree/master/work/tf/samples.

Do you think we could accommodate them in TVM as docs & utils?
Or
Make them as test cases but deactivate?

[tqchen]

In the interest of minimum codebase, let us keep it as a separate thing, but we can generate the proto and download them in the testcase. Thanks @srkreddy1238

[srkreddy1238]

@tqchen
I am not talking about adding the protobuf files into TVM.
Also tensorflow research/slim models are nearly 40, adding all of them into test cases delays the CI time.
Some of them need building few utils from tensorflow source too (Resnet, Vgg) to prepare protobuf from checkpoint.
My interest here is about sharing the steps(tutorials) from tensorflow release to NNVM frontend import.

What do you suggest ?

[tqchen]

Maybe we can build up another set of test that runs nightly, and the normal fast test jobs to make sure general coverage

[tqchen]

Another possible approach is to provide the conversion dry run, which converts the model but does not run them. This can considerably reduce the test time and can be placed in the CI

[srkreddy1238]

+1 for nightly build.

Conversion dry run is not a straight forward as checkpoint to protobuf conversion need https://github.com/tensorflow/models (to generate initial protobuf) and freeze_graph built from tensorflow source to embed checkpoint into it.

I may revisit into simplifying/automating this later for nightly build. For now I will leave a doc @ docs/frontend/tensorflow.md.

[srkreddy1238]

@nishi-t welcome to review.

[nishi-t]

I found the case that causes KeyError in self._nodes[i] in my current work (PR #2001 ). I think that it is needed to handle KeyError. And, I'd appreciate your help.

For example:

    def check_split_concat(ishape, **kwargs):
        inp_array = np.random.uniform(size=ishape).astype(np.float32)
        with tf.Graph().as_default():
            in1 = tf.placeholder(shape=inp_array.shape, dtype=inp_array.dtype)
            splited = tf.split(in1, **kwargs)
            tf.concat(splited, axis=1)
            compare_tf_with_tvm(inp_array, 'Placeholder:0', 'concat:0')

    check_split_concat((5, 30), num_or_size_splits=[15, 15], axis=1)

In above code, the concat node has three inputs(split, split1, concat/axis), but split1 is not registered in _nodes.(the detail of graphdef in this case is here) Moreover, these split:X is increased depending on the number of split.

update: fix the link of my gist

[nishi-t]

Please don't care above my comment. I understand it is a problem which should be resolved separately from this PR. Thanks ;)

[srkreddy1238]

cc @nishi-t have another look pls.

[nishi-t]

Looks good to me

[yzhliu]

Thanks! this is now merged.