[High level OPT][RFC] NNVMv2 IR - Relay #1672

jroesch · 2018-08-29T00:39:15Z

[RFC]: Relay a new high level IR for TVM

Relay is a new high level intermediate representation (IR) intended to act as v2.0 of NNVM.

See #1673 for the full RFC details.

Python 2.7 Support

Our goal is to adapt the core IR code to support 2.7 as well as 3.X. We plan on addressing this near the end of the PR review window after other TODOs have been addressed.

Features

This PR was extracted from a larger work-in-progress version of Relay, and as such it still needs some additional features before it is ready to merge into TVM mainline. We are opening the PR in a "nearly finished" state so other contributors can provide review and feedback on the alpha design and implementation of Relay.

Core IR

Code Map

This section contains a high level overview of the code organization by file.

Core IR

include/tvm/relay/base.h and src/relay/ir/base.cc contain the base classes for Relay AST.
We reuse the tvm::Node infrastructure to enable Python interoperability.
include/tvm/relay/type.h and src/relay/ir/type.cc contain the type classes for the Relay Type AST.
include/tvm/relay/expr.h and src/relay/ir/expr.cc contain the expression classes for the Relay expression AST. This is a small functional language with functions, control-flow, operators, and constants.
include/tvm/relay/op.h and src/relay/ir/op.cc contains the definition of the Relay operator registry. Relay's operator registry stores high level information about operators such as type, and metadata useful for optimization similar to NNVM.
include/tvm/relay/environment.h and src/relay/ir/environment.cc contains the global environment, which contains all global functions and configuration options for the compiler.
include/tvm/relay/error.h
include/tvm/relay/expr_functor.h and include/tvm/relay/expr_visitor.h define traversal strategies over Relay expressions.
include/tvm/relay/logging.h contains a wrapper around logging which can be controlled with an environment variable. This is useful for extra logging information while debugging the compiler.
include/tvm/relay/pass.h exports the current set of Relay passes (only type inference right now).
include/tvm/relay/source_map.h src/relay/source_map.cc

Operators

src/relay/op/tensor/elemwise.cc provides examples of registering Relay operators.
src/relay/op/type_relations.cc and src/relay/op/type_relations.h defines common
type relations (i.e shape inference functions) for operators.

Type Inference

include/tvm/relay/pass/type_infer.h and src/relay/pass/type_infer.cc implement type inference, and type checking. Given a Relay expression (e) they infer a type (T) written (e : T) and check that e has type T. This process proceeds by building partial types for an expression and iteratively filling the type in by solving constraints introduced in the program.
src/relay/pass/incomplete_type.h introduces an "incomplete type" (known as a type variable in the programming language and compiler literature), this represents a portion of the type we do not yet know.
src/relay/pass/unifier.h and src/relay/pass/unifier.cc implement type unification. They solve equations between types which arise due to type inference/checking, the idea is that there are some unknown portions of the type (incomplete types) and our goal is to fill them in given a set of equations on types.
include/tvm/relay/pass/alpha_eq.h and src/relay/pass/alpha_eq.cc implements structural equivalence of Relay types and expressions. This is called alpha equivalence in the programming language and compiler literature.
src/relay/pass/resolve.h and src/relay/pass/resolve.cc implement a procedure which will resolve incomplete types into complete ones based on solutions in the unifier.
src/relay/pass/type_functor.h and src/relay/pass/type_visitor.h implement useful traversal strategies over types.
src/relay/pass/type_subst.h and src/relay/pass/type_subst.cc implement type substitution.

Python Core IR

python/tvm/relay/__init__.py is the entry point of the Relay namespace.
python/tvm/relay/_env.py exports the FFI to the Environment and python/tvm/relay/_env.pyi adds MyPy stubs.
python/tvm/relay/_ir_pass.py exports the FFI to the pass functions and python/tvm/relay/_ir_pass.pyi adds MyPy stubs.
python/tvm/relay/_make.py exports the constructor functions used in expr.py, etc.
python/tvm/relay/base.py, python/tvm/relay/env.py, python/tvm/relay/expr.py, and python/tvm/relay/type.py export their C++ counterparts to Python.
python/tvm/relay/ir_builder.py contains a first attempt at an IRBuilder for Relay.
python/tvm/relay/ir_pass.py contain the pass functions exported by Relay.

Python Operators

python/tvm/relay/op/__init__.py entry point into relay.op namespace.
python/tvm/relay/op/_make.py exposes functions which build calls to each op.
python/tvm/relay/op/tensor.py contains explicit definitions of operators which dispatch to the primitives defined in op/_make.py, this means Relay's Python API has explicit signatures for each op, and don't rely on *args and **kwargs.
python/tvm/relay/op/_tensor.py backend registry for compiler related data about ops in tensor.py.
python/tvm/relay/op/op.py exposes functionality on the Operator Registry.

Python Compiler to TVM RTS

python/tvm/relay/tvm_rts_backend.py contains a soon to be updated version of a compiler from Relay to the TVM RTS.

Timeline

Complete PR checklists.
Leave PR open for ~2 weeks for review and feedback while finishing up TODOs.
Merge PR with @tqchen when he returns from vacation.

Thanks

Finally I would like to thank all the people who helped out with the development of Relay. Although I am the author of most of these commits the work is derived from months of hard work from @MarisaKirisame @slyubomirsky @joshpoll @weberlo @tqchen and @ztatlock.

junrushao · 2018-08-29T03:24:14Z

Nice work! I believe that this is the first elegant and systematic approach in addressing the lack of Turing-completeness in deep learning systems. Will be happy to see the future development of Relay to support richer sets of features that could possibly benefit deep learning practitioners. Will roughly go through it in 24 hours and leave more comments.

BTW, @jroesch could you add a link or bibtex to the Relay paper in the description so that others could easily cite?

junrushao · 2018-08-29T04:00:42Z

This is pretty cool feature to make the dependent type shape of an NDArray generic, but is this able to be dynamic? More specifically, the shape depends on some intermediate result of computation?

Update: my concern is resolved after reading the code.

masahi · 2018-08-29T04:20:09Z

@jroesch how does it relate to HalideIR?

tqchen · 2018-08-29T05:54:36Z

Let us move general discussion to a RFC thread, and keep this thread for specific code discussion

jroesch · 2018-08-29T08:45:32Z

@masahi just moved the proposal bit to the RFC (see #1673) let's chat there.

junrushao · 2018-08-29T17:30:56Z

include/tvm/relay/environment.h

+ *  It contains all global functions, and configuration 
+ *  options.
+ *
+ *  Many operations require acess to the global


"acess" => "access"

zhiics · 2018-08-29T23:49:49Z

include/tvm/relay/environment.h

+#include <string>
+#include <vector>
+#include "./expr.h"
+#include "./type.h"


I think we probably should prefer absolute path and configure include path...

zhiics · 2018-08-30T01:13:08Z

src/relay/source_map.cc

+  }
+}
+
+std::string SourceFragment::SourceAt(Span sp, int max_lines) {


Does it make sense to default max_lines to 0, and return the current line in that case.

What about defaulting to 1? unless I'm missing something?

Sorry, I probably misunderstood it. 1 works if it means the current line.

zhiics · 2018-08-30T01:14:30Z

src/relay/source_map.cc

+  return source_slice;
+}
+
+SourceName SourceMap::AddSource(std::string file_name, std::string source) {


Just curious, do it make sense to allow source replacement, e.g. replace a line or a chunk of code?

I don't think so, the current design is to enable three use cases.

The user uses a Relay frontend (such as the one we prototyped) and we register the chunk of Python code written by the user.

We parse the IR from a textual format and are able to report directly against the file inputted.

We generate a source representation of a programmatic IR which we use to report errors against. The idea is the user builds a graph by hand either in Python or one of the frontends, we pretty print and then parse the result giving us a "source program" to show the error against.
This final one is very useful as we can trigger errors when trying to optimize and point directly to a cleanly printed representation of the program.

I see. Thanks.

zhiics · 2018-08-30T01:15:41Z

include/tvm/relay/environment.h

+  Function Lookup(const std::string & s);
+
+  /*! \brief Add a source fragment to the environment. */
+  SourceName AddSource(std::string file_name, std::string source);


Nit: make both of them const std::string& ?

include/tvm/relay/type.h

zhiics · 2018-08-30T01:18:00Z

src/relay/pass/unifier.h

+  explicit UnionFind(std::shared_ptr<tvm::Node> p) : NodeRef(p) {}
+
+  // no const so that union find can be mutable as a member of unifier
+  inline UnionFindNode* operator->() const {


I think there is no need to put inline before member functions, compiler will automatically inline them if it is necessary/profitable.

zhiics · 2018-08-30T01:19:41Z

include/tvm/relay/expr.h

+
+RELAY_DEFINE_NODE_REF(Function, FunctionNode, Expr);
+
+using Attrs = tvm::Attrs;


nit: Move it into class CallNode? Seems that's the only place you need it.

Yeah I think this is an old artifact of when this was typedef'd to something else. I will just remove and use tvm::Attrs.

I would also prefer tvm::Attrs

include/tvm/relay/expr_functor.h

zhiics · 2018-08-30T01:23:37Z

include/tvm/relay/expr_visitor.h

+    for (auto param : op->params) {
+      Expr param_expr = this->VisitExpr(param, args...);
+      if (const ParamNode* param_node = param_expr.as<ParamNode>()) {
+        auto param = GetRef<Param>(param_node);


change variable name? it has the same name as the auto param

masahi · 2018-08-30T08:12:35Z

Another style issue: it looks like & position in function arguments are basically random everywhere. They need to be consistent.

masahi · 2018-08-30T08:16:30Z

include/tvm/relay/pass/type_infer.h

+/*! \brief Ensures that an operator is well-formed with respect
+ * to Relay's type system.
+ */
+Op CheckOp(const Environment & env, const Op & op);


a more descriptive name would be helpful.

alex-weaver · 2018-08-30T12:12:48Z

include/tvm/relay/base.h

+/*!
+ * \brief we always used NodeRef for referencing nodes.
+ *
+ *  By default, NodePtr is a std::shared_ptr of node


Is this comment intended to say NodeRef rather than NodePtr ?

tqchen · 2018-08-31T19:51:41Z

include/tvm/relay/pass/type_infer.h

+ *  The pass produces a new expression with its checked_type
+ *  field populated and incomplete types resolved.
+ */
+#ifndef TVM_RELAY_PASS_TYPECHECKER_H_


TYPE_INFER_H_

tqchen · 2018-08-31T19:53:45Z

include/tvm/relay/source_map.h

+namespace tvm {
+namespace relay {
+
+struct SourceFragment {


need explaination of the example usage.

tqchen · 2018-08-31T19:54:06Z

include/tvm/relay/source_map.h

+struct SourceFragment {
+  std::string file_name;
+  std::vector<std::string> source_lines;
+


consider make it as a Node if we are supposed to access from python

tqchen · 2018-08-31T19:55:34Z

include/tvm/relay/expr_visitor.h

+};
+
+template <typename... Args>
+class ExprFVisitor : public ::tvm::relay::ExprFunctor<Expr(const Expr& n, Args...)> {


To make things consistent with low-level naming convention, this should be ExprMutator. See https://github.com/dmlc/tvm/blob/master/include/tvm/ir_mutator.h

Consider make function signature as the follows(which is consistent with tvm::IRMutator)

Expr Mutate_(const ConstOpNode* op, const Expr& self) { }

This removes the need of GetRef in the default leaf case

In general I would prefer to never use GetRef but it doesn't provide us the down-casted data which would be far more useful. Is it the extra argument useful enough to be passed everywhere?

tqchen · 2018-08-31T19:56:56Z

include/tvm/relay/expr_visitor.h

+
+  void VisitExpr_(const TupleNode* op, Args... args) override {
+    for (auto field : op->fields) {
+      this->VisitExpr(field, args...);


need to use std::forward(args)...

include/tvm/relay/logging.h

tqchen · 2018-08-31T19:58:52Z

include/tvm/relay/pass.h

+
+/*! The result of type checking an expression is a new expression
+ * with unambigous type information filled in, as well as it's
+ * checked type field populated with the result type.


document all parameters and return values

tqchen · 2018-08-31T19:59:46Z

include/tvm/relay/pass/alpha_eq.h

+#include "../expr.h"
+
+namespace tvm {
+namespace relay {


consider directly move the relay/pass.h for now

tqchen · 2018-08-31T20:00:04Z

include/tvm/relay/pass/alpha_eq.h

+namespace tvm {
+namespace relay {
+
+bool AlphaEqual(const Expr & e1, const Expr & e2);


style: const Expr&

tqchen · 2018-08-31T20:00:30Z

include/tvm/relay/pass/type_infer.h

+
+/*! \brief Ensures that an operator is well-formed with respect
+ * to Relay's type system.
+ */


move to relay/pass.h

tqchen · 2018-08-31T20:09:53Z

include/tvm/relay/expr_visitor.h

+};
+
+template <typename... Args>
+class ExprFVisitor : public ::tvm::relay::ExprFunctor<Expr(const Expr& n, Args...)> {


Consider make function signature as the follows(which is consistent with tvm::IRMutator)

Expr Mutate_(const ConstOpNode* op, const Expr& self) { }

This removes the need of GetRef in the default leaf case

joshpoll

Mostly comments on comments. Will leave more soon.

joshpoll · 2018-09-02T17:56:13Z

include/tvm/relay/base.h

+/*!
+ * \brief Macro to make it easy to define node ref type given node
+ * \param TypeName The name of the reference type.
+ * \param NodeName The internal contrainer name.


contrainer -> container

joshpoll · 2018-09-02T17:56:45Z

include/tvm/relay/base.h

+class Span;
+/*!
+ * \brief Stores locations in frontend source that generated a node.
+ *


include/tvm/relay/base.h

joshpoll · 2018-09-02T17:58:31Z

include/tvm/relay/base.h

+ */
+class SourceName;
+/*!
+ * \brief The source name in the Span


Is this an identifier or source code? The comment should make this clear.

joshpoll · 2018-09-02T17:59:42Z

include/tvm/relay/base.h

+ */
+class RelayNode : public Node {
+ public:
+  /*! \brief The debug information, can be null, check with span.defined() */


This isn't exactly true. It's just source map info. What about: "Location of node in source." or something.

include/tvm/relay/expr.h

joshpoll · 2018-09-02T18:15:54Z

include/tvm/relay/expr.h

+  /*! \return The corresponding tensor type of the data */
+  TensorType tensor_type() const;
+
+  /*! \return whether it is scalar(rank-0 tensor) */


Whether it is scalar (i.e., a rank-0 tensor).

include/tvm/relay/expr.h

joshpoll · 2018-09-02T18:18:11Z

include/tvm/relay/expr.h

+ * \brief Local variables used in the let expression.
+ * This is similar to Var that is being used in the low level tensor expression.
+ *
+ * \note Each LocalVar is bind only once and is immutable/


Are bind-once and immutability different? Also the comment should end in a period.

joshpoll · 2018-09-02T18:19:00Z

include/tvm/relay/expr.h

+/*! \brief Container for LocalVar */
+class LocalVarNode : public ExprNode {
+ public:
+  /*! \brief The name of the variable, this only acts as a hint. */


-> The name of the variable. This only acts as a hint.

What does hint mean? Who does it provide a hint for?

weberlo · 2018-09-01T19:24:25Z

include/tvm/relay/environment.h

+  void Update(const GlobalVar& var, const Function & func);
+  void Remove(const GlobalVar& var);
+
+  /*! \brief Lookup a global function by its variable. */


weberlo · 2018-09-01T20:37:38Z

include/tvm/relay/expr.h

+  /*! \brief The expression evaluated when condition is true. */
+  Expr true_value;
+  /*! \brief The expression evaluated when condition is false */
+  Expr false_value;


Should these be called true_expr and false_expr or maybe true_branch and false_branch?

include/tvm/relay/logging.h

weberlo · 2018-09-01T21:50:40Z

include/tvm/relay/op.h

+    }
+    return *this;
+  }
+  /*! \return The global single retistry */


include/tvm/relay/op.h

weberlo · 2018-09-02T00:36:09Z

src/relay/ir/op.cc

+  std::unordered_map<std::string, std::unique_ptr<GenericOpMap> > attr;
+  // frontend functions
+  std::vector<PackedFunc*> frontend_funcs;
+  // get singleton of the


Incomplete comment?

src/relay/pass/alpha_eq.cc

weberlo · 2018-09-02T05:24:53Z

src/relay/pass/unifier.cc

+}
+
+void UnionFindNode::unify(const IncompleteType &v1, const Type &t) {
+  RELAY_LOG(INFO) << "UnionFindNode::Unify v1=" << v1 << "t=" << t << std::endl;


Should "t=" be " t="?

src/relay/pass/unifier.h

weberlo · 2018-09-02T22:04:11Z

src/relay/source_map.cc

+  std::stringstream out;
+
+  // We need to move from 1 based indexing to zero based indexing.
+  int starting_line = sp->lineno;


What does this comment have to do with the line that follows?

yzhliu · 2018-09-04T21:16:11Z

include/tvm/relay/error.h

+namespace relay {
+
+struct Error : dmlc::Error {
+    Error(std::string msg) : dmlc::Error(msg) {}


2-space indent instead. and shall we use explicit?

yzhliu · 2018-09-04T21:16:22Z

include/tvm/relay/error.h

+};
+
+struct InternalError : Error {
+    InternalError(std::string msg) : Error(msg) {}


same as above.

yzhliu · 2018-09-04T21:26:03Z

include/tvm/relay/expr_visitor.h

+      auto type = this->VisitType(op->type);
+      return ParamNode::make(var, type);
+    } else {
+      throw dmlc::Error("the default param visitor has bug");


I guess it is better to use relay::Error instead.

yzhliu · 2018-09-04T21:26:18Z

include/tvm/relay/expr_visitor.h

+        auto ty_param_ref = GetRef<TypeParam>(ty_param);
+        ty_params.push_back(ty_param_ref);
+      } else {
+        throw dmlc::Error("the default func visitor has bug");


relay::Error

zhiics · 2018-09-04T01:35:15Z

include/tvm/relay/error.h

+namespace relay {
+
+struct Error : dmlc::Error {
+    Error(std::string msg) : dmlc::Error(msg) {}


const std::string&

zhiics · 2018-09-04T01:47:48Z

include/tvm/relay/error.h

+};
+
+struct InternalError : Error {
+    InternalError(std::string msg) : Error(msg) {}


zhiics · 2018-09-04T01:48:17Z

include/tvm/relay/error.h

+struct SpannedError {
+  std::string msg;
+  Span sp;
+  SpannedError(std::string msg, Span sp) : msg(msg), sp(sp) {}


zhiics · 2018-09-04T02:06:08Z

include/tvm/relay/expr.h

+ * \brief Global variable that leaves in the top-level environment.
+ * This is used to enable recursive calls between function.
+ *
+ * \note GlobalVar can only corresponds to functions.


s/corresponds/correspond

include/tvm/relay/expr.h

zhiics · 2018-09-04T03:03:41Z

include/tvm/relay/expr_functor.h

+ * \sa tvm/ir_functor.h
+ *
+ * \tparam FType function signiture
+ *  This type if only defined for FType with function signiture R(const Expr&,


zhiics · 2018-09-04T03:17:18Z

include/tvm/relay/type.h

+    kType     = 3,
+  };
+  /*!
+   * \brief The variable


s/The variable/d

zhiics · 2018-09-04T03:32:48Z

src/relay/pass/unifier.cc

+
+void UnionFindNode::debug() {
+  for (auto entry : this->uf_map) {
+    std::cout << entry.first << " = " << entry.second << std::endl;


Use LOG(INFO) instead of std::cout?

MarisaKirisame · 2018-09-05T23:14:54Z

python/tvm/relay/type.py

+    ShapeVar = 0
+    Shape = 1
+    BaseType = 1
+    Type = 2


enum Kind : int {
/*! \brief template variable in shape expression */
kShapeVar = 0,
kShape = 1,
kBaseType = 2,
kType = 3,
};
I am certain that this is an error

MarisaKirisame · 2018-09-05T23:16:26Z

src/relay/ir/op.cc

+  std::unordered_map<std::string, std::unique_ptr<GenericOpMap>> attr;
+  // frontend functions
+  std::vector<PackedFunc*> frontend_funcs;
+  // get singleton of the


comment is off

MarisaKirisame · 2018-09-05T23:16:38Z

src/relay/ir/op.cc

+  // get singleton of the
+  static OpManager* Global() {
+    static OpManager inst;
+    return &inst;


why not return OpManager&?

src/relay/ir/op.cc

MarisaKirisame · 2018-09-05T23:17:31Z

src/relay/ir/op.cc

+  if (auto ty_func = op.as<FuncTypeNode>()) {
+    return ty_func->type_params.size() != 0;
+  } else {
+    return false;


should this be an error instead?

MarisaKirisame · 2018-09-05T23:20:30Z

src/relay/op/type_relations.cc

+  return imm->value;
+}
+
+Array<Type> IdentityRel(const Array<Type>& types, int num_args) {


why does it need num_args when we can always get it from types?

Because the array contains input and output variables, we made it this way to simplify the calling convention between C++ and Python.

MarisaKirisame · 2018-09-05T23:27:17Z

src/relay/op/type_relations.cc

+              << (full_len - suffix_len - 1) << std::endl;
+    auto lower_bound = full_len - suffix_len - 1;
+
+    for (int64_t i = full_len - 1; i > lower_bound; i--) {


this will out of range immediately on lb of different size

suppose sh1.size() is 10, sh2.size() is 5
lb = 4
i originally = 9
will out of bound

MarisaKirisame · 2018-09-05T23:37:10Z

src/relay/op/type_relations.cc

+      std::cout << "Index i=" << i << std::endl;
+      auto dim1 = to_int(sh1[i]);
+      auto dim2 = to_int(sh2[i]);
+      if (dim1 != dim2) {


one of them can be 1 in this case, no need to immediately Check false

jroesch · 2018-09-06T10:54:31Z

Thanks for feedback going to try and get PR in a clean state tomorrow. I spent a lot of today adding the final features we need and looking into updating docs.

sergei-mironov · 2018-09-07T14:07:19Z

python/tvm/relay/tvm_rts_backend.py

+    def lookup(self, ident: ir.LocalId) -> NodeRef:
+        return self.id_map[ident]
+
+    def compile(self, func: ir.Function) -> None:


What is the advantages of keeping compile algorithm implemented in Python rather than in C++? I would say that keep it in C++ is better because it allows us to write DSL frontends for other languages more easily.

Eventually we will port it to C++, the idea was to demonstrate some passes in C++ and Python, as the alpha of Relay progresses more and more will be moved into C++.

tqchen · 2018-09-19T00:05:09Z

include/tvm/relay/expr.h

+  /*!
+   * \brief Stores the result of type inference(type checking).
+   *
+   * \note This can be undefined before type inference.


remove this note as it is now serialized

I believe this is still accurate, before type inference the node may still contain null.

include/tvm/relay/expr.h

tqchen · 2018-09-19T17:42:55Z

src/relay/op/type_relations.cc

+      rev_sh2++;
+    }
+
+    Array<HalideIR::Expr> larger;


Use ShapeExpr

tqchen · 2018-09-19T17:43:07Z

src/relay/op/type_relations.cc

+    Array<HalideIR::Expr> smaller;
+
+    for (int i = 0; i < (full_len - suffix_len); i++) {
+      smaller.push_back(tvm::ir::IntImm::make(HalideIR::Int(64), 1));


Int function is already defined

jroesch · 2018-09-19T22:00:47Z

I think this is now in a state where we should merge and address any other issues in follow-up PRs.

tqchen · 2018-09-19T22:58:47Z

Thanks, @yzhliu @jroesch @alex-weaver @zhiics @junrushao1994 @masahi @joshpoll @weberlo @grwlf @MarisaKirisame for reviews.

I merge this PR given that it reaches a state of test passing and most comments addressed. Let us get the balls rolling with follow up PRs

jroesch · 2018-09-19T23:12:32Z

I plan on opening two PRs one for the evaluation and one for the documentation and reference manual.

tqchen changed the title ~~Add initial version of the Relay IR~~ [High level OPT][RFC] NNVMv2 IR Aug 29, 2018

tqchen added the status: need RFC need RFC discussion label Aug 29, 2018

tqchen changed the title ~~[High level OPT][RFC] NNVMv2 IR~~ [High level OPT][RFC] NNVMv2 IR - Relay Aug 29, 2018

jroesch mentioned this pull request Aug 29, 2018

[RFC] NNVMv2 IR - Relay #1673

Closed

tqchen added status: RFC status: need review and removed status: need RFC need RFC discussion labels Aug 29, 2018

junrushao reviewed Aug 29, 2018

View reviewed changes

zhiics reviewed Aug 29, 2018

View reviewed changes

zhiics reviewed Aug 30, 2018

View reviewed changes

masahi reviewed Aug 30, 2018

View reviewed changes

alex-weaver reviewed Aug 30, 2018

View reviewed changes

tqchen requested changes Aug 31, 2018

View reviewed changes

joshpoll reviewed Sep 2, 2018

View reviewed changes

weberlo reviewed Sep 2, 2018

View reviewed changes

yzhliu reviewed Sep 4, 2018

View reviewed changes

zhiics reviewed Sep 4, 2018

View reviewed changes

jroesch force-pushed the relay branch from 40d71a3 to 888914a Compare September 5, 2018 23:18

MarisaKirisame requested changes Sep 5, 2018

View reviewed changes

MarisaKirisame reviewed Sep 5, 2018

View reviewed changes

sergei-mironov reviewed Sep 7, 2018

View reviewed changes

jroesch force-pushed the relay branch from a87c7b9 to 3a98c3d Compare September 7, 2018 22:08

jroesch added 3 commits September 18, 2018 16:33

Remove alpha_eq tests

defa8e4

Style fix in expr.cc

167d235

Fix & style

a6b6038

tqchen requested changes Sep 19, 2018

View reviewed changes

jroesch added 9 commits September 18, 2018 17:20

Fix more style issues

41098c4

Remove whitespace

bee4234

Address final comments modulo type_infer.cc

7e86f56

Address CR and work on 2.7 support

8339eaf

Move annotations to expr.pyi

a7b67ad

Fix bug introduced by refactor

a1734d7

Python2 and Python3 tests pass locally

4976808

Refactor type relation solver.

f074edd

Fix PyLint

d7835fa

tqchen requested changes Sep 19, 2018

View reviewed changes

jroesch added 3 commits September 19, 2018 13:24

Fix PyLint

9281077

Repair tests

b1c8ecd

Add initial version of concat

b5c5626

jroesch force-pushed the relay branch from 789f13c to b5c5626 Compare September 19, 2018 20:24

jroesch added 4 commits September 19, 2018 13:28

Fix issue in type_relations.cc

388b275

Fix PyLint ... again

81c9d6d

Fix issue with test case

4e0c7b8

Fix & style

052d619

tqchen approved these changes Sep 19, 2018

View reviewed changes

tqchen merged commit 51fe00f into apache:master Sep 19, 2018

jroesch mentioned this pull request Sep 21, 2018

[DOC][Relay]: Add API docs for Relay. #1750

Merged

FrozenGene pushed a commit to FrozenGene/tvm that referenced this pull request Dec 27, 2018

[High level OPT][RFC] NNVMv2 IR - Relay (apache#1672)

0801889

jroesch deleted the relay branch February 4, 2021 04:36


		RELAY_DEFINE_NODE_REF(Function, FunctionNode, Expr);

		using Attrs = tvm::Attrs;

[High level OPT][RFC] NNVMv2 IR - Relay #1672

[High level OPT][RFC] NNVMv2 IR - Relay #1672

Uh oh!

Conversation

jroesch commented Aug 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

[RFC]: Relay a new high level IR for TVM

Python 2.7 Support

Features

Core IR

Code Map

Core IR

Operators

Type Inference

Python Core IR

Python Operators

Python Compiler to TVM RTS

Timeline

Thanks

Uh oh!

junrushao commented Aug 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

junrushao commented Aug 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

masahi commented Aug 29, 2018

Uh oh!

tqchen commented Aug 29, 2018

Uh oh!

jroesch commented Aug 29, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

masahi commented Aug 30, 2018

Uh oh!

masahi Aug 30, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tqchen Aug 31, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

jroesch commented Aug 29, 2018 •

edited

Loading

junrushao commented Aug 29, 2018 •

edited

Loading

junrushao commented Aug 29, 2018 •

edited

Loading

masahi Aug 30, 2018 •

edited

Loading

tqchen Aug 31, 2018 •

edited

Loading

tqchen Aug 31, 2018 •

edited

Loading