Add a tutorial for control flow operators. by zheng-da · Pull Request #12340 · apache/mxnet

zheng-da · 2018-08-24T21:30:03Z

Description

This is to provide a tutorial for control flow operators.

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

Feature1, tests, (and when applicable, API doc)
Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

zheng-da · 2018-08-24T22:45:41Z

@aaronmarkham could you review this tutorial, please?

aaronmarkham

Nice tutorial. I have a few suggestions though.

aaronmarkham · 2018-08-24T23:16:09Z

+MXNet currently provides three control flow operators: `cond`, `foreach` and `while_loop`. Like other MXNet operators, they all have a version for NDArray and a version for Symbol. These two versions have exactly the same semantics. We can take advantage of this and use them in Gluon to hybridize models.
+
+In this tutorial, we use a few examples to demonstrate the use of control flow operators in Gluon and show how a model that requires control flow is hybridized.
+


Are there any prerequisites worth mentioning?

I don't think so.

aaronmarkham · 2018-08-24T23:48:20Z

+foreach(body, data, init_states, name) => (outputs, states)
+```
+
+It iterates over the first dimension of the input data (it can be an array or a list of arrays) and run the Python function defined in `body` for every slice from the input arrays. The signature of the `body` function is defined as follows:


aaronmarkham · 2018-08-24T23:50:05Z

+
+The inputs of the `body` function have two parts: `data` is a slice of an array (if there is only one input array in `foreach`) or a list of slices (if there are a list of input arrays); `states` are the arrays from the previous iteration. The outputs of the `body` function also have two parts: `outputs` is an array or a list of arrays; `states` is the computation states of the current iteration. `outputs` from all iterations are concatenated as the outputs of `foreach`.
+
+The pseudocode below illustrates the execution of `foreach`.


use "the following" instead of below (different consumption modes means the content isn't always below)

aaronmarkham · 2018-08-24T23:50:56Z

+```
+
+### Example 1: foreach works like map
+`foreach` can work like a map function of a functional language. In this case, the states of foreach can be an empty list, which means the computation doesn't carry computation states across iterations.


tick each use of foreach

aaronmarkham · 2018-08-24T23:52:00Z

+### Example 1: foreach works like map
+`foreach` can work like a map function of a functional language. In this case, the states of foreach can be an empty list, which means the computation doesn't carry computation states across iterations.
+
+In this example, we use `foreach` to add each element in an array by one.


use foreach with an array to increase each element's value by one

aaronmarkham · 2018-08-24T23:58:18Z

+```
+
+
+```python


why two different code blocks? Is there supposed to be some commentary here?

aaronmarkham · 2018-08-24T23:59:22Z

+from mxnet.gluon import HybridBlock
+```
+
+# foreach


I'd go to two hashes for your subsections and only use the single hash for the document title.

aaronmarkham · 2018-08-24T23:59:44Z

+res, states = lstm(rnn_data, [x for x in init_states], valid_length)
+```
+
+# while_loop


aaronmarkham · 2018-08-25T00:00:32Z

+    <NDArray 1 @cpu(0)>]
+
+
+# cond


aaronmarkham · 2018-08-25T00:01:06Z

+
+
+# cond
+`cond` is defined with the following signature:


It would be nice to talk about what it is and what it does before showing the signature. The same goes for the other sections/functions.

eric-haibin-lin · 2018-08-26T18:38:23Z

@@ -0,0 +1,390 @@
+


Will this be added to http://mxnet.incubator.apache.org/tutorials/index.html ?

how to do that?

Roshrini · 2018-08-27T17:45:34Z

@zheng-da @eric-haibin-lin @aaronmarkham Can you check if all the feebdack was addressed and is this PR ready to merge now?

eric-haibin-lin · 2018-08-27T18:23:40Z

my comments are addressed

zheng-da · 2018-08-27T18:32:23Z

@Roshrini I have addressed all of @aaronmarkham's comments.

* the first version. * fix. * add to test. * fix. * fix. * fix * fix. * fix. * add title. * add link * fix.

* Add a tutorial for control flow operators. (#12340) * the first version. * fix. * add to test. * fix. * fix. * fix * fix. * fix. * add title. * add link * fix. * Update ONNX API docs references (#12317) * update onnx API references * update descriptions * [MXAPPS-581] Disable an additional long test in the SD nightly (#12343) * Disable an additional test in the SD nightly that also takes over the timeout. * Documentation update related to sparse support (#12367) * Update sparse.md * Update sparse.md * Update csr.md * Update row_sparse.md * Update train.md

* the first version. * fix. * add to test. * fix. * fix. * fix * fix. * fix. * add title. * add link * fix.

zheng-da requested a review from szha as a code owner August 24, 2018 21:30

the first version.

cdfacfc

zheng-da force-pushed the control_flow_tutorial branch from 09ceac8 to cdfacfc Compare August 24, 2018 22:36

sandeep-krishnamurthy added Doc Python Backend Issues related to the backend of MXNet pr-awaiting-review PR is waiting for code review and removed Backend Issues related to the backend of MXNet labels Aug 24, 2018

zheng-da added 2 commits August 24, 2018 15:42

fix.

a0e08db

add to test.

bf5d333

zheng-da added 4 commits August 24, 2018 15:51

fix.

1c3818d

fix.

69e71a4

fix

f20c200

fix.

d485eaa

aaronmarkham approved these changes Aug 25, 2018

View reviewed changes

eric-haibin-lin reviewed Aug 26, 2018

View reviewed changes

zheng-da added 4 commits August 26, 2018 21:27

fix.

8e4dab0

add title.

07d40dd

add link

00bb353

fix.

110e32c

szha merged commit 9e48f70 into apache:master Aug 27, 2018

Roshrini pushed a commit to Roshrini/mxnet that referenced this pull request Aug 27, 2018

Add a tutorial for control flow operators. (apache#12340)

0b338d0

* the first version. * fix. * add to test. * fix. * fix. * fix * fix. * fix. * add title. * add link * fix.

Roshrini mentioned this pull request Aug 27, 2018

Cherry-pick few commits to release 1.3 branch #12375

Merged

4 tasks

anirudh2290 pushed a commit to anirudh2290/mxnet that referenced this pull request Sep 19, 2018

Add a tutorial for control flow operators. (apache#12340)

b4c2adf

* the first version. * fix. * add to test. * fix. * fix. * fix * fix. * fix. * add title. * add link * fix.

zheng-da deleted the control_flow_tutorial branch September 29, 2018 21:32

		MXNet currently provides three control flow operators: `cond`, `foreach` and `while_loop`. Like other MXNet operators, they all have a version for NDArray and a version for Symbol. These two versions have exactly the same semantics. We can take advantage of this and use them in Gluon to hybridize models.

		In this tutorial, we use a few examples to demonstrate the use of control flow operators in Gluon and show how a model that requires control flow is hybridized.


		The inputs of the `body` function have two parts: `data` is a slice of an array (if there is only one input array in `foreach`) or a list of slices (if there are a list of input arrays); `states` are the arrays from the previous iteration. The outputs of the `body` function also have two parts: `outputs` is an array or a list of arrays; `states` is the computation states of the current iteration. `outputs` from all iterations are concatenated as the outputs of `foreach`.

		The pseudocode below illustrates the execution of `foreach`.

		```


		```python

Conversation

zheng-da commented Aug 24, 2018

Description

Checklist

Essentials

Changes

Comments

Uh oh!

zheng-da commented Aug 24, 2018

Uh oh!

aaronmarkham left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Roshrini commented Aug 27, 2018

Uh oh!

eric-haibin-lin commented Aug 27, 2018

Uh oh!

zheng-da commented Aug 27, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants