Imperative regression output layers are broken

[asitstands]

I tried to implement a simple linear regression with ndarray in imperative style.

import mxnet as mx
from mxnet import nd

num_inputs = 3
num_outputs = 1
num_examples = 1000

def real_fn(X):
    return 2 * X[:, 0] - 3.4 * X[:, 1] + 4.2

train_data = nd.random_normal(shape=(num_examples, num_inputs))
noise = 0.01 * nd.random_normal(shape=(num_examples,))
train_label = real_fn(train_data) + noise

batch_size = 4
train_iter = mx.io.NDArrayIter(train_data, train_label, batch_size, shuffle=True, label_name='lin_reg_label')

weight = nd.random_normal(0, 0.01, shape=(num_outputs, num_inputs))
weight.attach_grad()
bias = nd.random_normal(0, 0.01, shape=(num_outputs, ))
bias.attach_grad()
optimizer = mx.optimizer.SGD(learning_rate=0.1, momentum=0.9)
optimizer_weight_st = optimizer.create_state(0, weight)
optimizer_bias_st = optimizer.create_state(0, bias)

for epoch in range(10):
	train_iter.reset()
	for batch in train_iter: # `batch` is a `DataBatch`
		with mx.autograd.record():
			fc0 = nd.FullyConnected(data=batch.data[0], weight=weight, bias=bias, num_hidden=num_outputs)
			out = nd.LinearRegressionOutput(data=fc0, label=batch.label[0]) # RMS cost function
		out.backward()
		optimizer.update(0, weight, weight.grad, optimizer_weight_st)
		optimizer.update(0, bias, bias.grad, optimizer_bias_st)

I got the following error message.

/usr/lib/python3.6/site-packages/mxnet/optimizer.py:136: UserWarning: WARNING: New optimizer mxnet.optimizer.NAG is overriding existing optimizer mxnet.optimizer.NAG
  Optimizer.opt_registry[name].__name__))
Traceback (most recent call last):
  File "Tmp.py", line 31, in <module>
    out = nd.LinearRegressionOutput(data=fc0, label=batch.label[0]) # RMS cost function
  File "<string>", line 52, in LinearRegressionOutput
  File "/usr/lib/python3.6/site-packages/mxnet/_ctypes/ndarray.py", line 92, in _imperative_invoke
    ctypes.byref(out_stypes)))
  File "/usr/lib/python3.6/site-packages/mxnet/base.py", line 148, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [20:58:23] src/imperative/./imperative_utils.h:123: Check failed: infertype.count(attrs.op) Operator LinearRegressionOutput is missing FInferType attribute

Stack trace returned 10 entries:
[bt] (0) /usr/lib/python3.6/site-packages/mxnet/libmxnet.so(dmlc::StackTrace[abi:cxx11]()+0x46) [0x7f380a187db6]
[bt] (1) /usr/lib/python3.6/site-packages/mxnet/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x28) [0x7f380a185be8]
[bt] (2) /usr/lib/python3.6/site-packages/mxnet/libmxnet.so(mxnet::imperative::SetShapeType(mxnet::Context const&, nnvm::NodeAttrs const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, mxnet::DispatchMode*)+0xc2e) [0x7f380b93416e]
[bt] (3) /usr/lib/python3.6/site-packages/mxnet/libmxnet.so(mxnet::Imperative::Invoke(mxnet::Context const&, nnvm::NodeAttrs const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&)+0x25a) [0x7f380b921cea]
[bt] (4) /usr/lib/python3.6/site-packages/mxnet/libmxnet.so(MXImperativeInvokeImpl(void*, int, void**, int*, void***, int, char const**, char const**)+0x1e4) [0x7f380bd24c44]
[bt] (5) /usr/lib/python3.6/site-packages/mxnet/libmxnet.so(MXImperativeInvokeEx+0x13c) [0x7f380bd2544c]
[bt] (6) /usr/lib/libffi.so.6(ffi_call_unix64+0x4c) [0x7f3811e561c8]
[bt] (7) /usr/lib/libffi.so.6(ffi_call+0x32a) [0x7f3811e55c2a]
[bt] (8) /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(_ctypes_callproc+0x2ce) [0x7f38120b654e]
[bt] (9) /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(+0x11f85) [0x7f38120b6f85]

I think the point of the error is "src/imperative/./imperative_utils.h:123: Check failed: infertype.count(attrs.op) Operator LinearRegressionOutput is missing FInferType attribute".

I also got the similar errors with LogisticRegressionOutput and MAERegressionOutput. With SoftmaxOutput I get no errors while it is not the regression that I want. The imperative style regression outputs may not be used frequently but anyway they could be useful sometimes and listed in the API document anyway. Please fix them.

[ZiyueHuang]

Thanks for reporting this issue. I fix it in the PR above. Then your code is ok on my machine.

By the way, I think mx.nd.waitall() should be in the last line otherwise the computation is not triggered.

[asitstands]

@ZiyueHuang Thanks, the fix works. For the waitall call, why do we need explicit synchronization here? In my machine the code works without it.