NPE in BucketingModule when providedLabel of DataBatch is not set

[milandesai]

Package: Scala
Mxnet Version: 1.1.0
Java Version: openjdk 1.8.0_131
Maven Version: [unused]
Scala Runtime: 2.12.4

We create a DataBatch without setting the providedLabel field. We then call BucketingModule.forward(...) with the data batch and get a NullPointerException:

java.lang.NullPointerException was thrown.
java.lang.NullPointerException
	at ml.dmlc.mxnet.DataDesc$.ListMap2Descs(IO.scala:252)
	at ml.dmlc.mxnet.module.BucketingModule.forward(BucketingModule.scala:313)

This happens because the implicit method ListMap2Descs tries to convert ListMap[String, Shape] into IndexedSeq[DataDesc] but fails because the former is null by default.

[lupesko]

Thanks for reporting the issue @milandesai

To help the community look into it, please provide more information as asked below (usually the new issue template asks to fill all of that in) - providing complete information in the most concise form is the best way to get help.

• Package used (Python/R/Scala/Julia):
• For Scala user, please provide:

1. Java version: (java -version)
2. Maven version: (mvn -version)
3. Scala runtime if applicable: (scala -version)

• Build info (Required if built from source)

1. Compiler (gcc/clang/mingw/visual studio):
2. MXNet commit hash (Paste the output of git rev-parse HEAD here.)
3. Build config: (Paste the content of config.mk, or the build command.)

• Minimum reproducible example
  (If you are using your own code, please provide a short script that reproduces the error. Otherwise, please provide link to the existing example.)

Thanks!

[lupesko]

@nswamy please label as "Scala" "Bug"
@lanking520 @andrewfayres @ddavydenko who are working on the Scala API

[lanking520]

Hi @milandesai could you please tell me which version of MXNet you are using?

we are also facing some issues about DataDesc in some other issues. And you are right, there is no good way to avoid this. The minimum fix you can take is using what I have tried with RNN example, adding a handler in this conversion function to let None/Empty ListMap being passed.

There will be an upcoming PR address this issue. Since it was a pain from v0.11.0. The currently planning is getting rid of ListMap and use DataDesc entirely so we can avoid the implicit conversion. Will keep track on this and see if that can help.

[milandesai]

Hi @lanking520, the mxnet version is 1.1.0, though I believe this issue should exist on master as well based on a quick check. I resolved this by modifying the source in IO.scala to simply do a null check. Will post a PR shortly for reference.

[milandesai]

PR: #11790

[lanking520]

Adding DataDesc in this PR: #11844

[lupesko]

#11790 merged. @milandesai can we close this issue?