add custom relu implementation

[akshay326]

thanks to @matbesancon for #91, added a very trivial example of implementing ReLU as a layer in a NN trained on MNIST dataset using Flux.jl
There are some issues pending, specifically training on the complete dataset if very slow, and gives a JuMP warning:

Warning: The addition operator has been used on JuMP expressions a large number of times. 
This warning is safe to ignore but may indicate that model generation is slower than necessary. 
For performance reasons, you should not add expressions in a loop.
Instead of x += , use append!(x,y) to modify x in place. 
If y is a single variable, you may also use push!(x, coef, y) in place of x += coef*y

[akshay326]

also, current code works on latest Julia build (1.6.0). Seems Julia 1.0 supports only Tracker based Flux (while the latest one supports Zygote.jl)

[codecov]

Codecov Report

Merging #92 (e1e199e) into master (a2ab035) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #92   +/-   ##
=======================================
  Coverage   81.20%   81.20%           
=======================================
  Files           6        6           
  Lines         782      782           
=======================================
  Hits          635      635           
  Misses        147      147           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a2ab035...e1e199e. Read the comment docs.

[matbesancon]

we should make this into a documentation page instead I think, it will also build it at every PR

[odow]

The warning suggests you are doing some arithmetic outside the JuMP macros. I couldn't see anything obvious here, so it must be somewhere inside DiffOpt.jl.

[blegat]

This is where the warning comes from I believe, this should be fixed by jump-dev/MutableArithmetics.jl#87

[akshay326]

solved. seems there's a set_objective_coefficient method. + speeds up training a bit

[blegat]

we should make this into a documentation page instead I think, it will also build it at every PR

I agree, it would be helpful to run it at every commit with Literate

[matbesancon]

matrix_relu better than myRelu? Also write out the model in the docstring

[akshay326]

changing it

[matbesancon]

status on this one? Should we merge it or make it a doc page first?

[akshay326]

@matbesancon should we close this (without merging ?)
#95 already contains the changes introduced in this PR

[matbesancon]

#95 will be lighter to review if this gets merged first.
Things missing:

• Some comments / explanations
• Adding it to the tests

[matbesancon]

otherwise there is a risk we have this example breaking at some point

[matbesancon]

bump @be-apt to add the example to CI

[akshay326]

closing this. duplicate code of #95