Redefines value_and_pullback_function to return a value and a pullback function

[sethaxen]

Fixes part of #4. It's a breaking change, but the version number is given a -DEV suffix, since we should also handle #34 before releasing. (EDIT: Ref #36)

[codecov-commenter]

Codecov Report

Merging #35 (fae5c75) into master (979fa04) will decrease coverage by 1.00%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #35      +/-   ##
==========================================
- Coverage   80.05%   79.04%   -1.01%     
==========================================
  Files           2        2              
  Lines         396      377      -19     
==========================================
- Hits          317      298      -19     
  Misses         79       79              

Impacted Files	Coverage Δ
src/AbstractDifferentiation.jl	76.99% <100.00%> (-1.23%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 979fa04...fae5c75. Read the comment docs.

[sethaxen]

@mohamed82008 in #36 (comment)

This PR needs more thinking. I think when using the Jacobian as a primitive, deriving the value_and_pullback function won't be possible. Only the value and pullback_function is. So it might make sense to define the value_and_pullback_function/value_and_pushforward_function to throw when the Jacobian is used as a primitive. That is users shouldn't expect to use these 2 functions for such a backend or at least the methods won't be define automatically.

I'm not sure I understand what you mean. value_and_pushforward_function is not touched in this PR. And the tests already include examples where Jacobian is used as a primitive and value_and_pushforward_function is tested. e.g. this works with this PR:

julia> using ForwardDiff, AbstractDifferentiation

julia> struct ForwardDiffBackend1 <: AD.AbstractForwardMode end

julia> AD.primal_value(::ForwardDiffBackend1, ::Any, f, xs) = ForwardDiff.value.(f(xs...))

julia> AD.@primitive function jacobian(ab::ForwardDiffBackend1, f, xs)
           if xs isa Number
               return (ForwardDiff.derivative(f, xs),)
           elseif xs isa AbstractArray
               out = f(xs)
               if out isa Number
                   return (adjoint(ForwardDiff.gradient(f, xs)),)
               else
                   return (ForwardDiff.jacobian(f, xs),)
               end
           elseif xs isa Tuple
               error(typeof(xs))      
           else
               error(typeof(xs)) 
           end
       end

julia> ba = ForwardDiffBackend1();

julia> val, back = AD.value_and_pullback_function(ba, identity, randn(10))
([0.05771077577524417, -0.6046897384683757, -0.43893636554383, 1.008345527744184, -1.3448119234793194, 0.0541982394477022, 0.2593213719269091, 1.6426658474014317, 0.651265342373578, -0.795563384561261], AbstractDifferentiation.var"#21#23"{ForwardDiffBackend1, typeof(identity), Tuple{Vector{Float64}}}(ForwardDiffBackend1(), identity, ([0.05771077577524417, -0.6046897384683757, -0.43893636554383, 1.008345527744184, -1.3448119234793194, 0.0541982394477022, 0.2593213719269091, 1.6426658474014317, 0.651265342373578, -0.795563384561261],)))

julia> back(randn(10))
([-1.2729554614899294, 0.1378713003865544, 0.2690893343964529, 0.9206297829092358, -1.3913076264330144, 0.4862468894828594, -1.1728445800434935, 0.6753224021990761, 0.14583421961218776, 0.732158895744132],)

[devmotion]

This change would be very helpful for other PRs such as #93: #93 (comment)

[gdalle]

@devmotion should we close this one?

[devmotion]

Probably. We can always re-open if it is preferred over #93.