Higher-Order functions: Low-hanging fruit #222
Description
We've yet to properly figure out how to make ChainRules play nicely with higher-order functions, see JuliaDiff/ChainRulesCore.jl#68.
However, this doesn't stop us from implement a load of rules for functions that ChainRules knows how to handle explicitly. For example, we have a rule for sin, so we can implement map(sin, x) ourselves for carefully chosen types of x.
A pre-requisite for this is JuliaDiff/ChainRulesCore.jl#127 which would let us get hold of all of the rules that we've implemented. It'll be necessary to be able to filter them for appropriateness so that we know what they're applicable to, in particular we need to know
- the arity of the function to which the rule applies
- the element types to which it applies
This would be a big win for e.g. Zygote where this is done, but with a piece-meal approach.