From d588084cb456baa684c4114a27379a2e866b7a62 Mon Sep 17 00:00:00 2001 From: Guillaume Dalle <22795598+gdalle@users.noreply.github.com> Date: Sun, 27 Mar 2022 22:35:11 +0200 Subject: [PATCH 1/4] Update README.md Add explanation about backends and small example (see #31) --- README.md | 37 ++++++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 5219d33..f94ce30 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,42 @@ using AbstractDifferentiation ## `AbstractDifferentiation` backends -To use `AbstractDifferentiation`, first construct a backend instance `ab::AD.AbstractBackend` using your favorite differentiation package in Julia that supports `AbstractDifferentiation`. For higher order derivatives, you can build higher order backends using `AD.HigherOrderBackend`. For instance, let `ab_f` be a forward-mode automatic differentiation backend and let `ab_r` be a reverse-mode automatic differentiation backend. To construct a higher order backend for doing forward-over-reverse-mode automatic differentiation, use `AD.HigherOrderBackend((ab_f, ab_r))`. To construct a higher order backend for doing reverse-over-forward-mode automatic differentiation, use `AD.HigherOrderBackend((ab_r, ab_f))`. +To use `AbstractDifferentiation`, first construct a backend instance `ab::AD.AbstractBackend` using your favorite differentiation package in Julia that supports `AbstractDifferentiation`. +The following backends are made available by `AbstractDifferentiation` if their corresponding package is loaded (thanks to [Requires.jl](https://github.com/JuliaPackaging/Requires.jl)): + +- `AD.ForwardDiffBackend()` for [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl) +- `AD.FiniteDifferencesBackend()` for [FiniteDifferences.jl](https://github.com/JuliaDiff/FiniteDifferences.jl) +- `AD.ReverseDiffBackend()` for [ReverseDiff.jl](https://github.com/JuliaDiff/ReverseDiff.jl) +- `AD.ReverseRuleConfigBackend(ruleconfig)` for any [ChainRules.jl](https://github.com/JuliaDiff/ChainRules.jl)-compatible reverse mode backend +- `AD.ZygoteBackend()` for [Zygote.jl](https://github.com/FluxML/Zygote.jl) +- `AD.TrackerBackend()` for [Tracker.jl](https://github.com/FluxML/Tracker.jl) + +Here's an example: + +```julia +julia> using AbstractDifferentiation + +julia> ab = AD.ZygoteBackend() +ERROR: UndefVarError: ZygoteBackend not defined +Stacktrace: + [1] getproperty(x::Module, f::Symbol) + @ Base ./Base.jl:35 + [2] top-level scope + @ REPL[3]:1 + +julia> using Zygote + +julia> ab = AD.ZygoteBackend() +AbstractDifferentiation.ReverseRuleConfigBackend{Zygote.ZygoteRuleConfig{Zygote.Context}}(Zygote.ZygoteRuleConfig{Zygote.Context}(Zygote.Context(nothing))) + +julia> f(x) = log(sum(exp, x)) +f (generic function with 1 method) + +julia> AD.gradient(ab, f, rand(10)) +([0.07163448353282538, 0.08520350535348796, 0.09675622487503996, 0.1522744408520505, 0.12174662595572318, 0.07996969757526722, 0.07832665607158593, 0.11001685581681672, 0.06691909637037166, 0.1371524135968315],) +``` + +For higher order derivatives, you can build higher order backends using `AD.HigherOrderBackend`. For instance, let `ab_f` be a forward-mode automatic differentiation backend and let `ab_r` be a reverse-mode automatic differentiation backend. To construct a higher order backend for doing forward-over-reverse-mode automatic differentiation, use `AD.HigherOrderBackend((ab_f, ab_r))`. To construct a higher order backend for doing reverse-over-forward-mode automatic differentiation, use `AD.HigherOrderBackend((ab_r, ab_f))`. ## Backend-agnostic interface From 4673f59eb0c80cecfc6ed6c8c3238b5642258a97 Mon Sep 17 00:00:00 2001 From: Guillaume Dalle <22795598+gdalle@users.noreply.github.com> Date: Mon, 28 Mar 2022 00:24:19 +0200 Subject: [PATCH 2/4] Update README.md Co-authored-by: David Widmann --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f94ce30..0f80c71 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,8 @@ using AbstractDifferentiation ## `AbstractDifferentiation` backends To use `AbstractDifferentiation`, first construct a backend instance `ab::AD.AbstractBackend` using your favorite differentiation package in Julia that supports `AbstractDifferentiation`. -The following backends are made available by `AbstractDifferentiation` if their corresponding package is loaded (thanks to [Requires.jl](https://github.com/JuliaPackaging/Requires.jl)): +The following backends are temporarily made available by `AbstractDifferentiation` as soon as their corresponding package is loaded (thanks to [Requires.jl](https://github.com/JuliaPackaging/Requires.jl)). +In the long term, these backend objects are meant be defined within their respective packages to enforce the `AbstractDifferentiation` interface. - `AD.ForwardDiffBackend()` for [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl) - `AD.FiniteDifferencesBackend()` for [FiniteDifferences.jl](https://github.com/JuliaDiff/FiniteDifferences.jl) From 831b2c70d56157e0ad6575ad203a94d71593d90d Mon Sep 17 00:00:00 2001 From: Guillaume Dalle <22795598+gdalle@users.noreply.github.com> Date: Mon, 28 Mar 2022 07:33:10 +0200 Subject: [PATCH 3/4] Update README.md Clarify differences between backends --- README.md | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 0f80c71..23d53c1 100644 --- a/README.md +++ b/README.md @@ -20,30 +20,23 @@ using AbstractDifferentiation ## `AbstractDifferentiation` backends To use `AbstractDifferentiation`, first construct a backend instance `ab::AD.AbstractBackend` using your favorite differentiation package in Julia that supports `AbstractDifferentiation`. -The following backends are temporarily made available by `AbstractDifferentiation` as soon as their corresponding package is loaded (thanks to [Requires.jl](https://github.com/JuliaPackaging/Requires.jl)). -In the long term, these backend objects are meant be defined within their respective packages to enforce the `AbstractDifferentiation` interface. +In particular, you may want to use `AD.ReverseRuleConfigBackend(ruleconfig)` for any [ChainRules.jl](https://github.com/JuliaDiff/ChainRules.jl)-compatible reverse mode differentiation package. + +The following backends are temporarily made available by `AbstractDifferentiation` as soon as their corresponding package is loaded (thanks to [Requires.jl](https://github.com/JuliaPackaging/Requires.jl)): - `AD.ForwardDiffBackend()` for [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl) - `AD.FiniteDifferencesBackend()` for [FiniteDifferences.jl](https://github.com/JuliaDiff/FiniteDifferences.jl) - `AD.ReverseDiffBackend()` for [ReverseDiff.jl](https://github.com/JuliaDiff/ReverseDiff.jl) - `AD.ReverseRuleConfigBackend(ruleconfig)` for any [ChainRules.jl](https://github.com/JuliaDiff/ChainRules.jl)-compatible reverse mode backend -- `AD.ZygoteBackend()` for [Zygote.jl](https://github.com/FluxML/Zygote.jl) - `AD.TrackerBackend()` for [Tracker.jl](https://github.com/FluxML/Tracker.jl) +- `AD.ZygoteBackend()` for [Zygote.jl](https://github.com/FluxML/Zygote.jl), which is a special case of `AD.ReverseRuleConfigBackend` + +In the long term, these backend objects (and many more) will be defined within their respective packages to enforce the `AbstractDifferentiation` interface. Here's an example: ```julia -julia> using AbstractDifferentiation - -julia> ab = AD.ZygoteBackend() -ERROR: UndefVarError: ZygoteBackend not defined -Stacktrace: - [1] getproperty(x::Module, f::Symbol) - @ Base ./Base.jl:35 - [2] top-level scope - @ REPL[3]:1 - -julia> using Zygote +julia> using AbstractDifferentiation, Zygote julia> ab = AD.ZygoteBackend() AbstractDifferentiation.ReverseRuleConfigBackend{Zygote.ZygoteRuleConfig{Zygote.Context}}(Zygote.ZygoteRuleConfig{Zygote.Context}(Zygote.Context(nothing))) From eb8296fe07b98a41831d53350533dafa38a34b15 Mon Sep 17 00:00:00 2001 From: Guillaume Dalle <22795598+gdalle@users.noreply.github.com> Date: Mon, 28 Mar 2022 07:33:35 +0200 Subject: [PATCH 4/4] Update README.md Remove duplicate --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 23d53c1..42125b8 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,6 @@ The following backends are temporarily made available by `AbstractDifferentiatio - `AD.ForwardDiffBackend()` for [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl) - `AD.FiniteDifferencesBackend()` for [FiniteDifferences.jl](https://github.com/JuliaDiff/FiniteDifferences.jl) - `AD.ReverseDiffBackend()` for [ReverseDiff.jl](https://github.com/JuliaDiff/ReverseDiff.jl) -- `AD.ReverseRuleConfigBackend(ruleconfig)` for any [ChainRules.jl](https://github.com/JuliaDiff/ChainRules.jl)-compatible reverse mode backend - `AD.TrackerBackend()` for [Tracker.jl](https://github.com/FluxML/Tracker.jl) - `AD.ZygoteBackend()` for [Zygote.jl](https://github.com/FluxML/Zygote.jl), which is a special case of `AD.ReverseRuleConfigBackend`