Variational inference using reverse-mode AD

[jar600]

VKM: "variational inference on a toy Bayesian linear regression example, with a naive mean field [variational] approximation, using reverse-mode AD [on a subset of Metaprob]" (see https://github.com/probcomp/metaprob-clojure/milestones/5 and https://github.com/probcomp/metaprob-clojure/milestone/4 )

[jar600]

Hoping to get help from @marcoct on #16 #17 #18 and #19 .

[jar600]

Reverse-mode AD seems to be partially implemented in the metaprob-python source tree: https://github.com/probcomp/metaprob/blob/master/examples/ad/backpropagate.vnts (private, sorry).

I can't tell whether backpropagate.vnts is supposed to work, or whether, if it were to work, it would be the solution to this issue (as opposed to something more specific - the name suggests application to NNs).

[jar600]

AD in Clojure: https://github.com/log0ymxm/clj-auto-diff (this is a port of Jeff Siskind's implementation in Scheme - so as long as the port is faithful, the code ought to be very good!)

[zane]

Per @jar398, legacy code relating to this issue might be available in the python and converted directories at the legacy tag.