Re: [boost] New library proposal: Autodiff

On Thu, Dec 20, 2018 at 1:34 AM Raffi Enficiaud via Boost < boost@lists.boost.org> wrote:

it supports only compilation time expressions, is that correct? Would it be possible to construct expressions at runtime and then call the autodiff on that expression? I believe this would make the library extremely useful and comparable to whatever tensorflow or caffe have

Let's say one is interested in calculating the first 3 derivatives of a function f(x) with respect to x, and for a particular computation, the value of x is 10. The 3 must be known at compile-time, but the 10 can be decided at runtime: autodiff::variable x(10); Within the function body f, say x is squared: x*x. This operation must call the overloaded operator*() defined in autodiff.hpp. Similarly when calling basic mathematical functions such as exp(x) or tan(x) and compositions of such operations/function they must call those defined in autodiff.hpp. (You can also call boost's math functions whose parameters are template variables, such as tgamma(), but you will not get very good results on the derivatives since these functions were written to only approximate the value itself and not the derivatives. Fortunately it is quite easy to add functions whose derivatives are known to autodiff.hpp.) To answer your question, as long as the runtime-constructed expressions satisfy this, then it should work.

- does it handle vector/arrays/matrices already? It happens often that we have a vector function returning eg. an array, and we want the differential wrt. one element of that array. Same for matrices.

It does not directly included matrix multiplication, if that is what you are asking. However if each element of a matrix can be an autodiff::variable<> then the existing matrix multiplication logic should use the overloaded autodiff operator*() and it should all work. For example, calls are make to std::inner_product() in which the internal multiplication is calling the overloaded autodiff operator*(). Matt