New library proposal: Autodiff
Hello, An automatic differentiation https://en.wikipedia.org/wiki/Automatic_differentiation C++ library - Autodiff - is released under the Boost License and is proposed for inclusion into Boost: - Github: https://github.com/pulver/autodiff - Boost Library Incubator: http://blincubator.com/bi_library/autodiff-2/?gform_post_id=1716 Features: - Instances of autodiff variables satisfy Boost's Conceptual Requirements for Real Number Types https://www.boost.org/doc/libs/1_69_0/libs/math/doc/html/math_toolkit/real_c.... In fact the class and function template definitions are based upon the tables in this page. - No use of dynamic memory. The only member variable is a std::array<>. - Consistent with Boost's type promotion templates. When adding/multiplying/etc. variables of differing dimension number and sizes, the resulting data type is calculated at compile-time. - Single header-only file. - Intuitive and minimal API. Requirements: - C++17 compiler that supports constexpr if statements. There are a fair amount of calculations done at compile-time which would require messy SFINAE hacks to make this C++14-compatible. Todo: - A github build matrix that also includes clang and MSVC. - Additional documentation, including the mathematics. Feedback and endorsements for Boost Library inclusion are welcome and requested. Best regards, Matt
On Tue, Dec 18, 2018 at 1:09 PM Matt Pulver via Boost
- Github: https://github.com/pulver/autodiff
Minor nit, your local git credentials are not configured correctly, that's why your commits are not attached to your GitHub account in your commit log (set `git config --global user.email` and `git config --global user.name` to match your GitHub login). https://github.com/pulver/autodiff/commits/master
- C++17 compiler that supports constexpr if statements. There are a fair amount of calculations done at compile-time which would require messy SFINAE hacks to make this C++14-compatible.
I know my position is unpopular, but I don't think this is a good tradeoff. Supporting C++11 certainly requires more work on the author's part but that is a finite investment. The return on this is a practically unbounded amount of utility, as the number of users is normally expected to greatly outnumber the amount of authors (1 in this case). In fact you could argue that there is infinite utility in laboring to produce "messy SFINAE hacks", because otherwise users of C++14 and C++11 cannot use the library at all. Most users only care that the library works, and are unaffected by the particulars of the sausage-making. In general, I feel like it is a good engineering practice for Boost libraries to only require C++14 or C++17 when absolutely necessary. In this case it does not seem necessary. This year's developer surveys indicate that C++11 has the most widespread use: https://www.jetbrains.com/research/devecosystem-2018/cpp/ https://isocpp.org/files/papers/CppDevSurvey-2018-02-summary.pdf It seems like doing the more verbose C++11-style SFINAE is worth it, to have access to the larger base of C++ users. This opinion is my own, and I will note that (roughly speaking) the Boost library policy does not currently mandate support for any specific versions of C++. Regards
- C++17 compiler that supports constexpr if statements. There are a fair amount of calculations done at compile-time which would require messy SFINAE hacks to make this C++14-compatible.
I know my position is unpopular, but I don't think this is a good tradeoff. Supporting C++11 certainly requires more work on the author's part but that is a finite investment. The return on this is a practically unbounded amount of utility, as the number of users is normally expected to greatly outnumber the amount of authors (1 in this case). In fact you could argue that there is infinite utility in laboring to produce "messy SFINAE hacks", because otherwise users of C++14 and C++11 cannot use the library at all. Most users only care that the library works, and are unaffected by the particulars of the sausage-making.
In general, I feel like it is a good engineering practice for Boost libraries to only require C++14 or C++17 when absolutely necessary. In this case it does not seem necessary. This year's developer surveys indicate that C++11 has the most widespread use:
I get that, and haven't looked in detail at the code yet, but for the record, for Math heavy code, "if constexpr" is probably more useful than anything else in C++11 and 14.... the next "biggy" is C++20's is_constant_evaluated which is likely to make a lot of things that can't quite be fully constexpr yet, more or less possible in the future (Boost.Multiprecision for one). John. --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
John Maddock wrote:
I get that, and haven't looked in detail at the code yet, but for the record, for Math heavy code, "if constexpr" is probably more useful than anything else in C++11 and 14....
In the specific case of https://github.com/pulver/autodiff/blob/master/include/boost/math/autodiff.h... as far as I can see you can just drop the `constexpr` part from `if constexpr` on pre-17 compilers and everything would work. But that's just from a quick glance, I haven't tried it.
(since you know your position is unpopular, I'm sure you would be disappointed if noone replied ;-) On Wed, 19 Dec 2018, Vinnie Falco via Boost wrote:
- C++17 compiler that supports constexpr if statements. There are a fair amount of calculations done at compile-time which would require messy SFINAE hacks to make this C++14-compatible.
I know my position is unpopular, but I don't think this is a good tradeoff. Supporting C++11 certainly requires more work on the author's part but that is a finite investment.
Writing it may be a "finite" investment, but maintaining the resulting, way more complicated code is not. And I am not even counting the cases where this has an impact on the API.
The return on this is a practically unbounded amount of utility, as the number of users is normally expected to greatly outnumber the amount of authors (1 in this case). In fact you could argue that there is infinite utility in laboring to produce "messy SFINAE hacks", because otherwise users of C++14 and C++11 cannot use the library at all. Most users only care that the library works, and are unaffected by the particulars of the sausage-making.
Don't underestimate the importance of keeping the author of the library motivated. If supporting crappy old tools makes them lose the will to keep improving the library, we end up with one more unmaintained boost package...
In general, I feel like it is a good engineering practice for Boost libraries to only require C++14 or C++17 when absolutely necessary. In
Why stop at C++11? Why not stick to C?
this case it does not seem necessary. This year's developer surveys indicate that C++11 has the most widespread use:
https://www.jetbrains.com/research/devecosystem-2018/cpp/
https://isocpp.org/files/papers/CppDevSurvey-2018-02-summary.pdf
It seems like doing the more verbose C++11-style SFINAE is worth it, to have access to the larger base of C++ users. This opinion is my own, and I will note that (roughly speaking) the Boost library policy does not currently mandate support for any specific versions of C++.
The survey is about what people were using several months ago for all kinds of projects, including maintaining legacy code bases. However, we are interested in - what people will be using by the time the library has been reviewed and released (say a couple years) - projects that are likely to introduce a dependency to some new external library Both introduce a strong bias towards more recent tools and standards. -- Marc Glisse
I'm quite new to this mailing list and this topic is therefore very
interesting as it show the opinion of various people contributing to boost.
When I'm looking for a library for my - professional - projects, I tend to
put more interest into libraries that leverage new parts of the standard.
Firstly, it is a very good way to push for a more modern codebase and
improve my and my coworkers knowledge of a more future-proof way of using
C++. Don't underestimate the importance of learning. Secondly, people
dealing with old codebases are less likely to look for new libraries. They
usually mostly maintain code or refactor one piece at a time, but tend to
use already existing libraries, or internal ones: new libraries are used
for new, up-to-date projects. Don't develop stuff for people who don't need
it. Thirdly, if you wait for a standard to be mature to start developing
around it, it's going to be outdated by the time your library is fit for
production.
I'd be very interested to know what other, more experienced people think
about all this.
Obviously if new libraries are able to be compatible with more compilers,
it is a good thing, but remember that if we want to have people using
recent standards, then we have to make them relevant.
On Dec 19, 2018 21:40, "Marc Glisse via Boost"
- C++17 compiler that supports constexpr if statements. There are a fair amount of calculations done at compile-time which would require messy SFINAE hacks to make this C++14-compatible.
I know my position is unpopular, but I don't think this is a good tradeoff. Supporting C++11 certainly requires more work on the author's part but that is a finite investment.
Writing it may be a "finite" investment, but maintaining the resulting, way more complicated code is not. And I am not even counting the cases where this has an impact on the API.
The return on this is a practically unbounded amount of utility, as the number of users is normally expected to greatly outnumber the amount of authors (1 in this case). In fact you could argue that there is infinite utility in laboring to produce "messy SFINAE hacks", because otherwise users of C++14 and C++11 cannot use the library at all. Most users only care that the library works, and are unaffected by the particulars of the sausage-making.
Don't underestimate the importance of keeping the author of the library motivated. If supporting crappy old tools makes them lose the will to keep improving the library, we end up with one more unmaintained boost package...
In general, I feel like it is a good engineering practice for Boost libraries to only require C++14 or C++17 when absolutely necessary. In
Why stop at C++11? Why not stick to C?
this case it does not seem necessary. This year's developer surveys indicate that C++11 has the most widespread use:
https://www.jetbrains.com/research/devecosystem-2018/cpp/
https://isocpp.org/files/papers/CppDevSurvey-2018-02-summary.pdf
It seems like doing the more verbose C++11-style SFINAE is worth it, to have access to the larger base of C++ users. This opinion is my own, and I will note that (roughly speaking) the Boost library policy does not currently mandate support for any specific versions of C++.
The survey is about what people were using several months ago for all kinds of projects, including maintaining legacy code bases. However, we are interested in - what people will be using by the time the library has been reviewed and released (say a couple years) - projects that are likely to introduce a dependency to some new external library Both introduce a strong bias towards more recent tools and standards. -- Marc Glisse _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Wed, Dec 19, 2018 at 3:40 PM Marc Glisse via Boost
Don't underestimate the importance of keeping the author of the library motivated. If supporting crappy old tools makes them lose the will to keep improving the library, we end up with one more unmaintained boost package...
Thank you for articulating this point. In fact I wrote a closed-source
C++11 version of this library a few years ago with all the SFINAE code as
well as Windows and Linux-specific hacks due to slight differences in how
the compilers interpreted the obscure SFINAE code. Though it worked, I was
not happy with how much I had to butcher the project and additional changes
started to become increasingly difficult. It is due to the new C++17
compile-time logic that provides the opportunity to "do it right" and I am
happy with how much cleaner and maintainable this new version is.
On Wed, Dec 19, 2018 at 3:44 PM Peter Dimov via Boost
In the specific case of
https://github.com/pulver/autodiff/blob/master/include/boost/math/autodiff.h...
as far as I can see you can just drop the `constexpr` part from `if constexpr` on pre-17 compilers and everything would work. But that's just from a quick glance, I haven't tried it.
For example, the following will not compile with pre-17 compilers if
`constexpr` is dropped:
template
Matt Pulver wrote:
For example, the following will not compile with pre-17 compilers if `constexpr` is dropped:
template
constexpr size_t dimension ::depth() { if constexpr (is_dimension<RealType>::value) return 1 + RealType::depth(); else return 1; }
This specific case isn't hard to support either, although whether you would
want to invest the effort is up to you.
template
On 18.12.18 17:37, Matt Pulver via Boost wrote:
Hello,
An automatic differentiation https://en.wikipedia.org/wiki/Automatic_differentiation C++ library - Autodiff - is released under the Boost License and is proposed for inclusion into Boost:
- Github: https://github.com/pulver/autodiff - Boost Library Incubator: http://blincubator.com/bi_library/autodiff-2/?gform_post_id=1716
Features:
- Instances of autodiff variables satisfy Boost's Conceptual Requirements for Real Number Types https://www.boost.org/doc/libs/1_69_0/libs/math/doc/html/math_toolkit/real_c.... In fact the class and function template definitions are based upon the tables in this page. - No use of dynamic memory. The only member variable is a std::array<>. - Consistent with Boost's type promotion templates. When adding/multiplying/etc. variables of differing dimension number and sizes, the resulting data type is calculated at compile-time. - Single header-only file. - Intuitive and minimal API.
Requirements:
- C++17 compiler that supports constexpr if statements. There are a fair amount of calculations done at compile-time which would require messy SFINAE hacks to make this C++14-compatible.
Todo:
- A github build matrix that also includes clang and MSVC. - Additional documentation, including the mathematics.
Feedback and endorsements for Boost Library inclusion are welcome and requested.
Best regards, Matt
Hi, I like the idea very much, and it would be super useful. Having a quick look at the documentation: * it supports only compilation time expressions, is that correct? Would it be possible to construct expressions at runtime and then call the autodiff on that expression? I believe this would make the library extremely useful and comparable to whatever tensorflow or caffe have - does it handle vector/arrays/matrices already? It happens often that we have a vector function returning eg. an array, and we want the differential wrt. one element of that array. Same for matrices. Thanks, Raffi
On Thu, Dec 20, 2018 at 1:34 AM Raffi Enficiaud via Boost < boost@lists.boost.org> wrote:
it supports only compilation time expressions, is that correct? Would it be possible to construct expressions at runtime and then call the autodiff on that expression? I believe this would make the library extremely useful and comparable to whatever tensorflow or caffe have
Let's say one is interested in calculating the first 3 derivatives of a
function f(x) with respect to x, and for a particular computation, the
value of x is 10. The 3 must be known at compile-time, but the 10 can be
decided at runtime:
autodiff::variable
- does it handle vector/arrays/matrices already? It happens often that we have a vector function returning eg. an array, and we want the differential wrt. one element of that array. Same for matrices.
It does not directly included matrix multiplication, if that is what you are asking. However if each element of a matrix can be an autodiff::variable<> then the existing matrix multiplication logic should use the overloaded autodiff operator*() and it should all work. For example, calls are make to std::inner_product() in which the internal multiplication is calling the overloaded autodiff operator*(). Matt
-----Original Message----- From: Boost
On Behalf Of Matt Pulver via Boost Sent: Tuesday, December 18, 2018 5:38 PM Hello,
An automatic differentiation https://en.wikipedia.org/wiki/Automatic_differentiation C++ library - Autodiff - is released under the Boost License and is proposed for inclusion into Boost:
- Github: https://github.com/pulver/autodiff - Boost Library Incubator: http://blincubator.com/bi_library/autodiff-2/?gform_post_id=1716
Features:
- Instances of autodiff variables satisfy Boost's Conceptual Requirements for Real Number Types https://www.boost.org/doc/libs/1_69_0/libs/math/doc/html/math_toolkit/real_c.... In fact the class and function template definitions are based upon the tables in this page. - No use of dynamic memory. The only member variable is a std::array<>. - Consistent with Boost's type promotion templates. When adding/multiplying/etc. variables of differing dimension number and sizes, the resulting data type is calculated at compile-time. - Single header-only file. - Intuitive and minimal API.
Requirements:
- C++17 compiler that supports constexpr if statements. There are a fair amount of calculations done at compile-time which would require messy SFINAE hacks to make this C++14-compatible.
Todo:
- A github build matrix that also includes clang and MSVC. - Additional documentation, including the mathematics.
Feedback and endorsements for Boost Library inclusion are welcome and requested.
1) What exactly is mean by "Standard Library Support Requirements"? Do my types have to work with the actual Standard library functions (e.g. via implicit conversion to double) or does there have to be a function with the same name that can be found via ADL? 2) Any plan to support types, where the result of a math operation might not be the same as the operands? 3) Regarding the language standard: Don't use c++17 just for the sake of it, or because it simplifies a few lines of code. But if (contrary to Peter Dimov's statement) it actually lets you avoid lots of template machinery It probably also reduces compile times and I for one would be very happy to see a library that prioritizes implementation simplicity and compile times over support for outdated standards and compilers. Best Mike
Best regards, Matt
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Thu, Dec 20, 2018 at 12:38 PM mike via Boost
1) What exactly is mean by "Standard Library Support Requirements"?
That phrase is copied from: https://www.boost.org/doc/libs/1_69_0/libs/math/doc/html/math_toolkit/real_c... Those functions listed (fabs(), abs(), ceil(), ...) in that section of the code are requirements for satisfying Boost's "Conceptual Requirements for Real Number Types".
Do my types have to work with the actual Standard library functions (e.g. via implicit conversion to double) or does there have to be a function with the same name that can be found via ADL?
ADL should be sufficient. An update was made to the code today to make better use of ADL; in fact an example and test was just added using boost::multiprecision::cpp_dec_float_100. It's the second example on the current README.md: https://github.com/pulver/autodiff
2) Any plan to support types, where the result of a math operation might not be the same as the operands?
This is supported now, using the same mechanism of type-promotion as given
in
#include
3) Regarding the language standard: Don't use c++17 just for the sake of it, or because it simplifies a few lines of code.
But if (contrary to Peter Dimov's statement) it actually lets you avoid lots of template machinery It probably also reduces compile times and I for one would be very happy to see a library that prioritizes implementation simplicity and compile times over support for outdated standards and compilers.
A member of the mailing list Kedar B. is looking into making it C++11/14 compatible. Back when I wrote the first version in C++11, it was among my first forays into SFINAE coding and there was probably quite a bit of room for improvement. Matt
participants (8)
-
John Maddock
-
Marc Glisse
-
Matt Pulver
-
mike.dev@gmx.de
-
Peter Dimov
-
Quentin Chateau
-
Raffi Enficiaud
-
Vinnie Falco