Le 18/03/2017 à 22:52, Louis Dionne via Boost a écrit :

...

If there were several proposals, we could review all of them together. We already did that for the thread/futures proposals. Ah, that's exactly the kind of thing I was looking for, thanks. So we've had previous experience with this. Great.

...

* do we want a Hana style with "concepts" and with customizations? Do we want other data type than type lists? IIUC Peter's library works only with template aliases as data types and almost variadic class template with type parameters is a good candidate for a type list (even std::variant :( ) As far as I can tell, none of the proposals have Hana-style customization points and concepts, but I see no reason to explicitly request that they provide them. I'm happy as long as there's some way to interoperating with other libraries. I'm not requesting that the meta libraries customize Hana concepts as Boost.Hana has values. I would like we discuss if we adapting those meta-concepts to the meta world could be useful or not.

...

However I see only one official proposal (mp11 - Peter Dimov), even if there are other interesting libraries (as Meta, Brigand). You're right. I thought Bruno Dutra's Metal had been officially submitted, but not yet. In any case, it's just a matter of days or weeks. Who knows ;-)

Vicente

Vicente J. Botet Escriba wrote:

* do we want (can) to use as (HOMF) high-order meta-functions? if we nedd them at all). IIUC, Peter library doesn't works with HOMFs. * do we want a Hana style with "concepts" and with customizations? Do we want other data type than type lists? IIUC Peter's library works only with template aliases as data types and almost variadic class template with type parameters is a good candidate for a type list (even std::variant :( )

These two are the distinguishing features of mp11. All other libraries have chosen differently. To clarify, the mp11 approach does support higher-order metaprogramming, but it chooses to make the ordinary case easier at the expense of the HOMF case, which is made harder, whereas the other libraries choose to make HOMF easier, at the expense of taking quoted metafunctions as arguments, which requires aliases to be quoted. mp11, in contrast, requires quoted metafunctions to be de-quoted (Q::invoke) when passed to algorithms. To clarify further, std::variant is absolutely a supported typelist in mp11. If you have a std::variant V, you can remove the duplicate types from it with mp_unique<V>. This is again a deliberate design decision which keeps simple uses simple at the expense of making more complicated uses more complicated. With the other libraries, to remove the duplicates from a variant, you first have to convert it to the native typelist type, apply 'unique', then convert back to std::variant.

Le 18/03/2017 à 23:08, Peter Dimov via Boost a écrit :

Vicente J. Botet Escriba wrote:

...

* do we want (can) to use as (HOMF) high-order meta-functions? if we nedd them at all). IIUC, Peter library doesn't works with HOMFs. * do we want a Hana style with "concepts" and with customizations? Do we want other data type than type lists? IIUC Peter's library works only with template aliases as data types and almost variadic class template with type parameters is a good candidate for a type list (even std::variant :( )

These two are the distinguishing features of mp11. All other libraries have chosen differently. To clarify, the mp11 approach does support higher-order metaprogramming, but it chooses to make the ordinary case easier at the expense of the HOMF case, which is made harder, whereas the other libraries choose to make HOMF easier, at the expense of taking quoted metafunctions as arguments, which requires aliases to be quoted.

mp11, in contrast, requires quoted metafunctions to be de-quoted (Q::invoke) when passed to algorithms.

I have the impression that mp11 cannot use the meta function returned by a function as a parameter of an algorithm, but I'm missing something that it should be evident to you. How can we define a compose meta-function that could be used later as parameter? algo>

To clarify further, std::variant is absolutely a supported typelist in mp11. If you have a std::variant V, you can remove the duplicate types from it with mp_unique<V>. This is again a deliberate design decision which keeps simple uses simple at the expense of making more complicated uses more complicated. With the other libraries, to remove the duplicates from a variant, you first have to convert it to the native typelist type, apply 'unique', then convert back to std::variant.

I know. I just find it weird to take std::variant as a data type that models type list. I agree with you that given the definition you have used in mp11 (it is also the case of Brigand), std::variant models a mp11 type list. I'm not saying that it is not useful and I suspect that it should reduce the compile time. My main concern is that Boost.Hana has more data types than type list and that the algorithms are associated to concepts. IIUC, mp11 has only one concept (variadic template) and all the algorithms we can define on this very rich data structure. Maybe you are right and this is all we need. At a first glance I would prefer a meta library that is based on the design of Hana. I'm not saying here that I want the mixity of type and values. Of course, if the compile times are very different I could change my first impression. Vicente

2017-03-18 22:49 GMT+01:00 Vicente J. Botet Escriba via Boost < boost@lists.boost.org>:

Le 18/03/2017 à 21:32, Louis Dionne via Boost a écrit :

...

Dear Boost,

We've seen two metaprogramming libraries ask for a formal review on the list recently. I expect a third (Brigand) might join the party soon, and maybe even a fourth (Eric Niebler's meta). I think it would be wise to have a plan for how we're going to deal with this.

Before going further, I'd also like to clear something out. Some people have asked whether "classic" metaprogramming libraries still had their place given that we now have Hana. As the author of Hana, I think the answer is yes. While Hana is (IMHO) easier to use and more powerful, it also has an important shortcoming. Hana is too slow when dealing with very large inputs. This is due to the fact that it can also handle values, which is much heavier than dealing with types only. This can be alleviated to some extent, but the truth is that our execution model (the compiler) simply has to do more work when handling values.

Hence, I think Boost does need a modern classic TMP library to handle these cases where a library writer needs to handle gigantic type lists. I think marketing such a library as an advanced tool for library writers would be a service to the overall C++ community, but that's an orthogonal concern.

However, Boost needs one such library, not four. I think we can't just do 4 reviews and include all the libraries that pass in Boost; that would be a huge disservice to our users, who will be left with the burden of choosing. Heck, if we can't even make our mind, how can they?

So, how do we pick? Have we ever been in a similar situation where we have multiple competing libraries solving _exactly_ the same problem?

If there were several proposals, we could review all of them together. We already did that for the thread/futures proposals.

I believe that some of the things we need to answer are: * do we want (can) to use as (HOMF) high-order meta-functions? if we nedd them at all). IIUC, Peter library doesn't works with HOMFs. * do we want a Hana style with "concepts" and with customizations? Do we want other data type than type lists? IIUC Peter's library works only with template aliases as data types and almost variadic class template with type parameters is a good candidate for a type list (even std::variant :( ) * do we need lazy evaluation? * do we need lambdas? * do we want a C++11/C++14/C++17 library? .... other you can think of.

Maybe, during the process, to get the additional insight, we sould request of each of the candidates to compare the librearies with one another. They are experts in the domain, and should be able to quickly identify different tradeoffs taken by deiifferent libraries. Regards, &rzej;

On 2017-03-19 12:02, Peter Dimov via Boost wrote:

Paul Fultz II wrote:

...

It would be nice to see better collaboration between authors to build an unifying metaprogramming library, rather than competing against each other. Of course, this can be somewhat difficult as they have different fundamental concepts.

As I see it, it's the fundamental concepts competing against each other, and that's how it should be; how else we'll find out which approach is best?

Would not that be better to contain the battle among the gurus, the lib developers. So that they are forced to cooperate, to consider alternatives, to come up with one best (in their collective opinion) solution... and leave the user out of it? Otherwise users are forced to consider 4 solutions, to evaluate those and to decide (potentially incorrectly) which one to use. As a user I personally do not have the resources and the knowledge to do that. Taking/choosing a Boost lib I've always known that there was a lot of effort spent evaluating, optimizing, improving, etc. the lib. So that I get the best I can get.

On 3/18/17 19:48, Vladimir Batov via Boost wrote:

Would not that be better to contain the battle among the gurus, the lib developers. So that they are forced to cooperate, to consider alternatives, to come up with one best (in their collective opinion) solution... and leave the user out of it? Otherwise users are forced to consider 4 solutions, to evaluate those and to decide (potentially incorrectly) which one to use. As a user I personally do not have the resources and the knowledge to do that. Taking/choosing a Boost lib I've always known that there was a lot of effort spent evaluating, optimizing, improving, etc. the lib. So that I get the best I can get.

Users of these libraries are often library developers. Many of us have already implemented smallish TMP libraries that we use internally. I think the Boost community is the right community to review just this sort of library. Abusing compilers and meta programming rich libraries is what we are known for (for good or bad). I expect a combined review to tease out the best design choices and implementations and may even discover better solutions yet-to-be-discovered. micahel -- Michael Caisse Ciere Consulting ciere.com

On Mar 18, 2017, at 7:02 PM, Peter Dimov via Boost wrote:

Paul Fultz II wrote:

...

It would be nice to see better collaboration between authors to build an unifying metaprogramming library, rather than competing against each other. Of course, this can be somewhat difficult as they have different fundamental concepts.

As I see it, it's the fundamental concepts competing against each other, and that's how it should be; how else we'll find out which approach is best?

Well brigand and mpl11 seem very similar conceptually. There is a conceptual difference with Metal where everything is explicit. However, I don’t think we have to choose between a implicit and explicit API. Rather a library could provide both APIs, much like Boost.PP provides an API for both explicit and implicit(ie deduced) re-entrance. Furthermore, the simple API of mpl11 very likely could be implemented using Metal underneath. Perhaps instead of providing two different libraries we provide one library with metal::explicit_ and metal::implicit where the implicit API could follow mpl11 very closely(if thats what seems to be the best approach to the implicit part considering other APIs). Paul

On 19/03/2017 19:57, Louis Dionne via Boost wrote:

...

If Hana is the slow, "full fat" metaprogramming library,

Hana is not a "slow, full fat" metaprogramming library. Quite far from that. It's just that for pure type-level metaprogramming (which by the way is quite rarely needed nowadays with deduced return types), we can't possibly be as fast as Brigand, Metal or such type-level only solutions that is highly optimized for that precise use case.

Surely you forgot to append the condition "we can't possibly be as fast as ... *with current compilers*"?

...

is there space remaining for exactly one fast, "lightweight" metaprogramming library?

The problem is that the term metaprogramming is overloaded to mean several things. It can either mean pure type-level metaprogramming (MPL world), or algorithms on tuples (Fusion world). It HAPPENS that the latter is powerful enough to perform the former; this is what Hana allows and then you only need one metaprogramming library. However, that is heavier compile-time wise and does not scale as well as a library that was crafter with type computations only from the start. And if that was your question, I don't think you can do what Hana does (i.e. handle Fusion + MPL) while being significantly faster. You might be able to be some small constant factor faster, but that's it.

Again, Hana is slower than these other libraries because *current compilers* have been heavily optimised for type based metaprogramming, and not yet for deduced return type metaprogramming. But this won't be the situation soon. The Visual Studio team are actively targeting MSVC at Hana. Knowing them, even after it's compiling 100% clean they'll then go on to make MSVC *fast* with Hana. I don't doubt the clang guys are also thinking the same thing, that a Hana based benchmark could be real useful to shining a spotlight on code which is currently highly suboptimal in their compilers. (As an aside, a Hana based compiler performance benchmark would be awesome, you should write one with all that free time you have :) and put it online with graphs or something)

By the way, these libraries for pure type-level metaprogramming have the same problem, it's just that their constant factors are smaller than Hana's. Take any of these libraries and try to process a type list with 200K elements while doing something non-trivial. The fundamental problem is that we need language-level support for metaprogramming if we want to go completely crazy with it, and some of us are working exactly on this right now for C++next (or maybe the one after).

So I've got to ask the question: Are we not jumping the gun here with adding any metaprogramming library to Boost beyond Hana? If we wait a few years, the performance gap between Hana and any of these other libraries will probably close very significantly to the point of inconsequence. Let me put this another way: do any of the other metaprogramming libraries add *in pure API or capability terms* anything over Hana? Because if they add nothing new in terms of a better API, or able to do extra stuff, there is a *strong* argument that we should just wait and see what future compilers look like before adding more metaprogramming libraries. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Let me put this another way: do any of the other metaprogramming libraries add *in pure API or capability terms* anything over Hana?

I don't mean to derail the discussion again, but since you asked, Metal provides SFINAE-friendliness guarantees that Hana does not. Think std::enable_if on steroids.

Thanks for the info. However to continue to play devil's advocate, is that a highly motivating reason for an additional metaprogramming library over Hana in Boost when Concepts will ship in every major compiler this year or next? If someone were proposing a metaprogramming library which allowed me to write code which uses Concepts on a supporting compiler, but fell back to a C++ 14 emulation on older compilers, that I'd find that a compelling reason for an additional Boost metaprogramming library. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On Sun, Mar 19, 2017 at 4:22 PM, degski via Boost wrote:

I must say, I like Peters' 'simple' mp11 though, providing building blocks to more advanced stuff (I would say (generally) the STL-spirit).

I agree being simple is a plus, but IMO there is nothing exceptionally simple about mp11. Metal was designed to be as a simple building block as possible as well [1], the only difference is that it approaches this goal in different way. [1]: http://brunocodutra.github.io/metal/#concepts

Oswin Krause wrote:

For me the biggest question is: will it work natively with mpl::vector?

Not very well at the moment; mpl::vector is a list of fixed size 20, whereas mpl::vector::type is of size 3, but still fixed. It will only work with algorithms that don't need to change the size of the list. It would be possible in principle to make the algorithms take mpl::vector and return mp_list, I suppose.

Bruno Dutra wrote:

...

It would be possible in principle to make the algorithms take mpl::vector and return mp_list, I suppose.

You forgot to take into account that algorithms such as mpl::insert and mpl::erase return proxies that inherit from vectors and don't expose the elements in their types signatures, while being themselves valid instances of a Vector. This is the real deal breaker.

On MSVC, insert/erase return numbered vectors, but on g++ and clang++, you're right, they return some weird v_item sequences. What is our use case here though? From where would those mpl types come?

On Sun, Mar 19, 2017 at 7:43 PM, Oswin Krause < Oswin.Krause@ruhr-uni-bochum.de> wrote:

On 2017-03-19 17:13, Bruno Dutra via Boost wrote:

...

On Sun, Mar 19, 2017 at 4:22 PM, degski via Boost wrote:

I must say, I like Peters' 'simple' mp11 though,

...

providing building blocks to more advanced stuff (I would say (generally) the STL-spirit).

I agree being simple is a plus, but IMO there is nothing exceptionally simple about mp11.

Hi,

I would disagree. Conceptionally it is a lot easier to grasp as everything is defined either in language primitives or logical constraints.

Here are the concepts:

A list is any type of the form L. A set is a list where all elements are unique A map is a set of keys zipped with an arbitrary list of same size. A metafunction is a template alias with unspecified template parameters.

The definition of the concepts do not need to reference any primitive inside the library and thus it can work well with many other of the proposed libraries and libraries outside of boost, without having to jump through additional hoops (e.g. renaming/"copying" arguments).

You've pretty much described Metal, except that Metal requires lists to be specializations of metal::list, but that is far from a shortcoming, in fact there is a very good reason behind that design choice. What do you expect to happen if you try to erase a value from a template, such as std::unique_ptr, that has a default parameter using mp11? What about inserting an element? Templates with default parameters are valid lists if the template that represents a List is not specified, so algorithms must be well defined for all these cases, which is very cumbersome to handle on the library side for very little gain, not to mention that it often leads to surprising and unexpected behavior. For me the biggest question is: will it work natively with mpl::vector?

This would make it a clear winner in my book.

Working natively with mpl::vector is only possible through adaptors, either implicit in the library, or explicit to the user, because the type of the concept behind mpl::vector is unspecified by design. Personally I think there is very little gain in embedding such adaptors implicitly in any library, because they bring along a lot of legacy and make the life of maintainers miserable for basically no gain, since a explicit adaptor, such as from_mpl<>, has zero cost to the user. My choice for Metal was to provide metal::from_mpl, a metafunction that converts any MPL concept to its equivalent in Metal parlance. Bruno

Am 19.03.2017 um 23:00 schrieb Michael Caisse via Boost:

On 3/19/17 12:31, Bruno Dutra via Boost wrote:

...

On Sun, Mar 19, 2017 at 7:43 PM, Oswin Krause < Oswin.Krause@ruhr-uni-bochum.de> wrote:

...

On 2017-03-19 17:13, Bruno Dutra via Boost wrote:

...

On Sun, Mar 19, 2017 at 4:22 PM, degski via Boost wrote:

<snip features and criticisms of various libraries>

...

You've pretty much described Metal, except that Metal requires lists to be specializations of metal::list, but that is far from a shortcoming, in fact there is a very good reason behind that design choice.

<snip more things why-I-designed-my-library-this-way>

Here is a suggestions. I don't think this thread is the place for the authors to explain the benefits of their design/implementation/usage. There will be a lot of time for that.

In this thread, we are trying (as a community) to figure out the best approach to deal with multiple submissions of TMP libraries. Lets concentrate on that concern.

As mentioned before, I think some kind of comparison between such TMP libraries is required. A comparison between their features/concepts and especially the users' needs they try to address with these features/concepts. For example, from a user's point of view I would pretty much like to see a "drop-in" replacement for Boost.MPL which compiles (much) faster (and uses less RAM doing so). However, I am not really interested in using such a replacement by myself. I am instead interested in all existing Boost libraries that currently use Boost.MPL to "magically" become/compile faster (and use less RAM). Of course, if all Boost-libraries that use Boost.MPL are actively maintained and their maintainers would apply internal changes so that these libraries would use another (faster) TMP library (e.g. Boost.Hana or what might be adopted next), that would be fine, too. However, it probably is less work for the maintainer to only adjust some includes and maybe some typedefs to use a "drop-in" replacement instead of changing a lot of code to use another TMP library which is completely different from Boost.MPL.

Don't worry ... there will be plenty of opportunity for critique and defense (o;

michael

Just my "user's view". Deniz

On Mon, Mar 20, 2017 at 4:00 PM, Peter Dimov via Boost wrote:

...

As mentioned before, I think some kind of comparison between such TMP libraries is required.

Error C6301: passive voice used: "is required"

Who should produce this comparison?

I agree with Deniz if we can rephrase his words slightly, changing required to desirable. I have myself some knowledge of MPL and a little bit of knowledge of Brigand. But my knowledge is fragmented, having used only what I use in my projects. It would be quite nice to have an overview of the capabilities and differences of the different libraries. For metal and brigand that should not be to difficult as I know that the authors have followed both libraries. For your library, I read your excellent posts on metaprogramming, but I was unaware that it had become a public library before you submitted it to boost. /Peter

Le 19/03/2017 à 16:22, degski via Boost a écrit :

On 18 March 2017 at 14:32, Louis Dionne via Boost wrote:

...

However, Boost needs one such library, not four.

Maybe, maybe not. What would help though if there would be a (ONE) document outlining what the differences are and in what domain/context one should prefer on library over the other, complemented with pros and cons of each lib.

Including Hana. The best would be only one library for all TMP needs, but if there is more than one, such a document is of the essence. --- Loïc