I strongly suggest that the goal of using CMake with Boost begin with making Boost libraries, whether header-only or built, available to CMake projects. This also means that we have a way of testing whether or not such a solution actually works for any given Boost library. My own preference for testing whether or not such a solution would work or not would be to use our current b2/Boost Build test jamfiles, but if this is too difficult to do we still need some way to test the solution for each library so each library maintainer can see whether or not the solution we choose works for his library.

Are you talking about writing a CMake Config for installed boost? If so I'm strongly in favor of doing so! You'd need some BoostConfig.cmake files with appropriate contents. They are similar to pkgconfig files, so small text files. Those can be (mostly) generated by CMake when it is used to build boost and in the end they replace the FindBoost.cmake currently shipped with CMake (and which the CMake maintainers dislike) Starting from the "other side" would allow some experimentation on how Boost can be consumed by CMake projects and will show many pitfalls early. For example one would need to think about how to deal with the variants (static/dynamic, debug/release[actually simple], static/dynamic runtime, versions installed to same folder, ...) If this is solved, one can go further and implement the actual build system. Note that at this point naming conventions are already established and the current build system can be used to generate these files requiring less effort.

On 10/8/2018 6:15 AM, Mike Dev via Boost wrote:

...

-----Original Message----- From: Boost On Behalf Of Edward Diener via Boost Sent: Monday, October 1, 2018 8:00 PM

On 10/1/2018 1:07 PM, Robert Ramey via Boost wrote:

...

Background ==========

My personal knowledge of build systems is somewhat limited.

a) I don't really understand CMake all that well. I have made CMakeLists.txt files for both the serialization library and the safe numerics libraries as an aid to build and maintain IDE and make files used in running tests and posting results to CDash.

b) I don't really understand our current system, Boost Build, very well either. I've made jam files for building and running tests for the serialization and safe numerics libraries.

c) I recently reviewed C++Now 2017: Daniel Pfeifer's excellent video from C++Now 2017 “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk . I recommend it highly.

Despite my lack of knowledge in this area, or perhaps because of it, I've volunteered to direct the effort to evolve Boost so that users of Boost can avail themselves of the benefits of CMake. This effort will be organized as follows:

a) A discussion on the boost developers mailing list with the goal of reaching a consensus as to what benefits CMake can and should provide to users and developers of Boost. This effort will commence with the posting of this notice. I hope that this discussion can be more or less resolved within 30 days.

b) I will then synthesize from the above discussion a call for proposals which lists the requirements and requirements.

c) Those proposing CMake for boost will have approximately 90 days to prepare their proposals. Their proposal are expected to look similar to a boost library proposal. That is they are expected to have some (CMake) code, along with tests, documentation including tutorials and reference. This is only an expectation. It's conceivable that such a proposal might only contain conventions and and samples of what boost library should contain to fulfill the requirements. As with boost libraries, submissions are not required to totally complete. But they have to be sufficiently complete to convince reviewers the the submission fulfills the stated requirements and can be finished in good time.

d) After the 90 day period, submissions will be reviewed simultaneously. Note that this is at a variance from normal boost procedure of reviewing submissions one at a time. Since we expect to receive a number of submissions and only one can be accepted, the normal boost protocol can't really function. Target date for this review is 1 February 2019.

e) After the review period the review manager will prepare a report which includes the decision of which submission will be accepted into the official boost distribution. Currently, I, Robert Ramey, expect to be review manager. However, it's possible that this by the time the review is undertaken, this coveted position could be assigned to some more worthy candidate.

Scope, Requirements and Features ================================

We presume that submissions will fulfill the applicable requirements of any boost submission. In addition we would like to see the following addressed.

Library Build ------------- “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk Recommends that all libraries - including header only libraries have a CMakeList.txt file. What should this include for Boost Libraries?

Library Test ------------ Should facilities for "testing" be only done by developers? or should users who acquire library packages also be able to test libraries. Should CMake testing results posting be used - CDASH? Should we just skip the issue of Library Testing and continue to depend on Boost Build.

Library Packaging ----------------- Is the library packaging facility provide by CMake - CPACK - useful to boost. Should boost libraries be updated to support it?

Dependencies ------------ If the above is implemented, can we depend upon CMake to handle library dependencies? Or does some special functionality Do circular dependencies constitute a problem?

Modularity ---------- Currently boost is organized as a tightly integrated group of libraries. This organization manifests itself in a number of ways. We have a Boost "super project" in github. Boost libraries are "sub projects" of this super project. This known structure is exploited by and depended upon by boost tools. Should CMake support continue this policy/design. We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?

Documentation/testing --------------------- What support should be require for uses of Boost CMake implementation. Documentation, examples, templates for helping library developers. How should the CMake design/implementation be tested?

Other considerations -------------------- add your requirement/features here.

Useful Resources =============== “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk https://gist.github.com/mbinna/c61dbb39bca0e4fb7d1f73b0d66a4fd1 https://github.com/purpleKarrot/Boost.CMake https://www.youtube.com/watch?v=eC9-iRN2b04

Please try to keep this discussion on this list.

I know that there has been a lot going on on slack. But having it in two places makes it harder for me to review and summarize. Also slack doesn't conveniently maintain comment hierarchy. So please try to keep this discussion on this list.

I strongly suggest that the goal of using CMake with Boost begin with making Boost libraries, whether header-only or built, available to CMake projects. This also means that we have a way of testing whether or not such a solution actually works for any given Boost library. My own preference for testing whether or not such a solution would work or not would be to use our current b2/Boost Build test jamfiles, but if this is too difficult to do we still need some way to test the solution for each library so each library maintainer can see whether or not the solution we choose works for his library.

This should be our first goal and all other goals should be tabled until and when we can have a solution where this works flawlessly and there is a general agreement that whatever proposal is chosen to implement this is best.

Only when the above is fully accomplished should Boost then look into the goal of using CMake internally, perhaps for test, docs, building libraries, or what-not. We need to work this way because every solution is bound to end with comments that it does X but does not do Y or it does X and Y but does not do Z, and we all know where these endless arguments end: that we end up doing nothing because we start by trying to do too much and we can never reach agreement.

I agree with you in so far, that the focus should be on the interface (how to consume boost libraries) and less on the implementation details, but I see two practical reasons, why it is beneficial to immediately use cmake (at least) for building too:

1) Creating a CMake target for a separately compiled binary that can be consumed by the user is about the same effort as using cmake to build the library (if necessary) in which case you get the interface information for free (switching to cmake for unit-tests and docs is a different topic).

2) "making Boost libraries [...] available to CMake projects" for me also means I have an easy way to compile those libraries with specific compiler flags that match my project settings - otherwise, what is the advantage compared to the current mechanism that is already build into cmake? Ensuring consistent build flags is relatively easy if all libraries use the same build system (in this case cmake) but can become quite complex if you want to implement an automatic translation from cmake to b2 or vice versa.

So yes, while a library (build-) interface and consummation should be the focus of the review, I'd expect that at least building the libraries with cmake will be the simplest way to implement that interface.

Just to be completely clear: Just because the "reference implementation" of a cmake interface is probably a cmake-based build, that doesn't mean everyone has to actually adopt that reference implementation as long as the interface is consistent.

Boost Build has a feature which allows the programmer to test at build time what C++ features are supported for a particular compiler implementation and compile parameters, and change the build internally on-the-fly accordingly. I do not know if any Boost non-header only library uses this feature but if one does it will be difficult translating this to CMake terms, if such a similar feature exists in CMake. It is for such a reason that as soon as you start talking about using CMake for anything else but identifying Boost libraries for the purposes of end-user use of Boost libraries in their own CMake scripts, that you may run into much more work than you can imagine. You just can't tell individual library developers to translate all their own use of Boost Build into CMake, as if this will be simple for them to do, and they will just do it with little time spent and as if it will be no work for them. Boost Build, for better or worse, has some extensive features which have no easy correspondence to CMake. Unless you or someone else is willing to develop a bridge between Boost Build and CMake for features of Boost Build which libraries rely on, then individual Boost libraries which use such Boost Build features are going to find it very difficult to transition to CMake, possibly even for something as seemingly innocuous as building the library itself using a CMake script. This is why I have urged the move to CMake by Boost to proceed slowly by baby steps, rather than to try doing things that may take a huge initial effort to get done, and therefore may end up never accomplishing anything.

Mike

On 10/8/2018 6:46 PM, Richard via Boost wrote:

In article , Edward Diener via Boost writes:

...

Boost Build has a feature which allows the programmer to test at build time what C++ features are supported for a particular compiler implementation and compile parameters, and change the build internally on-the-fly accordingly. I do not know if any Boost non-header only library uses this feature but if one does it will be difficult translating this to CMake terms, if such a similar feature exists in CMake.

CMake has supported this for years, it is not a blocking issue.

Good ! Then I am sure you will be glad to be nominated as the person who converts all instances of the Boost Build use of this feature, in any Boost library which uses it, to the exact equivalent in CMake. It is great to know that you have volunteered to do this for Boost !

On 2018-10-09 05:13 PM, Richard via Boost wrote:

In article , Edward Diener via Boost writes:

...

Good ! Then I am sure you will be glad to be nominated as the person who converts all instances of the Boost Build use of this feature, You can nominate about whatever you like, but I am not volunteering for anything in boost. It takes forever to get anything done.

I think the point wasn't so much to nominate you, but rather to point out that a statement like the one you made is rater gratuitous to make if you don't have to back it up by actually doing the implied work. Stefan -- ...ich hab' noch einen Koffer in Berlin...

AMDG On 10/10/2018 10:32 AM, Richard via Boost wrote:

In article , Stefan Seefeld via Boost writes:

...

On 2018-10-09 05:13 PM, Richard via Boost wrote:

...

In article , Edward Diener via Boost writes:

...

Good ! Then I am sure you will be glad to be nominated as the person who converts all instances of the Boost Build use of this feature, You can nominate about whatever you like, but I am not volunteering for anything in boost. It takes forever to get anything done.

I think the point wasn't so much to nominate you,

Au contraire, he quite specifically nominates me in the first line I quoted and further implies that I will be happy about it.

The question he raised was whether or not CMake supports this kind of feature testing and I simply pointed out that it has been supported for many years.

Why he goes further and tries to volunteer me for work is beyond my understanding.

It's because you misread the post that you responded to:

...

...it will be difficult translating this to CMake terms, if such a similar feature exists in CMake.

CMake has supported this for years, it is not a blocking issue.

From Edward's later responses, it seems clear that he really did mean exactly what he said rather than "...if a similar feature *does not* exist...", which is what you responded to.

In Christ, Steven Watanabe

On 10/8/18 5:40 PM, Edward Diener via Boost wrote:

On 10/8/2018 6:46 PM, Richard via Boost wrote:

...

In article ,      Edward Diener via Boost writes:

...

Boost Build has a feature which allows the programmer to test at build time what C++ features are supported for a particular compiler implementation and compile parameters, and change the build internally on-the-fly accordingly. I do not know if any Boost non-header only library uses this feature but if one does it will be difficult translating this to CMake terms, if such a similar feature exists in CMake.

CMake has supported this for years, it is not a blocking issue.

Good ! Then I am sure you will be glad to be nominated as the person who converts all instances of the Boost Build use of this feature, in any Boost library which uses it, to the exact equivalent in CMake. It is great to know that you have volunteered to do this for Boost !

I don't think that at this point you can presume that such a person would be necessary. But then again I don't presume it wouldn't either. I'm going restate this as a suggested requirement: "Any new CMake implemenation for Boost shouldn't require non-trivial efforts on existing libraries by library developers." I'll take this under advisement. Robert Ramey

On 10/9/2018 7:09 PM, mike via Boost wrote:

...

-----Original Message----- From: Boost On Behalf Of Edward Diener via Boost Sent: Monday, October 8, 2018 10:22 PM

On 10/8/2018 6:15 AM, Mike Dev via Boost wrote:

...

...

-----Original Message----- From: Boost On Behalf Of Edward Diener via Boost Sent: Monday, October 1, 2018 8:00 PM

On 10/1/2018 1:07 PM, Robert Ramey via Boost wrote:

...

Background ==========

My personal knowledge of build systems is somewhat limited.

a) I don't really understand CMake all that well. I have made CMakeLists.txt files for both the serialization library and the safe numerics libraries as an aid to build and maintain IDE and make files used in running tests and posting results to CDash.

b) I don't really understand our current system, Boost Build, very well either. I've made jam files for building and running tests for the serialization and safe numerics libraries.

c) I recently reviewed C++Now 2017: Daniel Pfeifer's excellent video from C++Now 2017 “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk . I recommend it highly.

Despite my lack of knowledge in this area, or perhaps because of it, I've volunteered to direct the effort to evolve Boost so that users of Boost can avail themselves of the benefits of CMake. This effort will be organized as follows:

a) A discussion on the boost developers mailing list with the goal of reaching a consensus as to what benefits CMake can and should provide to users and developers of Boost. This effort will commence with the posting of this notice. I hope that this discussion can be more or less resolved within 30 days.

b) I will then synthesize from the above discussion a call for proposals which lists the requirements and requirements.

c) Those proposing CMake for boost will have approximately 90 days to prepare their proposals. Their proposal are expected to look similar to a boost library proposal. That is they are expected to have some (CMake) code, along with tests, documentation including tutorials and reference. This is only an expectation. It's conceivable that such a proposal might only contain conventions and and samples of what boost library should contain to fulfill the requirements. As with boost libraries, submissions are not required to totally complete. But they have to be sufficiently complete to convince reviewers the the submission fulfills the stated requirements and can be finished in good time.

d) After the 90 day period, submissions will be reviewed simultaneously. Note that this is at a variance from normal boost procedure of reviewing submissions one at a time. Since we expect to receive a number of submissions and only one can be accepted, the normal boost protocol can't really function. Target date for this review is 1 February 2019.

e) After the review period the review manager will prepare a report which includes the decision of which submission will be accepted into the official boost distribution. Currently, I, Robert Ramey, expect to be review manager. However, it's possible that this by the time the review is undertaken, this coveted position could be assigned to some more worthy candidate.

Scope, Requirements and Features ================================

We presume that submissions will fulfill the applicable requirements of any boost submission. In addition we would like to see the following addressed.

Library Build ------------- “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk Recommends that all libraries - including header only libraries have a CMakeList.txt file. What should this include for Boost Libraries?

Library Test ------------ Should facilities for "testing" be only done by developers? or should users who acquire library packages also be able to test libraries. Should CMake testing results posting be used - CDASH? Should we just skip the issue of Library Testing and continue to depend on Boost Build.

Library Packaging ----------------- Is the library packaging facility provide by CMake - CPACK - useful to boost. Should boost libraries be updated to support it?

Dependencies ------------ If the above is implemented, can we depend upon CMake to handle library dependencies? Or does some special functionality Do circular dependencies constitute a problem?

Modularity ---------- Currently boost is organized as a tightly integrated group of libraries. This organization manifests itself in a number of ways. We have a Boost "super project" in github. Boost libraries are "sub projects" of this super project. This known structure is exploited by and depended upon by boost tools. Should CMake support continue this policy/design. We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?

Documentation/testing --------------------- What support should be require for uses of Boost CMake implementation. Documentation, examples, templates for helping library developers. How should the CMake design/implementation be tested?

Other considerations -------------------- add your requirement/features here.

Useful Resources =============== “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk https://gist.github.com/mbinna/c61dbb39bca0e4fb7d1f73b0d66a4fd1 https://github.com/purpleKarrot/Boost.CMake https://www.youtube.com/watch?v=eC9-iRN2b04

Please try to keep this discussion on this list.

I know that there has been a lot going on on slack. But having it in two places makes it harder for me to review and summarize. Also slack doesn't conveniently maintain comment hierarchy. So please try to keep this discussion on this list.

I strongly suggest that the goal of using CMake with Boost begin with making Boost libraries, whether header-only or built, available to CMake projects. This also means that we have a way of testing whether or not such a solution actually works for any given Boost library. My own preference for testing whether or not such a solution would work or not would be to use our current b2/Boost Build test jamfiles, but if this is too difficult to do we still need some way to test the solution for each library so each library maintainer can see whether or not the solution we choose works for his library.

This should be our first goal and all other goals should be tabled until and when we can have a solution where this works flawlessly and there is a general agreement that whatever proposal is chosen to implement this is best.

Only when the above is fully accomplished should Boost then look into the goal of using CMake internally, perhaps for test, docs, building libraries, or what-not. We need to work this way because every solution is bound to end with comments that it does X but does not do Y or it does X and Y but does not do Z, and we all know where these endless arguments end: that we end up doing nothing because we start by trying to do too much and we can never reach agreement.

I agree with you in so far, that the focus should be on the interface (how to consume boost libraries) and less on the implementation details, but I see two practical reasons, why it is beneficial to immediately use cmake (at least) for building too:

1) Creating a CMake target for a separately compiled binary that can be consumed by the user is about the same effort as using cmake to build the library (if necessary) in which case you get the interface information for free (switching to cmake for unit-tests and docs is a different topic).

2) "making Boost libraries [...] available to CMake projects" for me also means I have an easy way to compile those libraries with specific compiler flags that match my project settings - otherwise, what is the advantage compared to the current mechanism that is already build into cmake? Ensuring consistent build flags is relatively easy if all libraries use the same build system (in this case cmake) but can become quite complex if you want to implement an automatic translation from cmake to b2 or vice versa.

So yes, while a library (build-) interface and consummation should be the focus of the review, I'd expect that at least building the libraries with cmake will be the simplest way to implement that interface.

Just to be completely clear: Just because the "reference implementation" of a cmake interface is probably a cmake-based build, that doesn't mean everyone has to actually adopt that reference implementation as long as the interface is consistent.

Boost Build has a feature which allows the programmer to test at build time what C++ features are supported for a particular compiler implementation and compile parameters, and change the build internally on-the-fly accordingly. I do not know if any Boost non-header only library uses this feature but if one does it will be difficult translating this to CMake terms, if such a similar feature exists in CMake.

As Richard pointed out, such features exist in cmake too and have been in common use for years. E.g. It has a built-in mechanism that lets you directly query for most of the post c++03 compiler features: (https://cmake.org/cmake/help/v3.13/prop_gbl/CMAKE_CXX_KNOWN_FEATURES.html) and as a more general feature, you can always check if a particular source file using some feature or type compiles or not: (https://cmake.org/cmake/help/v3.13/module/CheckCXXSourceCompiles.html)

Could you maybe point me to some examples where this feature is used so one could estimate how difficult a conversion is?

The feature is essentially programmed by config and predef to provide build-time testing of compiler support and testing of such things like the OS versions and compiler versions etc. See https://www.boost.org/doc/libs/1_68_0/libs/config/doc/html/boost_config/buil... for the explanation in config and https://www.boost.org/doc/libs/1_68_0/doc/html/predef/check_utilities.html for checking predef values at build time. The basics of the feature are offered by Boost Build as explained at https://boostorg.github.io/build/manual/develop/index.html#bbv2.reference.ru... in reference to the check-target-builds rule. Plenty of libraries use the feature during testing and I imagine that some of them may use the feature during building. Asking individual libraries to convert their tests or build to use some CMake equivalent, for every possible setting in config and predef which they use, is not realistic. I understand that CMake may have an equivalent technique, but who is going to program that technique to offer all the equivalents which config and predef currently offer ?

...

It is for such a reason that as soon as you start talking about using CMake for anything else but identifying Boost libraries for the purposes of end-user use of Boost libraries in their own CMake scripts,

Just consuming precompiled/pre-installed boost libraries in a cmake script is already possible thanks to the work of cmake contributors and the people providing boost packages for vcpkg, conan and other package management systems (some of those contributors are probably also boost library maintainers?). Those solutions have to keep playing catch up with the latest boost release if something significant changes (like name-mangling or a new library), but by and large they work quite well. Imho an official boost solution should provide a bit more functionality to justify the effort.

...

that you may run into much more work than you can imagine. You just can't tell individual library developers to translate all their own use of Boost Build into CMake, as if this will be simple for them to do, and they will just do it with little time spent and as if it will be no work for them.

Let's get one thing straight: Switching to cmake will require work, no one is denying that and it will not happen during the course of a single release. And if library developers are not willing to invest some time to do this, we can forget about it all together. However, it is my impression that many developers are willing to make that effort. All that is lacking is a clear decision of what the (semi-) final solution should look like to make sure the effort isn't wasted.

Now as I said. I'm not convinced, that only providing interface information is actually less work than building boost with cmake and letting it generate the cmake information automatically. That probably depends to a large degree if those cmake files can be auto generated by b2. However, just as there seem to be more complex build scenarios, there are probably scenarios where the auto-generated cmake-config files are not enough.

...

Boost Build, for better or worse, has some extensive features which have no easy correspondence to CMake.

Such as? If plain cmake is lacking any important features that are needed by boost, then a replacement is exactly what a boost cmake library should provide.

...

Unless you or someone else is willing to develop a bridge between Boost Build and CMake for features of Boost Build which libraries rely on, then individual Boost libraries which use such Boost Build features are going to find it very difficult to transition to CMake, possibly even for something as seemingly innocuous as building the library itself using a CMake script.

Let's not forget, that there are already solutions out there that build (almost?) all of boost with cmake. Here is one: https://github.com/boost-cmake/boost

Now instead of adopting and tweaking one of those existing solutions during review, you are imho asking for the development of yet another solution. I'm not sure if that is actually an easier path forward. That doesn't mean that an individual library author can't decide he wants to stick to b2 internally and just provide a cmake interface file, but we for the time being and

...

This is why I have urged the move to CMake by Boost to proceed slowly by baby steps, rather than to try doing things that may take a huge initial effort to get done, and therefore may end up never accomplishing anything.

That is kind of what I'm trying to achieve with my individual mini PRs. They are in no way a replacement for the existing boost build system, as they only cover a very narrow use case, but even those also build the boost libraries. Again, I don't think building is the issue (as I said, it has been done already) and if someone really encounters a problem, I'm sure people on the ML and elsewhere are willing to help. IMHO it will be far more difficult to agree all the bike shedding topics related to how exactly boost libraries should be made available to the consumer.

Also, proceeding by Baby steps means that boost will stay in limbo between cmake and boost build for quite some time, where library maintainers will have to maintain two systems and more or less constantly upgrading their CMake files. It doesn't have to happen all in one big step (I agree that that is not realistic), but considering that consumers can already use boost libraries in their cmake projects (albeit with a few pain points here and there) and that there are already proof of concepts for more fully fledged solutions out there, I think the initial goal should be a tad more ambitious.

Mike

On 10/9/18 4:58 PM, Andrey Semashev via Boost wrote:

...

in reference to the check-target-builds rule. Plenty of libraries use the feature during testing and I imagine that some of them may use the feature during building. Asking individual libraries to convert their tests or build to use some CMake equivalent, for every possible setting in config and predef which they use, is not realistic. I understand that CMake may have an equivalent technique, but who is going to program that technique to offer all the equivalents which config and predef currently offer ?

At this point I have no idea what it might take for a CMake fulfill such a requirement. But of course that's not relevant at this point. So this would be another suggested requirement: "Build and Test should be able to adjust target requirements according to the test results of boost/config - similar to how boost build does today." Again, This is noted. I'll take it under advismment when I craft the first cut of the "scope and requirements" proposal. Robert Ramey

...

Why do you think a potential Boost.CMake proposal cannot offer an equivalent of check-target-builds that internally uses the checks based on Boost.Config or Boost.Predef?

I do not think what you propose. I am just pointing out that this feature, used extensively from config and/or predef for a number of Boost libraries, needs to have an equally easy CMake equivalent so that converting the library's build or test jamfile to CMake is fairly trivial.

Sorry, but WHY exactly do you need to have the checks from Boost.Config/Predef in CMake? Do they exist in B2? In which form? Isn't Boost.Config/Predef a set of headers defining macros for consumption in source code?

...

Sorry, but WHY exactly do you need to have the checks from Boost.Config/Predef in CMake? Do they exist in B2? In which form? Isn't Boost.Config/Predef a set of headers defining macros for consumption in source code?

A great many Boost libraries use the checks from Boost.Config/Predef in their jamfiles. What do you propose to do when these jamfiles need to be converted for the purposes of using CMake for these libraries ?

Providing a CMake equivalent in the Boost.Config and Boost.Predef

What I meant was: I was only aware of a C++ part (header files) of Boost.Config/Predef. Can you point me to where those header defines are transported into bjam so they can be used by those other libs in their jamfiles? libraries for all the current checks which are offered by those libraries, and then using these equivalents by other libraries which currently use them Exactly this would be the proposal. I can comment on possible effort when I have seen how it is done in bjam.

For Predef https://github.com/boostorg/predef/tree/develop/tools/check with documentation for usage here https://www.boost.org/doc/libs/1_68_0/doc/html/predef/check_utilities.html.

For Config https://github.com/boostorg/config/tree/develop/checks and documentation for that here https://www.boost.org/doc/libs/1_68_0/libs/config/doc/html/boost_config/buil.... Thanks. Seeing this I approximate the work required to convert the checks of Config to ~1h, maybe 2h. Usage would be:   include(BoostConfigFeatures)   if(BOOST_FEATURE_cxx11_local_class_template_parameters)     add_executable(...)   endif()

Some bike-shedding required: - Name for the CMake file that contains this - Naming scheme for the results of the checks (the variable in the "if") - Eagerly test all or on demand:   boost_config_has(RESULT cxx11_local_class_template_parameters)   if(RESULT) add_executable(...) endif() Similar for predef:   include(BoostPredefCheck)   boost_predef_check(RESULT "BOOST_OS_WINDOWS != 0" "BOOST_OS_VMS != 0")   if(RESULT) add_executable(...) endif() Work is about the same and yes some checks can be omitted (e.g. boost_predef_check(RESULT "BOOST_OS_WINDOWS != 0") --> RESULT==WIN32 standard CMake variable)

Programmers are notorious for estimating the work to do any programming as about an order of magnitude less than it really takes <g>. True ;) So essence: Very low effort to port this to CMake due to existing bjam code What is the difference in CMake terms ? In Boost Build I am fairly sure the checks are done on demand, and usually involves some sort of compile/build/run/test-result cycle which would make pre-testing all checks in a library, and holding the result for further inquiry, pretty expensive. On demand: User calls a function and uses the result (see example) Eager: All checks executed on including the CMake file. (Note: Results are cached, hence this overhead is only once for all libraries for the mono-build), yes high overhead for a single lib, but this is what CMake also does to e.g. get compiler type, version, OS, ... and usage is easier (no additional calls, see examples)

On Tue, Oct 16, 2018 at 9:13 AM Edward Diener via Boost < boost@lists.boost.org> wrote:

On 10/16/2018 3:34 AM, Alexander Grund via Boost wrote:

...

Similar for predef: include(BoostPredefCheck) boost_predef_check(RESULT "BOOST_OS_WINDOWS != 0" "BOOST_OS_VMS != 0") if(RESULT) add_executable(...) endif()

Work is about the same and yes some checks can be omitted (e.g. boost_predef_check(RESULT "BOOST_OS_WINDOWS != 0") --> RESULT==WIN32 standard CMake variable)

Even if there is a standard CMake variable it would be worthwhile to have the check so that libraries which use the check can use the same syntax throughout.

Actually the CMake built-in is not equivalent to the Predef version AFAICT. For instance the Predef version can compare against version numbers, and not just zero and non-zero unlike the CMake WIN32 boolean var. The next closest CMake var would be CMAKE_SYSTEM_VERSION.. Although I do love how the format of the version number is not documented for that :-) Hence the closer equivalent would be "WIN32 && CMAKE_SYSTEM_VERSION != 0" (where 0 could be 8.1 maybe?). Also in B2 you wouldn't use Predef for checking the OS like above. You would just use the built in "<target-os>windows" feature. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net

On 10/10/2018 4:44 AM, Roger Leigh via Boost wrote:

On 09/10/2018 23:42, Edward Diener via Boost wrote:

...

On 10/9/2018 7:09 PM, mike via Boost wrote:

...

...

As Richard pointed out, such features exist in cmake too and have been in common use for years. E.g. It has a built-in mechanism that lets you directly query for most of the post c++03 compiler features: (https://cmake.org/cmake/help/v3.13/prop_gbl/CMAKE_CXX_KNOWN_FEATURES.html)

and as a more general feature, you can always check if a particular source file using some feature or type compiles or not: (https://cmake.org/cmake/help/v3.13/module/CheckCXXSourceCompiles.html)

Could you maybe point me to some examples where this feature is used so one could estimate how difficult a conversion is?

The feature is essentially programmed by config and predef to provide build-time testing of compiler support and testing of such things like the OS versions and compiler versions etc. See https://www.boost.org/doc/libs/1_68_0/libs/config/doc/html/boost_config/buil... for the explanation in config and https://www.boost.org/doc/libs/1_68_0/doc/html/predef/check_utilities.html for checking predef values at build time. The basics of the feature are offered by Boost Build as explained at https://boostorg.github.io/build/manual/develop/index.html#bbv2.reference.ru... in reference to the check-target-builds rule. Plenty of libraries use the feature during testing and I imagine that some of them may use the feature during building. Asking individual libraries to convert their tests or build to use some CMake equivalent, for every possible setting in config and predef which they use, is not realistic. I understand that CMake may have an equivalent technique, but who is going to program that technique to offer all the equivalents which config and predef currently offer ?

This is all supported by CMake for years as mike said, and it's utterly trivial to write such feature tests.  If you would like some examples, here's some I wrote for Xerces-C++:

https://github.com/apache/xerces-c/tree/trunk/cmake

For example, here's a simple test for const [it's a port of Autoconf logic] with CheckCXXSourceCompiles:   https://github.com/apache/xerces-c/blob/trunk/cmake/XercesConst.cmake this could be adapted for any feature you care to check, from constexpr to user-defined literals, or any other feature which can be checked by running the compiler.

This is a set of checks for integer type sizes: https://github.com/apache/xerces-c/blob/trunk/cmake/XercesIntTypes.cmake

This is a more complex set of checks for SSE intrinsics: https://github.com/apache/xerces-c/blob/trunk/cmake/XercesSSE2.cmake

Here's some checks to verify Boost is working properly: https://gitlab.com/codelibre/ome-common-cpp/blob/master/cmake/BoostChecks.cm...

Testing of linker version scripts: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L198

Checking include presence: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L212

Checking compiler keyword presence and fallbacks: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L226

Checking structure presence: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L226

Checking library symbol exports: https://gitlab.com/libtiff/libtiff/blob/master/CMakeLists.txt#L226

I don't think any of this is difficult.  If anything, you can likely reuse the existing source code used by the Boost.Build checks, maybe even sourced directly without any changes--just wrap it with check_cxx_source_compiles or other feature test macros.

My point is that you can not ask each library maintainer to re-invent what he already uses from Boost config and/or Boost predef. You need to reprogram this in CMake for each case in config and predef, and then provide a CMake-like interface to it for other libraries. I do not care how easy you find it in CMake, individual library maintainers are not going to spend time re-duplicating everything they need from config and predef just so they can build their library and/or run their library tests. We are talking about some 130+ potential Boost libraries we have to deal with, and any conversion to CMake that is non-trivial is going to be very unpopular.

On 10/15/2018 4:36 PM, Roger Leigh via Boost wrote:

On 10/10/18 15:23, Edward Diener via Boost wrote:

...

On 10/10/2018 4:44 AM, Roger Leigh via Boost wrote:

...

This is all supported by CMake for years as mike said, and it's utterly trivial to write such feature tests.  If you would like some examples, here's some I wrote for Xerces-C++:

[examples]

...

...

I don't think any of this is difficult.  If anything, you can likely reuse the existing source code used by the Boost.Build checks, maybe even sourced directly without any changes--just wrap it with check_cxx_source_compiles or other feature test macros.

My point is that you can not ask each library maintainer to re-invent what he already uses from Boost config and/or Boost predef. You need to reprogram this in CMake for each case in config and predef, and then provide a CMake-like interface to it for other libraries. I do not care how easy you find it in CMake, individual library maintainers are not going to spend time re-duplicating everything they need from config and predef just so they can build their library and/or run their library tests. We are talking about some 130+ potential Boost libraries we have to deal with, and any conversion to CMake that is non-trivial is going to be very unpopular.

The logic could be written to be either in a single place (which can provide CMake logic for other components to use), or duplicated in the different components.  Which is better depends upon how decoupled you wish the components to be.  If you always require a monolithic build, then one component could provide shared functionality.  Otherwise, duplication is better to avoid tight coupling.  As an example:

  https://gitlab.com/rleigh/ome-common-cpp/tree/master/cmake

are modules I routinely share between related components of a larger project.  Not the size of Boost though.

Nearly every library depends on Boost.Config. Boost.Predef was created much, much later than Boost.Config so I do not know how many libraries use its functionality. Creating support for Boost.Config check builds should be done once in Boost.Config and creating support Boost.Predef check builds should be done once in Boost.Predef. Expecting individual libraries to duplicate what is current in those libraries, for CMake, is not something the maintainers of those other libraries will want to do.

I looked at the Boost config and predef links Rene provided in another email in this thread.

The Boost.Config checks all look pretty straightforward.  Should be possible to iterate over them and store each one in a variable, then we could provide them in the exported configuration for dependent components to consume.

(While I see this component does have CMake support merged in, I don't see any configuration export.  Where is that handled?  Has anyone worked on it yet?)

For Boost predef this isn't much different; run the program and store the results, export the variables.

In both cases, CMake users can then simply run conditional logic such as:

  if (boost_feature)     list(APPEND sources foo.cpp)   endif()

or

  if (boost_feature2 AND boost_feature3 AND NOT boost_feature 4)      add_executable(foo …)      …   endif()

and so on.

Note that a number of the predef checks are already supported directly by CMake, such as   if(MSVC)   if(WIN32) etc.  While the predef checks can be carried over for completeness, they are not necessary.

I would be happy to work on some of this.  I'm not sure exactly how to do so however.  Where would I submit it?  Is CMake development discussion happening on any other lists or other places?  I don't want to tread on any toes or work at cross-purposes.

It would be best to mention that you would like to help convert the check builds functionality in those two libraries to CMake on the appropriate library's Github page. You could bring this up as Issues for each of the two libraries and eventually create PRs.

Regards, Roger

On 10/10/2018 12:09, mike wrote:

...

It is for such a reason that as soon as you start talking about using CMake for anything else but identifying Boost libraries for the purposes of end-user use of Boost libraries in their own CMake scripts,

Just consuming precompiled/pre-installed boost libraries in a cmake script is already possible thanks to the work of cmake contributors and the people providing boost packages for vcpkg, conan and other package management systems (some of those contributors are probably also boost library maintainers?). Those solutions have to keep playing catch up with the latest boost release if something significant changes (like name-mangling or a new library), but by and large they work quite well. Imho an official boost solution should provide a bit more functionality to justify the effort.

I'm mostly watching this from the sidelines (since other than building Boost itself I use neither b2 nor cmake), so take this with a grain of salt, but: I think that "onboarding" the consumption of Boost libraries into Boost could be valuable in itself with no need to justify additional functionality, simply because it eliminates that catch-up process and helps ensure things are correct before release instead of after. And Boost is probably in a better position to track library dependencies. Having said that, I'm not sure how feasible it is in practice simply due to historically package deployment being left entirely to outside parties (especially on Linux), thus there being significant variation in the wild.

On 10/10/18 1:24 AM, Roger Leigh via Boost wrote:

I think it's primarily useful for packaging standalone applications where there are no complex library dependencies.  Certainly for Windows and MacOS X it makes sense.  For packaging a set of libraries like Boost, I don't think it's as useful;

we already have good Boost packaging anyway, so I'd suggest ignoring it at least for now.

Hmmm - what is "Boost packaging" refering to? Robert Ramey

Regards, Roger

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 10.10.18 22:25, Roger Leigh via Boost wrote:

On 10/10/2018 16:02, Robert Ramey via Boost wrote:

...

On 10/10/18 1:24 AM, Roger Leigh via Boost wrote:

...

I think it's primarily useful for packaging standalone applications where there are no complex library dependencies.  Certainly for Windows and MacOS X it makes sense.  For packaging a set of libraries like Boost, I don't think it's as useful;

...

we already have good Boost packaging anyway, so I'd suggest ignoring it at least for now.

Hmmm - what is "Boost packaging" refering to?

I was referring to the existing packaging for the various Linux distributions, BSD ports, MacOS MacPorts/homebrew, vcpkg etc.  I simply meant that the problem is already taken care of by others for the most part, and that CPack doesn't bring much (if any) added value on top of that.

I believe it would be interesting to distribute boost via Launchpad/PPA for Ubuntu. CPack can generate debian packages for that. An example that (used to) work with multiple packages: https://bitbucket.org/renficiaud/yayi-debian-ppa/src/master/debian/rules Raffi

Here's an idea, we leave our distribution as-is, and let the community step forward and decide if and how they want to package individual

Vinnie Falco wrote: libraries. This is not possible at the moment because all headers are copied into boost/ and removed from libs/$lib/include. The 'community' can't package an individual library from the contents of a release.

On 10/11/18 11:38 AM, Vinnie Falco via Boost wrote:

On Mon, Oct 1, 2018 at 10:07 AM Robert Ramey via Boost wrote:

...

We distribute boost as "super project". Should we continue to do this or distribute libraries on a library by library basis. Does CMake/CPACK etc. make this "simple"?

Here's an idea, we leave our distribution as-is, and let the community step forward and decide if and how they want to package individual libraries. We can make it easier for them by making adjustments as needed. This way the stakeholders have a direct say in how it goes.

Right. This is really a question about deployment of modular Boost. The purpose of this thread is to solicit suggestions as to what a CMake system for Boost should include. I'm aware that CMake includes CPACK which I believe touches upon this subject. I hope that someone who knows more about this than I can supply more information as to whether we should include support for CPACK in the coming list of requirements contained in the forthcoming call for submissions. Robert Ramey

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 10/11/18 9:45 PM, Rene Rivera via Boost wrote:

Second, I think the scope and goals of supporting CMake, even though well intentioned, fall short of what Boost needs to do to resolve the varied afflictions that plague us.

Well, the scope, goals and requirements of CMake as it relates to Boost aren't decided yet. That is the current discussion. I don't believe that CMake will address all the afflictions that plague us. I believe it will make boost in someway better. That is my aspiration.

In what follows when I refer to: "users" I mean those programmers that obtain Boost and use the libraries in their projects, and "authors" as the programmers that develop and support the Boost Libraries and tools.

With that in mind, what I would like us to do is what I've mentioned many times in the past.. Move towards a truly modular Boost distribution and development that: a) supports users no matter what environment they choose to use Boost Libraries in, b) supports developers to flexibly develop and test their libraries, c) allows Boost to grow in a way that doesn't cripple development resources.

This is my goal as well. I believe that a CMake support of boost will take use closer to that goal.

I think that without a "fully normalized modular Boost" a CMake, or any other build system, effort is a waste of time.

Of course we differ. I don't believe it a grand (intelligent?) design. I believe in evolution. It is my intention that CMake support of boost will bring us close to the goal of a modular boost - which I've referred to in the past as Boost 2.0.

As it doesn't address the long term viability of Boost.

Not explicitly. But I believe it will contribute to the long term visibility of Boost.

And hence if we are going to spend considerable effort it should be towards modularity.

By keeping the scope relatively narrow, I hope to be able to make actual progress. This is a key feature of my proposal and strategy. Other more ambitious and grandiose initiatives have failed in the past because they were too ambitious and grandiose. This makes it impossible to reach consensus which is the bedrock of Boost. We've tried the marxist approach to move to Boost 2.0 and it's failed us. Now we're going to try capitalism - which accepts and embraces failure is an inevitable cost of moving forward.

In addition to the standard requirements that Robert mentioned here's what I think is required for a modular Boost:

1) Fully normalized library file structure that can be introspected.

This would be nice - but I don't think it's essential to making a CMake Boost integration.

2) Clear inter-library, and non-cyclic, dependency arrangement and information.

LOL - the hole grail! It's never going to happen. The advocates of this view have totally misunderstood the meaning and role of dependency in this context. Like the knights of old, they will continue to pursue this holy grail - always getting closer - but never capturing the prize. I've recently described my views on this in detail (for the Nth time!) on slack in a thread with Peter Dimov. And for the Nth time I've failed to convince anyone! LOL If he's sir galahad, maybe I'm don quijote - both pure of heart but totally misguided in different ways. (I'm just being entertaining here - I know I'm right and have proven it many times.)

3) Abandon the super-project structure as a development and distribution vehicle.

I agree with this. It's my current intention that the "scope and requirements" include the statement like: "Any CMake support of Boost should not presume any super project or similar directory structure outside of any particular library"

To telegraph my intent for when it's time for proposals.. I think that for #1 above I think we should be forward looking and follow the Pitchfork structure proposal http://bit.ly/2Edy4pC.

Interesting - but as I said before - out of scope.

As for build system.. I think I only have one additional requirement given the above:

1) We should support generating multiple build system files for distribution and development from the introspection of the modular Boost libraries to support the varied user and developer needs.

LOL - that could mean anything - maybe you should run for office! I think I get the idea but would phrase it differently: "A CMake integration should not conflict with the current build system or other boost tools" Robert Ramey

Hi Rene, Robert, all, On 10/12/18 3:04 PM, Rene Rivera via Boost wrote:

...

...

With that in mind, what I would like us to do is what I've mentioned many times in the past.. Move towards a truly modular Boost distribution and development that: a) supports users no matter what environment they choose to use Boost Libraries in, b) supports developers to flexibly develop and test their libraries, c) allows Boost to grow in a way that doesn't cripple development resources. This is my goal as well. I believe that a CMake support of boost will take use closer to that goal.

Again, I disagree, no build system will take us closer to that goal because it's not the build system at fault.

+10

...

...

I think that without a "fully normalized modular Boost" a CMake, or any other build system, effort is a waste of time. Of course we differ. I don't believe it a grand (intelligent?) design. I believe in evolution. It is my intention that CMake support of boost will bring us close to the goal of a modular boost - which I've referred to in the past as Boost 2.0.

I'm a big fan of incremental progress (which is my attempt at paraphrasing your use of the term "evolution"), and it is precisely for that reason that I think "modularity" must include organizational / administrative aspects (e.g. the ability to take certain decisions per project rather than monolithically for the whole organization) rather than only technical ones (e.g. the ability to build individual projects). Not only would a solution that involves some projects using CMake and some using Boost.Build be a much more convincing demonstration of modularity, it would also address a range of concerns voiced by others (including Edward Diener in https://lists.boost.org/Archives/boost/2018/10/243620.php and myself in https://lists.boost.org/Archives/boost/2018/09/243293.php et al.)

...

This is a key feature of my proposal and strategy. Other more ambitious and grandiose initiatives have failed in the past because they were too ambitious and grandiose.

This seems to be a question of perspective. To me a proposal to "switch from Boost.Build to CMake for the entirety of Boost" is a way more ambitious goal than to "change the Boost infrastructure to support individual projects to switch to CMake".

...

This makes it impossible to reach consensus which is the bedrock of Boost.

Consensus would be great, but we (the community) have demonstrated that it has its limits, i.e. it doesn't scale well. And of course, to state the obvious: This isn't a democracy. It's the developers and maintainers who put in the effort to develop and maintain individual projects who should have the ultimate power to decide.

...

...

1) Fully normalized library file structure that can be introspected. This would be nice - but I don't think it's essential to making a CMake Boost integration.

True it's not essential. But it would make any build system integration a magnitude easier (and I'm including b2 reintegration here).

"Provide the following integration points: ..." would be a much simpler set of requirements. But I agree of course with the general idea: establish fewer points to be standardized, to improve the ability to encapsulate and insulate "implementation details".

...

...

2) Clear inter-library, and non-cyclic, dependency arrangement and information.

+10

...

...

3) Abandon the super-project structure as a development and distribution vehicle.

+10

...

I agree with this. It's my current intention that the "scope and requirements" include the statement like:

"Any CMake support of Boost should not presume any super project or similar directory structure outside of any particular library"

Sounds good.. Just replace "CMake" with "build system".

Yes !! Stefan -- ...ich hab' noch einen Koffer in Berlin...