[cmake] Minimum viable cmakeification for Boost

Niall Douglas

19 Jun 2017 19 Jun '17

11:42 p.m.

I've finished the mockup which can be studied at: https://github.com/ned14/boost-bmv-cmake Just Boost.System was cmakeified. Extensive comments are throughout to explain every single line of cmake and why it's there. A small sample program is in the root of the repo to show how this cmake would be used by end user programs. Readme.md follows for those who don't like clicking on links: --- cut --- This is a mock up of a **bare minimum viable** cmakeification of the Boost C++ libraries for the community to study and evaluate. It only cmakeifies Boost.System alone, its upstream dependencies are stubbed/faked. A demo program which shows how one would link against Boost.System is in this directory (`example_client_program`) and it demos linking against boost::system as a header only library, as a static library and as a shared library. # Specific design goals of this mock up: - Modern, **highly reusable by unknown third party cmake** cmake3 only throughout. - Minimum possible cmake complexity, no cmake innovating, even at the cost of extra boilerplate. - No custom macros, functions, nor custom build logic. Only vanilla cmake3. Keep the learning curve for the build system as gentle as is possible. - **Strict** separation of non-fixed configuration and build. Child CMakeLists must only EVER define build and *fixed* configuration like "I need C++ 11 minimum". *Variable* configuration like naming, directory layout for outputs, optimisation flags etc is ALWAYS defined in the **rootmost** CMakeLists. NEVER in child CMakeLists. - All custom build logic should always be placed in rootlevel cmake scripts (command line programs written in cmake, runnable using `cmake -V`, these can take args etc) or rootlevel CMakeLists, or combinations thereof. # Stuff in there we might yet remove: - This cmake mockup explicitly documents dependencies between Boost libraries in cmake, and so when I link against say Boost.System, cmake will also link in the things Boost.System depends against. Moreover this is minimal, so ONLY the stuff minimally necessary to fulfil the need for Boost.System is built. - We allow those Boost libraries support it to present header only, static and dynamic linkage editions of themselves. This does complicate the CMakeLists however. # What we don't do: - Test the boundaries of cmake or push any limits of what's possible. - Bespoke libraries of common functions dragged in wherever. - Complex custom build logic which needs to be documented and learned off by library developers. - Do anything which gets in the way of CMakeLists reuse by any arbitrary third party cmake written by any Boost user. --- cut --- The code should be fairly self explanatory. Yes there are no tests support. We are limiting the scope of discussion to build only for now. Technical questions are welcome. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Show replies by date

David Sankel

20 Jun 20 Jun

1:07 a.m.

On Mon, Jun 19, 2017 at 7:42 PM, Niall Douglas via Boost < boost@lists.boost.org> wrote:

...

I've finished the mockup which can be studied at:

https://github.com/ned14/boost-bmv-cmake

Niall, thanks so much for contributing this. It is the most viable Boost CMake convention I've seen yet and I'm 100% behind it. It is super informative as well for learning the CMake 3.5 style. Things I very much like about this system: - The separation of concerns between library.cmake and systems's CMakeLists.txt. The former only consists of the library's content where the latter declares all dependencies. I suspect this would aid in reuse where someone would prefer to declare different dependencies (or dependencies in a different way). - The minimalism. I love how it is both immediately useful for people and omits bells and whistles. I think it is an excellent way to start for CMakeification of Boost. I have a few questions and comments below. In library.cmake, you use include commands like follows, include("build/headers.cmake") . How does 'include' know how to resolve this relative path? Will it always resolve to CMAKE_CURRENT_SOURCE_DIR? I'm surprised you didn't need to do this, include("${CMAKE_CURRENT_LIST_DIR}/build/headers.cmake") . In system's 'CMakeLists.txt', you have this, set(BOOST_LIBRARY_PROJECTS ${BOOST_LIBRARY_PROJECTS} ${PROJECT_NAME} PARENT_SCOPE) . I was thinking it may be better to use list(APPEND BOOST_LIBRARY_PROJECTS ${PROJECT_NAME}) , but then I realized that 'list' doesn't have a parent scope option. You'd have to add, set(BOOST_LIBRARY_PROJECTS ${BOOST_LIBRARY_PROJECTS} PARENT_SCOPE) , which doesn't seem like much of an improvement. In your top level 'CMakeLists.txt' file, I saw, add_subdirectory("boostorg" EXCLUDE_FROM_ALL) . I was wondering how you would deal with unused Boost targets that have dependencies that don't exist on the system (e.g. Boost.Python's Python dependency). I suppose the 'boostorg/CMakeLists.txt' could conditionally add targets by checking dependencies first, but then it seems like one would get the reverse problem (Boost.Python is really needed, but the Python dependency isn't there). In this case one would get a cmake-time error stating that boost::python doesn't exist. Unfortunately, there wouldn't be any explaination as to why. This isn't a big deal though by any stretch. The 'system/CMakeLists.txt' shows situational awareness by including dependencies with '..' and using 'BoostVersion.cmake'. This precludes 'system' from being buildable outside of the Boost tree. However, I can see how solving this problem is still up for debate and would be straightforward to add on top of what you have. -- David Sankel

P F

6:27 a.m.

...

On Jun 19, 2017, at 6:42 PM, Niall Douglas via Boost wrote:

I've finished the mockup which can be studied at:

https://github.com/ned14/boost-bmv-cmake

Just Boost.System was cmakeified. Extensive comments are throughout to explain every single line of cmake and why it's there. A small sample program is in the root of the repo to show how this cmake would be used by end user programs.

Here some feedback, I have: - Having things like `::hl` or `::sl` is quite strange. In cmake, I set `BUILD_SHARED_LIBS=On` when I want a shared library or I set it to `BUILD_SHARED_LIBS=Off` when I want a static library. There should only be one target `boost::config` or `boost::system` that I use. - Each project do things like `include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/BoostVersion.cmake”)` which is broken when built on its own. - Doing `if(NOT TARGET boost::assert::hl)` is wrong. A target does not have to exists to be able to link against it. Plus, it should call `find_package(boost_assert)` to bring in the target for when its not built in the superproject. - There is no installation of the targets - Doing `target_include_directories(boost_core-hl INTERFACE "include”)` is wrong as this will only add the include directories for the build. The include directories should be added for the build and for the installation.

Niall Douglas

12:17 p.m.

On 20/06/2017 07:27, P F wrote:

...

...
On Jun 19, 2017, at 6:42 PM, Niall Douglas via Boost wrote:

I've finished the mockup which can be studied at:

https://github.com/ned14/boost-bmv-cmake

Just Boost.System was cmakeified. Extensive comments are throughout to explain every single line of cmake and why it's there. A small sample program is in the root of the repo to show how this cmake would be used by end user programs.

Here some feedback, I have:

- Having things like `::hl` or `::sl` is quite strange. In cmake, I set `BUILD_SHARED_LIBS=On` when I want a shared library or I set it to `BUILD_SHARED_LIBS=Off` when I want a static library. There should only be one target `boost::config` or `boost::system` that I use.

BUILD_SHARED_LIBS is a cmake2-ism and should not be present in modern cmake. Moreover, cmake now supports header only libraries, and BUILD_SHARED_LIBS causes problems with eventual C++ Modules support. The ::hl suffix means "link against the header only library edition", the ::sl suffix means "link against the static library edition" and the ::dl suffix means "link against the dynamic library edition". Some bikeshedding over those names occurred off list. They don't hugely matter, they could be boost::system::header, boost::system::static and boost::system::shared if you wanted. Some bikeshedding also occurred over what boost::system with no suffix should mean. I recommend "whatever library edition the maintainer recommends as the best one" which is NOT necessarily the header only edition. For example, in the future with C++ Modules support the additional suffixs ::hlm, ::slm and ::dlm will be needed. Or longer named, boost::system::header_module, boost::system::static_module and boost::system::shared_module. So if C++ Modules were available, the maintainer might recommend the ::dlm variety, but recommend the ::hl variety if they weren't available. I think that will hugely vary per library as C++ Modules won't necessarily be always a win. Also, end users may wish to assemble all of Boost into a single C++ Module. We don't want to get in the way.

...

- Each project do things like `include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/BoostVersion.cmake”)` which is broken when built on its own.

That was intentional. Some bikeshedding occurred over that too with respect to which Boost library guaranteed to be a dependency for all other libraries ought to host the version setting script. Boost.Config is most likely. So the above is a placeholder.

...

- Doing `if(NOT TARGET boost::assert::hl)` is wrong. A target does not have to exists to be able to link against it. Plus, it should call `find_package(boost_assert)` to bring in the target for when its not built in the superproject.

That part is there solely to enable modular build. If you don't want to bother with modular build as this is a bare minimum viable cmakeification, you can eliminate it.

...

- There is no installation of the targets

Not necessary right now, so out of scope.

...

- Doing `target_include_directories(boost_core-hl INTERFACE "include”)` is wrong as this will only add the include directories for the build. The include directories should be added for the build and for the installation.

As I've repeatedly explained now, installation logic is best implemented by a rootmost CMakeLists. Not in the per-library CMakeLists. We do tell cmake what targets are used by installation via install(TARGETS ...). That's the bare minimum needed for other cmake to implement the remaining installation logic. (On wider picture stuff, install paths need to be absolute I've found, whereas build paths can usually be relative. Relative paths are much nicer during build. In a rootmost CMakeLists which implements some installation logic, you'd iterate the targets for the Boost libraries and configure each programmatically for the appropriate install for that library and that type of library. All the metadata you need to make decisions is available) Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

P F

1:32 p.m.

...

On Jun 20, 2017, at 7:17 AM, Niall Douglas via Boost wrote:

On 20/06/2017 07:27, P F wrote:

...
...
On Jun 19, 2017, at 6:42 PM, Niall Douglas via Boost wrote:

I've finished the mockup which can be studied at:

https://github.com/ned14/boost-bmv-cmake

Just Boost.System was cmakeified. Extensive comments are throughout to explain every single line of cmake and why it's there. A small sample program is in the root of the repo to show how this cmake would be used by end user programs.

Here some feedback, I have:

- Having things like `::hl` or `::sl` is quite strange. In cmake, I set `BUILD_SHARED_LIBS=On` when I want a shared library or I set it to `BUILD_SHARED_LIBS=Off` when I want a static library. There should only be one target `boost::config` or `boost::system` that I use.

BUILD_SHARED_LIBS is a cmake2-ism and should not be present in modern cmake.

No, cmake best practice including cmake 3: * Leave the control of `BUILD_SHARED_LIBS` to your clients[1]

...

Moreover, cmake now supports header only libraries, and BUILD_SHARED_LIBS causes problems with eventual C++ Modules support. The ::hl suffix means "link against the header only library edition", the ::sl suffix means "link against the static library edition" and the ::dl suffix means "link against the dynamic library edition".

Some bikeshedding over those names occurred off list. They don't hugely matter, they could be boost::system::header, boost::system::static and boost::system::shared if you wanted.

Some bikeshedding also occurred over what boost::system with no suffix should mean. I recommend "whatever library edition the maintainer recommends as the best one" which is NOT necessarily the header only edition. For example, in the future with C++ Modules support the additional suffixs ::hlm, ::slm and ::dlm will be needed. Or longer named, boost::system::header_module, boost::system::static_module and boost::system::shared_module.

So if C++ Modules were available, the maintainer might recommend the ::dlm variety, but recommend the ::hl variety if they weren't available. I think that will hugely vary per library as C++ Modules won't necessarily be always a win. Also, end users may wish to assemble all of Boost into a single C++ Module. We don't want to get in the way.

...
- Each project do things like `include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/BoostVersion.cmake”)` which is broken when built on its own.

That was intentional. Some bikeshedding occurred over that too with respect to which Boost library guaranteed to be a dependency for all other libraries ought to host the version setting script. Boost.Config is most likely. So the above is a placeholder.

Yes, but this breaks cmake best practice that says: * Make sure all your projects can be built as standalone and as a subproject of another project.

...

...
- Doing `if(NOT TARGET boost::assert::hl)` is wrong. A target does not have to exists to be able to link against it. Plus, it should call `find_package(boost_assert)` to bring in the target for when its not built in the superproject.

That part is there solely to enable modular build. If you don't want to bother with modular build as this is a bare minimum viable cmakeification, you can eliminate it.

What do you mean by modular build? There is only two ways it can be as standalone or as a modular build.

...

...
- There is no installation of the targets

Not necessary right now, so out of scope.

It is necessary.

...

...
- Doing `target_include_directories(boost_core-hl INTERFACE "include”)` is wrong as this will only add the include directories for the build. The include directories should be added for the build and for the installation.

As I've repeatedly explained now, installation logic is best implemented by a rootmost CMakeLists. Not in the per-library CMakeLists. We do tell cmake what targets are used by installation via install(TARGETS ...). That's the bare minimum needed for other cmake to implement the remaining installation logic.

Even if were implemented in the topmost, it would be broken when using `find_package` as the target is using the include directories in the build directory and not the install directory.

...

(On wider picture stuff, install paths need to be absolute I've found, whereas build paths can usually be relative. Relative paths are much nicer during build. In a rootmost CMakeLists which implements some installation logic, you'd iterate the targets for the Boost libraries and configure each programmatically for the appropriate install for that library and that type of library. All the metadata you need to make decisions is available)

I dont what you are saying here. Install paths should be relative or they should use the `$` generator expression in order to make the installation relocatable. [1]: https://de.slideshare.net/DanielPfeifer1/cmake-48475415

P F

1:51 p.m.

...

On Jun 20, 2017, at 8:32 AM, P F wrote:

...
On Jun 20, 2017, at 7:17 AM, Niall Douglas via Boost mailto:boost@lists.boost.org> wrote: That part is there solely to enable modular build. If you don't want to bother with modular build as this is a bare minimum viable cmakeification, you can eliminate it.

What do you mean by modular build? There is only two ways it can be as standalone or as a modular build.

Correction, I meant to say: there is only two ways it can be built as standalone or as a subproject of another project.

Niall Douglas

3:35 p.m.

...

...
...
- Having things like `::hl` or `::sl` is quite strange. In cmake, I set `BUILD_SHARED_LIBS=On` when I want a shared library or I set it to `BUILD_SHARED_LIBS=Off` when I want a static library. There should only be one target `boost::config` or `boost::system` that I use.

BUILD_SHARED_LIBS is a cmake2-ism and should not be present in modern cmake.

No, cmake best practice including cmake 3:

* Leave the control of `BUILD_SHARED_LIBS` to your clients[1]

That's exactly what I'm doing. A rootlevel CMakeLists can choose to observe global variables and apply them by choosing only the targets which are static or shared for build. Non-rootlevel CMakeLists should have no business ever touching, using, or relying on any global variable or state. Declaration only.

...

...
...
- Each project do things like `include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/BoostVersion.cmake”)` which is broken when built on its own.

That was intentional. Some bikeshedding occurred over that too with respect to which Boost library guaranteed to be a dependency for all other libraries ought to host the version setting script. Boost.Config is most likely. So the above is a placeholder.

Yes, but this breaks cmake best practice that says:

* Make sure all your projects can be built as standalone and as a subproject of another project.

Let me repeat myself because you didn't read my words: "So the above is a placeholder"

...

...
...
- Doing `if(NOT TARGET boost::assert::hl)` is wrong. A target does not have to exists to be able to link against it. Plus, it should call `find_package(boost_assert)` to bring in the target for when its not built in the superproject.

That part is there solely to enable modular build. If you don't want to bother with modular build as this is a bare minimum viable cmakeification, you can eliminate it.

What do you mean by modular build? There is only two ways it can be as standalone or as a modular build.

Ok, "exactly what is needed" build so. If I ask to link against Boost.System, only Boost.System and its dependencies get built by cmake. No unnecessary stuff. You don't actually need that for minimum viable cmake of course. You could have b2.exe call cmake to build the whole of Boost and place it in staging exactly as at present. But it seemed to me if one is to bother with cmakeification, you might as well formally specify the dependencies of each library and make possible "exactly what is needed build on demand". It's not much extra work, and it contributes much added value.

...

...
...
- There is no installation of the targets

Not necessary right now, so out of scope.

It is necessary.

No, it really isn't. If people want installation logic, it's trivially easy for them to write a rootlevel CMakeLists which implements that from what we've given them. No need for Boost to dictate what installation means or is by hard coding it into non-rootlevel CMakeLists.

...

...
...
- Doing `target_include_directories(boost_core-hl INTERFACE "include”)` is wrong as this will only add the include directories for the build. The include directories should be added for the build and for the installation.

As I've repeatedly explained now, installation logic is best implemented by a rootmost CMakeLists. Not in the per-library CMakeLists. We do tell cmake what targets are used by installation via install(TARGETS ...). That's the bare minimum needed for other cmake to implement the remaining installation logic.

Even if were implemented in the topmost, it would be broken when using `find_package` as the target is using the include directories in the build directory and not the install directory.

That's trivially easy to change at the rootlevel. Just iterate the targets and change the properties to whatever you feel they need to be. I also deliberately left open a customisation point on this where headers.cmake is auto-generated by rootlevel cmake script. It could inject $ and $ there if it chose. I think it unwise to hard code that stuff. It's non-fixed configuration. End users reusing our cmake may have custom install paths and needs. We should not be dictating to them what those should be. Let rootlevel CMakeLists implement that stuff.

...

...
(On wider picture stuff, install paths need to be absolute I've found, whereas build paths can usually be relative. Relative paths are much nicer during build. In a rootmost CMakeLists which implements some installation logic, you'd iterate the targets for the Boost libraries and configure each programmatically for the appropriate install for that library and that type of library. All the metadata you need to make decisions is available)

I dont what you are saying here. Install paths should be relative or they should use the `$` generator expression in order to make the installation relocatable.

For the output yes. I was referring to the input. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

paul

4:37 p.m.

On Tue, 2017-06-20 at 16:35 +0100, Niall Douglas via Boost wrote:

...

...
...
...
- Having things like `::hl` or `::sl` is quite strange. In cmake, I set `BUILD_SHARED_LIBS=On` when I want a shared library or I set it to `BUILD_SHARED_LIBS=Off` when I want a static library. There should only be one target `boost::config` or `boost::system` that I use.

BUILD_SHARED_LIBS is a cmake2-ism and should not be present in modern cmake.

No, cmake best practice including cmake 3:

* Leave the control of `BUILD_SHARED_LIBS` to your clients[1]

That's exactly what I'm doing. A rootlevel CMakeLists

The clients aren't necessarily the rootlevel cmake. They are mainly the ones who are invoking cmake.

...

can choose to observe global variables and apply them by choosing only the targets which are static or shared for build. Non-rootlevel CMakeLists should have no business ever touching, using, or relying on any global variable or state. Declaration only.

...
...
...
- Each project do things like `include("${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/BoostVersion.cmake”) ` which is broken when built on its own.

That was intentional. Some bikeshedding occurred over that too with respect to which Boost library guaranteed to be a dependency for all other libraries ought to host the version setting script. Boost.Config is most likely. So the above is a placeholder.

Yes, but this breaks cmake best practice that says:

* Make sure all your projects can be built as standalone and as a subproject of another project.

Let me repeat myself because you didn't read my words:

"So the above is a placeholder"

...
...
...
- Doing `if(NOT TARGET boost::assert::hl)` is wrong. A target does not have to exists to be able to link against it. Plus, it should call `find_package(boost_assert)` to bring in the target for when its not built in the superproject.

That part is there solely to enable modular build. If you don't want to bother with modular build as this is a bare minimum viable cmakeification, you can eliminate it.

What do you mean by modular build? There is only two ways it can be as standalone or as a modular build.

Ok, "exactly what is needed" build so. If I ask to link against Boost.System, only Boost.System and its dependencies get built by cmake. No unnecessary stuff.

That already happens when you use `EXCLUDE_FROM_ALL`, so cmake only builds the targets it needs.

...

You don't actually need that for minimum viable cmake of course. You could have b2.exe call cmake to build the whole of Boost and place it in staging exactly as at present.

But it seemed to me if one is to bother with cmakeification, you might as well formally specify the dependencies of each library and make possible "exactly what is needed build on demand". It's not much extra work, and it contributes much added value.

...
...
...
- There is no installation of the targets

Not necessary right now, so out of scope.

It is necessary.

No, it really isn't. If people want installation logic, it's trivially easy for them to write a rootlevel CMakeLists which implements that from what we've given them. No need for Boost to dictate what installation means or is by hard coding it into non-rootlevel CMakeLists.

Yes, but when we are building standalone the so called "non-rootlevel" is the rootlevel cmake.

...

...
...
...
- Doing `target_include_directories(boost_core-hl INTERFACE "include”)` is wrong as this will only add the include directories for the build. The include directories should be added for the build and for the installation.

As I've repeatedly explained now, installation logic is best implemented by a rootmost CMakeLists. Not in the per-library CMakeLists. We do tell cmake what targets are used by installation via install(TARGETS ...). That's the bare minimum needed for other cmake to implement the remaining installation logic.

Even if were implemented in the topmost, it would be broken when using `find_package` as the target is using the include directories in the build directory and not the install directory.

That's trivially easy to change at the rootlevel. Just iterate the targets and change the properties to whatever you feel they need to be.

I also deliberately left open a customisation point on this where headers.cmake is auto-generated by rootlevel cmake script. It could inject $ and $ there if it chose.

I think it unwise to hard code that stuff. It's non-fixed configuration. End users reusing our cmake may have custom install paths and needs.

This is to support custom install paths, and it will support relocatbility as well.

...

We should not be dictating to them what those should be. Let rootlevel CMakeLists implement that stuff.

For standalone, it is the rootlevel.

Peter Dimov

21 Jun 21 Jun

11:10 a.m.

Niall Douglas wrote:

...

The ::hl suffix means "link against the header only library edition", the ::sl suffix means "link against the static library edition" and the ::dl suffix means "link against the dynamic library edition".

The one remaining question I have here is how does a library link to its dependency. The obvious approach seems to be for filesystem::sl to link to system::sl and for filesystem::dl to link to system::dl, right? (And a hypothetical filesystem::hl would probably link to system::sl.) (Exceptions where the dependency is always ::dl or always ::sl notwithstanding. The question is about the case where it also supports both, or all three.)

Niall Douglas

12:13 p.m.

...

...
The ::hl suffix means "link against the header only library edition", the ::sl suffix means "link against the static library edition" and the ::dl suffix means "link against the dynamic library edition".

The one remaining question I have here is how does a library link to its dependency. The obvious approach seems to be for filesystem::sl to link to system::sl and for filesystem::dl to link to system::dl, right? (And a hypothetical filesystem::hl would probably link to system::sl.)

(Exceptions where the dependency is always ::dl or always ::sl notwithstanding. The question is about the case where it also supports both, or all three.)

That's the default convention I've always followed in my own code. But I agree it feels a little wrong, I can see where a dynamic library might want to use the static library edition of a dependency sometimes if they are setting -fvisibility=hidden. Because cmake targets propagate their dependencies to consumers, if the library developer says that the ::dl edition uses ::dl dependencies, and the end user wants it to use the ::sl dependency, unless the library developer adds a special target for them then you must resort to monkey patching the properties to change the hard coded dependency the library developer chose for you. As much as this might seem important, it isn't as much as it might seem in practice. Static libraries generally are not compiled to be linked into DLLs, so they don't mark their symbols as public etc, indeed in cmake they aren't even compiled with -fPIC by default. So the actual occasions where you do need to link static library editions into shared libraries happens less often in practice than you might think. Still, it leaves me feeling a little uncomfortable. Doing better in cmake introduces lots of extra custom logic though, you really need custom functions and macros to ease the pain :( Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

1:46 p.m.

Niall Douglas wrote:

...

...
The one remaining question I have here is how does a library link to its dependency. The obvious approach seems to be for filesystem::sl to link to system::sl and for filesystem::dl to link to system::dl, right? (And a hypothetical filesystem::hl would probably link to system::sl.) ...

...

That's the default convention I've always followed in my own code.

It's more complex than that. Even header-only libraries would need all three subtargets, for several reasons. One, we want the dependency enumeration to be scripted and therefore not depend on library specifics. Two, we can have chains pumpkin -> asio -> system, where I want pumpkin::static to propagate down to system::static, for which I'd need it to link to asio::static. There's also the question of the "maintainer-preferred" default subtarget. If plain asio maps to asio::header, as is natural because it _is_ ::header, this will propagate down to system and switch it into header-only mode, and I probably don't want that. So the default, even for header-only libraries, would need to be, f.ex. ::static. In addition to all that, we want some sort of error to be issued when the project links to both system::static and system::shared -- not sure how this is done in CMake. All in all, I think that if we keep this scheme, we need to drop ::header and stick to ::static and ::shared. This will eliminate the surprise of header-only libraries linking to ::header targets downstream. We may also think about the alternative of giving control of whether system means system::static or system::shared to root level, and only express dependencies using the unsuffixed form.

Niall Douglas

3:44 p.m.

On 21/06/2017 14:46, Peter Dimov via Boost wrote:

...

Niall Douglas wrote:

...
...
The one remaining question I have here is how does a library link to its > dependency. The obvious approach seems to be for filesystem::sl to link > to system::sl and for filesystem::dl to link to system::dl, right? (And > a hypothetical filesystem::hl would probably link to system::sl.) ...

...
That's the default convention I've always followed in my own code.

It's more complex than that. Even header-only libraries would need all three subtargets, for several reasons. One, we want the dependency enumeration to be scripted and therefore not depend on library specifics. Two, we can have chains pumpkin -> asio -> system, where I want pumpkin::static to propagate down to system::static, for which I'd need it to link to asio::static.

There's also the question of the "maintainer-preferred" default subtarget. If plain asio maps to asio::header, as is natural because it _is_ ::header, this will propagate down to system and switch it into header-only mode, and I probably don't want that. So the default, even for header-only libraries, would need to be, f.ex. ::static.

In addition to all that, we want some sort of error to be issued when the project links to both system::static and system::shared -- not sure how this is done in CMake.

All in all, I think that if we keep this scheme, we need to drop ::header and stick to ::static and ::shared. This will eliminate the surprise of header-only libraries linking to ::header targets downstream.

Before cmake 3.5 you couldn't add dependencies to header only targets on the basis of your reasoning. So end users had to know what a header only targets also needed to be linked against. However, it was fixed. It turns out that end users really like to be able to target_link_libraries() to some library and not have to care about what dependencies that library itself has or indeed anything else for the same reason as they like auto-linking. They don't want to have to care about that stuff. In the real world of course, most use of header only libraries is wrapped into a precompiled header. The number of individual users not using precompiled headers is probably higher, but in terms of corporate users it's rare not to see precompiled headers. One little trick I occasionally do for clients is to replace precompiled headers with static libraries and I often halve the compile times. People underrate static libraries.

...

We may also think about the alternative of giving control of whether system means system::static or system::shared to root level, and only express dependencies using the unsuffixed form.

That's another possibility, and I can see the attraction. But I suspect this is cart before horse, mountain out of molehill stuff. I think 98% of end users are wholly doing one of these three use cases: 1. All shared Boost libraries. 2. All static Boost libraries. 3. Precompiled headers/Header only Boost libraries. So keep it simple for now, support those three, see what happens after an extended beta programme. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

3:53 p.m.

Niall Douglas wrote:

...

But I suspect this is cart before horse, mountain out of molehill stuff. I think 98% of end users are wholly doing one of these three use cases:

1. All shared Boost libraries.

2. All static Boost libraries.

3. Precompiled headers/Header only Boost libraries.

Even if so, you still need to be able to answer the question "how do I link to Asio in such a way so that its dependency System is statically/dynamically linked?"

Niall Douglas

5:44 p.m.

On 21/06/2017 16:53, Peter Dimov via Boost wrote:

...

Niall Douglas wrote:

...
But I suspect this is cart before horse, mountain out of molehill stuff. I think 98% of end users are wholly doing one of these three use cases:

1. All shared Boost libraries.

2. All static Boost libraries.

3. Precompiled headers/Header only Boost libraries.

Even if so, you still need to be able to answer the question "how do I link to Asio in such a way so that its dependency System is statically/dynamically linked?"

I'm saying that ASIO header only users almost certainly will also want header only Boost.System. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Vinnie Falco

5:50 p.m.

On Wed, Jun 21, 2017 at 10:44 AM, Niall Douglas via Boost wrote:

...

I'm saying that ASIO header only users almost certainly will also want header only Boost.System.

I second that. I would venture a guess that the collective man-hours wasted by Boost.Asio users wrestling with their build scripts to also link in Boost.System greatly outweighs the time it would take for one person to make Boost.System header-only.

Thomas Heller

5:58 p.m.

Am 21.06.2017 7:44 nachm. schrieb "Niall Douglas via Boost" < boost@lists.boost.org>: On 21/06/2017 16:53, Peter Dimov via Boost wrote:

...

Niall Douglas wrote:

...
But I suspect this is cart before horse, mountain out of molehill stuff. I think 98% of end users are wholly doing one of these three use cases:

1. All shared Boost libraries.

2. All static Boost libraries.

3. Precompiled headers/Header only Boost libraries.

Even if so, you still need to be able to answer the question "how do I link to Asio in such a way so that its dependency System is statically/dynamically linked?"

I'm saying that ASIO header only users almost certainly will also want header only Boost.System. Absolutely not. I'd prefer to link against compiled libraries. If only to reduce compile times. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/ _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost

Peter Dimov

6:01 p.m.

Niall Douglas wrote:

...

...
Even if so, you still need to be able to answer the question "how do I link to Asio in such a way so that its dependency System is statically/dynamically linked?"

I'm saying that ASIO header only users almost certainly will also want header only Boost.System.

I don't think so (although now that it has been fixed to use Winapi instead of , maybe). But even if true, users of static library A that uses Asio will not necessarily want a header-only Boost.System, which means that static library A has to have a way to link to Asio "statically".

Vinnie Falco

6:13 p.m.

On Wed, Jun 21, 2017 at 10:58 AM, Thomas Heller via Boost wrote:

...

Absolutely not. I'd prefer to link against compiled libraries. If only to reduce compile times.

Nothing wrong with the OPTION to link against Boost.System, but header-only should be the default. Its easier for users and gives a better experience. On Wed, Jun 21, 2017 at 11:01 AM, Peter Dimov via Boost wrote:

...

...
...ASIO header only users almost certainly will also want header only Boost.System.

I don't think so (although now that it has been fixed to use Winapi instead of , maybe).

But even if true, users of static library A that uses Asio will not necessarily want a header-only Boost.System, which means that static library A has to have a way to link to Asio "statically".

For reference, here is the ONE .cpp file which makes Boost.System not header-only: https://github.com/boostorg/system/blob/develop/src/error_code.cpp#L14 and here's that one header it includes https://github.com/boostorg/system/blob/develop/include/boost/system/detail/... We make people link against Boost.System just for that? Especially when std::error_code is a built in for C++11 and later? It seems overkill. Boost.System should be header-only by default. Then, Asio users will never need a link dependency.

Peter Dimov

9:56 p.m.

Vinnie Falco wrote:

...

For reference, here is the ONE .cpp file which makes Boost.System not header-only: https://github.com/boostorg/system/blob/develop/src/error_code.cpp#L14

and here's that one header it includes https://github.com/boostorg/system/blob/develop/include/boost/system/detail/...

We make people link against Boost.System just for that?

No, for that: https://github.com/boostorg/system/blob/master/include/boost/system/detail/e... Or, in general, we use separate compilation to isolate the users of the library from the header dependencies required to build (but not use) the library.

...

Boost.System should be header-only by default.

In this case, after the recent fixes, maybe.

Niall Douglas

9:23 p.m.

On 21/06/2017 19:01, Peter Dimov via Boost wrote:

...

Niall Douglas wrote:

...
...
Even if so, you still need to be able to answer the question "how do I > link to Asio in such a way so that its dependency System is > statically/dynamically linked?"

I'm saying that ASIO header only users almost certainly will also want header only Boost.System.

I don't think so (although now that it has been fixed to use Winapi instead of , maybe).

But even if true, users of static library A that uses Asio will not necessarily want a header-only Boost.System, which means that static library A has to have a way to link to Asio "statically".

Surely users of static library A that uses ASIO will use the static library edition of ASIO and the static library edition of System? AFIO v1 used static library ASIO. Made a huge difference to compile times, it was my default choice for code development. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

9:51 p.m.

Niall Douglas wrote:

...

...
But even if true, users of static library A that uses Asio will not necessarily want a header-only Boost.System, which means that static library A has to have a way to link to Asio "statically".

Surely users of static library A that uses ASIO will use the static library edition of ASIO and the static library edition of System?

Eh. All right. Static library A uses header-only library B uses non-header-only library C. Are we clear now. :-)

Niall Douglas

10:21 p.m.

On 21/06/2017 22:51, Peter Dimov via Boost wrote:

...

Niall Douglas wrote:

...
...
But even if true, users of static library A that uses Asio will not necessarily want a header-only Boost.System, which means that static library A has to have a way to link to Asio "statically".

Surely users of static library A that uses ASIO will use the static library edition of ASIO and the static library edition of System?

Eh. All right.

Static library A uses header-only library B uses non-header-only library C. Are we clear now. :-)

Oh sure. But if that were the case, there is likely a very good reason for it, so it's desirable to be that way. The real problem is when the end user doesn't want the specific combination of hl-sl-dl chosen by the library devs for them. What I was saying what to supply all-::hl, all-::sl and all-::dl where possible, and that probably maps onto what 98% of end users will want. The number which want some weird mashup for ::hl, ::sl and ::dl variants is likely very low. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Thomas Heller

10:33 p.m.

On Thu, Jun 22, 2017 at 12:21 AM, Niall Douglas via Boost < boost@lists.boost.org> wrote:

...

On 21/06/2017 22:51, Peter Dimov via Boost wrote:

...
Niall Douglas wrote:

...
...
But even if true, users of static library A that uses Asio will not necessarily want a header-only Boost.System, which means that static library A has to have a way to link to Asio "statically".

Surely users of static library A that uses ASIO will use the static library edition of ASIO and the static library edition of System?

Eh. All right.

Static library A uses header-only library B uses non-header-only library C. Are we clear now. :-)

Oh sure. But if that were the case, there is likely a very good reason for it, so it's desirable to be that way.

The real problem is when the end user doesn't want the specific combination of hl-sl-dl chosen by the library devs for them. What I was saying what to supply all-::hl, all-::sl and all-::dl where possible, and that probably maps onto what 98% of end users will want. The number which want some weird mashup for ::hl, ::sl and ::dl variants is likely very low.

I can only speak for myself here. What my usecases are: - I don't care whether I got a combination of dynamic and static (which means I am on a platform supporting dynamic libraries) - I want a build with everything static That is, when I configure/run the build, I tell the build system this. Usually via the command line, in a similar way I set the compiler, additional flags and paths to dependencies. For CMake, this is, as far as I am concerned, done by setting specific cache variables, with autotools and b2, I pass the approrpriate flags. This is then honored by the specific build scripts. YMMV, of course. I am still unclear if this is the wrong approach in a modern, declarative world, or if there's a better approach. If there is one, which would it be?

...

Niall

-- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost

Daniel James

11:46 p.m.

On 21 June 2017 at 23:21, Niall Douglas via Boost wrote:

...

The real problem is when the end user doesn't want the specific combination of hl-sl-dl chosen by the library devs for them. What I was saying what to supply all-::hl, all-::sl and all-::dl where possible, and that probably maps onto what 98% of end users will want. The number which want some weird mashup for ::hl, ::sl and ::dl variants is likely very low.

I guess this might be a stupid question, but what would the 2% of users who want something different do?

Niall Douglas

22 Jun 22 Jun

12:02 a.m.

On 22/06/2017 00:46, Daniel James wrote:

...

On 21 June 2017 at 23:21, Niall Douglas via Boost wrote:

...
The real problem is when the end user doesn't want the specific combination of hl-sl-dl chosen by the library devs for them. What I was saying what to supply all-::hl, all-::sl and all-::dl where possible, and that probably maps onto what 98% of end users will want. The number which want some weird mashup for ::hl, ::sl and ::dl variants is likely very low.

I guess this might be a stupid question, but what would the 2% of users who want something different do?

You can do truly horrible monkey patching in cmake. So, iterating all targets and doing regex string match and replace on the internal strings which are built for you by the cmake commands, all of which are just convenient syntax sugar over the underlying string properties on targets. If you've seen any real world cmake in use in real world corporations, you'll likely have seen extended monkey patching logic before. It's fairly common. That's what the 2% would end up doing I would think. At least like most advanced cmake, how to do it is well documented on stackoverflow. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Daniel James

4:38 p.m.

On 22 June 2017 at 01:02, Niall Douglas via Boost wrote:

...

You can do truly horrible monkey patching in cmake. So, iterating all targets and doing regex string match and replace on the internal strings which are built for you by the cmake commands, all of which are just convenient syntax sugar over the underlying string properties on targets.

If you've seen any real world cmake in use in real world corporations, you'll likely have seen extended monkey patching logic before. It's fairly common. That's what the 2% would end up doing I would think. At least like most advanced cmake, how to do it is well documented on stackoverflow.

Thanks for the explanation. I think I'd be tempted to write my own build logic. It looks like some of the necessary data can be imported from your cmake files, and most boost libraries aren't that hard to build.

paul

21 Jun 21 Jun

6:28 p.m.

On Wed, 2017-06-21 at 18:53 +0300, Peter Dimov via Boost wrote:

...

Niall Douglas wrote:

...
But I suspect this is cart before horse, mountain out of molehill stuff. I think 98% of end users are wholly doing one of these three use cases:

1. All shared Boost libraries.

2. All static Boost libraries.

3. Precompiled headers/Header only Boost libraries.

Even if so, you still need to be able to answer the question "how do I link to Asio in such a way so that its dependency System is statically/dynamically linked?"

Ultimately, I don't think we should go down this route of providing multiple targets for the different build variants. This is not the way cmake is setup to work. Cmake requires different build tree for the different build variants. This will lead to further complications and frustrations for users. In the future, we can look for ways to "innovate" cmake to support multiple build variants, but for now we should stick to the standard cmake workflow if we are going to support cmake.

paul

4:54 p.m.

On Wed, 2017-06-21 at 16:44 +0100, Niall Douglas via Boost wrote:

...

On 21/06/2017 14:46, Peter Dimov via Boost wrote:

...
Niall Douglas wrote:

...
...
The one remaining question I have here is how does a library link to

its > dependency. The obvious approach seems to be for filesystem::sl to link > to system::sl and for filesystem::dl to link to system::dl, right? (And > a hypothetical filesystem::hl would probably link to system::sl.)

...

...
That's the default convention I've always followed in my own code.

It's more complex than that. Even header-only libraries would need all three subtargets, for several reasons. One, we want the dependency enumeration to be scripted and therefore not depend on library specifics. Two, we can have chains pumpkin -> asio -> system, where I want pumpkin::static to propagate down to system::static, for which I'd need it to link to asio::static.

There's also the question of the "maintainer-preferred" default subtarget. If plain asio maps to asio::header, as is natural because it _is_ ::header, this will propagate down to system and switch it into header-only mode, and I probably don't want that. So the default, even for header-only libraries, would need to be, f.ex. ::static.

In addition to all that, we want some sort of error to be issued when the project links to both system::static and system::shared -- not sure how this is done in CMake.

All in all, I think that if we keep this scheme, we need to drop ::header and stick to ::static and ::shared. This will eliminate the surprise of header-only libraries linking to ::header targets downstream.

Before cmake 3.5 you couldn't add dependencies to header only targets on the basis of your reasoning. So end users had to know what a header only targets also needed to be linked against.

Before cmake 3.0. CMake 3.0 introduced interface targets(mainly to support boost libraries), which allow you to link in usage requirements.

Peter Dimov

20 Jun 20 Jun

12:19 p.m.

"P F" wrote:

...

- Having things like `::hl` or `::sl` is quite strange. In cmake, I set `BUILD_SHARED_LIBS=On` when I want a shared library or I set it to `BUILD_SHARED_LIBS=Off` when I want a static library. There should only be one target `boost::config` or `boost::system` that I use.

Niall says that using the global BUILD_SHARED_LIBS is a cmake2ism, and that explicit targets are preferred nowadays.

...

- Doing `if(NOT TARGET boost::assert::hl)` is wrong. A target does not have to exists to be able to link against it.

The second sentence has nothing to do with the first. The if is just to detect if the target has already been declared, in order to avoid declaring it twice, like an (external) include guard.

Thomas Heller

12:30 p.m.

On Tue, Jun 20, 2017 at 2:19 PM, Peter Dimov via Boost < boost@lists.boost.org> wrote:

...

"P F" wrote:

- Having things like `::hl` or `::sl` is quite strange. In cmake, I set

...
`BUILD_SHARED_LIBS=On` when I want a shared library or I set it to `BUILD_SHARED_LIBS=Off` when I want a static library. There should only be one target `boost::config` or `boost::system` that I use.

Niall says that using the global BUILD_SHARED_LIBS is a cmake2ism, and that explicit targets are preferred nowadays.

Niall, can you point me to a reference stating this? Preferably some cmake documentation telling me what the right(tm) thing to do is. As a general observation, I see a lot of statements along the lines of "I state that XYZ is preferable over UVW", it would be nice to have to have background information (pros and cons anyone? what do the cmake authors/docs have to say about this?) on those statements so that everyone can form their own opinion instead of having to choose whom to trust about what's "standard" cmake.

...

- Doing `if(NOT TARGET boost::assert::hl)` is wrong. A target does not

...
have to exists to be able to link against it.

The second sentence has nothing to do with the first. The if is just to detect if the target has already been declared, in order to avoid declaring it twice, like an (external) include guard.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman /listinfo.cgi/boost

Thomas Heller

1:31 p.m.

Am 20.06.2017 2:30 nachm. schrieb "Thomas Heller" : On Tue, Jun 20, 2017 at 2:19 PM, Peter Dimov via Boost < boost@lists.boost.org> wrote:

...

"P F" wrote:

- Having things like `::hl` or `::sl` is quite strange. In cmake, I set

...
`BUILD_SHARED_LIBS=On` when I want a shared library or I set it to `BUILD_SHARED_LIBS=Off` when I want a static library. There should only be one target `boost::config` or `boost::system` that I use.

Niall says that using the global BUILD_SHARED_LIBS is a cmake2ism, and that explicit targets are preferred nowadays.

Niall, can you point me to a reference stating this? Preferably some cmake documentation telling me what the right(tm) thing to do is. https://cmake.org/cmake/help/v3.9/command/add_library.html The cmake docs still documents BUILD_SHARED_LIBS without giving any alternative. Any definite source on what a cmake2ism is or is not would be highly appreciated. As a general observation, I see a lot of statements along the lines of "I state that XYZ is preferable over UVW", it would be nice to have to have background information (pros and cons anyone? what do the cmake authors/docs have to say about this?) on those statements so that everyone can form their own opinion instead of having to choose whom to trust about what's "standard" cmake.

...

- Doing `if(NOT TARGET boost::assert::hl)` is wrong. A target does not

...
have to exists to be able to link against it.

The second sentence has nothing to do with the first. The if is just to detect if the target has already been declared, in order to avoid declaring it twice, like an (external) include guard.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman /listinfo.cgi/boost

Niall Douglas

3:08 p.m.

...

...
Niall says that using the global BUILD_SHARED_LIBS is a cmake2ism, and that explicit targets are preferred nowadays.

Niall, can you point me to a reference stating this? Preferably some cmake documentation telling me what the right(tm) thing to do is.

Any cmake construct which requires you to write an if() in a non-root-level CMakeLists is a cmake2-ism. There are a very few places in cmake3 remaining where no good alternative to global variables exist. BUILD_SHARED_LIBS is definitely not one of those. Shared libraries usually have different settings to static libs and again to header only libs. They have different relationships to their dependencies, different usage requirements for consumers. You can use generator expressions to encode those differences, but then you've hard coded them so external cmake no longer can easily override them. You have just made your non-root CMakeLists hard to reuse by cmake you didn't write nor design. Generator expressions are also hard to write and debug, are overwhelmingly confusing to read, and randomly don't work in some places depending on which build system generator you are using. So best avoid all of that complexity - place no complexity at all outside the rootlevel CMakeLists. Declaration only. Place all custom logic solely in rootlevel CMakeLists only.

...

Any definite source on what a cmake2ism is or is not would be highly appreciated.

As a general observation, I see a lot of statements along the lines of "I state that XYZ is preferable over UVW", it would be nice to have to have background information (pros and cons anyone? what do the cmake authors/docs have to say about this?) on those statements so that everyone can form their own opinion instead of having to choose whom to trust about what's "standard" cmake.

I believe Stephen Kelly is negotiating a book deal on this. It'll be some time before that book lands though. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

paul

4:28 p.m.

On Tue, 2017-06-20 at 16:08 +0100, Niall Douglas via Boost wrote:

...

...
...
Niall says that using the global BUILD_SHARED_LIBS is a cmake2ism, and that explicit targets are preferred nowadays.

Niall, can you point me to a reference stating this? Preferably some cmake documentation telling me what the right(tm) thing to do is.

Any cmake construct which requires you to write an if() in a non-root-level CMakeLists is a cmake2-ism.

So you mean like this is a cmake2-ism: if(NOT TARGET boost::assert::hl) add_subdirectory("../assert" "boost_assert" EXCLUDE_FROM_ALL) endif()

...

There are a very few places in cmake3 remaining where no good alternative to global variables exist. BUILD_SHARED_LIBS is definitely not one of those.

You should just write `add_library` and let the clients decide. I find no reference supporting your claim.

...

Shared libraries usually have different settings to static libs and again to header only libs. They have different relationships to their dependencies, different usage requirements for consumers. You can use generator expressions to encode those differences,

There is no need to use generator expressions since `BUILD_SHARED_LIBS` is a cmake variable.

...

but then you've hard coded them so external cmake no longer can easily override them. You have just made your non-root CMakeLists hard to reuse by cmake you didn't write nor design.

I do not follow this at all. What I presented does not have these problems.

Peter Dimov

2:20 p.m.

Thomas Heller wrote:

...

As a general observation, I see a lot of statements along the lines of "I state that XYZ is preferable over UVW", it would be nice to have to have background information (pros and cons anyone? what do the cmake authors/docs have to say about this?) on those statements so that everyone can form their own opinion instead of having to choose whom to trust about what's "standard" cmake.

That's the problem with CMake, there's no way for someone like me who does not follow it to get a definitive answer to what idiomatic CMake 3.5+ is. There are several articles about it, they all say more or less the same thing - use target_*. I get that. But that's not enough. I see a general (and very predictable) trend of moving from imperative to declarative. Programmers like imperative, but a proper build system really prefers declarative, so CMake is trying to evolve towards declarative. Take for example find_package. It's a command, find me this package, now. But it's much better if a library does not issue "find me this package" commands, but rather declares which packages it needs. So we get "best practice" hacks like redefining find_package. I'm sorry, this doesn't feel right.

P F

2:55 p.m.

...

On Jun 20, 2017, at 9:20 AM, Peter Dimov via Boost wrote:

Thomas Heller wrote:

...
As a general observation, I see a lot of statements along the lines of "I state that XYZ is preferable over UVW", it would be nice to have to have background information (pros and cons anyone? what do the cmake authors/docs have to say about this?) on those statements so that everyone can form their own opinion instead of having to choose whom to trust about what's "standard" cmake.

That's the problem with CMake, there's no way for someone like me who does not follow it to get a definitive answer to what idiomatic CMake 3.5+ is. There are several articles about it, they all say more or less the same thing - use target_*. I get that. But that's not enough.

But Daniel Pfeifer’s presentations go over more than use target_*.

...

I see a general (and very predictable) trend of moving from imperative to declarative. Programmers like imperative, but a proper build system really prefers declarative, so CMake is trying to evolve towards declarative.

Yes, in fact, with BCM it was written like this: bcm_boost_package(core VERSION 1.61.0 DEPENDS assert config ) So its very declarative, however, using custom functions like this is discouraged. So, I am planning to at least refactor it to say: add_library(boost_core INTERFACE) bcm_boost_depends(boost_core INTERFACE boost::assert boost::config) And it can take care of calling find_package and linking. Plus, this looks more cmakey. Although, I think the trend with cmake is moving away from subproject builds and towards standalone builds. This is why it supports exporting its targets so it can be consumed after installation.

...

Take for example find_package. It's a command, find me this package, now. But it's much better if a library does not issue "find me this package" commands, but rather declares which packages it needs.

A library may want to compile differently if a dependency isn't there, or even use completly different set of dependencies, which is best expressed imperatively the declaratively.

...

So we get "best practice" hacks like redefining find_package. I'm sorry, this doesn't feel right.

Redefining `find_package` is one way. Another way is to provide Find modules that override the behavior as well. Cmake also provides a `CMAKE_DISABLE_FIND_PACKAGE_<PackageName>` to disable certain packages, but its really only for optional dependencies. Ideally, there should be way in cmake to skip over these packages, that is to tell cmake that the package was already provided by another project in the build. For now, the easiest way, is overloading `find_package`.

Niall Douglas

3:51 p.m.

On 20/06/2017 15:55, P F via Boost wrote:

...

...
On Jun 20, 2017, at 9:20 AM, Peter Dimov via Boost wrote:

Thomas Heller wrote:

...
As a general observation, I see a lot of statements along the lines of "I state that XYZ is preferable over UVW", it would be nice to have to have background information (pros and cons anyone? what do the cmake authors/docs have to say about this?) on those statements so that everyone can form their own opinion instead of having to choose whom to trust about what's "standard" cmake.

That's the problem with CMake, there's no way for someone like me who does not follow it to get a definitive answer to what idiomatic CMake 3.5+ is. There are several articles about it, they all say more or less the same thing - use target_*. I get that. But that's not enough.

But Daniel Pfeifer’s presentations go over more than use target_*.

He was also addressing an audience with existing legacy cmake systems. If you are in the very fortunate position of starting from a completely clean sheet like we are, you'd do differently. You know you could just drop Daniel a line and ask for a bit of his time. He used to be on boost-dev, indeed he and I and Dave worked on the git conversion.

...

Although, I think the trend with cmake is moving away from subproject builds and towards standalone builds. This is why it supports exporting its targets so it can be consumed after installation.

That trend exists only because so much existing cmake is a very bad neighbour to other cmake, so you need to ring fence cmake-innovations away from one another as they trample on one another. Purely declarative cmake examines no global state and so isn't affected by other people's modifications of global state. It also changes no global state, so it is always a good neighbour. Therefore, unlike most existing cmake, it is always safe to use in subproject builds. I again reiterate that nobody is stopping anyone writing their own installation logic in a rootlevel CMakeLists. You can go ahead and implement target export and anything else you like on your own if that's what suits you. But most will simply do subproject builds. It's painfree and works without surprises on Windows. It also inherits the consumer's optimisation and build settings. End users will be delighted.

...

...
Take for example find_package. It's a command, find me this package, now. But it's much better if a library does not issue "find me this package" commands, but rather declares which packages it needs.

A library may want to compile differently if a dependency isn't there, or even use completly different set of dependencies, which is best expressed imperatively the declaratively.

You declare alternative targets for those situations and leave the rootlevel CMakeLists decide what is wanted. Non-rootlevel CMakeLists cannot be enforcing choices of how to best detect presence of dependencies on end users, they might have a custom build of some dependency in a custom path for example. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

paul

5:06 p.m.

On Tue, 2017-06-20 at 16:51 +0100, Niall Douglas via Boost wrote:

...

On 20/06/2017 15:55, P F via Boost wrote:

...
...
On Jun 20, 2017, at 9:20 AM, Peter Dimov via Boost wrote:

Thomas Heller wrote:

...
As a general observation, I see a lot of statements along the lines of "I state that XYZ is preferable over UVW", it would be nice to have to have background information (pros and cons anyone? what do the cmake authors/docs have to say about this?) on those statements so that everyone can form their own opinion instead of having to choose whom to trust about what's "standard" cmake.

That's the problem with CMake, there's no way for someone like me who does not follow it to get a definitive answer to what idiomatic CMake 3.5+ is. There are several articles about it, they all say more or less the same thing - use target_*. I get that. But that's not enough.

But Daniel Pfeifer’s presentations go over more than use target_*.

He was also addressing an audience with existing legacy cmake systems. If you are in the very fortunate position of starting from a completely clean sheet like we are, you'd do differently.

I am not sure who he is addressing, but what he suggest is how you should write cmake period. He never discussed any of things you are doing in his talk.

...

You know you could just drop Daniel a line and ask for a bit of his time. He used to be on boost-dev, indeed he and I and Dave worked on the git conversion.

Actually, a lot of what I wrote follow very closely follows to how he wrote it here: https://github.com/boost-cmake/boost-cmake/blob/master/listsfiles/libs/syste... CMakeLists.txt And this relies on `BUILD_SHARED_LIBS`.

...

...
Although, I think the trend with cmake is moving away from subproject builds and towards standalone builds. This is why it supports exporting its targets so it can be consumed after installation.

That trend exists only

The trend exists because this is what works for large-scale building, especially for 10000+ libraries.

...

because so much existing cmake is a very bad neighbour to other cmake, so you need to ring fence cmake-innovations away from one another as they trample on one another.

But what you are suggesting will be a bad neighbor. As I will not be able to build a library standalone.

...

Purely declarative cmake examines no global state and so isn't affected by other people's modifications of global state.

None of my project examine global state, whereas your example does by inspecting the existenct of targets, which is global state.

...

It also changes no global state, so it is always a good neighbour. Therefore, unlike most existing cmake, it is always safe to use in subproject builds.

Yes, and what I presented does this.

...

I again reiterate that nobody is stopping anyone writing their own installation logic in a rootlevel CMakeLists.

But that doesn't make the project work standalone, which is an important best practice for cmake.

...

You can go ahead and implement target export and anything else you like on your own if that's what suits you.

But most will simply do subproject builds.

I don't know about most. Most places I worked at built and installed boost, even on windows.

...

It's painfree and works without surprises on Windows.

I have found no problems using `find_package` on windows.

...

It also inherits the consumer's optimisation and build settings. End users will be delighted.

I don't think so when we say you must use boost as `add_subdirectory` and if you want to use `find_package` you are own your own. A lot of cmake users want the `find_package` support. Ultimately, we need to support both scenarios, what I presented does that. You can use it as `add_sudirectory`, you can build each project standalone, and you can use `find_package`. What you presented doesn't do those.

...

...
...
Take for example find_package. It's a command, find me this package, now. But it's much better if a library does not issue "find me this package" commands, but rather declares which packages it needs.

A library may want to compile differently if a dependency isn't there, or even use completly different set of dependencies, which is best expressed imperatively the declaratively.

You declare alternative targets for those situations and leave the rootlevel CMakeLists decide what is wanted. Non-rootlevel CMakeLists cannot be enforcing choices of how to best detect presence of dependencies on end users,

Using `find_package` doesn't as it is up to the clients to decide how it will find the dependencies through `CMAKE_PREFIX_PATH` and `<package>_DIR` variables that can be defined by the clients.

...

they might have a custom build of some dependency in a custom path for example.

Yes, which is what `mylib_DIR` is for.

...

Niall

Peter Dimov

5:26 p.m.

paul wrote:

...

The trend exists because this is what works for large-scale building, especially for 10000+ libraries.

This is what works when your build system is deficient, so it takes it a few hours to see that 9999 libraries haven't changed. Large-scale building done right is called Ninja.

paul

5:51 p.m.

On Tue, 2017-06-20 at 20:26 +0300, Peter Dimov via Boost wrote:

...

paul wrote:

...
The trend exists because this is what works for large-scale building, especially for 10000+ libraries.

This is what works when your build system is deficient, so it takes it a few hours to see that 9999 libraries haven't changed. Large-scale building done right is called Ninja.

At 10000+ you are dealing with at least 5 different buildsystems: autotools, cmake, b2, meson, and qmake. Its not feasible to maintain build scripts for 10000 dependencies. Nor is it reasonable to rebuild the same dependencies for every project.

Peter Dimov

5:58 p.m.

paul wrote:

...

At 10000+ you are dealing with at least 5 different buildsystems: autotools, cmake, b2, meson, and qmake.

Absolutely not. Everyone on this scale builds everything with a single (usually in-house) build system. The alternative is completely unmaintainable, and errors are irreproducible.

paul

6:31 p.m.

On Tue, 2017-06-20 at 20:58 +0300, Peter Dimov via Boost wrote:

...

paul wrote:

...
At 10000+ you are dealing with at least 5 different buildsystems: autotools, cmake, b2, meson, and qmake.

Absolutely not. Everyone on this scale builds everything with a single (usually in-house) build system.

I have not worked at a place that used a single build system. Nor do distros build like that. I am not saying it isn't done, just that it is not so common as you say.

...

The alternative is completely unmaintainable,

Well, I guess I should switch from my "unmaintainable" OS to windows.

...

and errors are irreproducible.

This has not been my experience. I have found "irreproducible" errors(ie it shows on one machine but not the other), but this was due to differnt kernel versions, which a single in-house build would've not fixed or helped either. Furthermore, tools like conan provide reproducible builds, but does not use a single build system either. So I dont think a single build system is always required for reproducible builds.

Peter Dimov

6:58 p.m.

paul wrote:

...

On Tue, 2017-06-20 at 20:58 +0300, Peter Dimov via Boost wrote:

...
paul wrote:

...
At 10000+ you are dealing with at least 5 different buildsystems: autotools, cmake, b2, meson, and qmake.

Absolutely not. Everyone on this scale builds everything with a single (usually in-house) build system.

I have not worked at a place that used a single build system. Nor do distros build like that.

Ah, we're talking about different things then. You're talking about building and shipping 10,000 libraries. I thought we're talking about building a project with 10,000 dependencies.

...

Well, I guess I should switch from my "unmaintainable" OS to windows.

That's just nonsense. The OS doesn't matter.

...

...
and errors are irreproducible.

This has not been my experience. I have found "irreproducible" errors(ie it shows on one machine but not the other), but this was due to differnt kernel versions, which a single in-house build would've not fixed or helped either.

Reproducible here refers to you get a bug report about build number 49121, but you're already on 56192 internally. You need to be able to rewind back to 49121. The entire state of the project is a unit, you can't treat dependencies as their own separate kingdoms which you build, "install", and forget about.

paul

7:27 p.m.

On Tue, 2017-06-20 at 21:58 +0300, Peter Dimov via Boost wrote:

...

paul wrote:

...
On Tue, 2017-06-20 at 20:58 +0300, Peter Dimov via Boost wrote:

...
paul wrote:

...
At 10000+ you are dealing with at least 5 different buildsystems: autotools, cmake, b2, meson, and qmake.

Absolutely not. Everyone on this scale builds everything with a single (usually in-house) build system.

I have not worked at a place that used a single build system. Nor do distros build like that.

Ah, we're talking about different things then. You're talking about building and shipping 10,000 libraries. I thought we're talking about building a project with 10,000 dependencies.

Well, of course, projects are work on, I need to ship them.

...

...
Well, I guess I should switch from my "unmaintainable" OS to windows.

That's just nonsense. The OS doesn't matter.

My OS is a large-scale build, which does not use a single build system.

...

...
...
and errors are irreproducible.

This has not been my experience. I have found "irreproducible" errors(ie it shows on one machine but not the other), but this was due to differnt kernel versions, which a single in-house build would've not fixed or helped either.

Reproducible here refers to you get a bug report about build number 49121, but you're already on 56192 internally. You need to be able to rewind back to 49121. The entire state of the project is a unit, you can't treat dependencies as their own separate kingdoms which you build, "install", and forget about.

If the dependencies are different between build 49121 and 56192, I would've update CI scripts in the repo with the different requirements. Otherwise the CI has no idea how to build it correctly. When I rollback to 49121, I just use the CI scripts for that commit.

Peter Dimov

7:38 p.m.

paul wrote:

...

My OS is a large-scale build, which does not use a single build system.

Good. I don't have an OS, so I can't relate.

...

...
Reproducible here refers to you get a bug report about build number 49121, but you're already on 56192 internally. You need to be able to rewind back to 49121. The entire state of the project is a unit, you can't treat dependencies as their own separate kingdoms which you build, "install", and forget about.

If the dependencies are different between build 49121 and 56192, I would've update CI scripts in the repo with the different requirements. Otherwise the CI has no idea how to build it correctly. When I rollback to 49121, I just use the CI scripts for that commit.

It's not entirely clear to me how just using the CI scripts for that commit would give you the build for that commit. The point of the install model is that the build of the project doesn't rebuild the dependencies, it uses the prebuilt ones.

paul

10:47 p.m.

On Tue, 2017-06-20 at 22:38 +0300, Peter Dimov via Boost wrote:

...

paul wrote:

...
My OS is a large-scale build, which does not use a single build system.

Good. I don't have an OS, so I can't relate.

...
...
Reproducible here refers to you get a bug report about build number 49121, but you're already on 56192 internally. You need to be able to rewind back to 49121. The entire state of the project is a unit, you can't treat dependencies as their own separate kingdoms which you build, "install", and forget about.

If the dependencies are different between build 49121 and 56192, I would've update CI scripts in the repo with the different requirements. Otherwise the CI has no idea how to build it correctly. When I rollback to 49121, I just use the CI scripts for that commit.

It's not entirely clear to me how just using the CI scripts for that commit would give you the build for that commit.

It gives the build environmnent to build it the same way.

...

The point of the install model is that the build of the project doesn't rebuild the dependencies, it uses the prebuilt ones.

Yes, the project does use the installed dependencies, and the dependencies are built during the setup of the build environment which of course is part of the CI scripts, which of course is checked into source control.

...

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boo st

Peter Dimov

11:30 p.m.

paul wrote:

...

Yes, the project does use the installed dependencies, and the dependencies are built during the setup of the build environment which of course is part of the CI scripts, which of course is checked into source control.

So it rebuilds everything from scratch? All 10,000 of it? The alternative here is, you have build 56192 checked out and built, someone says hey, take a look at this bug, you checkout build 49121, ./build, and the build system reliably rebuilds everything that has changed, and doesn't rebuild anything that hasn't, with the end result reliably being the same as the one you would have got had you rebuilt everything from scratch. It is no coincidence that such systems tend to come out of Google.

Niall Douglas

3:36 p.m.

On 20/06/2017 15:20, Peter Dimov via Boost wrote:

...

Thomas Heller wrote:

...
As a general observation, I see a lot of statements along the lines of "I state that XYZ is preferable over UVW", it would be nice to have to have background information (pros and cons anyone? what do the cmake authors/docs have to say about this?) on those statements so that everyone can form their own opinion instead of having to choose whom to trust about what's "standard" cmake.

That's the problem with CMake, there's no way for someone like me who does not follow it to get a definitive answer to what idiomatic CMake 3.5+ is. There are several articles about it, they all say more or less the same thing - use target_*. I get that. But that's not enough.

I see a general (and very predictable) trend of moving from imperative to declarative. Programmers like imperative, but a proper build system really prefers declarative, so CMake is trying to evolve towards declarative.

Take for example find_package. It's a command, find me this package, now. But it's much better if a library does not issue "find me this package" commands, but rather declares which packages it needs. So we get "best practice" hacks like redefining find_package. I'm sorry, this doesn't feel right.

A perfect summary. Thank you Peter. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

P F

8:31 a.m.

...

On Jun 19, 2017, at 6:42 PM, Niall Douglas via Boost wrote:

I've finished the mockup which can be studied at:

https://github.com/ned14/boost-bmv-cmake

So I setup a repo to build and install Boost.System as well, here: https://github.com/pfultz2/boost-cmake-demo This is following the best practices as explained in Daniel Pfeifer’s “Effective Cmake” talk(except the globbing subdirectories, I did that just to save time). It also very similar to the boost cmake repo written by Daniel Pfeifer and Stephen Kelley. This example can be used to build and install modular or in the superproject. Plus, it produces the find_package files, so you can do `find_package(boost_system)`.

Niall Douglas

noon

...

...
I've finished the mockup which can be studied at:

https://github.com/ned14/boost-bmv-cmake

So I setup a repo to build and install Boost.System as well, here:

https://github.com/pfultz2/boost-cmake-demo

This is following the best practices as explained in Daniel Pfeifer’s “Effective Cmake” talk(except the globbing subdirectories, I did that just to save time). It also very similar to the boost cmake repo written by Daniel Pfeifer and Stephen Kelley.

Half the stuff in your CMakeLists is non-mandatory configuration and can be relocated to the rootmost CMakeLists where it belongs.

...

This example can be used to build and install modular or in the superproject. Plus, it produces the find_package files, so you can do `find_package(boost_system)`.

"Installation" is non-mandatory configuration and will vary from end user to end user. It should be implemented in the rootmost CMakeLists. Not in library-level CMakeLists where it needlessly duplicates logic. My bare minimum cmake actually implements the bare minimum for installation, so it tells cmake what would need to be installed per project and does nothing more. Rootmost CMakeLists can then do whatever custom installation logic they choose. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

12:16 p.m.

"P F" wrote:

...

So I setup a repo to build and install Boost.System as well, here:

https://github.com/pfultz2/boost-cmake-demo

You're listing the dependencies three times in three separate locations, which will be error-prone and is harder to automate (unless the whole CMakeLists.txt is generated with boostdep.) You should be able to have dependencies.cmake with some_macro(dep1) some_macro(dep2) (or some_function) and then use that appropriately in CMakeLists.

P F

1:44 p.m.

...

On Jun 20, 2017, at 7:16 AM, Peter Dimov via Boost wrote:

"P F" wrote:

...
So I setup a repo to build and install Boost.System as well, here:

https://github.com/pfultz2/boost-cmake-demo

You're listing the dependencies three times in three separate locations, which will be error-prone and is harder to automate (unless the whole CMakeLists.txt is generated with boostdep.)

I agree, which is why I wrote BCM to take care of this part. Especially, since without it, there would be another list of dependencies to support pkgconfig.

...

You should be able to have dependencies.cmake with

some_macro(dep1) some_macro(dep2)

(or some_function) and then use that appropriately in CMakeLists.

Ideally, I can just create a list with dependencies and then iterate over them. I’ll try to add that to the repo.

2715

Age (days ago)

2718

Last active (days ago)

List overview

Download

50 comments

8 participants

participants (8)

Daniel James
David Sankel
Niall Douglas
P F
paul
Peter Dimov
Thomas Heller
Vinnie Falco