Header-only libraries linking to compiled libraries...

newer
Does boost have a way to get type...

older
Reminder: The master branch will...

Peter Dimov

2 Nov 2021 2 Nov '21

6:26 p.m.

... create problems (at least) for CMake users, because a project using a header-only library must link to its compiled dependencies by hand. (Important Note: this is not about the CMake build of Boost. It's about the b2 build of Boost, and the CMake configuration files it generates.) Compiled libraries don't have this problem, because they have CMake targets, and linking to the library target automatically links to the dependencies. For instance, linking to Boost::filesystem would automatically link to Boost::atomic (and on Windows, to bcrypt.lib) as needed. However, as pointed out in https://github.com/boostorg/boost_install/issues/54, header-only libraries such as Process or UUID have no CMake targets. Consequently, it's not possible for the CMake project to link to Boost::uuid and automatically inherit the dependency to bcrypt.lib, or to link to Boost::process and automatically acquire a dependency to Boost::filesystem. Since the CMake configuration files that are generated by `b2 install` define CMake targets based on the b2 targets, it's not possible to fix this just on the boost-install side. If we want Boost::process to appear as a target in CMake, we need boost_process to appear as a target in b2, that is, we need libs/process/build/Jamfile to exist. The easiest way to fix it - if we decide to do something about it - is to create dummy compiled libraries for each such header-only library that isn't really header-only because of dependencies. A number of Boost libraries are already header-only with a stub library for backward compatibility (System, DateTime, Regex), so this wouldn't be something novel.

Show replies by date

Niall Douglas

2 Nov 2 Nov

6:38 p.m.

On 02/11/2021 18:26, Peter Dimov via Boost wrote:

...

... create problems (at least) for CMake users, because a project using a header-only library must link to its compiled dependencies by hand.

[snip]

The easiest way to fix it - if we decide to do something about it - is to create dummy compiled libraries for each such header-only library that isn't really header-only because of dependencies. A number of Boost libraries are already header-only with a stub library for backward compatibility (System, DateTime, Regex), so this wouldn't be something novel.

The convention in cmake world is that INTERFACE libraries declare what dependencies they have in their INTERFACE_LINK_LIBRARIES export, and when someone tries to link against that INTERFACE library and they don't supply the dependencies, cmake fails the configure. If the end user wants to inject dummy targets to please cmake, that's for the cmake end user to decide, not Boost. Other end users may declare a library target alias to cmake to have those dependencies alias other libraries the end user has set up. All that is on the end cmake user to handle. Nothing to do with Boost. All Boost has to do to be a good actor is prefix its INTERFACE libraries with Boost:: namespacing in the dependencies list, and call it a day. Niall

Peter Dimov

6:56 p.m.

Niall Douglas wrote:

...

All Boost has to do to be a good actor is prefix its INTERFACE libraries...

There's no such thing. Neither FindBoost nor the b2-generated CMake config files define any INTERFACE libraries. (Except the catch-all Boost::boost/Boost::headers.) You didn't read the important note, did you? I knew I should have used all caps.

Niall Douglas

7:32 p.m.

...

Niall Douglas wrote:

...
All Boost has to do to be a good actor is prefix its INTERFACE

On 02/11/2021 18:56, Peter Dimov via Boost wrote: libraries...

...

There's no such thing. Neither FindBoost nor the b2-generated CMake config files define any INTERFACE libraries. (Except the catch-all Boost::boost/Boost::headers.)

You didn't read the important note, did you? I knew I should have used all caps.

No I read it. The correct solution here is that b2 doesn't misrepresent header only libraries to cmake as not being INTERFACE libraries. Fix that, and your semantic problems here go away. Also, cmake when it knows a library is INTERFACE will error out in lots more circumstances, and "do the right thing" in others. It's a good investment. Niall

Дмитрий Архипов

3 Nov 3 Nov

10:24 a.m.

В письме от вторник, 2 ноября 2021 г. 22:32:37 MSK пользователь Niall Douglas via Boost написал:

...

The correct solution here is that b2 doesn't misrepresent header only libraries to cmake as not being INTERFACE libraries.

I've skimmed through boost-install code and I think this is doable. There's already a hard-coded list of b2 targets that should map to INTERFACE CMake targets, it should be possible to extend the logic to any alias target. One thing to note is that alias targets aren't visible on the level CMake config module generation (inside boost-install) currently operates. This will have the effect that if header-only boost_foo depends on header-only boost_bar which in turn depends on compiled target boost_baz, CMake target Boost::foo will depend on Boost::baz, not on Boost::bar. Finally, there's a question of whether this should be done at all. CMake's FindBoost does not declare any targets for header-only libraries. Conan- generated CMake configs don't do that either. If a Boost user adds a dependency to Boost::process in his CML, he is now required to use Boost-generated CMake configs. This is OK when the build environment is controlled (e.g. closed source), but probably is not OK for open source libraries that depend on Boost.

Peter Dimov

3:07 p.m.

Дмитрий Архипов wrote:

...

В письме от вторник, 2 ноября 2021 г. 22:32:37 MSK пользователь Niall Douglas via Boost написал:

...
The correct solution here is that b2 doesn't misrepresent header only libraries to cmake as not being INTERFACE libraries.

I've skimmed through boost-install code and I think this is doable. There's already a hard-coded list of b2 targets that should map to INTERFACE CMake targets, it should be possible to extend the logic to any alias target.

It's not so simple because dependencies are gathered when generating the build variants (as they may depend on the current properties) and header-only libraries have none. I think I'll just remove BOOST_ALL_NO_LIB for now and restore the behavior of the FindBoost helper targets.

Дмитрий Архипов

4:31 p.m.

В письме от среда, 3 ноября 2021 г. 18:07:43 MSK пользователь Peter Dimov via Boost написал:

...

It's not so simple because dependencies are gathered when generating the build variants (as they may depend on the current properties) and header-only libraries have none.

alias boost_foo : boost_bar ; boost_foo has a dependency: boost_bar.

Peter Dimov

4:53 p.m.

Дмитрий Архипов wrote:

...

В письме от среда, 3 ноября 2021 г. 18:07:43 MSK пользователь Peter Dimov via Boost написал:

...
It's not so simple because dependencies are gathered when generating the build variants (as they may depend on the current properties) and header-only libraries have none.

alias boost_foo : boost_bar ;

boost_foo has a dependency: boost_bar.

Yeah, you don't say. It has no build variants though because it doesn't build anything.

Дмитрий Архипов

5:02 p.m.

В письме от среда, 3 ноября 2021 г. 19:53:47 MSK пользователь Peter Dimov via Boost написал:

...

...
В письме от среда, 3 ноября 2021 г. 18:07:43 MSK пользователь Peter Dimov via Boost написал:

...
It's not so simple because dependencies are gathered when generating the build variants (as they may depend on the current properties) and header-only libraries have none.

alias boost_foo : boost_bar ;

boost_foo has a dependency: boost_bar.

Yeah, you don't say. It has no build variants though because it doesn't build anything.

Maybe I misunderstood your point. There are no build variants for `alias` targets, but there are no build variants for `INTERFACE` targets either. And this actually can be used to tell apart header-only libraries and compiled in the generating rule. `lib` targets do produce variants which refer back to them, but `alias` targets do not.

Peter Dimov

5:21 p.m.

Дмитрий Архипов wrote:

...

В письме от среда, 3 ноября 2021 г. 19:53:47 MSK пользователь Peter Dimov via Boost написал:

...
...
В письме от среда, 3 ноября 2021 г. 18:07:43 MSK пользователь Peter Dimov via Boost написал:

...
It's not so simple because dependencies are gathered when generating the build variants (as they may depend on the current properties) and header-only libraries have none.

alias boost_foo : boost_bar ;

boost_foo has a dependency: boost_bar.

Yeah, you don't say. It has no build variants though because it doesn't build anything.

Maybe I misunderstood your point. There are no build variants for `alias` targets, but there are no build variants for `INTERFACE` targets either. And this actually can be used to tell apart header-only libraries and compiled in the generating rule. `lib` targets do produce variants which refer back to them, but `alias` targets do not.

A target can have different dependencies depending on the build variant. For instance, toolset=msvc may depend on Atomic, but toolset=gcc not; or address-model=32 may depend on foo32.lib, but address-model=64 on foo64.lib. That's why in boost-install the dependencies are determined when the build variant is processed (and emitted in the -config-variant-*.cmake file.) But header-only libraries don't have any such.

Boris Kolpackov

2 Nov 2 Nov

6:58 p.m.

Peter Dimov via Boost writes:

...

Since the CMake configuration files that are generated by `b2 install` define CMake targets based on the b2 targets, it's not possible to fix this just on the boost-install side. If we want Boost::process to appear as a target in CMake, we need boost_process to appear as a target in b2, that is, we need libs/process/build/Jamfile to exist.

The easiest way to fix it - if we decide to do something about it - is to create dummy compiled libraries for each such header-only library that isn't really header-only because of dependencies.

Wouldn't the proper way to fix this be to teach b2 first-class "binless" library targets (i.e., a library that doesn't actually have .a/.so)? This will also allow you to, say, have "binful" .so (for example, due to DllMain) but "binless" .a, if you ever want such a beast (no pun intended).

Peter Dimov

7:04 p.m.

Boris Kolpackov wrote:

...

Peter Dimov via Boost writes:

...
Since the CMake configuration files that are generated by `b2 install` define CMake targets based on the b2 targets, it's not possible to fix this just on the boost-install side. If we want Boost::process to appear as a target in CMake, we need boost_process to appear as a target in b2, that is, we need libs/process/build/Jamfile to exist.

The easiest way to fix it - if we decide to do something about it - is to create dummy compiled libraries for each such header-only library that isn't really header-only because of dependencies.

Wouldn't the proper way to fix this be to teach b2 first-class "binless" library targets (i.e., a library that doesn't actually have .a/.so)?

It might be possible to achieve something like that with alias targets, although I haven't tried.

René Ferdinand Rivera Morell

8:58 p.m.

On Tue, Nov 2, 2021 at 2:04 PM Peter Dimov via Boost wrote:

...

Boris Kolpackov wrote:

...
Peter Dimov via Boost writes:

...
Since the CMake configuration files that are generated by `b2 install` define CMake targets based on the b2 targets, it's not possible to fix this just on the boost-install side. If we want Boost::process to appear as a target in CMake, we need boost_process to appear as a target in b2, that is, we need libs/process/build/Jamfile to exist.

The easiest way to fix it - if we decide to do something about it - is to create dummy compiled libraries for each such header-only library that isn't really header-only because of dependencies.

Wouldn't the proper way to fix this be to teach b2 first-class "binless" library targets (i.e., a library that doesn't actually have .a/.so)?

It might be possible to achieve something like that with alias targets, although I haven't tried.

B2 alias targets _are_ real targets (in the meta-target sense of the B2 build abstractions). I haven't looked at how the cmake files are generated though. So not sure what that requires for it to work. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net

René Ferdinand Rivera Morell

9:01 p.m.

On Tue, Nov 2, 2021 at 3:58 PM René Ferdinand Rivera Morell < grafikrobot@gmail.com> wrote:

...

On Tue, Nov 2, 2021 at 2:04 PM Peter Dimov via Boost < boost@lists.boost.org> wrote:

...
Boris Kolpackov wrote:

...
Peter Dimov via Boost writes:

...
Since the CMake configuration files that are generated by `b2 install` define CMake targets based on the b2 targets, it's not possible to fix this just on the boost-install side. If we want Boost::process to appear as a target in CMake, we need boost_process to appear as a target in b2, that is, we need libs/process/build/Jamfile to exist.

The easiest way to fix it - if we decide to do something about it - is to create dummy compiled libraries for each such header-only library that isn't really header-only because of dependencies.

Wouldn't the proper way to fix this be to teach b2 first-class "binless" library targets (i.e., a library that doesn't actually have .a/.so)?

It might be possible to achieve something like that with alias targets, although I haven't tried.

B2 alias targets _are_ real targets (in the meta-target sense of the B2 build abstractions). I haven't looked at how the cmake files are generated though. So not sure what that requires for it to work.

PS. Projects are also real targets, in the same sense. So that might be an option to hook into also. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net

Andrey Semashev

9:58 p.m.

On 11/2/21 21:26, Peter Dimov via Boost wrote:

...

... create problems (at least) for CMake users, because a project using a header-only library must link to its compiled dependencies by hand.

(Important Note: this is not about the CMake build of Boost. It's about the b2 build of Boost, and the CMake configuration files it generates.)

Compiled libraries don't have this problem, because they have CMake targets, and linking to the library target automatically links to the dependencies. For instance, linking to Boost::filesystem would automatically link to Boost::atomic (and on Windows, to bcrypt.lib) as needed.

However, as pointed out in https://github.com/boostorg/boost_install/issues/54, header-only libraries such as Process or UUID have no CMake targets. Consequently, it's not possible for the CMake project to link to Boost::uuid and automatically inherit the dependency to bcrypt.lib, or to link to Boost::process and automatically acquire a dependency to Boost::filesystem.

Since the CMake configuration files that are generated by `b2 install` define CMake targets based on the b2 targets, it's not possible to fix this just on the boost-install side.

Is it possible to use CMakeLists.txt, if one is present for a library? I believe, we have those even for header-only libraries.

...

If we want Boost::process to appear as a target in CMake, we need boost_process to appear as a target in b2, that is, we need libs/process/build/Jamfile to exist.

The easiest way to fix it - if we decide to do something about it - is to create dummy compiled libraries for each such header-only library that isn't really header-only because of dependencies. A number of Boost libraries are already header-only with a stub library for backward compatibility (System, DateTime, Regex), so this wouldn't be something novel.

As I said in the Boost.UUID bug, I'm opposed to dummy compiled libraries where there shouldn't be one in the first place. Header-only libraries need to stay header-only, as in some cases this makes a difference between a library being used or not. As I also suggested in the bug, the problem needs to be solved in CMake and headers. Two immediate workarounds (besides reusing CMakeLists.txt, if possible) I see are: - Stop defining BOOST_ALL_NO_LIB in CMake config. - Ignore BOOST_ALL_NO_LIB (and library-specific equivalents) for linking non-Boost libraries. Either of those would allow auto-linking to hide the problem. The right solution would be to find a way to generate CMake targets for header-only libraries while still maintaining them header-only.

Peter Dimov

10:45 p.m.

Andrey Semashev wrote:

...

As I said in the Boost.UUID bug, I'm opposed to dummy compiled libraries where there shouldn't be one in the first place. Header-only libraries need to stay header-only, as in some cases this makes a difference between a library being used or not.

Header-only libraries would still be header-only. Having a dummy compiled library doesn't magically turn a library into a non-header-only one.

...

As I also suggested in the bug, the problem needs to be solved in CMake and headers. Two immediate workarounds (besides reusing CMakeLists.txt, if possible) I see are:

- Stop defining BOOST_ALL_NO_LIB in CMake config.

I'm not in support of this. Autolinking to Boost libraries when using a proper build system causes nothing but trouble. It does occasionally help hide problems (such as the project not linking to the right target), but that's not worth it. b2 defines BOOST_ALL_NO_LIB for the same reason.

...

- Ignore BOOST_ALL_NO_LIB (and library-specific equivalents) for linking non- Boost libraries.

We've been over this more than once, and we haven't gotten anywhere. IMO, this option should never have applied to autolinking system libraries, and it used not to, but then users who wanted to disable autolinking completely complained. There needs to be a way to enable or disable autolinking to Boost libraries separately from non-Boost ones. (This is not going to help GCC users on Windows, assuming we care about them.)

Andrey Semashev

11:08 p.m.

On 11/3/21 01:45, Peter Dimov via Boost wrote:

...

Andrey Semashev wrote:

...
As I said in the Boost.UUID bug, I'm opposed to dummy compiled libraries where there shouldn't be one in the first place. Header-only libraries need to stay header-only, as in some cases this makes a difference between a library being used or not.

Header-only libraries would still be header-only. Having a dummy compiled library doesn't magically turn a library into a non-header-only one.

If you make a library, and generate a CMake target for it, that library will be linked in user's project, however dummy it is. This means the library has to be packaged, and the package will be a dependency for downstream packages. No, just no. The library is header-only, let's keep it that way instead of creating a crutch for a build system.

...

...
As I also suggested in the bug, the problem needs to be solved in CMake and headers. Two immediate workarounds (besides reusing CMakeLists.txt, if possible) I see are:

- Stop defining BOOST_ALL_NO_LIB in CMake config.

I'm not in support of this. Autolinking to Boost libraries when using a proper build system causes nothing but trouble. It does occasionally help hide problems (such as the project not linking to the right target), but that's not worth it. b2 defines BOOST_ALL_NO_LIB for the same reason.

...
- Ignore BOOST_ALL_NO_LIB (and library-specific equivalents) for linking non- Boost libraries.

We've been over this more than once, and we haven't gotten anywhere. IMO, this option should never have applied to autolinking system libraries, and it used not to, but then users who wanted to disable autolinking completely complained. There needs to be a way to enable or disable autolinking to Boost libraries separately from non-Boost ones.

Agreed, we do need two separate knobs.

...

(This is not going to help GCC users on Windows, assuming we care about them.)

We do care about gcc (well, I do), and auto-linking would be an intermediate workaround, e.g. for 1.78, until we come up with a proper solution. This would be something better than what we had before.

Peter Dimov

3 Nov 3 Nov

2:26 a.m.

Andrey Semashev wrote:

...

...
...
- Stop defining BOOST_ALL_NO_LIB in CMake config.

I'm trying this, and it breaks my boost_install test of Boost.Log. That's because Log autolinks to DateTime for some reason. :-)

1119

Age (days ago)

1120

Last active (days ago)

List overview

Download

17 comments

6 participants

participants (6)

Andrey Semashev
Boris Kolpackov
Niall Douglas
Peter Dimov
René Ferdinand Rivera Morell
Дмитрий Архипов