Can we get rid of auto-linking?
Today I got bit by auto-linking again. As I forgot to define BOOST_ALL_NO_LIB in a CI build. I can't remember a time when auto-linking has been beneficial. And I've lost the number of times I've had to answer user questions with "Auto-linking is getting in the way. Define BOOST_ALL_NO_LIB and link normally." more times than I can count. If there was ever a benefit to auto-linking it has not surpassed the drawbacks. So.. Can we get rid of auto-linking? -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net
On Sat, Apr 27, 2024 at 4:56 PM René Ferdinand Rivera Morell wrote:
Can we get rid of auto-linking?
I've always felt that auto-linking should have been disabled by default. But then I remember users complaining when it is: https://github.com/boostorg/boost_install/issues/54 Glen
On 27/04/2024 22:08, Glen Fernandes via Boost wrote:
On Sat, Apr 27, 2024 at 4:56 PM René Ferdinand Rivera Morell wrote:
Can we get rid of auto-linking?
I've always felt that auto-linking should have been disabled by default.
But then I remember users complaining when it is: https://github.com/boostorg/boost_install/issues/54
The problem is one of "which of the many library variants do I link to?". My experience was always that users are terrible at picking the right one, and if they do do that, then the result is some very hard to track down errors arising from two different std libraries (memory managers) being in play. And that means more bug reports for us :( But.... if you're using Boost.Build, or even CMake to build everything at once then it's obviously not an issue. So... I'm not sure I have an answer for you, John.
John Maddock wrote:
On 27/04/2024 22:08, Glen Fernandes via Boost wrote:
On Sat, Apr 27, 2024 at 4:56 PM René Ferdinand Rivera Morell wrote:
Can we get rid of auto-linking?
I've always felt that auto-linking should have been disabled by default.
But then I remember users complaining when it is: https://github.com/boostorg/boost_install/issues/54
The problem is one of "which of the many library variants do I link to?". My experience was always that users are terrible at picking the right one, and if they do do that, then the result is some very hard to track down errors arising from two different std libraries (memory managers) being in play. And that means more bug reports for us :(
But.... if you're using Boost.Build, or even CMake to build everything at once then it's obviously not an issue.
It's a Rene-specific issue because he builds with b2 without the superproject, which means that the superproject is no longer defining BOOST_ALL_NO_LIB for him, but he hasn't fixed the Jamfiles of the libraries to define the library- specific BOOST_<LIBNAME>_NO_LIB macros (as I have in CMake.) And yes, I agree that not having autolinking will generate many more complaints that having it does today. (Also, usually when autolinking is on when it shouldn't be, nothing bad happens because the library is just linked to twice, which is harmless. When a problem appears, it's usually a symptom of something else, because that means that the headers think they should be linking to some library and the build system thinks it should be linking to some other library, which is a sign of a configuration mismatch which is almost always bad news.)
On Sun, Apr 28, 2024 at 4:16 AM Peter Dimov via Boost
John Maddock wrote:
On 27/04/2024 22:08, Glen Fernandes via Boost wrote:
On Sat, Apr 27, 2024 at 4:56 PM René Ferdinand Rivera Morell wrote:
Can we get rid of auto-linking?
I've always felt that auto-linking should have been disabled by default.
But then I remember users complaining when it is: https://github.com/boostorg/boost_install/issues/54
The problem is one of "which of the many library variants do I link to?". My experience was always that users are terrible at picking the right one, and if they do do that, then the result is some very hard to track down errors arising from two different std libraries (memory managers) being in play. And that means more bug reports for us :(
But.... if you're using Boost.Build, or even CMake to build everything at once then it's obviously not an issue.
It's a Rene-specific issue because he builds with b2 without the superproject, which means that the superproject is no longer defining BOOST_ALL_NO_LIB for him, but he hasn't fixed the Jamfiles of the libraries to define the library- specific BOOST_<LIBNAME>_NO_LIB macros (as I have in CMake.)
Except for the "It's a Rene-specific issue" part that's all true. :-)
And yes, I agree that not having autolinking will generate many more complaints that having it does today.
Perhaps. Although it's easy enough to find instances of people having problems with it.. https://cpplang.slack.com/archives/C41CWV9HA/p1697231475429529 https://stackoverflow.com/questions/27626891/is-visual-studio-adding-depende... https://stackoverflow.com/questions/27254718/maps-in-shared-memory-boost-int... https://stackoverflow.com/questions/44133002/boost-vs2017-linking-to-the-wro... https://stackoverflow.com/questions/51150197/cmake-generated-program-fails-t... But I guess I'll go fix everyone's Jamfile's now. Because authors failed to do that over the past decades. Probably because all build systems are terrible. But likely stemming from C++ building being a giant mess. (insert old-man-screaming-at-clouds image here) -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net
René Ferdinand Rivera Morell wrote:
But I guess I'll go fix everyone's Jamfile's now. Because authors failed to do that over the past decades. Probably because all build systems are terrible. But likely stemming from C++ building being a giant mess. (insert old-man- screaming-at-clouds image here)
It's because Jamroot defined BOOST_ALL_NO_LIB, so nobody else had to. I used to do the same in the CMake configs installed by b2 but switched to library-specific variables due to... people complaining that autolink didn't work. :-)
René Ferdinand Rivera Morell wrote:
Perhaps. Although it's easy enough to find instances of people having problems with it.. ... https://stackoverflow.com/questions/44133002/boost-vs2017-linking-to-the-wro... https://stackoverflow.com/questions/51150197/cmake-generated-program-fails-t...
Note how in both these cases autolink exposes a problem with BOOST_ALL_DYN_LINK not being defined, even though the user is trying to use a Boost .dll. Had BOOST_ALL_DYN_LINK been defined correctly, autolink would have caused no issues (because it would have linked to the same library the build system did.) (Forgetting to define BOOST_ALL_DYN_LINK correctly works... some of the time. When it doesn't, it's pretty hard to figure out what's wrong.)
I've always felt that auto-linking should have been disabled by default.
But then I remember users complaining when it is: https://github.com/boostorg/boost_install/issues/54
Note how in both these cases autolink exposes a problem with BOOST_ALL_DYN_LINK not being defined, even though the user is trying to use a Boost .dll. One reason against autolink: It decides the use of static vs dynamic
For CMake it doesn't really make sense to enable it, so this is a user error. You are supposed to link to what you use and the error is usually pretty clear if that happens at all. Same as with pretty much any other library. The unified header directory might mask such mistakes though. libraries depending on a preprocessor define, good luck getting that right in a portable way... Been bitten by that too, although IIRC it was Boost.Test where it wasn't exactly clear if "it" found the static or dynamic libraries especially given that "some systems" have import libraries that go with the dynamic ones but look like static ones.
Alexander Grund wrote:
Note how in both these cases autolink exposes a problem with BOOST_ALL_DYN_LINK not being defined, even though the user is trying to use a Boost .dll.
One reason against autolink: It decides the use of static vs dynamic libraries depending on a preprocessor define, good luck getting that right in a portable way...
The preprocessor define is necessary for things to work, so autolink is correct to decide based on it.
One reason against autolink: It decides the use of static vs dynamic libraries depending on a preprocessor define, good luck getting that right in a portable way... The preprocessor define is necessary for things to work, so autolink is correct to decide based on it. I am aware of that, I consider that define basically part of the autolink feature. Point is: Somewhere that define needs to be set. And in a generic or semi-automated environment it might be hard to be correct and could make accidental mixing of shared and static libs easier which poses a potential pitfall with non-trivial consequences (e.g. ODR violations with surprising behavior)
Alexander Grund wrote:
One reason against autolink: It decides the use of static vs dynamic libraries depending on a preprocessor define, good luck getting that right in a portable way... The preprocessor define is necessary for things to work, so autolink is correct to decide based on it. I am aware of that, I consider that define basically part of the autolink feature.
No, it's not part of the autolink feature. It needs to be set properly whether or not autolink is used. This is often overlooked because MSVC works hard to make broken things work, but there are limits. If you have exported variables and not just functions (which Boost.Test has), MSVC can't fix that for you. And MinGW doesn't fix anything for you so there you have to set the macro properly. Autolink simply turns not setting the macro into a linker error.
On 29/04/2024 11:41, Peter Dimov via Boost wrote:
Alexander Grund wrote:
One reason against autolink: It decides the use of static vs dynamic libraries depending on a preprocessor define, good luck getting that right in a portable way... The preprocessor define is necessary for things to work, so autolink is correct to decide based on it. I am aware of that, I consider that define basically part of the autolink feature. No, it's not part of the autolink feature. It needs to be set properly whether or not autolink is used.
Right, the whole __declspec(dllimport/dllexport) annotation needs to be set consistently across TU's, so there is always *some* macro or other which must be set consistently or "bad things may happen". All we're really doing is giving the macro a consistent name. John.
On 4/29/24 14:30, John Maddock via Boost wrote:
On 29/04/2024 11:41, Peter Dimov via Boost wrote:
Alexander Grund wrote:
One reason against autolink: It decides the use of static vs dynamic libraries depending on a preprocessor define, good luck getting that right in a portable way... The preprocessor define is necessary for things to work, so autolink is correct to decide based on it. I am aware of that, I consider that define basically part of the autolink feature. No, it's not part of the autolink feature. It needs to be set properly whether or not autolink is used.
Right, the whole __declspec(dllimport/dllexport) annotation needs to be set consistently across TU's, so there is always *some* macro or other which must be set consistently or "bad things may happen". All we're really doing is giving the macro a consistent name.
I'll add that at least for Boost.Log defining (or not defining) BOOST_LOG_DYN_LINK/BOOST_ALL_DYN_LINK also affects library namespace. Users will simply fail to link if they didn't define the macro correctly or inconsistently between TUs.
On April 27, 2024 11:56:34 PM René Ferdinand Rivera Morell via Boost
Today I got bit by auto-linking again. As I forgot to define BOOST_ALL_NO_LIB in a CI build. I can't remember a time when auto-linking has been beneficial. And I've lost the number of times I've had to answer user questions with "Auto-linking is getting in the way. Define BOOST_ALL_NO_LIB and link normally." more times than I can count. If there was ever a benefit to auto-linking it has not surpassed the drawbacks.
What issue did you have with auto-linking?
1st of all it is Windows specific, it does not apply to any other builds - so I still need to make build properly via cmake or other tools 2nd creates unobvious dependencies that you may not expect 3rd it problematic for libraries that not always require linking (or only partial functionality requires linking), especially when 1 library uses another 4th back ago I experienced different failures or incorrect naming - don't recall why (non-standard build?) Honestly when I use Boost on Windows I always define BOOST_ALL_NO_LIB since it vastly simplifies things. The critical problem with removing this feature from the beginning is breaking backward compatibility - something that I dislike even more than this feature. Artyom
What issue did you have with auto-linking?
On 4/28/24 18:53, Artyom Beilis via Boost wrote:
1st of all it is Windows specific, it does not apply to any other builds - so I still need to make build properly via cmake or other tools
True. But given that it is a Windows-only feature, it is popular among the Windows-only users. If a user is building his project only on Windows, e.g. in a Visual Studio project, he probably doesn't care about our build system - CMake or b2 - beyond building Boost, of course, and may rightfully rely on auto-linking to pull the right Boost libraries and their respective dependencies. In a sense, the auto-linking is a substitute for the dependency tracking feature of a build system. The problems begin when auto-linking doesn't work, or disabled, as was proposed by Rene. Suddenly, the user now needs to manually figure out all the dependencies himself, and specify those in his project. This may be a non-trivial task, as usually this is not documented, and the user has to parse through source code or go through trial and error. I vaguely remember that auto-linking didn't work in some cases (I think it was related to static libraries, but I may be mistaken). That's why I asked what the problem was.
2nd creates unobvious dependencies that you may not expect
This is not really a problem, or rather not more of a problem than with a common build system such as CMake. If you use a library and link with it, you accept whatever dependencies it brings, and you don't want to track these dependencies yourself. If you do care about dependencies, you will have to inspect them either way.
3rd it problematic for libraries that not always require linking (or only partial functionality requires linking), especially when 1 library uses another
Again, not really different from CMake or b2. If you link with a shared library, you get all its dependencies regardless of the part of it you are using. If you link with a static library, you only link with what you use (on TU granularity).
4th back ago I experienced different failures or incorrect naming - don't recall why (non-standard build?)
This sounds like a configuration mismatch that was (luckily) caught as a side product of auto-linking. Personally, I'm neutral to this feature as I'm not using Windows. But my understanding is that this feature is still popular among Windows users, so us disabling or removing it will probably result in lots of complaints.
participants (7)
-
Alexander Grund
-
Andrey Semashev
-
Artyom Beilis
-
Glen Fernandes
-
John Maddock
-
Peter Dimov
-
René Ferdinand Rivera Morell