Hi all, I'm in the process of trying to remove the Boost dependency from my project, as it is causing too many difficulties for non-developers trying to run my code. This has been mostly solved by switching to C++11 and using the std:: implementations instead, however rather than rewrite all my unit tests I am hoping I can use a standalone version of Boost.Test, and distribute the code alongside my own. Unfortunately I am not quite sure how to extract the Test library out of the main Boost distribution. I have compiled "bcp" and used it to copy "test" into a new folder, but it seems to copy a huge chunk of Boost as well, including a bunch of libraries (system, thread, date_time, etc.) and I end up with 29 MB of source files, which seems wrong (especially considering my own codebase is barely 500 kB). Am I doing something wrong? I was hoping to use the header-only version of Boost.Test and this does seem to work when I #include the system-wide version of Boost (and don't link to any libraries), so it would seem that I don't need all those libraries that bcp copies. Is there any other way to access a slimmed-down standalone version of Boost.Test? Many thanks, Adam.
Is there any other way to access a slimmed-down standalone version of Boost.Test?
It would be far easier to use CATCH and wrap it with Boost test macro equivalents. This will get you started: https://github.com/ned14/quickcpplib/blob/master/include/boost/test/unit_tes.... Note it contains an even lighter weight built in subset implementation of Boost.Test. It's good enough for most users. I just ported Outcome to Boost.Outcome by swapping this for Boost.Test, and everything "just worked". Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
Is there any other way to access a slimmed-down standalone version of Boost.Test?
It would be far easier to use CATCH and wrap it with Boost test macro equivalents.
Interesting. I don't think it will be practical for this codebase though, as I'd have to rewrite all the tests. I'm using Boost.Test's manual test registration (instead of the macros) because my tests all inherit from a base class which provides the tests, with the child classes loading different file format handlers and then running the base class tests against them. It doesn't look like Catch can handle this sort of thing without preprocessor hacks, so it'd probably end up being a huge rewrite. Cheers, Adam.
On 02/10/2017 01:59, Adam Nielsen wrote:
Is there any other way to access a slimmed-down standalone version of Boost.Test?
It would be far easier to use CATCH and wrap it with Boost test macro equivalents.
Interesting. I don't think it will be practical for this codebase though, as I'd have to rewrite all the tests. I'm using Boost.Test's manual test registration (instead of the macros) because my tests all inherit from a base class which provides the tests, with the child classes loading different file format handlers and then running the base class tests against them. It doesn't look like Catch can handle this sort of thing without preprocessor hacks, so it'd probably end up being a huge rewrite.
It's pretty straightforward to refactor that design to not rely so heavily on Boost.Test. But if you don't want to, Google Test offers almost exactly the same facility. A port to that ought to be very straightforward for the above design. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
Interesting. I don't think it will be practical for this codebase though, as I'd have to rewrite all the tests. I'm using Boost.Test's manual test registration (instead of the macros) because my tests all inherit from a base class which provides the tests, with the child classes loading different file format handlers and then running the base class tests against them. It doesn't look like Catch can handle this sort of thing without preprocessor hacks, so it'd probably end up being a huge rewrite.
It's pretty straightforward to refactor that design to not rely so heavily on Boost.Test. But if you don't want to, Google Test offers almost exactly the same facility. A port to that ought to be very straightforward for the above design.
It's not that I don't want to, it's just that I see it taking much more time than I really want to invest. Extracting Boost.Test (if it can be done in less than 29 MB of source files!) is definitely the easier option. Google Test looks like it's also a library you need to install first, so I'd end up in much the same place as I am now, having to worry about installing dependencies first. I like the idea with Catch that it's header only so you don't have to worry about any of that. If I was writing things from scratch I'd be seriously looking at Catch though! Cheers, Adam.
Hi Adam, I'm fairly new to the list so take my response with a grain of salt. I think you're facing a common challenge with the release strategy of just releasing tarballs for your library and having users compile themselves. When your user base is responsible for acquiring and compiling each of your project's dependencies for themselves, each layer of dependency adds significant complexity to the use of your project. Swapping out for header-only libs, particularly great ones like catch can improve the situation a lot, but I think the fundamental problem is inherent to your current delivery strategy. So, perhaps give that some additional thought. Gerald R. Wiltse jerrywiltse@gmail.com On Sun, Oct 1, 2017 at 11:24 PM, Adam Nielsen via Boost-users < boost-users@lists.boost.org> wrote:
Interesting. I don't think it will be practical for this codebase though, as I'd have to rewrite all the tests. I'm using Boost.Test's manual test registration (instead of the macros) because my tests all inherit from a base class which provides the tests, with the child classes loading different file format handlers and then running the base class tests against them. It doesn't look like Catch can handle this sort of thing without preprocessor hacks, so it'd probably end up being a huge rewrite.
It's pretty straightforward to refactor that design to not rely so heavily on Boost.Test. But if you don't want to, Google Test offers almost exactly the same facility. A port to that ought to be very straightforward for the above design.
It's not that I don't want to, it's just that I see it taking much more time than I really want to invest. Extracting Boost.Test (if it can be done in less than 29 MB of source files!) is definitely the easier option.
Google Test looks like it's also a library you need to install first, so I'd end up in much the same place as I am now, having to worry about installing dependencies first. I like the idea with Catch that it's header only so you don't have to worry about any of that.
If I was writing things from scratch I'd be seriously looking at Catch though!
Cheers, Adam.
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
I'm fairly new to the list so take my response with a grain of salt. I think you're facing a common challenge with the release strategy of just releasing tarballs for your library and having users compile themselves. When your user base is responsible for acquiring and compiling each of your project's dependencies for themselves, each layer of dependency adds significant complexity to the use of your project. Swapping out for header-only libs, particularly great ones like catch can improve the situation a lot, but I think the fundamental problem is inherent to your current delivery strategy. So, perhaps give that some additional thought.
Yes you're definitely right there, I'm just not sure there are that many alternatives. The way I'm doing things now is "the Linux way", so it's easy for people used to that, and easy for distro maintainers to package the software, but it can be painful for end users. Part of the problems have been that for whatever reason my code seems to have requirements that are somewhat unique, so I tend to find bugs in the dependent libraries I'm using. I report these and they get fixed, but of course it means that for a while I am depending on SVN/git versions of the libraries which means end users not only have to install an extra dependency, but it's already provided by their distro and they have to remove it in order to get the SVN/git version. Eventually this goes away once a release happens and the new version makes it into the various distros, but it's still a pain. I'm not really sure of any other way to do it, aside from the static binaries Gavin suggested. This isn't really "the Linux way" but I guess if it helps people why not. At any rate, my plans at the moment are to try to build a HTTP API for my code, stick it on Amazon Lambda, and write a web version of my program. I'll still have to deal with dependencies to some extent, but Lambda recommend uploading static binaries so at least the deployment part should be relatively pain free. And a web version means no more battling with Windows, Mac and Linux versions and their various compiler quirks - what a relief that will be :) Cheers, Adam.
On 2/10/2017 10:54, Adam Nielsen wrote:
I'm in the process of trying to remove the Boost dependency from my project, as it is causing too many difficulties for non-developers trying to run my code.
This should just be a matter of providing the corresponding binary dependencies along with the executable file when you make a release. To decrease the binary dependencies you can link Boost statically rather than dynamically. (If you're talking about issues with compiling your code, then you shouldn't be expecting non-developers to compile your code in the first place. That's what release zips/packages are for.)
I'm in the process of trying to remove the Boost dependency from my project, as it is causing too many difficulties for non-developers trying to run my code.
This should just be a matter of providing the corresponding binary dependencies along with the executable file when you make a release.
To decrease the binary dependencies you can link Boost statically rather than dynamically.
(If you're talking about issues with compiling your code, then you shouldn't be expecting non-developers to compile your code in the first place. That's what release zips/packages are for.)
That's ok for the Windows release (although it still causes me plenty of headaches I'd rather not have to put up with), but unfortunately it's not so simple under Linux. For Linux, most people expect to get binary packages from their distro maintainer. For niche projects like mine, they aren't going to carry my software, so I would have to maintain up-to-date installs of a dozen or so different distros just to build the binary packages. It's much easier for me to release a source tarball or a git repo, as most non-developers can manage ./configure && make install without too much difficulty. But it's a pain trying to figure out what distro someone is using and how they have named the Boost libraries, and whether they need to install some -dev variant as well, and so on. Then you still have the problem where every time the distro upgrades Boost to a new version, all the existing binaries break and have to be recompiled. You're right in that a static build would address many of these issues, and I must admit that hadn't occurred to me, but I am trying to encourage people to contribute to the codebase so I do still want them to be able to compile the source without wasting too much time figuring out dependencies. And of course I still have to deal with it all myself, and it's not easy. I use Travis CI for my testing, which among other things allows me to make sure my code compiles on a Mac, a platform I don't otherwise have access to. But Travis has an old version of Boost preinstalled that my code won't work with, and upgrading it gets me a version compiled without C++11 support, so I can't link with it. In order to compile my code, I have to download Boost and compile that first, then start compiling my own code. But that takes so long Travis thinks the build got stuck and terminates the job, causing a build failure. I actually went so far as to set up a GitHub repo[1] that through Travis, would build a Homebrew package of Boost, and upload it as a GitHub release, which I could then pull in before compiling my own code on a Mac through Travis. It's having to jump through hoops like that that are putting me off any dependencies, not just Boost. Header-only libraries are so much easier to deal with! Until C++11 it was worth the effort to get Boost installed, but having things like shared_ptr and bind made part of C++11 was a huge step forward. Now I use hardly any native Boost code - aside from Test and Program_Options - so it's at the point now where swapping these out for alternatives is starting to look like the easier option. Cheers, Adam. [1] https://github.com/Malvineous/travis-homebrew-bottle
On 2/10/2017 16:09, Adam Nielsen wrote:
For Linux, most people expect to get binary packages from their distro maintainer. For niche projects like mine, they aren't going to carry my software, so I would have to maintain up-to-date installs of a dozen or so different distros just to build the binary packages.
It's much easier for me to release a source tarball or a git repo, as most non-developers can manage ./configure && make install without too much difficulty. But it's a pain trying to figure out what distro someone is using and how they have named the Boost libraries, and whether they need to install some -dev variant as well, and so on.
It is possible to add some lines to your configure script to verify that a library -dev package is installed, which errors out with a reasonably clear error (which hopefully makes the users think to go install the package from their normal package manager). It's a one-liner for most libraries, but sadly it's not quite that simple for Boost since it doesn't provide pkg-config files out of the box. I'm sure there's probably still a way to do it though -- in particular if you assume that the Boost includes are in the system path then failing anything better you could just do a header-existence test (see AC_CHECK_HEADERS).
It's having to jump through hoops like that that are putting me off any dependencies, not just Boost. Header-only libraries are so much easier to deal with!
While that's understandable, having no dependencies means that (in most non-trivial work) you're duplicating things that already exist elsewhere, which can itself be a source of bugs and maintenance headaches (of course, dependencies can be maintenance headaches too). It's fairly common though, and one of the sources of NIH syndrome. Various solutions have been made for this sort of thing (source package managers and the like) but inevitably it just forms a collection of competing meta-dependencies.
It is possible to add some lines to your configure script to verify that a library -dev package is installed, which errors out with a reasonably clear error (which hopefully makes the users think to go install the package from their normal package manager).
It's a one-liner for most libraries, but sadly it's not quite that simple for Boost since it doesn't provide pkg-config files out of the box. I'm sure there's probably still a way to do it though -- in particular if you assume that the Boost includes are in the system path then failing anything better you could just do a header-existence test (see AC_CHECK_HEADERS).
I'm using AX_BOOST_* (part of the "autoconf-archive" package on my system) which works quite well. It seems more reliable than the previous boost.m4 I was using, which itself wasn't too bad at all. In my experience identifying the missing libraries hasn't been too much of a problem, it's mostly been about how to figure out what needs to be installed on a given system. Boost hasn't been too bad here for the end users (it's mostly me struggling any time I want to compile outside my local machine) but some of the other dependencies have been much worse (any time I have to depend on an SVN version to get a critical bug fix.)
It's having to jump through hoops like that that are putting me off any dependencies, not just Boost. Header-only libraries are so much easier to deal with!
While that's understandable, having no dependencies means that (in most non-trivial work) you're duplicating things that already exist elsewhere, which can itself be a source of bugs and maintenance headaches (of course, dependencies can be maintenance headaches too).
It's fairly common though, and one of the sources of NIH syndrome.
Various solutions have been made for this sort of thing (source package managers and the like) but inevitably it just forms a collection of competing meta-dependencies.
Yes you're right, either way you have maintenance overhead. However from where I'm sitting now the grass looks a lot greener on the other side, and you know what that means... Ask me again in a couple of years whether it was worth it or not :) In some ways I see the industry in general moving towards this trend. Header-only libraries mean you can lock in a version that works with your code, and upgrades won't happen behind your back, forcing you to adjust your own code to make sure it works with the latest version. Docker is now quite popular, and follows the same philosophy - locking in versions and configurations of dependencies so that you don't need to keep reconfiguring things all the time as new library versions are release. Node.js and other scripting languages now favour locking in specific versions of their dependencies for the same reason. You still have to update your code when you grab new versions of the libraries, but you can choose to do that when you have the time to spend on it, you're not forced into it just as a deadline approaches. So I see going header-only for my C++ dependencies as perfectly fitting in with this trend! Cheers, Adam.
On 01/10/2017 22:54, Adam Nielsen via Boost-users wrote:
Hi all,
I'm in the process of trying to remove the Boost dependency from my project, as it is causing too many difficulties for non-developers trying to run my code.
This has been mostly solved by switching to C++11 and using the std:: implementations instead, however rather than rewrite all my unit tests I am hoping I can use a standalone version of Boost.Test, and distribute the code alongside my own.
Unfortunately I am not quite sure how to extract the Test library out of the main Boost distribution. I have compiled "bcp" and used it to copy "test" into a new folder, but it seems to copy a huge chunk of Boost as well, including a bunch of libraries (system, thread, date_time, etc.) and I end up with 29 MB of source files, which seems wrong (especially considering my own codebase is barely 500 kB).
Try: bcp --boost=boost-root-directory boost/test output-path You still get a lot of dependencies - including chrono and system which are pulled in from libs/test/build/Jamfile.v2 via a dependency on Boost.Timer. Most of the extra dependencies are created because bcp follows every conceivable preprocessor path - so get dozens of dependencies only required by broken compilers etc. HTH, John. --- This email has been checked for viruses by AVG. http://www.avg.com
Le 01.10.17 à 23:54, Adam Nielsen via Boost-users a écrit :
Hi all,
I'm in the process of trying to remove the Boost dependency from my project, as it is causing too many difficulties for non-developers trying to run my code.
This has been mostly solved by switching to C++11 and using the std:: implementations instead, however rather than rewrite all my unit tests I am hoping I can use a standalone version of Boost.Test, and distribute the code alongside my own.
Unfortunately I am not quite sure how to extract the Test library out of the main Boost distribution. I have compiled "bcp" and used it to copy "test" into a new folder, but it seems to copy a huge chunk of Boost as well, including a bunch of libraries (system, thread, date_time, etc.) and I end up with 29 MB of source files, which seems wrong (especially considering my own codebase is barely 500 kB).
Am I doing something wrong? I was hoping to use the header-only version of Boost.Test and this does seem to work when I #include the system-wide version of Boost (and don't link to any libraries), so it would seem that I don't need all those libraries that bcp copies.
Is there any other way to access a slimmed-down standalone version of Boost.Test?
Hi, Currently this is not possible. But (as one of the boost.test maintainer) I see this as an important feature for boost.test, so I'll be working on it. The main chunks of dependencies boost.test relies on are: - boost.config that indicates many of the compiler capabilities. I believe we can come up a header that the user can adapt from a general one (as some autoconf like tool would do) - boost.mpl that can be replaced by c++11 features - boost.preprocessor that is heavily used for getting the macro magics. Apart from that, there are many utility functions, as John said for instance boost.chrono and boost.timer. I do not have a clear strategy for this, apart from duplicating those within boost.test. I understand that this is an urgent feature for you, although I cannot commit to a precise date, I will work on this. Best, Raffi
Many thanks, Adam.
participants (6)
-
Adam Nielsen
-
Gavin Lambert
-
Gerald Wiltse
-
John Maddock
-
Niall Douglas
-
Raffi Enficiaud