ATTN: Boost GSoC mentors and students: Continuous Integration for GSoC thoughts
Dear Boost, I spent the weekend experimenting with getting CI set up for the async file i/o library which will hopefully become Boost.AFIO through Paul's GSoC. I tried a number of free-for-open-source CI providers which integrate tightly with GitHub, and eventually settled with Travis CI. You can see the last commit CI results at https://travis-ci.org/ned14/TripleGit. As much as Travis CI has a primitive and clunky UI, unlike the other CIs it's based on snapshotting VMs with root privs which gives you unparalleled flexibility for testing config, so I was able to configure this to happen with each and every git push to github: 1. Test that code builds with GCC 4.6 and clang 3.2. 2. Test that unit and functional tests execute without error when built with GCC 4.6 and clang 3.2. 3. Run unit and functional tests with valgrind --tool=memcheck and test that no memory errors nor memory leaks occur with binaries built with GCC 4.6 and clang 3.2. 4. Install GCC 4.8 and compile everything using -fsanitize=thread (i.e. instrument using the port of clang's ThreadSanitizer v2 to GCC), then execute a full set of unit and functional tests looking for race conditions and if any, fail. You'll note I've marked that test as allowed to fail, because Boost v1.53 has a number of known race conditions when used as C++11 which are hopefully fixed in v1.54 :) Personally speaking, I think this depth and degree of CI is *invaluable* to modern software development practice. So the question becomes: are the Boost GSoC mentors and students interested in setting up CI for this year's GSoC projects, or at least the ones hosted on github at https://github.com/BoostGSOC? In case you're wondering how hard it is to set up Travis CI for your github project, you just need one file called .travis.yml. You can see the one I wrote at https://github.com/ned14/TripleGit/blob/master/.travis.yml. Thoughts? Niall --- Opinions expressed here are my own and do not necessarily represent those of BlackBerry Inc.
On Mon, Jun 3, 2013 at 4:59 PM, Niall Douglas
1. Test that code builds with GCC 4.6 and clang 3.2.
2. [...] with GCC 4.6 and clang 3.2.
3. [...] GCC 4.6 and clang 3.2.
Just curious: is Visual Studio support not included because not deemed important? Or not supported by this particular lib? Or not supported by Travis CI for some reason? (licensing?) Thanks, --DD
2013/6/4 Dominique Devienne
On Mon, Jun 3, 2013 at 4:59 PM, Niall Douglas
wrote:
1. Test that code builds with GCC 4.6 and clang 3.2.
2. [...] with GCC 4.6 and clang 3.2.
3. [...] GCC 4.6 and clang 3.2.
Just curious: is Visual Studio support not included because not deemed important? Or not supported by this particular lib? Or not supported by Travis CI for some reason? (licensing?)
Not supported by the platform. Travis-CI's build slaves run Ubuntu 12.04.
2013/6/4 Daniel Pfeifer
2013/6/4 Dominique Devienne
On Mon, Jun 3, 2013 at 4:59 PM, Niall Douglas
wrote:
1. Test that code builds with GCC 4.6 and clang 3.2.
2. [...] with GCC 4.6 and clang 3.2.
3. [...] GCC 4.6 and clang 3.2.
Just curious: is Visual Studio support not included because not deemed important? Or not supported by this particular lib? Or not supported by Travis CI for some reason? (licensing?)
Not supported by the platform. Travis-CI's build slaves run Ubuntu 12.04.
For minimal Windows support, you may go this route: * Install these packages: binutils-mingw-w64-i686 gcc-mingw-w64-i686 g++-mingw-w64-i686 wine * CC=i686-w64-mingw32-gcc CXX=i686-w64-mingw32-g++ * Link everything static, use the link flags -static-libgcc -static-libstdc++ * run tests with wine. I have set this up once with CMake, but I don't use it anymore. I can dig up an example if someone is interested. Cheers, Daniel
Daniel Pfeifer
2013/6/4 Daniel Pfeifer
2013/6/4 Dominique Devienne
On Mon, Jun 3, 2013 at 4:59 PM, Niall Douglas
wrote:
1. Test that code builds with GCC 4.6 and clang 3.2.
2. [...] with GCC 4.6 and clang 3.2.
3. [...] GCC 4.6 and clang 3.2.
Just curious: is Visual Studio support not included because not deemed important? Or not supported by this particular lib? Or not supported by Travis CI for some reason? (licensing?)
Not supported by the platform. Travis-CI's build slaves run Ubuntu 12.04.
For minimal Windows support, you may go this route:
* Install these packages: binutils-mingw-w64-i686 gcc-mingw-w64-i686 g++-mingw-w64-i686 wine * CC=i686-w64-mingw32-gcc CXX=i686-w64-mingw32-g++ * Link everything static, use the link flags -static-libgcc -static-libstdc++ * run tests with wine.
I have set this up once with CMake, but I don't use it anymore. I can dig up an example if someone is interested.
I for one would love to see how you got this working. Boost is always my stumbling block with cross-compiling for Windows on Linux with CMake. I can never get FindBoost to find Boost. Alex -- Swish - Easy SFTP for Windows Explorer (http://www.swish-sftp.org)
2013/6/11 Alexander Lamaison
Daniel Pfeifer
writes: 2013/6/4 Daniel Pfeifer
2013/6/4 Dominique Devienne
On Mon, Jun 3, 2013 at 4:59 PM, Niall Douglas
wrote:
1. Test that code builds with GCC 4.6 and clang 3.2.
2. [...] with GCC 4.6 and clang 3.2.
3. [...] GCC 4.6 and clang 3.2.
Just curious: is Visual Studio support not included because not deemed important? Or not supported by this particular lib? Or not supported by Travis CI for some reason? (licensing?)
Not supported by the platform. Travis-CI's build slaves run Ubuntu 12.04.
For minimal Windows support, you may go this route:
* Install these packages: binutils-mingw-w64-i686 gcc-mingw-w64-i686 g++-mingw-w64-i686 wine * CC=i686-w64-mingw32-gcc CXX=i686-w64-mingw32-g++ * Link everything static, use the link flags -static-libgcc -static-libstdc++ * run tests with wine.
I have set this up once with CMake, but I don't use it anymore. I can dig up an example if someone is interested.
I for one would love to see how you got this working. Boost is always my stumbling block with cross-compiling for Windows on Linux with CMake. I can never get FindBoost to find Boost.
Hi Alex, I have set up a minimal example here: https://github.com/purpleKarrot/CrossTest cheers, Daniel
Daniel Pfeifer
2013/6/11 Alexander Lamaison
Daniel Pfeifer
writes: 2013/6/4 Daniel Pfeifer
2013/6/4 Dominique Devienne
On Mon, Jun 3, 2013 at 4:59 PM, Niall Douglas
wrote:
1. Test that code builds with GCC 4.6 and clang 3.2.
2. [...] with GCC 4.6 and clang 3.2.
3. [...] GCC 4.6 and clang 3.2.
Just curious: is Visual Studio support not included because not deemed important? Or not supported by this particular lib? Or not supported by Travis CI for some reason? (licensing?)
Not supported by the platform. Travis-CI's build slaves run Ubuntu 12.04.
For minimal Windows support, you may go this route:
* Install these packages: binutils-mingw-w64-i686 gcc-mingw-w64-i686 g++-mingw-w64-i686 wine * CC=i686-w64-mingw32-gcc CXX=i686-w64-mingw32-g++ * Link everything static, use the link flags -static-libgcc -static-libstdc++ * run tests with wine.
I have set this up once with CMake, but I don't use it anymore. I can dig up an example if someone is interested.
I for one would love to see how you got this working. Boost is always my stumbling block with cross-compiling for Windows on Linux with CMake. I can never get FindBoost to find Boost.
Hi Alex,
I have set up a minimal example here: https://github.com/purpleKarrot/CrossTest
Thanks Daniel. I'm using your setup now but the last step is the one I'm struggling with, as usual: compiled Boost libraries. Have you any advice for cross-compiling these as part of a Boost client project's CMake build? Alex -- Swish - Easy SFTP for Windows Explorer (http://www.swish-sftp.org)
2013/6/3 Niall Douglas
Dear Boost,
I spent the weekend experimenting with getting CI set up for the async file i/o library which will hopefully become Boost.AFIO through Paul's GSoC. I tried a number of free-for-open-source CI providers which integrate tightly with GitHub, and eventually settled with Travis CI. You can see the last commit CI results at https://travis-ci.org/ned14/TripleGit.
As much as Travis CI has a primitive and clunky UI, unlike the other CIs it's based on snapshotting VMs with root privs which gives you unparalleled flexibility for testing config, so I was able to configure this to happen with each and every git push to github:
1. Test that code builds with GCC 4.6 and clang 3.2.
2. Test that unit and functional tests execute without error when built with GCC 4.6 and clang 3.2.
3. Run unit and functional tests with valgrind --tool=memcheck and test that no memory errors nor memory leaks occur with binaries built with GCC 4.6 and clang 3.2.
4. Install GCC 4.8 and compile everything using -fsanitize=thread (i.e. instrument using the port of clang's ThreadSanitizer v2 to GCC), then execute a full set of unit and functional tests looking for race conditions and if any, fail. You'll note I've marked that test as allowed to fail, because Boost v1.53 has a number of known race conditions when used as C++11 which are hopefully fixed in v1.54 :)
Personally speaking, I think this depth and degree of CI is *invaluable* to modern software development practice.
I could not agree more. One thing you may want to add is coverage analysis with https://coveralls.io/: If you build with gcc, pass "-fprofile-arcs -ftest-coverage" both to the compiler and the linker. After running the tests, you can analyze the coverage with gcov and send it to coveralls. To collect and upload, I use the following script: https://github.com/purpleKarrot/Karrot/blob/develop/test/coveralls.in (Note that gcov 4.6 does not support all flags that I use in the script, I use it wiht 4.8 only). cheers, Daniel
participants (4)
-
Alexander Lamaison
-
Daniel Pfeifer
-
Dominique Devienne
-
Niall Douglas