On 3 Jan 2015 at 10:36, Tom Kent wrote:
I can try to set things up like this again, but there are two reasons I was thinking about changing: 1) It seems that a lot of the gcc/clang versions have the same pass/fail results as the other versions of that compiler, so we're not adding a ton (though we may be adding the crucial points where a compiler change breaks something...is this something that devs have seen in practice?)
For my automated testing of parts of Boost I only bother testing builds with multiple compiler versions, and I don't bother with the unit testing which is generally only done with one clang (thread and UB sanitiser) and one gcc (right now usually 4.8). The build testing across compilers is very valuable, and you'd be amazed how often some random version breaks e.g. GCC 4.7 only. I've only ever seen unit testing break once on a single compiler only, that was on GCC 4.8.0. Obviously different code bases may experience different things.
2) The new version of my runners (Ubuntu 14.04) doesn't easily support the old gcc versions, it would take a lot of effort to get some of the old ones running. Clang is even worse...it is much more difficult to get even two versions running side by side now.
That's because the LLVM maintainer of the Ubuntu repos only added multi-version support from 3.5 onwards or so due to a feature request from me. To get earlier versions of clang which coexist happily on the same Ubuntu 14.04, look into the hATrayflood repo on launchpad. He's hacked clangs right back to 2.9 through to 3.6, and they all coexist. I also have GCCs back to 4.6 all coexisting, I think that's from the Ubuntu toolchain repo on launchpad. I don't need earlier GCCs, so can't say much about those.
Instead of running lots of different versions, I was thinking about running various compatibility options of the two main compilers...I've seen some other test runners with things like libc++, c++11, c++14, etc. My real question, would developers be better served by these options than different versions? Would they prefer both with longer revisit times between each test type? Other thoughts?
For Boost.Thread you can see the build dashboard at https://github.com/boostorg/thread/. As you'll see, we test 03, 11 and 14 as we have been surprised by random breakage. Instead of testing libc++ on Linux, I'd suggest a FreeBSD 10 runner which is also surprisingly good at catching problems on OS X. I've found you can simply use the FreeBSD 10 default compiler, though note that the auto detection logic in bootstrap.sh is broken on FreeBSD and has been for some time (force the toolset to clang, it'll work). Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/