Actually cmake's support for fail tests is richer than Boost.Build's.
Here is cmake to implement simple compile fail testing where a regex on the error messages from the compiler must be matched for the test to pass:
https://github.com/ned14/quickcpplib/blob/master/cmakelib/QuickCppLibMakeSta...
Thanks ! I will look at it. It does seem like a great deal of code just to run a bunch of tests.
Only four lines or so are needed to make a compile-fail test in cmake. You can extract those out from my code easily enough. The reason for all the code is that QuickCppLib automates everything. You simply point it at a directory tree laid out in boost format, fill in a few parameters like <library name>, and whoosh it all just works with a complete package of support for docs, sanitiser tests, C++ Modules, precompiled headers, versioning, mangling, the whole kaboosh. But that's cmake innovation. And I wouldn't recommend that for Boost. You guys need something as simple and as minimal as possible which matches, as close as possible, Boost.Build. Otherwise you'll never finish converting over the build system, you'll get 60% there and that long tail will become insurmountable given it's volunteer time. We saw exactly the same happen with the git migration, it's that long tail which kills you by causing people to quit the whole project rather than put up with the never ending criticism. I personally think that a build directory per configuration design is too dissimilar to Boost.Build that such a conversion could ever be finished without injecting paid death march hours. But I won't be doing the work, so it's up to those who are to choose their design poison.
This cmake reads the regex from the first line of the fail test's source code.
There is lots of flexibility in cmake's fail test design. You can make tests which must compile but must fail to link. You can make tests which must complete within a certain time period. Lots of flexibility.
ctest generates XML of all the test results. That can be merged with JUnit XML from the individual tests easily enough. And the whole lot sent up to CDash e.g. http://my.cdash.org/index.php?project=Boost.Outcome&date=2017-07-23 where fails and successes can be drilled into, and the cause figured out.
I have a feeling that CTest is not usable for the current Boost testing system, but maybe I am wrong.
CTest I think is more than able. The sole thing I've ever found it falls badly down on is running tests on real remote devices e.g. Android ones. It really wants to run on-device, not remotely. Still, it can be beaten into submission using evil trickery. CDash probably isn't up to Boost's requirements. But the XML CTest outputs is standardised, it's not even Kitware's XML design. So the tooling currently producing the Boost regression matrix etc ought to be very easy to teach to understand CTest XML output, and thus all becomes well. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/