Edward Diener wrote:
On 5/11/2015 12:09 PM, John Maddock wrote:
On 11/05/2015 16:46, Adam Wulkiewicz wrote:
I propose to go further with this and to mark the compilation failures for which we know the reason of the failure in a special way: - "file" - file too big (reported on Windows, e.g. http://www.boost.org/development/tests/develop/developer/output/MinGW-w64-4-...)
- "time" - time limit exceeded (e.g. http://www.boost.org/development/tests/develop/developer/output/teeks99-03b-...)
I saw them in many libraries and I think it would be very useful for libraries having many tests like Math or Geometry.
+1, time limit exceptions are a big issue for the Math and Multiprecision libs... and yes, I've already spent a lot of time refactoring to make them smaller, but they still persist. I suspect many of these are caused by virtual machines with insufficient RAM allocated, that then end up thrashing the HD: many of the tests that time out compile really pretty quickly here (even on obsolete hardware).
I see some time-out errors in VMD testing. The code makes heavy use of the preprocessor but no single test should last more than 5 minutes ( 300 seconds ).
I also would love to see time limit exceeded marked differently, and maybe some way of having regression testers to increase the time limit when those failures occur.
I think it is great work to have the regression test matrix marked more specifically.
I created a PR with the change: https://github.com/boostorg/regression/pull/13 Feel free to write your suggestions. Regards, Adam