Paul A. Bristow wrote:
-----Original Message-----
From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Adam Wulkiewicz E.g. I'm thinking about upgrading the Summary page http://www.boost.org/development/tests/develop/developer/summary.html to also display specific failure type, i.e. the most significant one. Right now it's all yellow. Would it be useful for someone? The hierarchy could be: comp > link > run > fail > cerr > time > file Yellow is a bit discouraging, for example for Boost.Math where the details are much more encouragingly green.
Exactly. After seing it I personally get an impression that Boost is not working in general, which of course is not true.
Is it easily possible to show the fraction of tests and/or platforms that pass?
(However I accept that the requirements are different for libraries will zillions of tests compared to those with few).
For now I prepared a PR adding the above proposal: https://github.com/boostorg/regression/pull/16 If it's merged we'll see if it helps and how much. When the summary is generated all of the tests from all libraries are checked anyway. This is done to generate the statistics shown on the top of the Summary page (Unusable, Regressions and New failures). And later when the table itself is generated. So, it wouldn't be a problem to gather some additional statistics for a library per toolset. But I don't have a clear view what should be presented and how. Well I have a few ideas: 1. In a cell put a percent of failing tests with a color of the type of the most significant one. I don't know if this would be clear enough because the numbers are in general hard to analyse on a first sight. 2. Instead of the numbers there could be some character representation of the percent of the tests that are failures like: [||| ] [--- ] or something similar. 3. Use some wierd unicode characters for the indicator. Or explicitly write fraction as: ¼, ½, ¾. 4. Or more graphically. The cells for failures probably could be divided into 2 colors vertically. The height of the yellow/orange bar inside a cell could indicate the percentage of the failures (with 4 possible heights: <=25%, <=50%, <=75% and <=100%). With this we could put the name of the most significant failure in the cell. This could be nice actually but it's hard to predict how clear would it be. It requires some testing. Do you have some specific idea? Regards, Adam