Re: [boost] Enhancing regression tests summary

Beman Dawes wrote:

On Fri, Oct 31, 2014 at 11:31 AM, Rene Rivera wrote:

...

On Fri, Oct 31, 2014 at 10:18 AM, Robert Ramey wrote:

...

Adam Wulkiewicz wrote

...

Hi,

Some time ago I was writing about regression summaries enhancements, that I'd be nice to see more meaningful list of errors, etc. I asked questions, even prepared a PR for regression, to no avail.

So I developed a simple program downloading summaries, test logs and saving the enhanced result. You may check it out here: https://github.com/awulkiew/summary-enhancer Very nice - but rather than downloading and enhancing the results, wouldn't it be simpler to just enhance the current program. That is clone the the current results display tool, incorporate the enhancements, and then merge the changes into the current tool?

+1

+1

Of course it would be simpler. In fact I proposed this some time ago (http://boost.2283326.n4.nabble.com/testing-Proposal-regression-tests-results...). But in practice there wasn't enough feedback/help. At that Point I had some questions about the way how summary pages are generated. Which tools are used exactly, what parameters are passed etc. I found C++ code of a tool generating summary pages, modified it and tested on locally generated XMLs (by the run.py). I prepared a PR (https://github.com/boostorg/boost/pull/25) adding simple enhancements to check if my locally-tested program will work with the official setup. I have no access to the machine on which the pages are generated or even to the actual files sent by the runners so can't do any real testing. Even though I wasn't sure if this C++ generator is the one that had been used to generate the official summary pages, I wanted to have something that actually works. In fact I suspect that they're generated by the XSLT. It's because there are some attributes on summary pages that couldn't be generated by the C++ tool. On the other hand they can be found in the XSLT files. Furthermore, I suspect that summary pages for develop and master branches may be generated somehow differently because for some libraries the pages for master are missing (e.g. Geometry, Spirit) but this is a wild guess. So finally I decided to implement something myself, something that would be useful for me and anyone interested but didn't require anyone's involvement. In the same time I wanted to develop a proof of concept showing what I have in mind. So if you have any ideas how to move this forward and add this functionality to the officially used tool, I'm open to suggestion. Regards, Adam