[testing][regression] Heads up: compilation warnings are available for passing tests.
Hi, For passing tests which are expected to pass, if warnings are generated during the compilation they can be seen after clicking on the "pass" link. If the link isn't there it means that there were no warnings. I hope it will allow us to improve the quality of the libraries even more. Regards, Adam
On Wed, Jun 3, 2015 at 7:26 AM, Adam Wulkiewicz
For passing tests which are expected to pass, if warnings are generated during the compilation they can be seen after clicking on the "pass" link. If the link isn't there it means that there were no warnings. I hope it will allow us to improve the quality of the libraries even more.
I noticed one serious problem with this change.. When there are warnings on a test the has the always show output option on it.. The test output is not available (it usually would be available from that same pass link). For an example of the problem take a look at the Predef < http://www.boost.org/development/tests/develop/developer/predef.html> results for any of the info_* tests. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
2015-06-03 16:33 GMT+02:00 Rene Rivera
On Wed, Jun 3, 2015 at 7:26 AM, Adam Wulkiewicz
wrote:
For passing tests which are expected to pass, if warnings are generated during the compilation they can be seen after clicking on the "pass" link. If the link isn't there it means that there were no warnings. I hope it will allow us to improve the quality of the libraries even more.
I noticed one serious problem with this change.. When there are warnings on a test the has the always show output option on it.. The test output is not available (it usually would be available from that same pass link). For an example of the problem take a look at the Predef < http://www.boost.org/development/tests/develop/developer/predef.html> results for any of the info_* tests.
Sorry, but I don't understand the nature of the problem. The output for a passing test was not available before. Would you prefer to see the full output for the passing test if warnings were detected? Generating separate file for each passing test with warnings would result in a massive files bloat. I tested it for gcc6.0 and the disk usage increased from 30MB to 120MB because a lot of small files was generated. Of course I could do it if that was acceptable. Or is it something else? Regards, Adam
On Wed, Jun 3, 2015 at 4:11 PM, Adam Wulkiewicz
2015-06-03 16:33 GMT+02:00 Rene Rivera
: On Wed, Jun 3, 2015 at 7:26 AM, Adam Wulkiewicz < adam.wulkiewicz@gmail.com
wrote:
For passing tests which are expected to pass, if warnings are generated during the compilation they can be seen after clicking on the "pass" link. If the link isn't there it means that there were no warnings. I hope it will allow us to improve the quality of the libraries even more.
I noticed one serious problem with this change.. When there are warnings on a test the has the always show output option on it.. The test output is not available (it usually would be available from that same pass link). For an example of the problem take a look at the Predef < http://www.boost.org/development/tests/develop/developer/predef.html> results for any of the info_* tests.
Sorry, but I don't understand the nature of the problem.
The output for a passing test was not available before.
Would you prefer to see the full output for the passing test if warnings were detected? Generating separate file for each passing test with warnings would result in a massive files bloat. I tested it for gcc6.0 and the disk usage increased from 30MB to 120MB because a lot of small files was generated. Of course I could do it if that was acceptable.
Or is it something else?
Something else.. There's this build & testing feature of allowing one to mask tests as always providing the full test output regardless of it passing (or failing). Which one can activate by adding "<test-info>always_show_run_output" to the test requirements. For example see here https://github.com/boostorg/predef/blob/develop/test/build.jam#L52. The effect was that the pass cell would link to a page with the full compile+link+run output for the test. That functionality is now totally gone (even for passing tests without warnings that have the always_show_run_output option active). -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
2015-06-03 23:27 GMT+02:00 Rene Rivera
On Wed, Jun 3, 2015 at 4:11 PM, Adam Wulkiewicz
wrote:
2015-06-03 16:33 GMT+02:00 Rene Rivera
: On Wed, Jun 3, 2015 at 7:26 AM, Adam Wulkiewicz < adam.wulkiewicz@gmail.com
wrote:
For passing tests which are expected to pass, if warnings are generated during the compilation they can be seen after clicking on the "pass" link. If the link isn't there it means that there were no warnings. I hope it will allow us to improve the quality of the libraries even more.
I noticed one serious problem with this change.. When there are warnings on a test the has the always show output option on it.. The test output is not available (it usually would be available from that same pass link). For an example of the problem take a look at the Predef < http://www.boost.org/development/tests/develop/developer/predef.html> results for any of the info_* tests.
Sorry, but I don't understand the nature of the problem.
The output for a passing test was not available before.
Would you prefer to see the full output for the passing test if warnings were detected? Generating separate file for each passing test with warnings would result in a massive files bloat. I tested it for gcc6.0 and the disk usage increased from 30MB to 120MB because a lot of small files was generated. Of course I could do it if that was acceptable.
Or is it something else?
Something else.. There's this build & testing feature of allowing one to mask tests as always providing the full test output regardless of it passing (or failing). Which one can activate by adding "<test-info>always_show_run_output" to the test requirements. For example see here < https://github.com/boostorg/predef/blob/develop/test/build.jam#L52>. The effect was that the pass cell would link to a page with the full compile+link+run output for the test. That functionality is now totally gone (even for passing tests without warnings that have the always_show_run_output option active).
It should be fixed now. For these tests the full output is always generated. If warnings were also detected the cell color is different. Let me know if there are other issues which must be handled. Regards, Adam
AMDG On 06/03/2015 03:11 PM, Adam Wulkiewicz wrote:
2015-06-03 16:33 GMT+02:00 Rene Rivera
: *When there are warnings on a test the has the always show output option on it.* (emphasis mine)
Sorry, but I don't understand the nature of the problem.
The output for a passing test was not available before.
Usually not, but it can be enabled for specific tests. In Christ, Steven Watanabe
2015-06-03 14:26 GMT+02:00 Adam Wulkiewicz
Hi,
For passing tests which are expected to pass, if warnings are generated during the compilation they can be seen after clicking on the "pass" link. If the link isn't there it means that there were no warnings. I hope it will allow us to improve the quality of the libraries even more.
This is really good. How about changing the tint of the green color a bit for these cases? Regards, &rzej
2015-06-03 17:27 GMT+02:00 Andrzej Krzemienski
2015-06-03 14:26 GMT+02:00 Adam Wulkiewicz
: Hi,
For passing tests which are expected to pass, if warnings are generated during the compilation they can be seen after clicking on the "pass" link. If the link isn't there it means that there were no warnings. I hope it will allow us to improve the quality of the libraries even more.
This is really good. How about changing the tint of the green color a bit for these cases?
Initially I thought that it might be distracting. But indeed it's more clear this way. So it's done. Regards, Adam
2015-06-05 0:14 GMT+02:00 Adam Wulkiewicz
2015-06-03 17:27 GMT+02:00 Andrzej Krzemienski
: 2015-06-03 14:26 GMT+02:00 Adam Wulkiewicz
: Hi,
For passing tests which are expected to pass, if warnings are generated during the compilation they can be seen after clicking on the "pass" link. If the link isn't there it means that there were no warnings. I hope it will allow us to improve the quality of the libraries even more.
This is really good. How about changing the tint of the green color a bit for these cases?
Initially I thought that it might be distracting. But indeed it's more clear this way. So it's done.
Regards, Adam
Adam, Would it be possible to keep the passing tests with warnings green, and only *slightly* change its tint? Currently they are white, and it sends a strong signal that something is wrong with all these tests. I tried to follow the links, but the majority of warnings are silly, buggy, false-positive, or inadequate. For instance "warning: rvalue references are a C++11 extension" I never intend to fix them, and at the same time, I do want to send the message that the library is (or becomes) bug-free with the help of green color. Sorry for changing my mind like that. Regards, &rzej
2015-06-05 15:28 GMT+02:00 Andrzej Krzemienski
2015-06-05 0:14 GMT+02:00 Adam Wulkiewicz
: 2015-06-03 17:27 GMT+02:00 Andrzej Krzemienski
: 2015-06-03 14:26 GMT+02:00 Adam Wulkiewicz
Hi,
For passing tests which are expected to pass, if warnings are generated during the compilation they can be seen after clicking on the "pass" link. If the link isn't there it means that there were no warnings. I hope it will allow us to improve the quality of the libraries even more.
This is really good. How about changing the tint of the green color a bit for these cases?
Initially I thought that it might be distracting. But indeed it's more clear this way. So it's done.
Regards, Adam
Adam, Would it be possible to keep the passing tests with warnings green, and only *slightly* change its tint? Currently they are white, and it sends a strong signal that something is wrong with all these tests. I tried to follow the links, but the majority of warnings are silly, buggy, false-positive, or inadequate. For instance "warning: rvalue references are a C++11 extension" I never intend to fix them, and at the same time, I do want to send the message that the library is (or becomes) bug-free with the help of green color.
Sorry for changing my mind like that.
The warnings should be marked with yelowish green color. You were probably seing the tests for master baranch. For a reason unknown to me more time is needed to update the CSS file for master regression results. Please see it again. Is the color ok for you? Or is it too different from the lightgreen used for passing tests? You could also create a PR in Regression setting the color in the CSS file. Regards, Adam
2015-06-05 20:34 GMT+02:00 Adam Wulkiewicz
2015-06-05 15:28 GMT+02:00 Andrzej Krzemienski
: 2015-06-05 0:14 GMT+02:00 Adam Wulkiewicz
: 2015-06-03 17:27 GMT+02:00 Andrzej Krzemienski
: 2015-06-03 14:26 GMT+02:00 Adam Wulkiewicz < adam.wulkiewicz@gmail.com :
Hi,
For passing tests which are expected to pass, if warnings are generated during the compilation they can be seen after clicking on the "pass" link. If the link isn't there it means that there were no warnings. I hope it will allow us to improve the quality of the libraries even more.
This is really good. How about changing the tint of the green color a bit for these cases?
Initially I thought that it might be distracting. But indeed it's more clear this way. So it's done.
Regards, Adam
Adam, Would it be possible to keep the passing tests with warnings green, and only *slightly* change its tint? Currently they are white, and it sends a strong signal that something is wrong with all these tests. I tried to follow the links, but the majority of warnings are silly, buggy, false-positive, or inadequate. For instance "warning: rvalue references are a C++11 extension" I never intend to fix them, and at the same time, I do want to send the message that the library is (or becomes) bug-free with the help of green color.
Sorry for changing my mind like that.
The warnings should be marked with yelowish green color. You were probably seing the tests for master baranch. For a reason unknown to me more time is needed to update the CSS file for master regression results. Please see it again.
Is the color ok for you? Or is it too different from the lightgreen used for passing tests? You could also create a PR in Regression setting the color in the CSS file.
Ok, after F5 in my browser, I can see the green color. It looks perfect! Thank you for doing it, I have already found one genuine warning in my library. Regards, &rzej
participants (4)
-
Adam Wulkiewicz
-
Andrzej Krzemienski
-
Rene Rivera
-
Steven Watanabe