On 05.11.19 17:03, Paul A Bristow via Boost wrote:
[snip]
Actually I like the change - did we need 'framework'?
"#me too" but this is not the right time to introduce any breaking changes, especially without notice. It is used nowhere in boost.test but I have not checked within boost (and certainly not outside) before making that change (that was also unrelated to the issue being solved). Some legacy code, a proper deprecation should be emitted, at least in the change logs. I do not know if namespaces can have deprecation attributes, that would be handy.
Anyone using this is perhaps relying on a detail?
(The need for it in the Math test case was to ensure that the BOOST_TEST_MESSAGE("my_message") were actually visible. I think it would be better if messages were visible by default, not something one needs to ask for specially. I also was unconvinced the setting the environment variable BOOST_TEST_MESSAGE worked but I'll check this out and raise a bug report.)
I've finally found this exchange explaining the behaviour.
http://boost.2283326.n4.nabble.com/test-Problem-with-BOOST-TEST-MESSAGE-td46... 8.html
I think it is very counter-intuitive that messages only emerge to the normal output if there is an error or --log_level="message"
And that
#define BOOST_TEST_LOG_LEVEL "message"
does not appear to have the same effect as --log_level="message"
The current documentation never mentions the usage through a macro as you are using in the snippet above. Please see below about the relevant sections.
which is also surprising, and in my book a buglet?
I suggest that this behaviour should be changed so that BOOST_TEST_MESSAGE *always* produces output.
BOOST_AUTO_TEST_CASE(test_message) { BOOST_TEST_MESSAGE("\nTest BOOST_TEST_MESSAGE.\n"); } // BOOST_AUTO_TEST_CASE(test_message)
boost_test_message_test.vcxproj -> I:\Cpp\math\x64\Release\boost_test_message_test.exe
should output
Running 1 test case...
Test BOOST_TEST_MESSAGE.
*** No errors detected
I tend to disagree with that. What I am expecting from my tests is the report of "true positives", ie. assertions that fails for good reasons like regressions. All the rest I consider as information that distracts, or information printed only for non-regular/normal use like debugging. I would for instance put a message or a warning that I can easily activate if something goes wrong and needs my attention, until I fix the issue. Then I would go back to the normal mode. This is also the approach taken by tools like ctest or b2: normal logging level prints only a very high level status, verbose logs are redirected to file for potential inspection, details are printed on error. The behaviour of Boost.Test in that regard is documented in various places, mostly in the runtime arguments (command line) and in the logging part: * https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/test_out... * https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/test_out... * https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/test_out... (even with its own section) * https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/utf_refe... (indicates the environment variable as well) * https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/runtime_... although the section about environment variables can be clarified
Otherwise I am sure I will not be the only person confused.
At the very least the documentation should highlight that ones expectation may not match what actually happens.
I have to say that the current doc is sometimes more than complete with some redundant parts and confusing information. If you think of a better way to organize it or a better wording ("me not English native"), please let me know, I would be happy to improve it. Thanks! Raffi PS: and sorry about breaking your code again so close to the beta.