Boost.Test (1.34.0) - expected failures with BOOST_AUTO_TEST_SUITE()
Hello, I am trying to use BOOST_AUTO_TEST_SUITE() + BOOST_AUTO_TEST_CASE() and mark a test case for having expected failures. I am not able to use the BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES() macro inbetween the BOOST_AUTO_TEST_SUITE() and BOOST_AUTO_TEST_SUITE_END() macros, and I haven't found a way to be able to tag a test-suite-bound auto-test-case from outside that scope. Does anybody know if this is possible at all? Could it be possible, programmatically, from within the test-case? Lars J
Lars J. Aas <parawaiter <at> gmail.com> writes:
Hello,I am trying to use BOOST_AUTO_TEST_SUITE() + BOOST_AUTO_TEST_CASE()
Does anybody know if this is possible at all? Could it be possible,
and mark a test case for having expected failures. I am not able to use the BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES() macro inbetween the BOOST_AUTO_TEST_SUITE() and BOOST_AUTO_TEST_SUITE_END() macros, and I haven't found a way to be able to tag a test-suite-bound auto-test-case from outside that scope. Can you provide an error messages? programmatically, from within the test-case? Lars J You can always register some test cases manually. Gennadiy
On 9/21/07, Gennadiy Rozental <rogeeff@gmail.com> wrote:
Can you provide an error messages?
Sure. The wrapmsvc.exe compiler is just rewriting gcc-like arguments to pass to the Visual C++ tools, so this is actually with the Visual C++ 8.0compiler. /cygdrive/c/home/larsa/svn/work/Coin-3/cfg/wrapmsvc.exe -I../../Coin-3/include -I../../Coin-3/include/Inventor/annex -I../include -I../include/Inventor/annex -I../../Coin-3/testsuite -g /Z7 /Fdtestsuite.pdb /EHsc -I/cygdrive/c/Coin3D-8//include -I/cygdrive/c/Coin3D-8//include/Inventor/annex -DCOIN_DLL /MDd -g -c fieldsSoSFBitMask.cpp fieldsSoSFBitMask.cpp fieldsSoSFBitMask.cpp(29) : error C2143: syntax error : missing ';' before '<' fieldsSoSFBitMask.cpp(29) : error C2913: explicit specialization; 'SoSFBitMask_TestSuite::boost::unit_test::ut_detail::auto_tc_exp_fail' is not a specialization of a class template fieldsSoSFBitMask.cpp(29) : error C2059: syntax error : '<' fieldsSoSFBitMask.cpp(29) : error C2143: syntax error : missing ';' before '{' fieldsSoSFBitMask.cpp(29) : error C2447: '{' : missing function header (old-style formal list?) fieldsSoSFBitMask.cpp(31) : error C2039: 'auto_test_unit_registrar' : is not a member of 'SoSFBitMask_TestSuite::boost::unit_test::ut_detail' fieldsSoSFBitMask.cpp(31) : error C2146: syntax error : missing ';' before identifier 'textinput_registrar' fieldsSoSFBitMask.cpp(31) : error C4430: missing type specifier - int assumed. Note: C++ does not support default-int fieldsSoSFBitMask.cpp(31) : error C2039: 'make_test_case' : is not a member of ' SoSFBitMask_TestSuite::boost::unit_test' fieldsSoSFBitMask.cpp(31) : error C3861: 'make_test_case': identifier not found fieldsSoSFBitMask.cpp(31) : error C2027: use of undefined type 'SoSFBitMask_TestSuite::boost::unit_test::ut_detail::auto_tc_exp_fail' fieldsSoSFBitMask.cpp(29) : see declaration of 'SoSFBitMask_TestSuite::boost::unit_test::ut_detail::auto_tc_exp_fail' fieldsSoSFBitMask.cpp(31) : error C2065: 'value' : undeclared identifier fieldsSoSFBitMask.cpp(31) : error C4430: missing type specifier - int assumed. Note: C++ does not support default-int fieldsSoSFBitMask.cpp(31) : error C2078: too many initializers Lines 18-32 looks like this: 18: BOOST_AUTO_TEST_SUITE(SoSFBitMask_TestSuite); 19: 20: BOOST_AUTO_TEST_CASE(initialized) 21: { 22: SoSFBitMask field; 23: BOOST_CHECK_MESSAGE(SoSFBitMask::getClassTypeId() != SoType::badType(), 24: "SoSFBitMask class not initialized"); 25: BOOST_CHECK_MESSAGE(field.getTypeId() != SoType::badType(), 26: "SoSFBitMask class not initialized"); 27: } 28: 29: BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES(textinput, 1); 30: 31: BOOST_AUTO_TEST_CASE(textinput) 32: { [...] The code is auto-extracted from the actual source files and built separately from the class library that is tested, and I am hoping for a solution where I don't have to interpret what I extract to set up a lot of hardcoded/tweaked Boost.Test internals, but can just rely on the provided macros as is from the library source code. The reason I use test-suites and not just plain test-cases is that I do many similar tests for a lot of different classes, and none of them could have the same name if they weren't sub-sectioned into a testsuite for each class. Now, as you can see from the second error message, BOOST_AUTO_TEST_SUITE(arg) opens up 'arg' as a namespace, and then BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES() tries to specialize the boost::unit_test::ut_detail::auto_tc_exp_fail struct but fails to do so since we are inside the testsuite namespace. That is one problem. Moving the ..._EXPECTED_FAILURES() macro above the BOOST_AUTO_TEST_SUITE() macro so it happens in the root namespace makes things compile, but the predeclared "textinput_in" struct (from within the _EXPECTED_FAILURES macro) becomes a different struct type than the one that ends up in the namespace, so the auto_tc_exp_fail<> lookup just goes to the fallback when the testcase with expected failures is registered. So I did a lot of rewriting so the predeclaration ended up in the same namespace to try to fix this, but it still didn't work. It worked if I dropped the TEST_SUITE wrapper macros and just placed things in the testsuite namespace. It works all the way until I add the following line from BOOST_AUTO_TEST_SUITE() inside the testsuite namespace: BOOST_AUTO_TC_REGISTRAR( suite_name )( BOOST_TEST_SUITE( BOOST_STRINGIZE( suite_name ) ) ); - then the failures became unexpected again. So I figured that maybe a test-suite also would have to have the number of expected failures registered on its test_unit object, but I didn't get it working trying to do that either.
Does anybody know if this is possible at all? Could it be possible, programmatically, from within the test-case? Lars J
You can always register some test cases manually.
All the tests are auto-extracted from random files and compiled as separate object files, and the framework doesn't really know which files it is linked against, so it would definitely be most convenient if I could rely on the auto-registering functionality. There is no way to get hold of the "currently-being-executed" test_unit object? Maybe it would already be too late to update its expected-failures property, but it would be the simplest solution for me if that was the case - just create a macro that incremented that property by one, and use that macro just above the test expected to fail... Lars J
Ok then... Maybe it should be documented that BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES() doesn't work together with BOOST_AUTO_TEST_SUITE() usage. Lars J
Lars J. Aas <parawaiter <at> gmail.com> writes:
On 9/21/07, Gennadiy Rozental <rogeeff <at> gmail.com> wrote:
Hi, Lars, Sorry for delay. I am busy these days.
Can you provide an error messages?
Sure.
Does anybody know if this is possible at all? Could it be
Thanks for the report. Unfortunately current implementation of the expected failres support for auto test cases can't be easily modified to work within another namespace. I had to rework it and now I have proper solution. I'll try to commit it to svn once I figure out how. possible,programmatically, from within the test-case? Lars J
You can always register some test cases manually.
All the tests are auto-extracted from random files and compiled as separate object files, and the framework doesn't really know which files it is linked against, so it would definitely be most convenient if I could rely on the auto- registering functionality. There is no way to get hold of the "currently-being-executed" test_unit object?
framework has a method for an access to the currently executed test case. Gennadiy
On 9/30/07, Gennadiy Rozental <rogeeff@gmail.com> wrote:
Lars J. Aas <parawaiter <at> gmail.com> writes:
On 9/21/07, Gennadiy Rozental <rogeeff <at> gmail.com> wrote: Hi, Lars,
Sorry for delay. I am busy these days.
No problem. Lack of fast response usually means that there's no easy solution, which was answer enough for me ;) Unfortunately current implementation of the expected failres support for auto
test cases can't be easily modified to work within another namespace. I had to rework it and now I have proper solution. I'll try to commit it to svn once I figure out how.
Will this mean that the macro-API will be changed for the next version, or have you been able to handle it behind the scenes? Learning Boost.Test, I got the impression (from trying to build the examples google could dig up) that the API has really changed a lot from version to version, so when writing test code you really need to peg the boost version you use.
There is no way to get hold of the "currently-being-executed" test_unit object?
framework has a method for an access to the currently executed test case.
I'll have to dig into that then to see if I can use it. Thanks, Lars J
participants (3)
-
Gennadiy Rozental -
Lars J. Aas