{Boost-Test] How to specify order test suites are run.
I'm running Boost 1_58_0 on Windows 7.1, 64-bit, with MinGW 4.9.2, 64-bit. I've just started writing tests for my project and have one test suite working. I'm using BOOST_AUTO_TEST_CASE and BOOST_AUTO_TEST_SUITE. There will be many more test suites to come. They are not LOGICALLY dependent on each other, but if some fail, the other tests will be a waste of time. Therefore, I would like to be able to specify the order in which they are run. Is there a way to do that, or does choosing BOOST_AUTO_TEST_SUITE take it all out of my hands? Merrill Cornish
Hi Merrill, from what you describe, using the BOOST_REQUIRE asserts should do the job. See the documentation: http://www.boost.org/doc/libs/1_58_0/libs/test/doc/html/utf/testing-tools.ht... The test execution aborts, if predicate evaluated in the BOOST_REQUIRE level tools returns false. Best - Peter On 06/01/2015 12:18 AM, Merrill Cornish wrote:
I'm running Boost 1_58_0 on Windows 7.1, 64-bit, with MinGW 4.9.2, 64-bit.
I've just started writing tests for my project and have one test suite working. I'm using BOOST_AUTO_TEST_CASE and BOOST_AUTO_TEST_SUITE. There will be many more test suites to come. They are not LOGICALLY dependent on each other, but if some fail, the other tests will be a waste of time. Therefore, I would like to be able to specify the order in which they are run.
Is there a way to do that, or does choosing BOOST_AUTO_TEST_SUITE take it all out of my hands?
Merrill Cornish _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Peter, Thanks for the reply. I am familiar with the REQUIRE family of checks. However, that kills everything at the lowest level immediately. I have a more subtle problem. Here's an example: For a class being tested, I have a test case to validate the constructor. There are many individual checks in that test case. Just because some individual checks might fail is not a reason to abort the entire constructor test. So, I used CHECKs rather than REQUIRE. Continuing on, there are a number of other test cases for other functionality in the class. However, if the constructor can't be trusted, I could end up with a wild set of "errors" reported in the later test cases. If functionality A is needed by B, B is needed by C, C is needed by D, and so on. Then a logical testing sequence is A, B, C, D. If there is no guaranteed testing order, then, say, the test for D would have to do at least a cursory re-check of A, B, and C so that D would know it's checking what it thinks it's checking. I used Boost Test once before ~5 years ago. It only had manual registration of test cases and test suites (as I remember). That would have provided the ordering I need. However, the auto registration feature of the new version is so much more convenient, I was hoping to use it. The REQUIRE functionality is sort of "one strike and everyone is out." If there is an ordering, then you can start reading the log from the top down. If you really do find something fatal, then you would know to ignore the rest of the test output. Otherwise, all of the other test output can give useful information. I'm guessing that I will have to revert to manual registration. Merrill Cornish
Merrill Cornish
There will be many more test suites to come. They are not LOGICALLY dependent on each other, but if some fail, the other tests will be a waste of time. Therefore, I would like to be able to specify the order in which they are run.
Upcoming Boost.Test v3 has this feature in a form of depend_on decorator. It will enforce the order between test unit (making sure you do not introduce cycles) and also will only run test unit if all the prerequisites (including dependent test units success) are satisfied. You can give it a try using develop branch of Boost.Test. Gennadiy
participants (3)
-
Gennadiy Rozental
-
Merrill Cornish
-
Peter Steinbach