Boost::Test v. CppUnit
Howdy - I've been using CppUnit and various parts of Boost for a little over a year. The 1.3.0 rlease includes support for unit testing, a la CppUnit, JUnit, etc. At first glance, I can't find any fundamental differences between CppUnit and Boost's unit testing framework. I'm sure that CppUnit could benefit from a code review or two by Boost's luminaries, but I'm still surprised that a new framework was created instead of trying to improve CppUnit. Can anyone state a few areas where CppUnit and Boost::test differ? In general, I tend to prefer Boost's solutions over others', but I don't see a clear cut case to be made for Boost over CppUnit, vis a vis unit testing. I'd be happy to change my mind, however. :-) Jon -- Jon Stewart stew1@ellipsis.cx
There was a thread very similar to this in comp.lang.c++.moderated only a few days ago IIRC. Search for CppUnit in the archives. Gennadiy (the author of boost::test) provided answers. // Fredrik Jon Stewart wrote:
Howdy -
I've been using CppUnit and various parts of Boost for a little over a year. The 1.3.0 rlease includes support for unit testing, a la CppUnit, JUnit, etc. At first glance, I can't find any fundamental differences between CppUnit and Boost's unit testing framework. I'm sure that CppUnit could benefit from a code review or two by Boost's luminaries, but I'm still surprised that a new framework was created instead of trying to improve CppUnit.
Can anyone state a few areas where CppUnit and Boost::test differ? In general, I tend to prefer Boost's solutions over others', but I don't see a clear cut case to be made for Boost over CppUnit, vis a vis unit testing. I'd be happy to change my mind, however. :-)
Jon
I tend to like Boost.Test more. The only advantage (or disadvantage) of CppUnit is that it is a JUnit port. Hence if you know JUnit already learning CppUnit is easier. But then again, either of them are not hard to learn. I prefer Boost.Test because: 1. It is more flexible in terms of test case arrangements, you can use free functions, classes, whatever. 2. BOOST_CHECK and BOOST_REQUIRE (CppUnit does not provide anything similiar to BOOST_CHECK) 3. Boost framework correclty reports all non C++ exceptions (memory violations, division by zero etc.) while in CppUnit you get very helpful "Unknown exception" message. 4. Boost tools (CHECK, REQUIRE etc) more robust and provide better reporting capabilities (for instance you can check if two containers are equual) 5. CppUnit is no longer evolves, and it certainly misses some functionality, and has some bugs. I am sure this is not a full list, I just switched from CppUnit to Boost.Test and my Boost.Test knowledge is not thorough enough. Amoung things I am missing in boost.test -- ability to call test suites by name. Well to be fair, CppUnit does not provide such functionality out of the box either. Also it is planned for next version of the Boost.Test. --Kirill
Can anyone state a few areas where CppUnit and Boost::test differ? In general, I tend to prefer Boost's solutions over others', but I don't see a clear cut case to be made for Boost over CppUnit, vis a vis unit testing. I'd be happy to change my mind, however. :-)
Jon Stewart stew1@e...
Hi, Jon. I am not that big expert in CppUnit. If you could provide a feature list for CppTest I may do feature by feature comparison. But from what I view after perfunctory look on docs the feature lists seems incomparable. Though again I may be wrong. Regards, Gennadiy.
I am not that big expert in CppUnit. If you could provide a feature list for CppTest I may do feature by feature comparison. But from what I view after perfunctory look on docs the feature lists seems incomparable. Though again I may be wrong.
Hi Gennadiy, I'd be happy to provide you with a feature list. It may take a day or two. Boost typically provides libraries that beat the pants off anything else out there, and I'm sure that you're working to make Boost::Test the best unit testing framework available for C++. However, unlike many of the other libraries that Boost provides, a reasonably functional and popular substitute (CppUnit) existed. I'm curious as to what led you to write your own rather than contribute to CppUnit and try to get it incorporated into Boost (similar to how Spirit was incorporated)? Do you have different design goals for Boost::Test? cheers, Jon -- Jon Stewart stew1@ellipsis.cx
At 12:37 PM 6/10/2003, Jon Stewart wrote:
I am not that big expert in CppUnit. If you could provide a feature list for CppTest I may do feature by feature comparison. But from what I view after perfunctory look on docs the feature lists seems incomparable. Though again I may be wrong.
Hi Gennadiy,
I'd be happy to provide you with a feature list. It may take a day or two.
Boost typically provides libraries that beat the pants off anything else out there, and I'm sure that you're working to make Boost::Test the best unit testing framework available for C++. However, unlike many of the other libraries that Boost provides, a reasonably functional and popular substitute (CppUnit) existed. I'm curious as to what led you to write your own rather than contribute to CppUnit and try to get it incorporated into
Boost (similar to how Spirit was incorporated)? Do you have different design goals for Boost::Test?
Gennadiy took over an existing library, and then contributed a seriously major upgrade supporting full unit testing (while still preserving simpler forms of testing when appropriate). And that is one of the major advantages of Boost.Test compared to packages that were available at the time we started the initial design. The three levels of use that Boost.Test supports are: -- Execution monitoring - this isn't really "testing" functionality, but rather is a tremendous help for production programs. It it unifies several disparate error reporting mechanisms, and permits unattended operation. I can't imagine putting a job into overnight production use without Boost.Test execution monitoring. -- Simple test scenarios. Not all testing needs a full unit-test suite. Boost.Tests' test tools do a really nice job for light or medium duty testing. Minimal learning curve. -- Full unit-test. Heavier-duty testing to meet heavier-duty needs. Lot's of test packages attack this level, but leave you hanging for the lower levels. In a way, the whole of Boost.Build and the Boost regression testing system are yet another layer which fits nicely on top of the Boost.Test layers. So that's a bit of background. In a nutshell, Boost.Test servers a whole hierarchy of needs. --Beman
Hello all,
I am trying to use the boost::ublas library in my code to speed up
things, but I was a bit suspicious about the results. So I ran my own
tests, and I found that ublas is not as fast as the C implementation,
but is actually about twice slower (for my test, which are very
limited). Please find the source file below.
I am not very familiar with expression template. Am I doing something
wrong? Are the results specific to this test?
Thanks a lot.
Alexis
////////// CODE
#include <iostream>
#include "boost/numeric/interval.hpp"
#include "boost/numeric/interval/io.hpp"
#include "boost/numeric/ublas/config.hpp"
#include "boost/numeric/ublas/vector.hpp"
#include "boost/numeric/ublas/matrix.hpp"
#include "boost/numeric/ublas/vector_expression.hpp"
#include "boost/numeric/ublas/io.hpp"
#include "boost/timer.hpp"
#include "boost/progress.hpp"
namespace ublas = boost::numeric::ublas;
typedef int value_type;
#define N 14
typedef ublas::vector
Hi Alexis, you wrote:
I am trying to use the boost::ublas library in my code to speed up things, but I was a bit suspicious about the results.
You've read the documentation's remarks about abstraction penalty, haven't you?
So I ran my own tests, and I found that ublas is not as fast as the C implementation, but is actually about twice slower (for my test, which are very limited). Please find the source file below. I am not very familiar with expression template. Am I doing something wrong?
No (as far as I see).
Are the results specific to this test?
In a certain sense. [snip parts of code]
typedef int value_type;
Interesting data type ;-)
#define N 14 #define K 100000000
[snip other parts of code] I've been playing with these parameters and two compilers. For GCC 3.2.1 (compiler options -NDEBUG -O3) I see the following results on my box: n = 14, k = 100000000 c t=4.5 assign prod t=6.58 n = 14*14, k = 10000000 c t=5.59 assign prod t=7.76 n = 14*14*14, k = 1000000 c t=8.89 assign prod t=12.59 n = 14*14*14*14, k = 100000 c t=70.88 assign prod t=70.11 For ICC 7.1 (again compiler options -NDEBUG -O3): n = 14, k = 100000000 c t=3.16 assign prod t=6.22 n = 14*14, k = 10000000 c t=4.49 assign prod t=6.05 n = 14*14*14, k = 1000000 c t=6.91 assign prod t=10.19 n = 14*14*14*14, k = 100000 c t=70.37 assign prod t=70.78 It looks like the inner loops of the 'C' and C++ code are identical. I'm unsure if the abstraction penalty for small sizes can be eliminated by the compiler writers or if it justifies a 'tiny vector/matrix' abstraction a la Blitz++ (resulting in a corresponding compile time abstraction penalty ;-) Best, Joerg
Thanks for the answer :o) On Wednesday, Jun 11, 2003, at 22:48 Europe/London, jhr.walter@t-online.de wrote:
You've read the documentation's remarks about abstraction penalty, haven't you?
Yes, I did, I even read Vandervoorde and Josuttis "C++ templates" (though I havent grasped all the concepts yet ;o) )
Are the results specific to this test?
In a certain sense.
What I meant was "are the results not that good because of any of the functions I am using (element_prod in particular, since its quite new)?" But I've got my answer, thanks! I didnt think the optimizer would do such a "bad" job for such a small formula. Lets hope gcc catch up quickly! Sorry, I forgot to mention my config: gcc 3.1, OSX and G3 800 (definitely not as fast as yours!!) Thanks again Alexis [Non-text portions of this message have been removed]
participants (7)
-
Alexis -
Beman Dawes -
Fredrik Blomqvist -
Gennadiy E. Rozental -
jhr.walter＠t-online.de -
Jon Stewart -
klapshin