On 23/05/2016 14:29, Vinnie Falco wrote:
I'm reading this thread about doctest and I checked out the library. The first thing I noticed was macros and odd syntax. For example:
TEST_CASE("testing the factorial function") { CHECK(factorial(1) == 1); CHECK(factorial(2) == 2); CHECK(factorial(10) == 3628800); }
This looks like non-standard C++ and heavy on macros which turned me off to Boost.Test for the same reasons. I considered using Boost.Test for yet another library I am about to release and I gave up when I saw the baffling use of macros.
Generally macros are used for assertions because you want to include __LINE__ (and typically also __FILE__) in the assertion failure so that you can find *which* assertion failed (often the condition alone is not sufficiently unambiguous), and there isn't really any better way to get those AFAIK. (Sometimes it's possible to extract the failure location from a thrown exception, but this is highly compiler-dependent and non-standard -- and you're not interested in that specific point, but in one of its callers.) Using macros for the test case definition is more avoidable but is still generally done to avoid boilerplate (in this case, presumably at least auto-registration).
People seem to be lavishing praise on these test libraries so I am asking sincerely, what am I missing here? I used my own unit test framework which is header-only, customizable, doesn't require macros in the tests, and has a test structure which is easy to generate XML reports from. Here it is: https://github.com/vinniefalco/Beast/blob/master/extras/beast/unit_test/suit...
Your version seems to not capture line numbers and thus requires coming up with a unique string reason for each assertion, which is cumbersome.
To write a test you subclass from `beast::unit_test::suite` and implement the `run()` override, and then call `expect()` for each condition that should be true. I was able to get Beast to almost 97% coverage using this simple system (see https://codecov.io/gh/vinniefalco/Beast). Here's an example of a test: https://github.com/vinniefalco/Beast/blob/master/test/core/basic_streambuf.c...
This also requires listing each test twice (once in implementation, once in run()), and each suite twice (one in implementation, once in BEAST_DEFINE_TESTSUITE... which is one of those macros you didn't like, isn't it?). Requiring separate listing is "old school" test runner design; it went out of fashion because it's too error-prone (it's easy to forget to list a test case, and if you're not ensuring that tests fail before you make them pass then you might think the test passed instead of simply not being run). This design also makes it harder to run or debug individual test cases within a suite. (I don't know how easy doctest makes this, but most of the popular test runners can do this very easily, without recompiling.) You can comment out some of them but this requires a rebuild and is also error-prone since you have to remember to uncomment them again later. If it works for you, that's fine -- but these are some of the reasons why test frameworks generally avoid those things and use macros and auto-registration instead.