When you want to apply a test-driven design approach, you will probably find that almost none of the unit-test frameworks really fits your needs. So, if your company does not already use a testing framework, it is worth developing your own framework!
Below is a step-by-step development of a unit-testing framework. It is basically a summary of Kevlin Henney's "Raw TDD" workshop I attended during ACCU 2015. The workshop was so revealing, that I had to record it here.
Let's assume you're testing the function
bool is_leap_year(int year);
A naive way of testing is through the use of assert() calls:
int main()
{
cout << "odd years are not leap year\n";
assert( !is_leap_year(2001) );
cout << "multiples of 4 but not 100 are leap years\n";
assert( is_leap_year(1996) );
cout << "multiples of 4 and 100 are not leap years";
assert( !is_leap_year(1900) );
cout << "multiples of 400 are leap years";
assert( is_leap_year(800) );
cout << "All tests passed\n";
}
However, imagine that some months later, you make a change to is_leap_year(). A lot of tests could fail. Unfortunately, you'll only see the first that fails, as assert() will abort at first failure.
Finally, the scope of the tests is too large. Any resources allocated at the beginning of main() are only freed at the end of the function. This may create side-effects between the tests.
void odd_years_are_not_leap_year() {
assert( !is_leap_year(2001) );
}
void multiples_of_4_but_not_100_are_leap_years() { ... }
void multiples_of_4_and_100_are_not_leap_years() { ... }
void multiples_of_400_are_leap_years() { ... }
int main()
{
odd_years_are_not_leap_year();
multiples_of_4_but_not_100_are_leap_years();
multiples_of_4_and_100_are_not_leap_years();
multiples_of_400_are_leap_years();
cout << "All tests passed\n";
}
The disadvantage of this attempt is that it is often difficult to express a message in duck-typed or camel-case.
Also, the tests need to be defined and called. Forget to call a test function, and you may think that a test is successful !
In order to make sure that each test you write will be called, use the following skeleton:
...
test_case tests[] =
{
... // list test case functions here
};
int main()
{
for (auto test : tests) {
test();
}
}
Each test is written as a lambda function and stored as a std::function.
Then, we iterate this array of test cases, calling each test.
Here's how you may implement the unit test using the skeleton above:
struct test_case {
const char *name;
std::function<void ()> run;
};
{
{
{
{
assert(is_leap_year(800));
}
};
int main()
{
for (auto test : tests) {
std::cout << "Testing " << test.name << "\n";
test.run();
}
In the snippet above, each test case is composed of a string that states its name, and a test function. This helps in reporting which tests are running.
We have solved the problem of forgetting to call a rest, but still, even if a single test fails, the program stops.
We need a way to record all failures, without stopping.
To do this, just replace assert() calls with a macro ASSERT() that throws an exception when the test fails:
Instead of aborting on test failure, we simply print a warning message, and continue to the next test:
The constructs above result to the following, final framework.
The final framework contains two basic checks: ASSERT() and CATCH().
These are checks that will fire if an assertion is not satisfied, or an exception is not thrown, when it was expected to throw.
Then, we iterate this array of test cases, calling each test.
const char *name;
std::function<void ()> run;
};
test_case tests[] = {
"odd years are not leap years", []
assert(!is_leap_year(2001));
},
"multiples of 4 but not 100 are leap years", []
assert(is_leap_year(1996));
},
"multiples of 4 and 100 are not leap years", []
assert(!is_leap_year(1900));
},
"multiples of 400 are leap years", []
assert(is_leap_year(800));
}
};
{
for (auto test : tests) {
std::cout << "Testing " << test.name << "\n";
test.run();
}
std::cout << "All tests passed" << std::endl;
}We have solved the problem of forgetting to call a rest, but still, even if a single test fails, the program stops.
We need a way to record all failures, without stopping.
struct failure {
const char *expression;
int line;
};
#define ASSERT(condition) \
void((condition) ? 0 : \
throw failure( { "ASSERT(" #condition ")", __LINE__ } ))
for (auto test : tests)
{
try
{
test();
}
catch ( failure & caught )
{
std::cout << "Test Assertion failed:\n"
<< caught.expression << "\n"
<< " at line " << caught.line << "\n";
}
}
The final framework contains two basic checks: ASSERT() and CATCH().
These are checks that will fire if an assertion is not satisfied, or an exception is not thrown, when it was expected to throw.
#include "is_leap_year.hpp"
#include <cassert>
#include <iostream>
#include <functional>
struct failure
{
const char *expression;
int line;
};
struct test_case
{
const char *name;
std::function<void()> run;
};
#define ASSERT(condition) \
void((condition) ? 0 : \
throw failure( { "ASSERT(" #condition ")", __LINE__ } ))
#define CATCH(expression, exception) \
try \
{ \
(expression); \
throw failure({ "CATCH(" #expression ", " #exception ")", __LINE__ }); \
} \
catch(exception &) \
{\
}\
catch(...) \
{ \
throw failure({ "CATCH(" #expression ", " #exception ")", __LINE__ }); \
}
test_case tests[] = {
"odd years are not leap years", []
{
ASSERT(!is_leap_year(2001));
},
"multiples of 4 but not 100 are leap years", []
{
ASSERT(is_leap_year(1996));
},
"multiples of 4 and 100 are not leap years", []
{
ASSERT(!is_leap_year(1900));
},
"mulptiples of 400 are leap years", []
{
ASSERT(is_leap_year(800));
}
};
int main()
{
bool ok = true;
for (auto &&test : tests)
{
try
{
std::cout << "Testing " << test.name << "\n";
test.run();
}
catch ( failure &caught )
{
ok = false;
std::cout << "Test Assertion failed:\n"
<< caught.expression << "\n"
<< " at line " << caught.line << "\n";
}
}
if(ok) {
std::cout << "All tests passed" << std::endl;
}
}
No comments:
Post a Comment