Writing C++ unit tests with Catch2

In my book, Modern C++ Programming Cookbook, I discussed several testing frameworks for C++, more precisely, Boost.Test, Google Test, and Catch (which stands for C++ Automated Test Cases in a Header). Since the publishing of the book, a new version of Catch, called Catch2 has been released. This provides new functionalities, but also a series of breaking changes with Catch, including the drop of support for pre-C++11 compilers. For a list of changes, you can see the Catch2 release notes. Catch2 is available as a single-header library, is open-sources and cross-platform, and written for C++11/14/17/latest. In this article, I want to give a brief example of how you can write tests for C++ using Catch2.

In order to do so, I will consider the fizzbuzz game. This is a numbers game for children; one child is supposed to say a number and the other has to answer with:

• fizz, if the number is divisible by 3,
• buzz, if the number if divisible by 5,
• fizzbuzz, if the number if divisible by both 3 and 5, or
• the number itself in all other cases.

The function fizzbuzz() below implements this game:

#include <string>

std::string fizzbuzz(int const number)
{
   if (number != 0)
   {
      auto m3 = number % 3;
      auto m5 = number % 5;
      if (!m5 && !m3) { return "fizzbuzz"; }
      else if (!m5) { return "buzz"; }
      else if (!m3) { return "fizz"; }
   }
   return std::to_string(number);
}

This is what I want to test and the first test I would try is the result for number 0. In Catch2, you can write the test cases as follows:

TEST_CASE("Test with zero", "[classic]")
{
   REQUIRE(fizzbuzz(0) == "0");
}

The TEST_CASE macro defines a test case, called "Test with zero" here, and may associate tags to the case, such as [classic] in this example. Tags are used for selecting what test cases to run. REQUIRE is an assertion macro that determines the test to fail if the condition is not satisfied. There are several other assertion macros and you can read about them here.

Of course, we need more tests for this function, and the following test case called "Test positives" defines more:

TEST_CASE("Test positives", "[classic]")
{
   SECTION("Test all up to 10") {
      REQUIRE(fizzbuzz(1) == "1"); 
      REQUIRE(fizzbuzz(2) == "2");
      REQUIRE(fizzbuzz(3) == "fizz");
      REQUIRE(fizzbuzz(4) == "4");
      REQUIRE(fizzbuzz(5) == "buzz");
      REQUIRE(fizzbuzz(6) == "fizz");
      REQUIRE(fizzbuzz(7) == "7");
      REQUIRE(fizzbuzz(8) == "8");
      REQUIRE(fizzbuzz(9) == "fizz");
      REQUIRE(fizzbuzz(10) == "buzz");
   }

   SECTION("Test all multiples of 3 only up to 100") {
      for (int i = 3; i <= 100; i+=3) {
         if (i % 5) REQUIRE(fizzbuzz(i) == "fizz");
      }
   }

   SECTION("Test all multiples of 5 only up to 100") {
      for (int i = 5; i <= 100; i += 5) {
         if (i % 3) REQUIRE(fizzbuzz(i) == "buzz");
      }
   }

   SECTION("Test all multiples of 3 and 5 up to 100") {
      for (int i = 15; i <= 100; i += 15) {
         REQUIRE(fizzbuzz(i) == "fizzbuzz");
      }
   }
}

This is a little bit different than the test case above because it makes use of another macro called SECTION. This introduces a test function. Test functions can be nested (without limit) and they form a tree structure with test cases on root nodes and test function on inner and leaf nodes. When a leaf test functions is run, the entire code from the root test case to the leaf test function is executed. As a result, when multiple test functions (i.e. section) share common code, that code is executed for each section. This makes it unnecessary to have fixtures with setup and teardown code.

Here is yet another test case, this time for negative numbers:

TEST_CASE("Test negatives", "[classic]")
{
   REQUIRE(fizzbuzz(-1) == "-1");
   REQUIRE(fizzbuzz(-2) == "-2");
   REQUIRE(fizzbuzz(-3) == "fizz");
   REQUIRE(fizzbuzz(-4) == "-4");
   REQUIRE(fizzbuzz(-5) == "buzz");
   REQUIRE(fizzbuzz(-6) == "fizz");
   REQUIRE(fizzbuzz(-7) == "-7");
   REQUIRE(fizzbuzz(-8) == "-8");
   REQUIRE(fizzbuzz(-9) == "fizz");
   REQUIRE(fizzbuzz(-10) == "buzz");
}

Catch2 is automatically registering test cases and no additional work is necessary for that. Moreover, Catch2 can supply a main() function with all that is necessary for the setup of the framework. All you have to do for that is define the macro CATCH_CONFIG_MAIN before including the Catch2 header.

#define CATCH_CONFIG_MAIN
#include "catch.hpp"

Of course, you can supply your own main() but in this case you need to call into Catch2 yourself. This, however, allows you to tweak the configuration or provide your own command line options. To supply your own implementation of main() you need to define the macro CATCH_CONFIG_RUNNER. You can read more about this options here: Supplying main() yourself.

You can execute the tests simply by running your application.

If you have errors in your tests, this is how they are reported (to the console):

There is a multitude of command line options, that allow you to configure what tests are running, how they are running, how the results are reported, etc. You can read about them here: Command line. Here are some additional examples:

• Showing results for successful tests too (with arguments -s)

  

• Showing compact results, including for successful tests too (with arguments -s -r compact)

  

• Showing results in a JUnit XML Report ANT format (with arguments -r junit)

  

If you prefer to write your tests using a BDD approach, you can still do that with Catch2. The following are examples for testing the fizzbuzz() function.

SCENARIO("BDD test with zero", "[bdd]")
{
   WHEN("The number is 0") {
      THEN("The result is 0") {
         REQUIRE(fizzbuzz(0) == "0");
      }
   }
}

SCENARIO("BDD test any number", "[bdd]")
{
   GIVEN("Any positive number")
   {
      WHEN("The number is 1") {
         THEN("The result is 1") {
            REQUIRE(fizzbuzz(1) == "1");
         }
      }
      WHEN("The number is 2") {
         THEN("The result is 2") {
            REQUIRE(fizzbuzz(2) == "2");
         }
      }
      WHEN("The number is 3") {
         THEN("The result is fizz") {
            REQUIRE(fizzbuzz(3) == "fizz");
         }
      }
      WHEN("The number is 4") {
         THEN("The result is 4") {
            REQUIRE(fizzbuzz(4) == "4");
         }
      }
      WHEN("The number is 5") {
         THEN("The result is buzz") {
            REQUIRE(fizzbuzz(5) == "buzz");
         }
      }
      WHEN("The number is a multiple of 3 only") {
         THEN("The result is fizz") {
            for (int i = 3; i <= 100; i += 3) {
               if (i % 5) REQUIRE(fizzbuzz(i) == "fizz");
            }
         }
      }
      WHEN("The number is a multiple of 5 only") {
         THEN("The result is buzz") {
            for (int i = 5; i <= 100; i += 5) {
               if (i % 3) REQUIRE(fizzbuzz(i) == "buzz");
            }
         }
      }
      WHEN("The number is a multiple of 3 and 5") {
         THEN("The result is fizzbuzz") {
            for (int i = 15; i <= 100; i += 15) {
               REQUIRE(fizzbuzz(i) == "fizzbuzz");
            }
         }
      }
   }
   GIVEN("Any negative number") {
      WHEN("The number is -1") {
         THEN("The result is -1") {
            REQUIRE(fizzbuzz(-1) == "-1");
         }
      }
      WHEN("The number is -2") {
         THEN("The result is -2") {
            REQUIRE(fizzbuzz(-2) == "-2");
         }
      }
      WHEN("The number is -3") {
         THEN("The result is fizz") {
            REQUIRE(fizzbuzz(-3) == "fizz");
         }
      }
      WHEN("The number is -4") {
         THEN("The result is -4") {
            REQUIRE(fizzbuzz(-4) == "-4");
         }
      }
      WHEN("The number is -5") {
         THEN("The result is buzz") {
            REQUIRE(fizzbuzz(-5) == "buzz");
         }
      }
   }
}

This is possible because the macro SCENARIO resolves to TEST_CASE and the macros GIVEN, WHEN, AND_WHEN, THEN, AND_THEN resolve to SECTION. Here is how they are defined in the framework:

// "BDD-style" convenience wrappers
#define SCENARIO( ... ) TEST_CASE( "Scenario: " __VA_ARGS__ )
#define SCENARIO_METHOD( className, ... ) INTERNAL_CATCH_TEST_CASE_METHOD( className, "Scenario: " __VA_ARGS__ )

#define GIVEN( desc )    SECTION( std::string("   Given: ") + desc )
#define WHEN( desc )     SECTION( std::string("    When: ") + desc )
#define AND_WHEN( desc ) SECTION( std::string("And when: ") + desc )
#define THEN( desc )     SECTION( std::string("    Then: ") + desc )
#define AND_THEN( desc ) SECTION( std::string("     And: ") + desc )

Again, if we want to run all tests, we can just execute the app without any filters.

However, you can also specify the name of a test, or a pattern to select that test to be executed, or removed from execution. In the following example we execute the test case called "Test positives". You can do the same with BDD test cases (i.e. scenarios). However, you need to prefix the name with "Scenario: ", such as in "Scenario: BDD test any number". This is necessary, because of the way the SCENARIO macro is defined (i.e. #define SCENARIO( ... ) TEST_CASE( "Scenario: " __VA_ARGS__ )).

You can also list all the tags defined for your test cases. You do that with the -t option. In this example, we have 3 test cases tagged [classic] and two tagged [bdd].

You can use the tag names to execute all the test cases associated with them. In the following example we run all the test cases tagged [bdd].

There are other features that the framework provides, including string conversions, logging macros, event listeners, reporters, CI and Build system integration.

You can find the project on Github with reference documentation to help you write tests for your C++11/14/17 code.