Catch2

Catch2 is a light-weight testing framework. The name stands for C++ Automated Test Cases in Headers (version two). CLion supports Catch versions 1.7.2 and later.

As well as Boost.Test, Catch2 doesn't provide mocking functionality. However, you can combine it with standalone mocking frameworks such as Hippomocks, FakeIt, or Trompeloeil.

Catch2 basics

If you are not familiar with Catch/Catch2, you can find a description of its main concepts below:

Sample test

The following example shows a simple test written with Catch2:

#define CATCH_CONFIG_MAIN // provides main(); this line is required in only one .cpp file
#include "catch_amalgamated.hpp"

int theAnswer() { return 6*9; } // function to be tested

TEST_CASE( "Life, the universe and everything", "[42][theAnswer]" ) {
REQUIRE(theAnswer() == 42);
}

In the above example, Life, the universe and everything is a free-form test name, which must be unique. The second argument of the TEST_CASE macro is a combination of two tags, [42] and [theAnswer]. Both test name and tags are regular strings that are not limited to be valid C++ identifiers. You can run collections of tests by specifying a wildcarded test name or a tag expression.

Notice the assertion line REQUIRE(theAnswer() == 42). Unlike other frameworks, Catch2 doesn't have a collection of asserts to capture various conditional forms. Instead, it parses the actual C/C++ code of the conditional expression and also uses it to describe the result:

...Failure:
REQUIRE(theAnswer() == 42)
with expansion:
54 == 42

The REQUIRE macro aborts a test on failure, while the alternative CHECK macro only reports the failure and lets the test carry on. Within both of these macros, you can use all C++ comparison operators and pass the arguments in any order.

Sections

Another important feature of Catch2 is the way to organize tests in cases and sections (while the class-based fixture mechanism is also supported). Take a look at this example from the documentation:

TEST_CASE( "vectors can be sized and resized", "[vector]" ) {

// initialization block executed for each section
std::vector<int> v( 5 );
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 5 );
// end of initialization block

SECTION( "resizing bigger changes size and capacity" ) {
v.resize( 10 );

REQUIRE( v.size() == 10 );
REQUIRE( v.capacity() >= 10 );
}
SECTION( "resizing smaller changes size but not capacity" ) {
v.resize( 0 );

REQUIRE( v.size() == 0 );
REQUIRE( v.capacity() >= 5 );
}
}

In the above snippet, TEST_CASE is executed from the start for each SECTION. Two REQUIRE statements at the top of the TEST_CASE enforce that size is 5 and capacity is at least 5 at the entry of each section. This way, shared objects are allocated on stack and there is no need to create a fixture class for them. On each run through a TEST_CASE, Catch2 executes one section and skips the others. Next time, it executes the second section, and so on.

Sections can be nested to create a sequence of checking operations. Each leaf section (a section with no nested sections inside) is executed once. When a parent section fails, it prevents child sections from running. For example:

SECTION( "reserving bigger changes capacity but not size" ) {
v.reserve( 10 );

REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 10 );

// verify that attempting to reserve a smaller capacity changes nothing
SECTION( "reserving smaller again does not change capacity" ) {
v.reserve( 7 );
REQUIRE( v.capacity() >= 10 );
}
}

Catch2 also supports the alternative BDD-style syntax for test cases and sections.

Template tests

Catch2 supports type-parametrized test cases in the form of the following macros:

TEMPLATE_TEST_CASE( test name , tags, type1, type2, ..., typen )
TEMPLATE_PRODUCT_TEST_CASE( test name , tags, (template-type1, template-type2, ..., template-typen), (template-arg1, template-arg2, ..., template-argm) )
TEMPLATE_LIST_TEST_CASE( test name, tags, type list )

These macros behave in the same way as regular TEST_CASE, but are run for every type or type combination. For more information, refer to Type-parametrized test cases.

In addition to type-parametrized test cases, Catch2 also provides TEMPLATE_TEST_CASE_SIG and TEMPLATE_PRODUCT_TEST_CASE_SIG for creating signature-based parametrized test cases, which have similar syntax with the additional signature argument:

TEMPLATE_TEST_CASE_SIG( test name , tags, signature, type1, type2, ..., typen )

For more information, refer to Signature-based parametrized test cases.

Working with Catch2 in CLion

Add Catch2 to your project

Follow the instructions in this guide to start with Catch using its CMake integration.

Create a Catch run/debug configuration

Go to Run | Edit Configurations, click and select Catch from the list of templates.
Specify the configuration settings:
- Set the configuration name in the Name field. This name will be shown in the list of the available run/debug configurations.
- Select the Tags/Test option to run a test for the particular tags or all the tags. Select the Pattern option to run all the tests for a particular pattern.
- Specify tags in the Tags fields. This field is available only when the Tags/Test option is selected.
- Select the desired test from the Test list. This option is available only when one or more tags have been provided.
- Specify the pattern name in the Pattern field. This field is available only when the Pattern option is selected.
- In the Target field, select the desired target from the list of available targets.
Save the configuration, and it's ready for running or debugging .

Run tests

In CLion, there are several ways to start a run/debug session for tests, one of which is using special gutter icons. These icons help quickly run or debug a single test or a whole suite/fixture:
Gutter icons also show test results (when already available): success or failure .
When you run a test/suite/fixture using gutter icons, CLion creates a temporary Catch configuration, which is greyed out in the list of configurations. To save a temporary configuration, select it in the Edit Configurations dialog and press :

Explore results

When you run tests, CLion shows the results and the process in the built-in test runner window. The test tree shows all the tests while they are being executed one by one. The test runner window includes:

progress bar with the percentage of tests executed so far,
tree view of all the running tests with their status and duration,
tests' output stream,
toolbar with the options to rerun failed tests, export or open previous results saved automatically , sort the tests alphabetically to easily find a particular test, or sort them by duration to understand which test ran longer than others.

Last modified: 13 November 2023