Catch
Catch2 is a light-weight testing framework. The name stands for C++ Automated Test Cases in Headers (version two). CLion supports Catch versions 1.7.2 and later.
As well as Boost.Test, Catch2 doesn't provide mocking functionality. However, you can combine it with standalone mocking frameworks such as Hippomocks, FakeIt, or Trompeloeil.
Catch2 basics
If you are not familiar with Catch/Catch2, you can find a description of its main concepts below:
Sample test
The following example shows a simple test written with Catch2:
In the above example, Life, the universe and everything
is a free-form test name, which must be unique. The second argument of the TEST_CASE
macro is a combination of two tags, [42]
and [theAnswer]
. Both test name and tags are regular strings that are not limited to be valid C++ identifiers. You can run collections of tests by specifying a wildcarded test name or a tag expression.
Notice the assertion line REQUIRE(theAnswer() == 42)
. Unlike other frameworks, Catch2 doesn't have a collection of asserts to capture various conditional forms. Instead, it parses the actual C/C++ code of the conditional expression and also uses it to describe the result:
The REQUIRE
macro aborts a test on failure, while the alternative CHECK
macro only reports the failure and lets the test carry on. Within both of these macros, you can use all C++ comparison operators and pass the arguments in any order.
Sections
Another important feature of Catch2 is the way to organize tests in cases and sections (while the class-based fixture mechanism is also supported). Take a look at this example from the documentation:
In the above snippet, TEST_CASE
is executed from the start for each SECTION
. Two REQUIRE
statements at the top of the TEST_CASE
enforce that size
is 5 and capacity
is at least 5 at the entry of each section. This way, shared objects are allocated on stack and there is no need to create a fixture class for them. On each run through a TEST_CASE
, Catch2 executes one section and skips the others. Next time, it executes the second section, and so on.
Sections can be nested to create a sequence of checking operations. Each leaf section (a section with no nested sections inside) is executed once. When a parent section fails, it prevents child sections from running. For example:
Catch2 also supports the alternative BDD-style syntax for test cases and sections.
Template tests
Catch2 supports type-parametrized test cases in the form of the following macros:
TEMPLATE_TEST_CASE( test name , tags, type1, type2, ..., typen )
TEMPLATE_PRODUCT_TEST_CASE( test name , tags, (template-type1, template-type2, ..., template-typen), (template-arg1, template-arg2, ..., template-argm) )
TEMPLATE_LIST_TEST_CASE( test name, tags, type list )
These macros behave in the same way as regular TEST_CASE
, but are run for every type or type combination. For more information, refer to Type-parametrized test cases.
In addition to type-parametrized test cases, Catch2 also provides TEMPLATE_TEST_CASE_SIG
and TEMPLATE_PRODUCT_TEST_CASE_SIG
for creating signature-based parametrized test cases, which have similar syntax with the additional signature argument:
For more information, refer to Signature-based parametrized test cases.
Working with Catch2 in CLion
Add Catch2 to your project
Follow the instructions in this guide to start with Catch using its CMake integration.
Run/debug the automatically created Catch configuration
CLion detects the Catch tests in your project and creates a run/debug configuration for them.
You can use this configuration right away to run or debug all the Catch tests in your project:
Add a new Catch run/debug configuration
Go to Run | Edit Configurations.
Click and select Catch from the list of templates.
Set the configuration name in the Name field. This name will be shown in the list of the available run/debug configurations.
Select the Tags/Test option to run a test for the particular tags or all the tags. Select the Pattern option to run all the tests for a particular pattern.
Specify tags in the Tags fields. This field is available only when the Tags/Test option is selected.
Select the desired test from the Test list. This option is available only when one or more tags have been provided.
Specify the pattern name in the Pattern field. This field is available only when the Pattern option is selected.
In the Target field, select the desired target from the list of available targets.
Click Apply to save the configuration.
Select the newly created configuration in the switcher and run or debug it.
Run tests
The quickest way to run or debug a single test or a whole suite/fixture is by using the gutter icons:
Gutter icons also show test results (when already available): success or failure .
When you run a test/suite/fixture using gutter icons, CLion creates a temporary Catch configuration, which is greyed out in the list of configurations. To save a temporary configuration, select it in the dialog and click :
Explore test results
When you run tests, CLion shows the results and the process in the built-in test runner window. The test tree shows all the tests while they are being executed one by one. The test runner window includes:
progress bar with the percentage of tests executed so far,
tree view of all the running tests with their status and duration,
tests' output stream,
toolbar with the options to rerun failed tests, import/export or open previous results saved automatically , sort the tests alphabetically to easily find a particular test, or sort them by duration to understand which test ran longer than others.
Specify random seed
If you are using randomization in your Catch tests, you can set the initial seed by specifying the --rng-seed='time'|'random-device'|'number'
parameter. See this section in the Catch documentation for details.
In the main menu, go to
and select any Catch configuration. The seed parameter will be used for the entire test run.In the Program arguments field, set the
--rng-seed
flag:Run the configuration, and you will see the fixed seed in the test tree:
Use the context menu to rerun the tests with the same fixed seed:
Troubleshooting: long test names produce incorrect listings and terminate test run
If there are long test names in your project, this may break the listing and terminate the whole test run (see this Catch issue). Try one of the following solutions:
Open the catch_user_config.hpp file and change the
#define CATCH_CONFIG_CONSOLE_WIDTH
parameter from80
to800
.Open the CatchConfigOptions.cmake file and change the config generator from
set(CATCH_CONFIG_CONSOLE_WIDTH "80" ...)to
set(CATCH_CONFIG_CONSOLE_WIDTH "800" ...)(Least recommended) Disable the test list extraction in CLion: clear the
cidr.test.prepare.test.list
registry key. Use Ctrl+Shift+A to open the Registy dialog.