Run/Debug Configuration: TestNG

The TestNG run/debug configuration is used to launch the tests that comply with the TestNG framework. The dialog consists of several tabs.

Common options

Name

Specify a name for the run configuration to quickly identify it among others when editing or running.

Allow multiple instances

Allow running multiple instances of this run configuration in parallel.

By default, it is disabled, and when you start this configuration while another instance is still running, IntelliJ IDEA suggests stopping the running instance and starting another one. This is helpful when a run configuration consumes a lot of resources and there is no good reason to run multiple instances.

Store as project file

Save the file with the run configuration settings to share it with other team members. The default location is .idea/runConfigurations. However, if you do not want to share the .idea directory, you can save the configuration to any other directory within the project.

By default, it is disabled, and IntelliJ IDEA stores run configuration settings in .idea/workspace.xml.

Configuration tab

The composition of this tab depends on the selected testing scope: package, project, and so on.

Item	Description
Test kind	All in package – run all tests in a package. In the Package field, specify the fully qualified name of the package. Pattern: run a set of test classes or specific methods within test classes. This set may include classes located in the same or different directories, packages, or modules. Declarations must be separated with `\|\|`. Each class in this field must be represented by its fully qualified name: Include the package name and class name Use `,` as a separator for methods Use `$` as a separator for inner classes You can type class names manually, or click on the right (or press `Shift+Enter`) and search for classes you want to add in the dialog that opens. Example: `packageName.ClassName$InnerClassName,methodName` You may also specify the required classes using regular expressions. For example, if you want to exclude all integration tests that have `IT` in their names, type `^(?!.IT.).*$`. You can create a suite test, that is, a bundle of several test classes that will be run together. To create a suite test class, click on the right and type the test classes you want to be run as a suite. As a result, a new class will be created with the `@Suite` annotation. Group – run a test group. In the Group field, specify the group you want to run. Suite – run a test suite. In the Suite field, enter the corresponding testng.xml file. In whole project – the IDE will look for the tests in all the modules In single module – the IDE will look for the tests only in the module that is selected in the Use classpath of module field. Across module dependencies – the same as the previous option plus the modules that depend on that module. Class – run a test class. In the Class field, enter the fully qualified name of the test class you want to run. Method – run a test method.
Output directory	The directory in which test reports will be generated.

Item

Description

Test kind

All in package – run all tests in a package. In the Package field, specify the fully qualified name of the package.
Pattern: run a set of test classes or specific methods within test classes. This set may include classes located in the same or different directories, packages, or modules. Declarations must be separated with ||.
Each class in this field must be represented by its fully qualified name:
- Include the package name and class name
- Use , as a separator for methods
- Use $ as a separator for inner classes
You can type class names manually, or click on the right (or press Shift+Enter) and search for classes you want to add in the dialog that opens.
Example: packageName.ClassName$InnerClassName,methodName
You may also specify the required classes using regular expressions.
For example, if you want to exclude all integration tests that have IT in their names, type ^(?!.*IT.*).*$.
You can create a suite test, that is, a bundle of several test classes that will be run together. To create a suite test class, click on the right and type the test classes you want to be run as a suite. As a result, a new class will be created with the @Suite annotation.
Group – run a test group. In the Group field, specify the group you want to run.
Suite – run a test suite. In the Suite field, enter the corresponding testng.xml file.
- In whole project – the IDE will look for the tests in all the modules
- In single module – the IDE will look for the tests only in the module that is selected in the Use classpath of module field.
- Across module dependencies – the same as the previous option plus the modules that depend on that module.
Class – run a test class. In the Class field, enter the fully qualified name of the test class you want to run.
Method – run a test method.

Output directory

The directory in which test reports will be generated.

JDK Settings

Item	Description
Test runner parameters	Arguments to be passed to the test runner. Use the same rules as for specifying the VM options.
Working directory	Specify the working directory to be used for running the application. This directory is the starting point for all relative input and output paths. By default, the working directory is the project root.
Environment variables	Click to open the Environment Variables dialog where you can create variables and specify their values.
Use module path	This checkbox is available for projects with JPMS modules (Java 9 and later). It allows you to select how to run your tests: on a class or on a module path.
JRE	The JRE to be used.
Shorten command line	Select a method that will be used to shorten the command line if the classpath gets too long, or you have many VM arguments that exceed your OS command line length limitation. The choice of option depends on the class loader implementation. Note that some frameworks do not support JAR manifest, while other frameworks with custom class loaders will not work well with the classpath.file option. none: IntelliJ IDEA will not shorten a long classpath. If the command line exceeds the OS limitation, IntelliJ IDEA will be unable to run your application and will display a message suggesting that you specify the shortening method. JAR manifest: IntelliJ IDEA will pass a long classpath via a temporary classpath.jar. The original classpath is defined in the manifest file as a `class-path` attribute in classpath.jar. Note that you will be able to preview the full command line if it was shortened using this method, not just the classpath of the temporary classpath.jar. classpath.file: IntelliJ IDEA will write a long classpath into a text file. @argFiles (java 9+): the command-line length limitations will be overcome using the @argument files option. This setting is shared if you select to share your run/debug configuration, so the same method will be applied for your team members irrespective of their operating system.

Parameters

Item	Description
Parameters
Properties file	Specify the .properties file to be passed to TestNG.
Name - Value	Additional parameters as key - value pairs.
Listeners
	Use these icons to make up a list of listeners.

Code Coverage tab

Use this tab to configure code coverage monitoring options.

Item	Description
Packages and classes to record code coverage data	Click and select Add Class or Add Package to specify classes and packages to be measured. You can also remove classes and packages from the list by selecting them in the list and clicking the button.

Logs

The following options are related to logging the execution of this configuration. For more information, refer to View logs.

Item	Description
Specify logs to be shown in console	Specify which log files to display while running the application. Click to add a new log. In the Edit Log Files Aliases dialog, configure the following: Alias: The name of the tab where the log will be displayed. Log File Location: Specify the path to the log file or an Ant pattern. If several files of a rolling log match the pattern, IntelliJ IDEA will display the most recent one. Show all files coverable by pattern: Show all logs that match the pattern. For logs in the table, you can configure the following options: Is Active: Display the specified log file. Skip Content: Do not display old log messages from previous runs.
Save console output to file	Save the console output to the specified location. Type the path manually or click the browse button and point to the desired location in the dialog that opens.
Show console when a message is printed to stdout	Activate the console when the application writes to the standard output stream.
Show console when a message is printed to stderr	Activate the console when the application writes to the standard error stream.

Before Launch

In this area, you can specify tasks to be performed before starting the selected run/debug configuration. The tasks are performed in the order they appear in the list.

Item	Description
Add before launch task	Enable this option to add one of the following available tasks: Launch Web Browser: select this option to have a browser started. In the dialog that opens, select the type of the browser and provide the start URL. Also, specify if you want the browser be launched with JavaScript debugger. Run External tool: run an external application. In the dialog that opens, select one or multiple applications you want to run. If it is not defined in IntelliJ IDEA yet, add its definition. For more information, refer to External tools. Run Another Configuration: select to execute another run/debug configuration and wait until it finishes before starting the current configuration. If you want to run several configurations in parallel, use a compound run/debug configuration. Run Remote External Tool: add a remote SSH external tool. Run Gradle task: run a Gradle task. In the dialog that opens, specify the task and provide additional configuration if necessary. Build: select to compile the specified module. The Build Module action will be executed. If an error occurs during compilation, IntelliJ IDEA won't attempt to start the run/debug configuration. Build Project: select to compile the entire project. The Build Project action will be executed. If an error occurs during compilation, IntelliJ IDEA won't attempt to start the run/debug configuration. Build, no error check: the same as the Build option, but IntelliJ IDEA will try to start the run/debug configuration irrespective of the compilation results. Build Artifacts: select this option to build an artifact or artifacts. In the dialog that opens, select the artifact or artifacts that should be built. Run Maven Goal: select this option to run a Maven goal. In the dialog that opens, select the goal to be run. Run Grunt task: select this option to run a Grunt task. In the Grunt task dialog that opens, specify the Gruntfile.js where the required task is defined, select the task to execute, and specify the arguments to pass to the Grunt tool. Specify the location of the Node.js interpreter, the parameters to pass to it, and the path to the grunt-cli package. Run gulp task: select this option to run a Gulp task. In the Gulp task dialog that opens, specify the Gulpfile.js where the required task is defined, select the task to execute, and specify the arguments to pass to the Gulp tool. Specify the location of the Node.js interpreter, the parameters to pass to it, and the path to the gulp package. Run npm script: select this option to execute an npm script. In the NPM Script dialog that opens, specify the npm run/debug configuration settings. Compile TypeScript: select to run the built-in TypeScript compiler and thus make sure that all the changes you made to your TypeScript code are reflected in the generated JavaScript files. In the TypeScript Compile Settings dialog that opens, select or clear the Check errors checkbox to configure the behaviour of the compiler in case any errors are detected: If the Check errors checkbox is selected, the compiler will show all the errors and the run configuration will not start. If the Check errors checkbox is cleared, the compiler will show all the detected errors but the run configuration still will be launched. Disconnect Data Source: select this option if you want to disrupt the connection to a data source before the run/debug configuration is run.
Open run/debug tool window when started	Depending on the type of configuration, open the Run, Debug, or Services tool window when you start this run configuration. If this option is disabled, you can open the tool window manually: View \| Tool Windows \| Run or `Alt+4` View \| Tool Windows \| Debug or `Alt+5` View \| Tool Windows \| Services or `Alt+8`
Focus run/debug tool window when started	Focus on the run configuration tool window when the tests are running.
Show the run/debug configuration settings before start	Show the run configuration settings before actually starting it.

Last modified: 03 July 2024