Code Coverage

Code coverage is a number of metrics that measure how your code is covered by unit tests. Code coverage information displays for those build configurations, where this metric is configured in the Code Coverage section of a Build Runner (Build Configuration Home Page > Edit Build Configuration Settings > 3.Runners). In TeamCity the code coverage is available for Ant and Ipr build runners. The engine is based on the EMMA open-source toolkit.

View the code coverage summary in the Overview tab of the Build Results Home Page; detailed report displays in the Code Coverage tab. Chart for code coverage is also available in the Statistics of the build configuration.

EMMA Integration Notes

There are several steps when collecting coverage with EMMA:

1. After each compilation step (with javac/javac2), build agent invokes EMMA to instrument compiled classes and to store the location of the source files. As a result, coverage.em file is created in the build checkout directory, which contains classes metadata. The collected source paths of java files are used to generate the final HTML report.

2. Test run. On this stage, actual runtime coverage information is collected. This process results in creation of the file coverage.ec. If there are several test tasks, data is appended to coverage.ec.

3. Report generation. When build ends, TeamCity generates HTML coverage report, creates a zip file with the report (coverage.zip) and uploads it to the server. It also generates and uploads summary report - coverage.txt file, and original coverage.e(c|m) files to allow viewing coverage from TeamCity plugin for IntelliJ IDEA.

Code Coverage Troubleshooting

No coverage, there is a message: EMMA: no output created: metadata is empty

Please make sure that all your classes (whose coverage is to be evaluated) are recompiled during the build. Usually, this requires adding a kind of "clean" task at the beginning of the build.

java.lang.NoClassDefFoundError: com/vladium/emma/rt/RT

This message appears when your build loads EMMA-instrumented class files in runtime, and it cannot find emma.jar file in classpath. For test tasks, like junit or testng, TeamCity adds emma.jar to classpath automatically. But for other tasks, this is not the case and you might need to modify your build script or to exclude some classes from instrumentation.

If your build runs a java task, which uses your own compiled classes, you'll have to either add emma.jar to classpath of the java task, or to ensure that classes used in your java task are not instrumented. Another point, you should run your java task with attribute fork=true.

Corresponding emma.jar file can be taken from buildAgent/plugins/coveragePlugin/lib/emma.jar. For a typical build, corresponding include path would be ../../plugins/coveragePlugin/lib/emma.jar

To exclude classes from compilation, use settings for EMMA instrumentation task. TeamCity UI has a field to pass these parameters to EMMA, labeled "Coverage instrumentation parameters". To exclude some package from instrumenting, use syntax like: -ix -com.foo.task.*,+com.foo.*,-*Test*, where package com.foo.task.* contains files for your custom task.

EMMA coverage results are unstable

Please make sure that your junit task has attribute fork=true. The recommended combination of attributes is "fork=true forkmode=once".