The Ultimate List of Code Coverage Tools
Code coverage is the number of lines, statements, or blocks of code tested by your automated tests. It’s a crucial statistic for determining the effectiveness of your quality assurance operations. Code coverage indicates how much of your application is not covered by automated tests and thus susceptible to bugs. Code coverage methods are usually expressed as a percentage, i.e., percentage of code coverage. The closer to 100 percent of code coverage, the better when it comes to demonstrating test coverage to your bosses, a tool for code coverage (along with other trade skills, of course) comes in handy.
Table of content:
- 25 Code Coverage Tools
- How to Choose a Code Coverage Tool
Many open-source and commercial tools have been developed throughout the years to meet the code coverage requirements of every software development project. There’s a code coverage analysis tool for everyone, whether you’re a single developer working on a side project at home, a major corporation with a large DevOps staff, or working on QA for a startup. Many programming languages provide code coverage tools, including many popular QA tools. They’re connected with build tools like Ant, Maven, and Gradle, as well as continuous integration (CI) systems like Jenkins, project management platforms like Jira, and a slew of other software development tools.
We’ve compiled a list of 25 code coverage tools that include alternatives for:
- Open Source Code Coverage Tools
- Commercial Code Coverage Tools
- Quality Assurance and Other Tools with Code Coverage as a Feature
- Code Coverage Tools for Other Programming Languages
Open Source Code Coverage Tools
Several open-source code coverage tools are available, but they are not equal. Some are deprecated, while others are still being developed, and each takes a unique approach to code coverage.
Cobertura
Cobertura is a popular open-source code coverage tool based on the now-retired jCoverage (jcov). It allows you to run tasks using Ant, Maven, or the Cobertura CLI. It’s also included in a number of QA products and continues to exist in different incarnations. Cobertura, like EMMA, is no longer actively maintained and does not support current Java versions.
Programming languages: Java
Developer: Steven Christou
Cost: Free
CodeCover
A “testing technique that analyses the program structure and derives test data from the program logic/code” is known as glass box testing. CodeCover is an open-source Java and COBOL glass-box testing tool. Statement, branch, loop, and stringent condition coverage are all measured by CodeCover.
Ant, Jenkins, JUnit, Eclipse, and other development and testing tools are nicely integrated with CodeCover. The Eclipse Public License governs its use (EPL).
| Programming languages: | Java & COBOL |
| Developers: | Team at University of Stuttgart |
| Cost: | Free |
Coverage.py
Coverage.py is a Python code coverage tool. It keeps track of which sections of your Python scripts have been executed and analyzes the source code to find code that should have been performed but wasn’t.
Programming Languages: Python
Developer: Originally developed by Gareth Rees, maintained by Ned Batchelder (and others) since 2004
Cost: Free
EMMA
Emma is one of the most well-known and widely used code coverage tools. EMMA is the first tool that comes up when searching for code coverage tools on Google. EMMA coverage tests Java program coverage and supports a variety of coverage types, including class, method, line, and block. EMMA can be used in real-time or offline mode. EMMA is not actively updated and does not support the current version of Java, despite its popularity and mature feature set. It is, however, still widely used, and its legacy lives on in the numerous tools and companies it has inspired.
Programming languages: Java
Developer: Vlad Roubtsov
Cost: Free
Gretel
Gretel is a tool for checking residual test coverage. The most recent version (from 2002) includes statement coverage monitoring, which shows which lines of Java have been executed and which code has been justify untested.
Programming languages: Java
Developer: University of Oregon
Cost: Free
Hansel
Hansel is a JUnit module that extends the testing framework to provide code coverage. Much of Hansel’s source code is derived from Gretel, which was previously mentioned, albeit Hansel is JUnit compatible, whereas Gretel is not. Hansel goes beyond the standard code coverage tool by letting developers know how much of the test’s piece of code was expected to be covered.
Programming languages: Java
Developer: Niklas Mehner
Cost: Free
JaCoCo
A demand Part of the Eclipse Foundation replaced EMMA as the code coverage tool that powers the EclEmma plugin, although the name of the plugin remains the same. The depreciation of EMMA and Cobertura has led to an increase in demand for an actively maintained code coverage tool.-JaCoCo was created just for this reason. It can combine with Ant, Maven, Gradle, Jenkins, Visual Studio, etc. It’s one of the few open-source projects that’s still being developed.
Programming languages: Java
Developers: EclEmma team
Cost: Free
JCov
Another open-source code coverage tool is JCov. It is test framework agnostic and supports both offline and on-the-fly instrumentation code (code instrumentation). It is currently not being actively developed.
Programming languages: Java
Developers: YP engineering team
Cost: Free
NUnit
FirstPartners.net created NUnit, which uses Java, XML, and XSLT to assess your JUnit tests in your source project(s), giving you a view of your code and revealing how excellent and thorough your JUnit tests are.
Programming languages: Java
Organization: FirstPartners.net
Cost: Free
PITest
The majority of the code coverage tools on this page look for test coverage for each line, statement, or branch of code (branch coverage). They can only show out which lines of code haven’t been tested in this way. On the other hand, mutation testing can tell you how many lines of code are checked and how effective your automated tests are at finding issues. PITest promises to be the “gold standard test coverage” for mutation testing for Java and the JVM.
PITest accomplishes this by altering your code and then running unit tests on it (runtime). The unit test is valuable if it fails following a code change. However, if the unit test does not fail, it cannot detect the code change and must be modified. PITest is free and open-source, with support for And, Maven, Gradle, Eclipse, and InteliJ.
Programming languages: Java
Developers: Henry Coles
Cost: Free
Quilt
Quilt is a Java software development tool for calculating code coverage designed to work with the JUnit testing framework. It can be employed with Ant, Maven, and other tools in addition to JUnit. The original Quilt was split into three different variations. Quilt 0.6, the most recent version, has primarily new code but is based on the original.
Programming languages: Java
Developers: David Dixon-Peugh
Cost: Free
Serenity BDD
Serenity BDD is a solution for automated acceptance testing that adds code coverage as a feature. It allows you to build tales and epics for each behavior route and track testing coverage for each of them. The code coverage results are far more useful than counting how many lines of code are covered since code coverage is matched to user activity. It works with all standard tools and QA tools like Sauce Labs, BrowserStack, Appium, Jenkins, and Jira. Serenity BDD offers an integrated testing suite based on Selenium, and code coverage is included as a bonus if you use the full suite.
Programming languages: Java, Groovy
Organization: Wakaleo Consulting
Cost: Free
Commercial Code Coverage Tools
While open-source tools are extremely capable, you may require the guarantee of a commercial product for large-scale project management tools. There are quite a few nice options here.
Atlassian Clover
- It examines the code coverage of Java and Groovy applications.
- It integrates with CI tools like Bamboo, Hudson, and Jenkins and builds tools like Ant and Maven.
- Clover is compatible with any Eclipse-based IDE or IntelliJ IDEA.
Programming languages: Java, Groovy
Organization: Atlassian
Cost: $300 – $8,000+
Bullseye Coverage
BullseyeCoverage is a C/C++ code analyzer that determines how to test your source code. It’s used to improve software quality in mission-critical systems such as corporate applications, medical, industrial control, communications, aerospace, and defense. It works with anything written in C and C++, including kernel and system-level code.
Programming languages: C, C++
Organization: Bullseye Testing Technology
Cost: $800 for a 1-year license and up
Froglogic Coco
Coco evaluates C, C++, C#, and Tcl code coverage. It contains Visual Studio add-ins and connects with all major build, CI, and test tools. It works in tandem with FrogLogic’s Squish UI unit testing tool.
Programming languages: C, C++, C# and Tcl code
Organization: FrogLogic
Cost: Not disclosed
Testwell CTC++
Testwell CTC++ is a well-established product that has been in use for more than 25 years and is still being actively developed. As its name suggests, CTC++ is a code coverage tool for C, C+, Java, and C#. Line, statement, function, decision, multi-condition, modified condition/decision coverage (MC/DC), and condition coverage are all covered by it.
It is employed in aerospace, healthcare, and transportation because of its early start and enterprise-friendly capabilities. It’s simple to configure for a variety of uses, and the host-target add-on allows it to be used in embedded programs.
Programming languages: C, C+, Java, and C#
Organization: Testwell, acquired by Verifysoft Technology in 2013
Cost: Not disclosed
NCover
NCover comes in three flavors: a desktop edition with a powerful.NET code coverage solution, Code Central, NCode’s fully integrated, enterprise-class code coverage solution for the entire team, and Collector, which extends Code Central to QA and allows you to construct servers with an automatic collection.
Programming languages: .NET
Organization: gnoso
Cost: Not disclosed (free trials available)
QA and Other Tools with Code Coverage as a Feature
Considering code coverage is an important element of the QA process, it’s no surprise that some of the most popular QA systems include it as a feature. Here are a few examples of tools that include code coverage as part of their package.
Intel C++ Compiler 15.0
Intel’s Intel C++ Compiler 15.0 has a code coverage tool that tells developers how much of an application’s source code is utilized under a given workload. This tool shows dynamic execution counts for each basic block of the application, offers differential coverage data, and visually displays code coverage data with a customizable coloring scheme.
Programming languages: C++
Organization: Intel
Cost: Not disclosed
Parasoft Jtest
Parasoft offers a number of testing tools for tasks such as load testing, security testing, and more. C, C++, Java, and.NET are all supported. Jtest, for example, is one of its products that gives code coverage. Jtest covers code by line, statement, block, decision/branch, path, and MC/DC, as well as condition. This is a wonderful alternative if you’re concerned about your application’s compliance and security.
Code coverage can be broken out by file, class, and function coverage/method in Jtest. It tracks unit tests and integration testing, plugin, server-side, automated end-to-end functional testing i.e. functional tests, scripted testing, and user acceptance testing.
It is mature and interfaces with all essential tools like build tools, CI tools, and IDEs as one of the oldest tools. It tracks a wide variety of indicators, such as errors, inefficiencies, and policy breaches, and generates code coverage report(s)on them. However, the way it is presented is somewhat restrictive, requiring you to sign up in order to read the documentation or gain a better understanding of the product. This isn’t developer-friendly; it’s more suited to traditional businesses.
Programming languages: C, C++, Java, and .NET
Organization: Parasoft
Cost: Not disclosed
SpiraTeam
Inflectra’s SpiraTeam is an Application Lifecycle Management (ALM) product that focuses on increasing code coverage testing and, as a result, code quality. Its ability to track code coverage by requirement is one of its primary advantages. It can track how successfully your code is tested (with a tester) against the requirements you start with, and you can view part of your code coverage metrics per the requirement if you take a project management approach to code coverage. If one spends a lot of time preparing their software projects, this is a wonderful solution (as you should). SpiraTeam also offers a range of pricing choices, as well as the ability to operate your instance on-premise or have it hosted by SpiraTeam.
Programming languages: Any programming language
Organization: Inflectra
Cost: $9.99/mo – $3,999.99 for the hosted option, and $99.99 – $29,999.99 for the on-premise option
Vector Software
Vector Software’s VectorCAST embedded software testing tools platform, a collection of tools that automates testing across the SDLC (software development life cycle), includes code coverage.
Programming languages: C, C++
Organization: Vector Software
Cost: Not disclosed (free trial available)
Code Coverage Tools for Other Programming Languages
The majority of the tools mentioned thus far are designed for Java, C, and C++ programs. On the other hand, other code coverage tools are designed for less often used technologies.
Devel::Cover
Key features of Devel::Cover
Devel::Cover is a Perl application code coverage tool. It examines statement, branch, condition, subroutine, and pod code coverage. It’s free and open-source and Perl’s only code coverage tool. It isn’t as advanced as the other tools on this list, and it lacks integrations with other systems. It’s still being worked on, but not as aggressively as it once was.
Programming languages: Perl
Developer: Paul Johnson
Cost: Free
dotCover
Key features of dotCover
dotCover is also a.NET-only company. It examines unit test coverage and is incorporated into Visual Studio. It can display code coverage directly in the code editor, which is useful. DotCover is included in JetBrains’ ReSharper Ultimate package and is not available separately.
Programming languages: .NET
Organization: JetBrains
Cost: $11.90/mo – $39.90/mo for ReSharper Ultimate product.
OpenCover
OpenCover is a free code coverage tool for.NET versions two and up. It is only compatible with Windows. However, you have a few alternatives for code coverage in Visual Studio, including one that’s built-in.
Programming languages: .NET2 and above
Developer: Community/Shaun Wilde
Cost: Free
Visual Studio
Microsoft’s Visual Studio is a popular IDE. It features built-in tools for collecting code coverage common metric(s), although this functionality is only available in the Visual Studio Enterprise edition.
Programming languages: C, C++
Organization: Microsoft
Cost: $45 monthly for the hosted option, $499 – $11,99 for licensing
How to Choose a Code Coverage Tool
There are numerous factors to consider when selecting a test code coverage tool. To begin, think about your programming language. Java (java code coverage) is supported by most code coverage testing tools, followed by C, C++, and some.NET.
It would be best to consider whether you want to use an open-source or a commercial solution. You could easily make do with an open-source code coverage tool if you’re working on a tiny application and code coverage isn’t a major need. Make sure, however, that the tool you choose is still being actively developed and has a thriving developer community. If you’re utilizing the source tool for mission-critical apps at your company, you’ll most likely require a commercial tool that’s still under development to provide responsive customer service.
A code coverage tool should be well-integrated with a wide range of existing development and QA tools so that your QA team will adopt it and the code coverage metric(s) it gives are valuable.
Code coverage measurements are valuable only if you can exchange and evaluate them to make informed decisions (decision coverage mechanisms). You need quick access to code coverage report(s) and data while utilizing a code coverage tool. Exporting measurements, integrating with other source tool(s), sharing code coverage information across active development teams, and analyzing findings within the tool should all be possible.
Each tool uniquely approaches code coverage. Some follow the tried-and-true method of tracking lines, statements, branches, and application code blocks, then reporting the proportion of the code covered by automated unit tests. Many of the tools on this list, on the other hand, take a different approach to code coverage, focusing on requirements, mutation testing, and glass box testing. It would be appropriate to think about which of these approaches corresponds to your growth philosophy.
Finally, consider code coverage as part of your overall code quality assurance plan.
Code coverage by itself will not provide all of the answers on how to enhance your tests and decrease defects, but it will put you on the right path. If your code coverage is lingering around 50%, it’s time to pay attention and realize that your test suite has to be enlarged. This is critical for goods still in the early stages of development. Even if you grow your test suite to cover 90% of your code base, you may still have inefficient tests that fail to uncover bugs. When you need to assess the code quality of your automated tests, you must look beyond code coverage to other technologies.
Additional Information on Code Coverage
- The term “code coverage” refers to how well your test suite covers your source code. i.e., to what extent does the set of test cases cover the source code?
- Test-coverage- A simple command-line tool for gathering code coverage data from Dart tests.
- Coverage: Another free command-line tool that collects data about the code coverage of your project.
- jscoverage: This free tool, written entirely in JavaScript, is an excellent companion for checking code coverage on both the browser and server sides of your application.
- Line coverage refers to the number of lines in your program that have been executed.
- Condition coverage: Refers to how many Boolean expression(s) in your software have been validated
- Github- Provides PHP code coverage information gathering, processing, and display functionality.
- Gcov is an open-source code coverage analysis and statement-by-statement profiling tool that is available for free.
- Path coverage : Refers to how many pathways of the control structure(s)in your program have been run (if statements or loops i.e. loop coverage)
