ArtsAutosBooksBusinessEducationEntertainmentFamilyFashionFoodGamesGenderHealthHolidaysHomeHubPagesPersonal FinancePetsPoliticsReligionSportsTechnologyTravel

Software Testing Metrics

Updated on January 9, 2018
tamarawilhite profile image

Tamara Wilhite is a technical writer, an industrial engineer, a mother of two, and a published sci-fi and horror author.


Quality is ideally quantified by clearly defined metrics. In the area of software quality, there are two major types of metrics. The first class of metrics measure software test execution. How much testing has been done to date, and how much remains to be done? The second set of metrics attempt to quantify the quality of the software itself.

What are the advantages and disadvantages of the different types of software testing metrics?

Software Test Execution Metrics

Software test execution metrics attempt to quantify the progress of software testing. This metric is driven by the need to track progress on Gantt charts and project schedules.

The simplest software test execution metrics are captured by measuring the number of software test procedure steps completed, the number or percentage of test cases completed, and the percentage of software code tested.

Measuring Progress by the Number of Steps

Advantages of Tracking Testing Via the Number of Steps

  • The number of steps performed is easily tracked.
  • This metric can be used with automated test procedures as well as human testing. In the case of automated testing, the number of steps and percentage complete can be traced by the number of tests completed out of the total test procedure.

Disadvantages of Counting the Number of Steps

  • Long software test procedures with a failure at the last step look artificially good. If the final step in a procedure is a failure, the test metric may still say 95% complete.
  • The metric itself changes as test procedure changes. Adding missing steps to test procedures or a few additional checks will cause the percentage of steps executed to go down if testers are at the same place in the procedure.
  • If the software test procedure is simplified and unnecessary tests dropped, the productivity change makes the software quality look worse unless there were defects caught as a result of the dropped software test steps.
  • Less thorough test procedures can look artificially good when this metric is used

Counting the Test Cases Completed to Measure Testing Progress

Advantages of this Software Testing Metric

  • This method of tracking software testing progress is not inflated by tests with many steps.
  • You can consolidate UAT, unit testing and integration testing as test cases into this metric. This method of tracking the progress of testers easily incorporates unit testing, integrated NHA testing and customer acceptance testing as one or more test cases. The number of steps or even the procedure sub-contractors use is irrelevant to the person gathering the metrics, simply the fact that it is done or not yet complete.
  • If one test case is added due to a gap found by engineering, the overall metrics of the project are not appreciably changed whether the new test is passed or a failure.

Disadvantages of This Metric

  • The metric does not reflect the time involved to complete it; someone who completes three small tests may be seen as more productive than someone slogging through a long test sequence.
  • When new defects are found and the number of test cases is increased, the percentage of test cases completed goes down though this effort increases overall software quality.
  • One bad variable or dependency can cause multiple test cases to fail.
  • A particularly problematic test case with multiple failures may be worked on by software engineers who are reducing the number and severity of errors in that one code module or function, but the metric itself remains stuck at the same level.

Defect density metrics and defects per lines of code attempt to standardize defect rates for software programs of all sizes.
Defect density metrics and defects per lines of code attempt to standardize defect rates for software programs of all sizes. | Source

Tracking the Percentage of Code Tested

Advantages of Using This Metric

  • When this software testing method is used, test procedures are written to test all of the code, including rarely used functions.
  • This method of tracking the progress of software testing forces testers to review every code change to ensure that it is all tested, and they may find code that is redundant and easily removed from the app. Code is reviewed by more people, and that is generally a good thing.

Disadvantages of Using This Metric

  • It may be easy to calculate the number of lines of code, but it is harder to determine how many test steps are necessary to verify it all.
  • This metric only tracks progress made in testing, not the quality of the code.
  • It may take a complex series of steps to test rarely used functionality test procedures may not reflect actual user behavior with the application. Thorough testing of a complex software module won’t move the percentage completed bar much, though it can and should be done.

Metrics for the Defects Found in Software Testing

The number of defects found during software testing reflects the quality of the program or the system on which it operates. Ideally, the number of defects found over time should go down as issues are resolved.

There are several ways of tracking and measuring the defects found during software testing. The simplest method is measuring software quality via a direct defect count. A second method is calculating the software defect density. The defects per Lines of Code Metric rivals the software defect density metric.

Tracking Software Testing Defects via Defect Count

Advantages of the Defect Count Metric

  • The defect count is easy to measure. Did you come across a problem or error during testing? Then total up the number of software defects they reported.
  • It is easy to track over time. The metric will not change appreciably if the test procedure is lengthened.

Disadvantages of the Defect Count Metric

  • This metric does not take software complexity into account nor the amount of testing to be performed.
  • The defect count metric may not rate the severity of defects.
  • A number of cosmetic defects and minor errors can inflate the metric total without affecting software quality.
  • Test escapes aren't counted.
  • You need to make sure the defect count isn't inflated by multiple people reporting the same issue.
  • The defect count metric may not take into account whether the defect comes from legacy code, new changes to the application, supporting software applications or the operating system itself. Teasing out the root cause of the defects found could allow the official defect count to come down, but then time is spent determining whether or not to count a defect in the defect count.

Measuring Software Defects Using Defect Densities

The defect density measures the number of defects per unit of software size. This software testing metric is similar to defects per lines of code but does not necessarily count lines of code. It could also look at the executable file size or other variables.

Advantages of the Defect Density Metric

  • This software metric takes the size of a software application into account.
  • If the number of defects remains stable while additional code modules are added, this metric accurately reflects relatively improving overall software quality.
  • Whether basing the density on the lines of code can be counted as opportunities for defects or file size, this metric can be used to determine if your software quality reaches six sigma quality levels.

Disadvantages of the Defect Density Metric

  • Software code reuse may decrease the defect density, while the metric doesn't show the high ratio of defects in new code.
  • The density defect metric does not always reflect the severity of the defects it counts.
  • If developers are able to eliminate software modules and streamline the code faster than the number of defects are reduced, this metric says quality dropped though the application is actually being improved.
  • In theory, software with a virus embedded in it improves the software program's quality by increasing its apparent size.

The Defects per Lines of Code Metric

Advantages of the Defects per Lines of Code Metric

  • This is a simple metric to calculate. Compilers can determine the lines of code present if the programmer cannot.
  • The metric accurately reflects the fact that two programs with an equal number of defects are not equivalent in quality if one contains more code than the other.

Disadvantages of the Defects per Lines of Code Metric

  • Not all code is created equal. Very short programs with serious errors and long programs with a number of minor errors get the same score.
  • There are jokes that programmers paid by the line of code write excessively long programs instead of shorter and more elegant code. And then there is the argument about whether or not documentation within the code can be used to make this metric look better, by interspersing a book among the functional code. The quality metric looks better, but the compilation time and run time go up.
  • Writing additional code to solve a problem looks better when this metric is used compared to finding a solution that does not increase the program’s size.


This website uses cookies

As a user in the EEA, your approval is needed on a few things. To provide a better website experience, uses cookies (and other similar technologies) and may collect, process, and share personal data. Please choose which areas of our service you consent to our doing so.

For more information on managing or withdrawing consents and how we handle data, visit our Privacy Policy at:

Show Details
HubPages Device IDThis is used to identify particular browsers or devices when the access the service, and is used for security reasons.
LoginThis is necessary to sign in to the HubPages Service.
Google RecaptchaThis is used to prevent bots and spam. (Privacy Policy)
AkismetThis is used to detect comment spam. (Privacy Policy)
HubPages Google AnalyticsThis is used to provide data on traffic to our website, all personally identifyable data is anonymized. (Privacy Policy)
HubPages Traffic PixelThis is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.
Amazon Web ServicesThis is a cloud services platform that we used to host our service. (Privacy Policy)
CloudflareThis is a cloud CDN service that we use to efficiently deliver files required for our service to operate such as javascript, cascading style sheets, images, and videos. (Privacy Policy)
Google Hosted LibrariesJavascript software libraries such as jQuery are loaded at endpoints on the or domains, for performance and efficiency reasons. (Privacy Policy)
Google Custom SearchThis is feature allows you to search the site. (Privacy Policy)
Google MapsSome articles have Google Maps embedded in them. (Privacy Policy)
Google ChartsThis is used to display charts and graphs on articles and the author center. (Privacy Policy)
Google AdSense Host APIThis service allows you to sign up for or associate a Google AdSense account with HubPages, so that you can earn money from ads on your articles. No data is shared unless you engage with this feature. (Privacy Policy)
Google YouTubeSome articles have YouTube videos embedded in them. (Privacy Policy)
VimeoSome articles have Vimeo videos embedded in them. (Privacy Policy)
PaypalThis is used for a registered author who enrolls in the HubPages Earnings program and requests to be paid via PayPal. No data is shared with Paypal unless you engage with this feature. (Privacy Policy)
Facebook LoginYou can use this to streamline signing up for, or signing in to your Hubpages account. No data is shared with Facebook unless you engage with this feature. (Privacy Policy)
MavenThis supports the Maven widget and search functionality. (Privacy Policy)
Google AdSenseThis is an ad network. (Privacy Policy)
Google DoubleClickGoogle provides ad serving technology and runs an ad network. (Privacy Policy)
Index ExchangeThis is an ad network. (Privacy Policy)
SovrnThis is an ad network. (Privacy Policy)
Facebook AdsThis is an ad network. (Privacy Policy)
Amazon Unified Ad MarketplaceThis is an ad network. (Privacy Policy)
AppNexusThis is an ad network. (Privacy Policy)
OpenxThis is an ad network. (Privacy Policy)
Rubicon ProjectThis is an ad network. (Privacy Policy)
TripleLiftThis is an ad network. (Privacy Policy)
Say MediaWe partner with Say Media to deliver ad campaigns on our sites. (Privacy Policy)
Remarketing PixelsWe may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.
Conversion Tracking PixelsWe may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.
Author Google AnalyticsThis is used to provide traffic data and reports to the authors of articles on the HubPages Service. (Privacy Policy)
ComscoreComScore is a media measurement and analytics company providing marketing data and analytics to enterprises, media and advertising agencies, and publishers. Non-consent will result in ComScore only processing obfuscated personal data. (Privacy Policy)
Amazon Tracking PixelSome articles display amazon products as part of the Amazon Affiliate program, this pixel provides traffic statistics for those products (Privacy Policy)
ClickscoThis is a data management platform studying reader behavior (Privacy Policy)