Software Quality Standards
Software quality refers how well software conforms to functional requirements, how many defects it has and how well it is designed. Several standards organizations have issued their own software quality standards.
IEEE Standards for Software Quality
IEEE 730 is the IEEE standard for software quality assurance plans, and it was updated in 2002. IEEE 730 is shared by ANSI International. IEEE says that IEEE 730 meets the software quality assurance plan requirements set by IEEE/EIA 12207.1.
IEEE 983 was the IEEE guide for planning software quality assurance. IEEE 983 has been withdrawn. IEEE 1298 gave the requirements for managing software quality, but it has been withdrawn.
IEEE 1061 is the standard for defining and gathering software quality metrics. Though issued in 1998, IEEE 1061 is still active.
IEEE 1465 on software package quality requirements and testing has been withdrawn. It has been replaced by ISO standard 12119.
ISO Standards for Software Quality
Most ISO standards for software quality were drawn up by the ISO, although the ISO has also adopted many IEC software quality standards.
ISO 25000 is called the guide to SQuaRE; this stands software product quality requirements and evaluation. The ISO 25000 family is the 2010 set of standards for systems and software engineering. ISO 25001 is the standard for software product quality planning and management. ISO 25012 is the data quality model for SQuaRE.
ISO 25030 gives the quality requirements for software engineering. ISO 25040 gives the quality evaluation process for software. ISO 25045 describes the process of evaluating software modules for recoverability. ISO 25041, released in 2012, is an evaluation guide for SQuaRE for software developers and evaluators.
ISO/IEC 9126 was the quality model used for software engineering. IEC 9126 was made obsolete with the release of ISO/IEC 25010 in 2011.
ISO/IEC 14598 was the standard for software product evaluation. ISO/IEC 14598-1 has been replaced by ISO/IEC 25040.
ISO/IEC 25021 outlines the qualitative methods to measure software quality. ISO/IEC 25051 gives the quality requirements for COTS or commercial off the shelf software. For example, ISO 25051 would apply to commercial pattern recognition software being considered for military applications. This standard replaces ISO/IEC 1211.
ISO/IEC 25062 gives the common industry format or CIMF for software usability test reports.
IEC Standards for Software Quality
Not all IEC software quality standards have been adopted by the ISO. IEC 62814 addresses the quality assurance and dependability testing of reused components, including reused software components. IEC 62628 gives guidance on determining software dependability.
Military Standards for Software Quality
MIL-S-52779 was published in 1976; this mil spec gives the software quality program requirements for Department of Defense contractors. Version A was published in 1979. The nine main requirements of the software quality assurance program under this standard include work tasking procedures, software configuration management, testing, corrective actions, library controls, program design, software documentation, audits and software methodologies.
MIL-STD-1679 is the military specification for software development. Version A of MIL-STD-1679 was published in 1983. MIL-STD-SQAM is the DoD standard for software quality assessment and measurement.
Department of Defense STD 2168 is the federal software quality assurance standard for defense system software. DoD std 2167 is the U.S. Department of Defense standard for software documentation. However, DoD STD 2168 is not a military specification. DoD STD 2168 was absorbed into MIL-STD-498.
Related Software Standards
ISO/IEC 90003 describes how the quality improvement standards of ISO 9001 to software at all stages of its lifecycle.
Unlike the ISO 25000 standards, ISO 90003 applies to software even when it is being maintained, used regularly and debugged after release. This standard replaces ISO 9000-3, though the ISO 90003 standard has sections that lift language directly from the old standard.
Six Sigma quality levels can be achieved with software quality metrics based on defect counts. For example, software metrics counting the number of defects per thousand lines of code can approach six sigma quality levels. Software interfaces can reach six sigma quality levels if there are fewer than 3.4 errors per million transactions or opportunities.
ISO 15288 describes the system life-cycle process. ISO 12207 is the software life cycle process.
ISO 16085 is the standard for software and system engineering risk management during its life cycle, including its maintainability and support. These standards are referenced in many ISO software quality standards.
ISO 14625 is the standard for the performance and quality of ground support equipment and software for space systems. ISO 14300 is the product assurance and quality assurance standard for space products.
IEEE’s standard glossary for software engineering terms is contained in IEEE 729. This terminology is used throughout IEEE, IEC and ISO standards.
Related Software Testing Metrics
Seeding models are used to estimate the number estimate the number of defects in a software application. In software testing, the number of program errors suspected to still exist is called the Number of Errors Left or NOEL metric. This is, however, simply an estimate. There is always the possibility that there are no defects left to find or the defect is too minor to be worth the extended effort to search for it.
The estimated time to fix or ETTF metric is an estimate as to how long it would take to fix an error if it were found. In some cases, if the ETTF is small while the time it would take to find statistically probably but unknown errors is great drives developers to abandon the search and agree to fix it later if a problem actually occurs later.
IEEE Std 1633 is the IEEE recommended practice on software reliability and predicting the errors and defects in software using software reliability models.
Gaps in Today’s Software Quality Standards
Software quality standards rarely address quality issues such as the maintainability or how easily defects are found and fixed, software serviceability or how easily software is inspected and serviced or upgraded, and software repairability, a measure of how easily software can be restored after it fails.
While the ease of upgrading software and debugging are important to users, these factors have not yet been quantified and standardized like software quality metrics that measure the relative number of bugs in a software application.