Performance engineering is the preferred approach for achieving performance levels consistent with business demands.
IT executives face increasing demands from the corporation to show the return on information technology. Yet the cost of quality (CoQ) has seldom been considered in the cost-benefit analysis. CoQ principles applied routinely and effectively in manufacturing are now emerging as important factors in the application value equation. The "good" part of the good, fast, and cheap equation is getting its due, along with improvement programs such as Six Sigma, TQM, and the like.
Our research shows that in 1998, more than 85% of IT shops conducted return on investment (ROI) analyses of their IT expenditures (up from less than 50% in 1996). Furthermore, with more than 30% of IT organisations now involved in process improvement initiatives (such as Software Engineering Institute's Capability Maturity Model, Six Sigma, ISO 9000-based approaches), many companies have faced the challenge of showing ROI for improved productivity and quality.
By 2001/02, software process improvement will be on the rise again, with renewed interest in quality and agility measures. In the meantime, ROI analyses will become increasingly common, with a growing interest in being able to express the cost of quality and agility.
Applying cost of quality to software
When CoQ was developed for manufacturing, the assumption was that it was always cheaper to do the job right the first time. However, software now challenges this assumption with its requisite changeability. It is supposed to change - otherwise, its functionality would be realised in the hardware. Software adjusts in order to respond to changes in its functionality, quality, or even its purpose. Both software professionals and their customers have come to rely on the ability of software to accommodate changes, generating a whole new meaning to "doing the job right".
IT shops must understand the costs of quality software before they can justify investments in improving quality. The following questions must be answered:
- How much does defective software because of poor quality cost the organisation?
- How good or bad is the software quality in the IT application portfolio?
- How much does it cost to produce quality software?
The costs of quality can be placed in three primary categories:
1. Cost of nonconformance
In dealing with nonconformance, both pre-release and post-delivery quality issues must be addressed. For pre-release, typical cost items include defect management, re-work, re-reviews, and re-testing. For post-delivery, technical support, defect notification, fixes, and updates must be considered.
2. Cost of evaluating conformance
Evaluating conformance entails assessing the quality level of the software produced, both in terms of identifying the nonconformance situations as well as providing quality control checks. To determine the level of nonconformance and understand the condition of the software, normal cost items include inspections, testing, software quality assurance, and reviews. For quality control checks, typical costs include product quality audits and criteria for "go/no-go" decisions. For many IT organisations, costs associated with assessing quality levels are frequently overlooked.
3. Cost of preventing poor quality, or ensuring good quality
To prevent quality issues from arising, effort is expended to set quality goals and establish standards of performance as well as performance zones. It is also important to develop and execute a process improvement program for a quality process to develop quality products. Typical costs include training, process improvement initiatives, metrics collection and analysis, and defining release criteria for acceptance testing.
Cost-benefit analysis for cost of quality
For software quality, the end game centres on controlling costs and managing according to the appropriate quality performance zones. Once the three categories of quality costs are in place, several benefits can be realised.
First, organisation quality costs can and must be compared to the benefits derived. The costs can be compared to the overall development, delivery, and, ultimately, maintenance and support costs. Previously masked costs associated with software quality are revealed and visible to IT management. Economic trade-offs with quality become apparent, leading to more effective decision making. The bottom-line effects of quality programs and process improvements become measurable, visible, and actionable.
Once IT executives see and understand the real cost of software quality, more thorough analysis will lead to proactive positions regarding the competitive environment. Quality costs are baselined and benchmarked against industry norms. Application development and maintenance organisations are then able to deal with software quality in an economic context, thus having a bottom-line effect.
Most CoQ techniques are applied to ROI retrospectively, because most IT executives want to know the payoff of their investments in process improvement. However, we recommend IT organisations consider some other applications and use CoQ to:
- Determine the potential costs and risks of specific quality tradeoffs on critical software projects
- Determine potential legal exposure associated with customer-experienced defects (eg, Y2K exposures)
- Develop a basis for budgeting quality management functions (control and assurance)
- Couple software quality cost data with production cost data to demonstrate the value of staff efforts to the corporate bottom line
- Compare the proposed process improvement initiatives and select those that are most cost-effective
Bottom Line: With increased demand for IT ROI and most IT organisations likely to experience software quality woes post-Y2K, cost of quality (CoQ) techniques are emerging as an effective way to communicate the value of quality initiatives and identify quality initiative candidates. CoQ enables users to understand the economic tradeoffs of delivering quality software.
CW360 would like to hear from you
Where do you draw the line between cost and quality? How do you balance the value equation?
E-mail CW360.com and share your views >>
This was first published in October 2001