1. Standard / procedure description
    1. Purpose of software quality metrics

Software quality is the ratio to which software has a desired combination of qualities. This desired combination of qualities must be clearly determined; otherwise, estimation of quality is left to a hunch. For the purposes of this standard, determining software quality for a system is equal to determining a list of software quality traits necessary for that system. An apt collection of software metrics must be recognised in order to measure the software quality attributes.

The purpose of software metrics is to make assessments throughout the software life cycle as to whether the software quality requirements are being met. The use of software metrics reduces subjectivity in the assessment of software quality by providing a quantitative basis for making decisions about software quality.

However, the use of software metrics does not eliminate the need for human judgement in software evaluations. The use of software metrics within an organisation or project is expected to have a beneficial effect by making software quality more visible.

More specifically, the use of metrics within the methodology of this standard allows an organisation to:

To accomplish these aims, both process and product metrics should be represented in the system metrics plan.

 
    2. Software quality metrics framework

The software quality metrics framework (See Figure 1 below) begins with the establishment of quality requirements by the assignment of various quality attributes. All attributes which define the quality requirements must be agreed upon by the project team, and definitions established. Quality factors, which represent management and user oriented views are then assigned to the attributes, then sub-factors, if necessary, assigned to each factor. Associated with each factor is a direct metric, which serves as a quantitative representation of a quality factor. For example, a direct metric for the factor reliability could be mean time to failure. Each factor must have one or more associated direct metrics and target values, such as one hour of execution time, that is set by project management. Otherwise, there is no way to determine whether the factor has been achieved.

At the second level of the hierarchy are the quality sub-factors, which represent software-oriented attributes that indicate quality. These are obtained by decomposing each factor into measurable software attributes. Sub-factors are independent attributes of software, and therefore may correspond to more than one factor (refer to Appendix A for further explanation). The sub-factors are concrete attributes of software that are more meaningful than factors to technical personnel, such as analysts, designers, programmers, testers, and maintainers. The decomposition of factors into sub-factors facilitates objective communication between the manager and the technical personnel regarding the quality objectives.

At the third level of the hierarchy the sub-factors are decomposed into metrics used to measure system products and processes during the development life cycle. Direct metric values (factor values) are typically unavailable or expensive to collect early in the software life cycle. Therefore, metrics on the third level, that are validated against direct metrics, are used to estimate factor values early in the software life cycle.

The framework, in a top-down fashion, facilitates:

On the other hand, the framework, in a bottom-up fashion, enables the managerial and technical personnel to obtain feedback by:

The framework is designed to be flexible. It permits additions, deletions, and modifications of factors, sub-factors, and metrics. Each level may be expanded to several sub-levels. The framework can thus be applied to all systems and can be adapted as appropriate without changing the basic concept.

Figure 1 Software Quality Metrics Framework
    1. The software quality metrics methodology
      1. Introduction

The software quality metrics methodology is a systematic approach to establishing quality requirements and identifying, implementing, analysing and validating process and product software quality metrics for a software system. It spans the entire software life cycle and comprises five steps.

These steps shall be applied iteratively because insights gained from applying a step may show the need for further evaluation of the results of prior steps.
    1. Establish Software Quality Requirements - A list of quality factors is selected, prioritised and quantified at the outset of system development or system change. These requirements shall be used to guide and control the development of the system and, on delivery of the system, to assess whether the system met the quality requirements specified in the contract.
    2. Identify Software Quality Metrics - The software quality metrics framework is applied in the selection of relevant metrics.
    3. Implement the Software Quality Metrics - Tools are procured or developed, data is collected, and metrics are applied at each phase of the software life cycle.
    4. Analyse the Software Quality Metrics Results - The metrics results are analysed and reported to help control the development and assess the final product.
    5. Validate the Software Quality Metrics - Predictive metrics results are compared to the direct metrics results to determine whether the predictive metrics accurately "measure" their associated factors.

The documentation produced as a result of the preceding steps is shown in Table 1