CO3808 and CO2403

Product Quality, Testing, Reviews and Standards

Software Product Quality Ensuring that the required level of quality is

achieved in a software product by defining appropriate quality standards and procedures ensuring that these are followed developing a ‘quality culture’

This involves: Quality assurance -procedures and standards Quality planning - for a particular project Quality control - ensuring that procedures and standards

are followed This is a separate activity from project management

to ensure independence and objectivity

What is Quality and Can it be Measured?

Quality is difficult to pin down, it is based heavily upon expectation and perception

There are some key words that can be associated with Quality Software

Safety Understandability PortabilitySecurity Testability UsabilityReliability Adaptability ReusabilityResilience Modularity EfficiencyRobustness Complexity Learnability

Product Testing Determining whether a product meets its

requirements – is it fit for purpose? Involves developing a testing strategy A testing strategy will require some, or all of the

following processes Verification & Validation (ensuring that the data entering the

system is accurate and reliable) Defect testing (finding and fixing bugs) Integration testing (linking objects, components and modules) Test Reports (confirmations that a particular product meets a

particular specification) Testing workbenches (tools that can assist with the testing

process)

What should a test strategy do?

Consider the overall system and determine the key functional and non-functional requirements

These must be met if a system is to be considered a success, so a testing strategy should be designed to ensure (and prove) that they can be verified and validated

A testing strategy should be designed ‘around’ the system in question and is intended to identify what is to be tested and the methods by which that testing can take place

A testing strategy is not simply about defect testing, it should consider wider project issues such as:

Assessing and Evaluating Risks Issues of criticality, performance and reliability Legal issues Security considerations Concurrency issues, system demand, scalability, portability,

backups and recovery, disaster management

Product Standards Standards are the key to

effective quality management

They may be international, national, organisational or specific project standards

Product standards define characteristics that all components should exhibit

Process standards define how the software process should be enacted

Example Product Standards• Design Review Form• Document Naming

Standards• Interface Design Standards• Object Variable and

Procedure Naming• Procedure header format• Java or C++ style standards• Project plan format• Change Request Form

StandardsWhy are standards important They encapsulate best practice in an attempt to avoid the

repetition of past mistakes They provide a framework for quality assurance process –

products cab be checked for standard compliance Provide continuity in the workplace - new staff can understand

the organisation by understanding the standards applied

Problems with Standards Not seen as relevant and up-to-date by experienced software

engineers (unless they developed the standards!) Often seen to involve too much bureaucratic form filling Can be very specific to an organisation and are therefore

unsupported by software tools so tedious manual work is involved to maintain standards

The quality review process The principal method of validating the quality of a

process or of a product Group examination of part or all of a process or system

and its documentation to find potential problems There are several different types of review with

different objectives Code Inspections, for defect identification and removal at

module level (Product focussed) Progress Reviews, to assess how the overall project is

progressing (product and process focus) Quality Reviews, to examine if the product is matching its

requirements and also to assess whether quality procedures are being followed (at all levels of a project)

Continued… A group of people carefully examine part or all of a software system

and its associated documentation before they are “signed off” as being complete

Code, designs, specifications, test plans, standards, etc. can all be reviewed

The objective is the discovery of system defects and/or inconsistencies Comments made during the review should be classified:

No action. No change to the software or documentation is required. Refer for repair. Designer or programmer should correct an identified fault. Reconsider overall design. The problem identified in the review impacts

other parts of the design. Some overall judgement must be made about the most cost-effective way of solving the problem.

Requirements and specification problems may have to be referred back to the client

Summary To determine if a project is a success, something has to

be measured against an agreed benchmark The quality of a product can be assessed against defined

quality attributes Product Quality can be determined by how well a product

meets its specification (functional and non-functional) Standards provide a framework against which to assess

individual components of a system Quality reviews provide the opportunity to assess quality

against the standards does the product meet the quality attributes does the product fulfil the user requirements have standards been adhered to (do they need changing?)