Requirements Validation

Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory 1

Requirements Validation

Section FourVersion: 1.0

Mehr 1383


ي نگاه اجمال

ازه�ا، ه�دف از انج�ام يص ني تش�خيابي�ح مفه�وم ارزيتش�ر1.آن

ازهاي نيابي ارزين فاکتورهاييتع2.

ازهاي نيابي ارزيهايژگي ويبررس3.

ازهاي نيابيند ارزي فرآيکهاي تکنيبررس4.


مجموعه پرسشها

انج�ام ي اس�ت و ب�ا چ�ه ه�دفي ب�ه چ�ه مع�نياعتبارس�نج1. شود؟يم

و2. نييه��ايژگيچه از اعتبارس��نجي مرحل��ه در يازه��ا شوند؟ي ميابيارزست؟يازها چي ني اعتبارسنجيکهايتکن3. کنند؟ي شرکت مي در جلسات بازنگريچه افراد4.ست؟يازها چي ني در اعتبارسنجينقش نمونه ساز5.ست؟يازها چي نيت در بازنگري نکات حائز اهم6.


(10- 5 يدهاي مدرس ) اساليراهنما

اس�ال• ايدهايمجموع�ه مع�رفي ب�ه قس�مت کلين ي Verification and Validationاز ب�ه ي� پ�ردازد. علت ني م

Requirement Validation تس�ت مختل�ف س�طوح و رد. ي گ�ي ق�رار مين قس�مت م�ورد بررس�يسيس�تم در ا

ک�ه در انج�ام ييازه�ا و فاکتوره�اي نيابي�و ه�دف از ارز شود.يت مطرح هستند مشخص مين فعاليا


What is V&V?• Purpose of V&V is to deliver a product that works

properly and performs to customer expectations.

• Verification is the process of confirming deliverable hardware and software are in compliance with the functional, performance, and design requirements.(Have we built the system right?)

• Validation is the process of providing evidence that a systems meets the needs of the User.(Have we built the right system?)

What we are trying to do is to establish confidence in the system, support equipment, test equipment, and people.

What


V&V

• The definition of V&V encompasses many of the activities that refer as Software Quality Assurance (SQA).

• V&V encompasses a wide array of SQA activities that include:– Formal technical reviews – Quality audits– Performance monitoring– Simulation– documentation review– testing


Different levels testing

UnitTesting

Integration Testing

Co

de

Integration Testing

Acceptance Testing

Validation Testing

Des

ign

Req

uir

emen

ts


Requirements validation

• Concerned with demonstrating that the requirements define the system that the customer really wants. (Define the right system)

• Pickup any problem before resources are committed to addressing the requirements.

• Requirements error costs are high so validation is very important– Fixing a requirements error after delivery may cost

up to 100 times the cost of fixing an implementation error

What


Relative Cost to Fix an Error

Phase in Which Requirement is Found

Cost Ratio

Requirements 1

Design 3-6

Coding 10

Development Testing 15-40

Acceptance Testing 30-70

Operation 40-1000

Why


Requirements checking

• Validity. Does the system provide the functions which best support the customer’s needs?

• Consistency. Are there any requirements conflicts?

• Completeness. Are all functions required by the customer included?

• Realism. Can the requirements be implemented given available budget and technology

• Verifiability. Can the requirements be checked?

What


(31-12 يدهاي مدرس ) اساليراهنما

س�تم مش�خص يک سي� يازه�اي نيه�ايژگين قس�مت ويدر ا•ايم ب�ه و ش�وند مي پرداخت�ه موض�وع ک�ه ين ش�ود

رد ک�ه ي انج�ام گ�يد ب�ه ص�ورتي�س�تم باي سيازه�اي نيبررس�ص داده يس�تم تش�خيک سي� ي ک�ه ب�راييازه�ايک از ني�ه�ر

ها را دارا باشند.يژگين وي شوند همه ايمق�ابل• ، وض�وح ب�ودن، کام�ل ثب�ات، يدقت، تس�ت، ت

ت ير و ق�ابلي�يت تغيک مش�کل مش�خص، ق�ابلي�ارتب�اط ب�ا ه�ر ين قس�مت ب�راي هس�تند ک�ه در اييه�ايژگي ويريگ�يپني� از فرآيک و مستندس�ازيازه�ا تع�ين�د و آنه�ا يي ن

گردند.ي ميمعرف


Characteristics of requirements

• Correct• Unambiguous• Complete• Testable• Consistent• Deal only with the problem• Modifiable• Traceable

What


Correct

• Each requirement statement accurately represents the functionality required of the system to be built.

• Example (of an incorrect requirement):• Problem domain (real life) states that

policeman ID numbers are in the range [10000…) and

• the requirements document specifies that each policeman has an ID number.

When




What


Unambiguous• The difficulty of ambiguity stems from the use of natural

language which in itself is inherently ambiguous.

• There is one and only one interpretation for every requirement.

• Requirement statements should be short, explicit, precise and clear.

• A glossary should be used when a term used in a particular context could have multiple meanings (I.e. “the user”).

• Formal requirements languages help reduce ambiguity.

When


Unambiguous (cont. )

• Example of ambiguity– The TVRS shall store changes made in the details of

a traffic report as soon as the data is entered.

• Disambiguation:– The TVRS shall store changes made in the details of

a traffic report if and only all input fields are valid and user approved saving of data

How




What


Complete• A requirements document is complete if it

includes all of the significant requirements, relating to – functionality,– performance,– design constraints attributes – external interfaces.

• No sections are marked “To be determined” (TBD).

• Conforms to the company standards.

When




What


Testable

• A requirements document is testable (verifiable) if and only if every requirement statement in it is testable.

• A requirement is testable if and only if there is some finite cost-effective way in which a person or machine can check to see if the software product satisfies that requirement.

When


Testable (cont.)

• Example of a non-verifiable requirement:– The TVRS shall complete storage of data within a

reasonable time of the user confirming a “Save” sequence.

• Example of a verifiable requirement:– The TVRS shall complete storage of data within 5

seconds of the user confirming a “Save” sequence, 80% of the time.

How




What


Consistent

• Three types of conflicts:– Different terms used for the same object:

• F323 and a “Policeman Details Form” might be used to describe the same form.

– Characteristics of objects:• In one part of the requirements document:

– “A policeman ID shall consist of decimal digits only”,

• while in another part– “Incase the policeman ID consists of non-

alphanumerical characters, display an error message”.

When


Consistent (cont.)

– Logical or temporal faults: “A follows B” in one part, “A and B occur simultaneously” in another.

• “TVRS shall support removal of a policeman record from the personal database” vs. “TVRS shall support read-only access to policeman details”.

Do clients know what a database

is?

When




What


Deal only with the problem

• Requirements should state “what” is required at the appropriate system level, not “how”.– In some cases, a requirement may dictate how a task

is to be accomplished.

• Avoid telling the designer “how” to do this job, instead state “what” has to be accomplished.

• Requirements should be understood by the clients as well as the developers.

When




What


Modifiable

• A requirements document is modifiable if its structure and style are such that changes can be made easily, completely and consistently:– Easy to use organization – table of contents,

index and cross references.– No redundancy – a requirement should not be

in more than one place.

When




What


Traceable

• Each requirement should be contained in a single, numbered paragraph so that it may be referred to in other documents:

– Backward traceability - implies that we know where every requirement exists

• Each requirement explicitly references its source in previous documents.

– Forward traceability – all documents to follow will be able to reference each requirement.

When


Traceable (cont.)

• Example:– 2.1 Functional requirements:

• 2.1.1 TVRS initialization:– …– 2.1.15 TVRS shall display the user login window (see

section 2.1.2.2)

• 2.1.2 TVRS user interfaces:– 2.1.2.1 All user interaction with the TVRS shall occur by

means of a graphical user interface.– 2.1.2.2 User login window:

» …

How



اس�ال• ايدهايمجموع�ه تکني قس�مت در ييکه�اين ک�ه رد را ي گ�يازه�ا م�ورد اس�تفاده ق�رار مي نيابي�ن�د ارزيفرآ

د.ي نماي ميمعرفعن�وان يبازنگر• ب�ه تکنيکي از ارزيکه�اي ايابي� در ن ي

، ش�رکت يردد. زم�ان انج�ام ب�ازنگريگ�يح ميقس�مت تش�رب�ازنگر در جلس�ات و ويکنن�دگان ه�ر يه�ايژگي از ي� ک

نک�ات و بايآنه�ا ک�ه ب�ازنگري� در ق�رار يد توج�ه م�ورد شود.يل شرح داده مين قسمت به تفصيرند، در ايگ


Requirements validation techniques

• Requirements reviews• Prototyping• Acceptance tests• Model Validation and Automated consistency a

nalysis

What


Requirements reviews• Requirements review is a systematic manual

analysis of the requirements

• Regular reviews should be held while the requirements definition is being formulated

• Both client and contractor staff should be involved in reviews

• Reviews may be formal (with completed documents) or informal.

What


General Guidelines for Review• Good communications between developers,

customers and users can resolve problems at an early stage

• Always conduct reviews in a meeting format, although the meeting participants might prepare some reviews on their own.

• Continuously check what is produced to make sure the product quality is as high as possible. Checkpoints are provided for this purpose; refer to the checkpoints for each analysis activity.

What


Roles in Review Meetings

• a person has essential knowledge of the business or technology domain and detailed knowledge of the applied facilitation and modeling techniques: – Requirements Reviewer – Requirements Specifier – System Analyst

• possibly at milestones such as the beginning or end of a Phase: – Stakeholders - customers and end-users (Where possible) – Change Control Manager (Where reviewing Change Requests) – Test Designer (Optional) – Software Architect (Optional, usually in Inception and

Elaboration) – Project Manager (Optional, usually at Phase Start)

Who


Review checks

• Verifiability. Is the requirement realistically testable?

• Comprehensibility. Is the requirement properly understood?

• Traceability. Is the origin of the requirement clearly stated?

• Adaptability. Can the requirement be changed without a large impact on other requirements?

What


RUP Checkpoints: Requirements Attributes

– Has the correct set of requirements attributes been used as specified in the Requirements Management Plan?

– Have attributes been set up for each requirement type to account for the following, where applicable, for each requirement?

• Tracking status? • Benefit? • Rationale? • Level of effort to implement? • Type and amount of each type of risk involved in implementing?

– Schedule risk? – Resource risk? – Technical risk?

• Stability of the requirement? • Target release? • Assignment? • Marketing input? • Development input? • Revision history? • Location? • Reasons for change? • Inconclusive requirements?

– Have all traceabilities been set up as specified for the project in the Requirements Management Plan?

•

What


(60 -40يدهاي مدرس ) اساليراهنما

يابي� اس�ت ک�ه در ارزيي از ابزاره�ايکي ينمون�ه س�از•نيص ص�حيتش�خ سي� يازه�ايح اس�تفاده يک م�ورد س�تم

رد.ي گيقرار مب يا و مع�اي�، مزاي نمون�ه س�ازين قس�مت ب�ه مع�رفيدرا•

س�اخت نمون�ه ه�ا وج�ود ي ک�ه ب�راي مختلفيآن، روش�هاو و يه��ايژگيدارد، آن اس��تفاده م��ورد روش، ه��ر ب�ه ک�ار مييابزاره�ا رود ، ي ک�ه در س�اخت نمون�ه ه�ا

شود.يپرداخته م




nalysis

What


System prototyping

• Prototyping is the rapid development of a system

• Using an executable model of the system to check requirements.

• In the past, the developed system was normally thought of as inferior in some way to the required system so further development was required

• Now, the boundary between prototyping and normal system development is blurred and many systems are developed using an evolutionary approach

What


Uses of system prototypes

• The principal use is to help customers and developers understand the requirements for the system– Requirements elicitation. Users can experiment

with a prototype to see how the system supports their work

– Requirements validation. The prototype can reveal errors and omissions in the requirements

• Prototyping can be considered as a risk reduction activity which reduces requirements risks

What


Prototyping benefits

• Misunderstandings between software users and developers are exposed

• Missing services may be detected and confusing services may be identified

• A working system is available early in the process• The prototype may serve as a basis for deriving a

system specification• The system can support user training and system

testing

Why


Prototyping benefits

• Improved system usability• Closer match to the system needed• Improved design quality• Improved maintainability• Reduced overall development effort

Why


Prototyping in the software process

• Evolutionary prototyping– an initial prototype is produced and refined

through a number of stages to the final system

• Throw-away prototyping– a practical implementation of the system is

produced to help discover requirements problems and then discarded. The system is then developed using some other development process

What


Prototyping objectives

• Evolutionary prototyping – to deliver a working system to end-users. – The development starts with those requirements

which are best understood.

• Throw-away prototyping– to validate or derive the system requirements.– The prototyping process starts with those

requirements which are poorly understood

What


Evolutionary prototyping

• Must be used for systems where the specification cannot be developed in advance e.g. AI systems and user interface systems

• Based on techniques which allow rapid system iterations

• Verification is impossible as there is no specification.

• Validation means demonstrating the adequacy of the system.

What



Build prototypesystem

Develop abstractspecification

Use prototypesystem

Deliversystem

Systemadequate?

YES

N

What


Evolutionary prototyping advantages

• Accelerated delivery of the system– Rapid delivery and deployment are

sometimes more important than functionality or long-term software maintainability

• User engagement with the system– Not only is the system more likely to meet

user requirements, – they are more likely to commit to the use of

the system

Why



• Specification, design and implementation are inter-twined

• The system is developed as a series of increments that are delivered to the customer

• Techniques for rapid system development are used such as CASE tools and 4GLs

• User interfaces are usually developed using a GUI development toolkit

How


Evolutionary prototyping problems

• Management problems– If management processes assume a

waterfall model of development– Specialist skills are required which may not

be available in all development teams• Maintenance problems

– Continual change tends to corrupt system structure so long-term maintenance is expensive

• Contractual problems

What


Prototypes as specifications

• Some parts of the requirements (e.g. safety-critical functions) may be impossible to prototype and so don’t appear in the specification

• An implementation has no legal standing as a contract

• Non-functional requirements cannot be adequately tested in a system prototype

What


Throw-away prototyping

• Used to reduce requirements risk• The prototype is developed from an initial

specification, delivered for experiment then discarded

• The throw-away prototype should NOT be considered as a final system– Some system characteristics may have been left

out– There is no specification for long-term maintenance– The system will be poorly structured and difficult to

maintain

What


Prototype delivery

• Developers may be pressured to deliver a throw-away prototype as a final system

• This is not recommended– It may be impossible to tune the prototype

to meet non-functional requirements– The prototype is inevitably undocumented– The system structure will be degraded

through changes made during development– Normal organisational quality standards

may not have been applied

Why not


Rapid prototyping techniques

• Various techniques may be used for rapid development– Dynamic high-level language development– Database programming– Component and application assembly

• These are not exclusive techniques - they are often used together

• Visual programming is an inherent part of most prototype development systems

What


Component and application assembly

• Prototypes can be created quickly from a set of reusable components plus some mechanism to ‘glue’ these component together

• The composition mechanism must include control facilities and a mechanism for component communication

• The system specification must take into account the availability and functionality of existing components

What


Visual programming

• Scripting languages such as Visual Basic support visual programming where the prototype is developed by creating a user interface from standard items and associating components with these items

• A large library of components exists to support this type of development

• These may be tailored to suit the specific application requirements

What


Visual programming with reuse

File Edit Views Layout Options Help

GeneralIndex

Hypertextdisplay componentDate component

Range checkingscript

Tree displaycomponent

12th January 2000

3.876

Draw canvascomponent

User promptcomponent +

script

What


Problems with visual development

• Difficult to coordinate team-based development• No explicit system architecture• Complex dependencies between parts of the

program can cause maintainability problems

What


User interface prototyping• It is impossible to pre-specify the look and feel of a

user interface in an effective way. Prototyping is essential

• UI development consumes an increasing part of overall system development costs

• User interface generators may be used to ‘draw’ the interface and simulate its functionality with components associated with interface entities

• Web interfaces may be prototyped using a web site editor

What



•Acceptance Test يابي� ارزيکه�اي از تکنيکي ب�ه عن�وان ردد. ح�دود انج�ام اين تس�ت يگ�يح مين قس�مت تش�ريدر ا

Beta و Alpha Testing شود و در نهايت يمشخص مTestingشوند.ي مي تست نرم افزار معرفي برا

ش�ده و از طري�ق آن ي م�دلها بررس�يسپس اعتبارس�نج• ش�ده و م�وارد اس�تفاده ي تحليلي بررس�يکيفيت م�دلها

شود.ياز اعتبارسنِجe مدلها بيان م




nalysis

What


Acceptance Test

• Enable the customer to validate all requirements

• Conducted by end user

• Can range from informal “test drive” to a planned and systematically executed series of tests

• If software is developed as a product to be used by many customers a process called alpha and beta testing is used.


Alpha Testing

• Conducted at the developer’s site by a customer

• Software is used in a natural setting with the developer “looking over the shoulder” of the user and recording errors and usage problems.


Beta Testing

• Conducted at one or more customer sites by the end-user of the software.

• The developer is generally not present.• Live application of the software in an

environment that can not be controlled by the developer.

• Customer records all problems that are encountered and reports them to the developer.

• Developer modifies software and then prepare for release of the software product.




nalysis

What


Model Validation• Validating the quality of analysis models

• Is used to check the consistency of a structured requirements description

• E.g. Static analysis to verify that communication paths exist between objects that in stakeholder domain exchange data.

• If formal specification is used, using formal reasoning to prove properties of the requirements specification (e.g. completeness)

What


پاسخ به پرسشها

ياست و با چه هدف يبه چه معن ياعتبارسنِج1.شود؟ يانجام م

يازها در مرحله اعتبارسنِجياز ن ييهايژگيچه و2.شوند؟ يم يابيارز

ست؟يازها چين ياعتبارسنِج يکهايتکن3.

کنند؟ يشرکت م يدر جلسات بازنگر يچه افراد4.

ست؟يازها چين يدر اعتبارسنِج ينقش نمونه ساز5.

ست؟يازها چين يت در بازنگرينکات حائز اهم 6.


What is V&V?• Purpose of V&V is to deliver a product that works

properly and performs to customer expectations.

• Verification is the process of confirming deliverable hardware and software are in compliance with the functional, performance, and design requirements.(Have we built the system right?)

• Validation is the process of providing evidence that a systems meets the needs of the User.(Have we built the right system?)

What we are trying to do is to establish confidence in the system, support equipment, test equipment, and people.

What






کنند؟ يشرکت م يدر جلسات بازنگر يچه افراد4.

ست؟يازها چين يدر اعتبارسنِج ينقش نمونه ساز5.

ست؟يازها چين يت در بازنگرينکات حائز اهم 6.


Requirements checking

• Validity. Does the system provide the functions which best support the customer’s needs?

• Consistency. Are there any requirements conflicts?

• Completeness. Are all functions required by the customer included?

• Realism. Can the requirements be implemented given available budget and technology

• Verifiability. Can the requirements be checked?

What






کنند؟ي شرکت مي در جلسات بازنگريچه افراد4.

ست؟يازها چي ني در اعتبارسنجينقش نمونه ساز5.

ست؟يازها چي نيت در بازنگري نکات حائز اهم6.




nalysis

What


Roles in Review Meetings

• a person has essential knowledge of the business or technology domain and detailed knowledge of the applied facilitation and modeling techniques: – Requirements Reviewer – Requirements Specifier – System Analyst

• possibly at milestones such as the beginning or end of a Phase: – Stakeholders - customers and end-users (Where possible) – Change Control Manager (Where reviewing Change Requests) – Test Designer (Optional) – Software Architect (Optional, usually in Inception and

Elaboration) – Project Manager (Optional, usually at Phase Start)

Who


Uses of system prototypes

• The principal use is to help customers and developers understand the requirements for the system– Requirements elicitation. Users can experiment

with a prototype to see how the system supports their work

– Requirements validation. The prototype can reveal errors and omissions in the requirements

• Prototyping can be considered as a risk reduction activity which reduces requirements risks

What


General Guidelines for Review• Good communications between developers,

customers and users can resolve problems at an early stage

• Always conduct reviews in a meeting format, although the meeting participants might prepare some reviews on their own.

• Continuously check what is produced to make sure the product quality is as high as possible. Checkpoints are provided for this purpose; refer to the checkpoints for each analysis activity.

What

Requirements Validation

Documents

Transcript of Requirements Validation