WARNING! THIS PRESENTATION CONTAINS CONDENSED MATTER PR-101005-a-JGU, October 7th, 2010 J. Gutleber...

128
WARNING! THIS PRESENTATION CONTAINS CONDENSED MATTER PR-101005-a-JGU, October 7th, 2010 J. Gutleber 1

Transcript of WARNING! THIS PRESENTATION CONTAINS CONDENSED MATTER PR-101005-a-JGU, October 7th, 2010 J. Gutleber...

J. Gutleber1

WARNING!THIS PRESENTATION CONTAINS CONDENSED MATTER

PR-101005-a-JGU, October 7th, 2010

J. Gutleber2

Systems Engineeringwith Enterprise Architect

October 15th, 2010Johannes Gutleber

Roland Moser

PR-101005-a-JGU, October 7th, 2010

J. Gutleber3

PR-101005-a-JGU, October 7th, 2010

- Model

J. Gutleber4

What is Systems Engineering?• Delivers a product and a process for producing it.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber5

Iterative V-Model

• Can be seen as an extension to the V-Model• Phases may trigger changes in a previous phase

• E.g. Design problem triggers requirement change

• Iterative process• Concurrent work in each of

the steps possible

PR-101005-a-JGU, October 7th, 2010

J. Gutleber6

What is covered by Enterprise Architect?

PR-101005-a-JGU, October 7th, 2010

J. Gutleber7

Requirements Gathering

PR-101005-a-JGU, October 7th, 2010

J. Gutleber8

Goals

• Problem to be solved and goals to be achieved must be stated in clear, unambiguous manner.

• Problem statement explains • needs, • goals, • capabilities, • scope of the system, • concept of operations, • stakeholders, • deliverables, • key decisions to be made

PR-101005-a-JGU, October 7th, 2010

J. Gutleber9

Inception Condition

• If there is no problem statement stop here.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber10

Properties of Problem Statement

• No reference to specific implementations.• Similar to a patent

• Serve actually as good examples for problem statements

• Defines the top-level function or service

PR-101005-a-JGU, October 7th, 2010

J. Gutleber11

Problem Statement Example

• Original Example• The system shall hold together 2 to 20 pieces of A4, 100g paper.

• Improved Example• I typically produce reports consisting of 2 to 20 pieces of A4, 100g

paper. The pages frequently get out of order and become mixed up with pages of other reports.

PR-101005-a-JGU, October 7th, 2010

Stable, paper clip, fold corner, put pages in a folder, glue together,

The above plus:Number pages, ring binder, throw

away old reports, keep them electronically, …

J. Gutleber12

Next steps

• Response to the GOAL statement is not necessarily a system!• Procedure in a manual• Organization

• Requirements must reflect the GOAL statement• If goal not clear, requirements will result in a solution that is not

wanted!

• GOALS and PROBLEM statement are the MOST IMPORTANT ITEMS in the process.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber13

FREE YOUR MINDFROM SOLUTIONS!

PR-101005-a-JGU, October 7th, 2010

J. Gutleber14

Stakeholders

• Can be anybody!• Users, Operators, Regulatory agencies, Victims, Sponsors, Maintainers,

Architects and designers, Managers, testers, risk managers, purchasing officers, environment, implementers, owners, and many more!

• All relevant stakeholders need to be • Identified• Involved in requirements finding process

• Common sense is important!!!• Stakeholders may be represented• Only the “needed” stakeholders should be involved• Don’t underestimate the effort (similar to selecting a jury for court)

PR-101005-a-JGU, October 7th, 2010

J. Gutleber15

OUR WORKING EXAMPLE

PR-101005-a-JGU, October 7th, 2010

J. Gutleber16

Problem Statement• The Little Endians need a convenient and cheap way to get to

the Big Endians.• The Little Endians are separated from the Big Endians

by a river that is 100 meters wide.

PR-101005-a-JGU, October 7th, 2010

• During validation, it must be checked if the initial goals are fulfilled.

• Problem statement may include vague terms like “cheap”, “convenient”. The requirements derived must not, since the vague terms must be clarified by then.

J. Gutleber17

WHAT should we do?

PR-101005-a-JGU, October 7th, 2010

J. Gutleber18

Requirements

• Specify what has to be done to reach the goals• Requirements must be solution independent• Existing best-practices can be taken into consideration

• Specified as design constraints

PR-101005-a-JGU, October 7th, 2010

J. Gutleber19

What is a Requirement?

PR-101005-a-JGU, October 7th, 2010

A statement that identifies a capability or function needed by a system in order to satisfy its customer’s needs.

Statement,short, concise WHAT not how

There’s reason and

justification

There’s a scope

Somebody needs it

J. Gutleber20

Holy Rule

• For every requirement ask yourself WHY it is needed• If the question cannot be answered it is not a requirement• Until not clearly answered iterate on the same requirement• Provide the justification and possibly the verification

procedure for the requirement in a “Rationale”.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber21

Not Necessarily Technical

• Requirements may not necessarily be technical• May also describe a condition or capability to

• Solve a problem• Achieve an objective• Satisfy a contract• Satisfy a standard• Satisfy another specification

PR-101005-a-JGU, October 7th, 2010

J. Gutleber22

Types of Requirements

• Functional• What, how well and under what conditions

a capability needs to be provided• Example: Paint the house

• Non-functional• Take into consideration quality criteria

• Performance• Cost

• Example: Perform the painting in less than 1 hour.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber23

The Tool

• Can do everything (almost)

PR-101005-a-JGU, October 7th, 2010

J. Gutleber24

Enterprise Architect

• What if offers• Modeling environment• Covers the full systems engineering lifecycle• Offers different modeling languages and concepts

• What we use it for• Database of requirements• Database of design components• Database of test cases• Database of hazards, effects, harms, risks and mitigation strategies• Linking of all of the above• Generating reports for all of the above

PR-101005-a-JGU, October 7th, 2010

J. Gutleber25

Availability

• MedAustron has licenses for WP CO• Additional licenses are bought on demand by WP CO• Tool is made available by SparxSystems Web page

• http://www.sparxsystems.com.au/

• Reports and plugins are maintained by WP CO• Cosylab provides development services and support

• User manuals are under preparation

PR-101005-a-JGU, October 7th, 2010

J. Gutleber26

Prepare Enterprise Architect Model

PR-101005-a-JGU, October 7th, 2010

• Download the EA model template fromhttp://indico.cern.ch/conferenceDisplay.py?confId=110606

• Additional material is available in the MACS SVN repository• http://svnweb.cern.ch/trac/macs/• Log in and go to the “browser” following the path

/trunk/sdp/documents/Requirements/RequirementsDocumentTemplate/• Contains

• Model template• Requirements guidelines• Example for this session

J. Gutleber27

Starting EA

PR-101005-a-JGU, October 7th, 2010

Project Browser

Diagram Toolbox

Document/Diagram View Requirements/

Component contents

J. Gutleber28

Enterprise Architect Project Browser

PR-101005-a-JGU, October 7th, 2010

Requirements

Design

Testing

Risk Management

• Contains 4 main packages

J. Gutleber29

Enterprise Architect Model Structure

PR-101005-a-JGU, October 7th, 2010

• Each main package contains• Introduction (predefined)

• Purpose• Scope• Definitions, Acronyms and Abbreviations• References• Overview

• Description• General description of the system

in regards to the main package• Package specific packages

• E.g. Requirements chapter

J. Gutleber30

Linked document

PR-101005-a-JGU, October 7th, 2010

• Right-click package and select Linked Document• <CTRL>+<ALT>+D

• Select Text indented by 1.5 cm template• in case no linked document exists yet

• Enter text to be stored in this section• Document is embedded in model!

J. Gutleber31

Linked document (2)

PR-101005-a-JGU, October 7th, 2010

J. Gutleber32

Recommendations

• Use any word editor of your choice• Copy an paste into the RTF editor of Enterprise Architect• Create images with a drawing tool

• MS-Visio using FMC stencils is recommended!• Save in <package>/img directory• Copy-Paste the images into the RTF editor

PR-101005-a-JGU, October 7th, 2010

J. Gutleber33

Report Templates

• To view entered data and to distribute them, reports are generated.

• Press “F8”• The templates need to be customized for each model

• Requirements, Architecture & Design, Testing, Risk Management• Provide Title, Document ID, Version, Authors, …• Content elements can also be included or excluded

• Include diagrams, drawings, more/less details

PR-101005-a-JGU, October 7th, 2010

J. Gutleber34

Edit Requirements Template

PR-101005-a-JGU, October 7th, 2010

Edit template

Overwrites specified File

View existing file

J. Gutleber35

Prepare Title Page (1)

PR-101005-a-JGU, October 7th, 2010

• Revise title page (add persons, change title, etc.)• Select Edit -> Edit Page Header/Footer• Change document identifier, revision number and

status

J. Gutleber36

Prepare Title Page (2)

PR-101005-a-JGU, October 7th, 2010

Don’t Dothis at home!

J. Gutleber37

Prepare Version Page

PR-101005-a-JGU, October 7th, 2010

• Change document identifier, revision number and status

• Update version history• Save and Close• Version history must be maintained manually!

• Whatever is released on PIMS gets a new version number

J. Gutleber38

Add Requirements!

PR-101005-a-JGU, October 7th, 2010

J. Gutleber39

Concept

• Create several diagrams• Organization is left to the user

• “Drag & Drop” requirements into diagram• Provide requirements properties

• Identifier• Statement• Other properties such as priority, stability

• Optionally aggregate requirements

PR-101005-a-JGU, October 7th, 2010

J. Gutleber40

Add Requirements Diagram

PR-101005-a-JGU, October 7th, 2010

• Right click on the package and select Add -> Add Diagram• Select Extended -> Requirements• Select in left sidebar More Tools … -> Requirements

J. Gutleber41

Add Requirement• Drag and drop Requirement from diagram toolbox (Left side!)• Move Requirement to the correct package

PR-101005-a-JGU, October 7th, 2010

J. Gutleber42

Main Requirement Properties• Double click on requirement and enter the following properties

• Identifier= ID + (Short Description)• Stereotype = Functional or Performance• Description = Requirement statement• Status = Is if approved or proposed?

PR-101005-a-JGU, October 7th, 2010

J. Gutleber43

Additional Properties (Manual)• Rationale – Why do we need the requirement?• Assigned To – who tracks the requirement?• Benefit – how important is it?• Effort – estimate in person weeks to realize• Risk – use “Difficulty” in main properties instead• Stability – probability to change told by developer• Target Release – when to provide capability

PR-101005-a-JGU, October 7th, 2010

J. Gutleber44

Linking Files

• Double click on requirement, design component

• Select Tab Files• Select file• Remove absolute path• Add files with

• Copy and paste (pictures)• Add links to files• Add Hyperlinks

PR-101005-a-JGU, October 7th, 2010

J. Gutleber45

Add hyperlink

• Select design component• Select file to use for linking• Click Add hyperlink button• Remove absolute path

PR-101005-a-JGU, October 7th, 2010

J. Gutleber46

Update Dependencies• Select Aggregate link• Drag from B to A if A aggregates B!• Example:

• A) Little and Big Endians use the system to cross the river• B) The system must have the capability to let pedestrians cross the river

PR-101005-a-JGU, October 7th, 2010

B

A

J. Gutleber47

Requirements Report

PR-101005-a-JGU, October 7th, 2010

• Select Requirements package• Press <F8> and select requirements template• Generate and View

J. Gutleber48

Requirements Language• Human language is used to formulate requirements• Common language is needed

• Restricted vocabulary• Project specific terms in a glossary• Acronyms defined in a list

• See http://cern.ch/macs/glossary• Access is granted for contributors• Contribution is required (biggest problem)

PR-101005-a-JGU, October 7th, 2010

Requirements definition and terms used with different meanings are the biggest

sources of confusion and conflicts!

J. Gutleber49

Common Terms

• Regulated by RFC 2119 • And many other standards such as MIL, IEEE, …

• Importance is to use a reduced set of terms• MUST, SHOULD, MAY• MUST NOT, SHOULD NOT

• Place at beginning of phrase• Avoid fanciness and personal style choices

• MUST vs. SHALL vs. WILL

PR-101005-a-JGU, October 7th, 2010

J. Gutleber50

Primary Terms to Use

• MUST• The definition is an absolute requirement of the specification

• SHOULD• There may exist valid reasons in particular circumstances to ignore a

particular item, but the full implications must be understood and carefully weighed before choosing a different course.

• MAY• means that an item is truly optional. One vendor may choose to

include the item because a particular marketplace requires it or because the vendor feels that it enhances the product while another vendor may omit the same item.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber51

Shall I use Must?

• IETF defines SHALL and MUST as synonyms• Oxford English dictionary defines SHALL as a necessary

condition := ‘will have to’, ‘must’• Avoid any confusions!• We use MUST only!

PR-101005-a-JGU, October 7th, 2010

J. Gutleber52

The Problem with May

• An implementation which does not include a particular option MUST be prepared to interoperate with another implementation which does include the option, though perhaps with reduced functionality.

• In the same vein an implementation which does include a particular option MUST be prepared to interoperate with another implementation which does not include the option (except, of course, for the feature the option provides.)

PR-101005-a-JGU, October 7th, 2010

J. Gutleber53

Issues With Requirements

• The “terms” MUST NOT be used to try to impose a particular method on implementors where the method is not required for interoperability.

• The effects on security of not implementing a MUST or SHOULD, or doing something the specification says MUST NOT or SHOULD NOT be done may be very subtle.

• Therefore, the implications of following or not following the requirements must be elaborated.• This is independent of any risk management, medical device or safety

system. It is good engineering practice!

PR-101005-a-JGU, October 7th, 2010

J. Gutleber54

Endian Requirements Act 1

• Functions• Little and Big Endians use the system to cross the river.• The system must accept 1) Endians, 2) Animals and 3) cars.• The system must transport users in both directions.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber55

Requirements for Requirements

• Each requirement must be• Needed• Unambiguous and Understandable• Complete• Correct• Traceable• Modifiable• Verifiable• Ranked for importance and stability

• A description of the verification procedure should be included• Is equivalent to the validation test plan

PR-101005-a-JGU, October 7th, 2010

J. Gutleber56

Necessary

• Requirement states what I really need.• Recommendation:

• Once the requirement is written ask yourself again • 1) what you really need and • 2) if what you need is expressed by the requirement.• Ask yourself about the origin of the requirement. If it cannot clearly be

identified, the requirement is probably not needed.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber57

Unambiguous

• Reader can draw only one interpretation of it (Difficult!)

• Multiple readers arrive at same interpretation (More Difficult!)

• Recommendations:• Avoid subjective words like: “user friendly”, “easy”, “small”, “fast”,

“slow”, “large”, “state-of-the-art”, “minimize”, “maximize”.• Write short, simple sentences• Provide a test-case for the requirement

• BUT! You may define how to measure “user-friendly”• Have the software used by 100 people and collect satisfaction

between 1 and 5. If 50% give 1 or 2 call it “user-friendly”.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber58

Complete

• No necessary information should be missing• No requirement should be missing

• What is not written will not be provided eventually

PR-101005-a-JGU, October 7th, 2010

J. Gutleber59

Correct

• One requirement describes one functionality to be delivered.• Reference for correctness is the “source of the requirement”

• Where does it come from? Who needs it?• Customer, designer

• If a user requirement conflicts with a system requirement, at least one of the two is not correct.• Example: (UR) The system must be able to import files of all major

graphic file formats. (SR) The system must be delivered in less than 2 weeks.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber60

Traceable

• To fulfill “correctness” each requirement is linked to its source• To ensure that the requirement is implemented• Sources can be

• Reference to the person that stated the requirement (user, designer)• Reference to a use case

• Sources can be specified in the “Rationale”• Sources are tracked by the “Assigned To” person• Trace to design components and tests

• To ensure that the requirement is eventually implemented • To ensure that all requirements are implemented

PR-101005-a-JGU, October 7th, 2010

J. Gutleber61

Modifiable

• Requirements can be changed and the change history must be traceable.

• Consequences:• Each requirement must be uniquely identified (identifier)• Requirements should not be deleted, but invalidated (to be added)• A change history must be available or each change becomes a new

requirement, invalidating the original one.• Requirements should be grouped so to ease navigation

PR-101005-a-JGU, October 7th, 2010

J. Gutleber62

Audit View

• View->Other Project Tools->Audit View• Use with care!

• Records a lot of data.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber63

Verifiable

• It must be possible to specify a test for the requirement.• Called the “Verification procedure”

• Validation• Each requirement must eventually be covered by a verification

procedure to complete a system validation test within its “operational environment”. Since this anyway needs to be done it is good practice to specify the “verification procedures” as soon as possible.

• Recommendation:• If no verification procedure can be specified or the verification

procedure turns out to be difficult to specify, the requirement is probably not 1) feasible, 2) complete, 3) unambiguous.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber64

Ranked for Importance and Stability

• Help preparing a schedule and working on requirements, design and implementation concurrently.

• Recommendation:• Stability: approved, proposed• Three priority levels: high, medium, low priority• Applies to work iterations• In each iteration, parallel work on requirement, design,

implementation• High means a “must” a.s.a.p.• “Medium” means it can appear later• “Low” is a nice to have and could also be dropped by keeping the system’s

basic functionality

PR-101005-a-JGU, October 7th, 2010

J. Gutleber65

Types of Requirements Specifications• User Requirements Specification (URS)

• From the user’s point of view. Pure “WHAT”. A system does not exist.• Example: Persons want to buy beverages in a metro station.

• System Requirements Specification (SRS)• A single system has already been identified. The requirements

describe “WHAT” the system has to do.• Example: Requirements on a coffee vending machine.

• Interface Requirements Specification (IRS)• An SRS exists or a system exists, but can be adapted to meet new

needs of additional users or to specify interactions between systems• Example 1: The coffee vending machine must send sales data to the

owner company of the machine.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber66

Examples for Requirement Improvement• The system must be compatible with Windows XP.• Better: All application software must run on Windows XP

natively without the need for any extra intervention.• The system must store the username and user id in a

relational database.• Better: The system must keep user information persistently

stored. • The system must provide a measurement value at 10 Hz.

• Ask yourself WHAT you really need and WHAT you intend to do with the value!

• Better: After having failed, the system must keep the last measurement recorded and provide it upon request.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber67

Endian Requirements Act 2

• Design Constraint• The solution set must include a bridge.

Rationale: Bridges are cost effective and convenient ways to cross rivers like the one in the problem statement.

• Improved Functions• Little and Big Endians use the system to cross the river.• Better: People use the system to cross the river.• The system must accept 1) Endians, 2) Animals and 3) cars.• Better:

The system must transport vehicles from bank to bank• The system must transport pedestrians from bank to bank• The system must transport users in both directions.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber68

Non-Functional Requirements

• Express quality capabilities that define the functions in more detail.

• Standard list provided for systems requirement specification

PR-101005-a-JGU, October 7th, 2010

System sizeTimeConcurrent operationStates and ModesOperationMonitoringSafetyUsability

ReliabilityLifetimeAvailabilityMean Time To RepairFailure NotificationOperating System PlatformHardware PlatformDesign Constraint

DocumentationUser InterfaceHardware InterfaceSoftware InterfaceCommunication InterfaceLicenseLegal, CopyrightApplicable Standards

J. Gutleber69

Don’t Fall in the Template Trap!!!• Fill in only those topics that you find useful and that apply!• Do not fill in anything in any topic!• Remove unused topics!

• What NASA found:• […] It was obvious that the standard had been

used to drive the analysis, with no attempt to tailor the content to the real problem at hand. Some sections of the document were filled with meaningless information simply because the section was included in the standard and apparently the author did not want to leave it blank.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber70

Architecture and Design

PR-101005-a-JGU, October 7th, 2010

J. Gutleber71

Bridge Architecture

• 3 Spans, 2 pilars, 1 anchor• Materials: iron, steel, concrete

PR-101005-a-JGU, October 7th, 2010

River

J. Gutleber72

Bridge Components

• Functional components: beams, stringers, deck• Performance components: compression and tension members• Additional performance components: struts, bracing control

deflection and avoid buckling compression members

PR-101005-a-JGU, October 7th, 2010

J. Gutleber73

As Built

PR-101005-a-JGU, October 7th, 2010

J. Gutleber74

Architecture and Design

• Every design component traces to at least one requirement• Design components can be grouped and hierarchically

organized

PR-101005-a-JGU, October 7th, 2010

req Trace Requirements to Components

BRI-CO-0010 (Multiple pedestrians)

(from Concurrent operation)

BRI-CO-0020 (multiple vehicles)

(from Concurrent operation)

«specification»BRI-DE-0010.20

(Span)

req Organization of Design Components

«specification»BRI-DE-0010.20

(Span)

«specification»BRI-DE-0010.20.10

(floor beams)

«specification»BRI-DE-0010.20.20

(Stringers)

«specification»BRI-DE-0010.20.30

(deck)

ComponentAggregation

Tracing Components to Requirements

J. Gutleber75

Observe the 100% Rule

• The sum of all subcomponents makes up 100% of the components!

• Example: • Bicycle is made up of wheels, frame, steering, saddle, transmission• Transmission is made up of chain, derailleur, pedals, tooth-wheel

• Note: What is evident for physical systems is non necessarily evident for software systems or combined systems!

PR-101005-a-JGU, October 7th, 2010

J. Gutleber76

Why Are We Doing This

• A design component without a trace to a requirement has no reason to exist

• Document the high-level architecture of the system• All design components• Aggregations/assemblies• Relations among components• Safety components documentation in scope of risk management

• System Verification• Test on component traces back to requirements• Attention: This is not system validation!

PR-101005-a-JGU, October 7th, 2010

J. Gutleber77

Add Diagrams

PR-101005-a-JGU, October 7th, 2010

• Right click on the package and select Add -> Add Diagram• Select Extended -> Requirements• Following diagrams shall be created

• Organization of design Components• Trace Requirements to Components

J. Gutleber78

Add Design Component

• Select in left sidebar More Tools … -> Component• Drag and drop Component from Toolbox• (Instead of Components also Document Artefacts can be used)

• Example: Risk Management Plan, Risk Management Report

PR-101005-a-JGU, October 7th, 2010

J. Gutleber79

Main Component Properties• Move component to “top”-component• Double click on component and

enter the following properties• Name = BRI-DE-0010 (Bridge)• Stereotype = specification• Description = Textual description

PR-101005-a-JGU, October 7th, 2010

J. Gutleber80

Organization of Design Components

PR-101005-a-JGU, October 7th, 2010

• Drag Design Components into diagram from Project Browser• Connect Components with Aggregate link to create hierarchies

J. Gutleber81

Notations Used

PR-101005-a-JGU, October 7th, 2010

A

B

A aggregates B

J. Gutleber82

Trace Requirements to Design Components

PR-101005-a-JGU, October 7th, 2010

• Drag Requirements into diagram from Project Browser• Connect Requirements and Components with Realization link• Hide connectors by selection and pressing delete

J. Gutleber83

Notations Used

PR-101005-a-JGU, October 7th, 2010

A

B

B extends A

A

B

B realizes A

J. Gutleber84

Model Validation• Validation is implicitly part of the V-model• Tool allows checking if

• Each requirement is covered by a design component• Each design component is implementing a requirement

• Select Package Requirement or Architecture & Design• Press <CTRL>+<ALT>+V

PR-101005-a-JGU, October 7th, 2010

J. Gutleber85

Configure Model Validation• Project -> Model Validation -> Configure…• Deselect All• Select “Requirements Management”

PR-101005-a-JGU, October 7th, 2010

J. Gutleber86

Architecture and Design Report

PR-101005-a-JGU, October 7th, 2010

• Select Architecture & Design package

• Press <F8> Generate Report

• Select Architecture & Design template

• Generate and View

From “linked document”

created with <CTRL>+<Alt>+D

Requirements trace

J. Gutleber87

Testing

PR-101005-a-JGU, October 7th, 2010

J. Gutleber88

What is Testing?

• Testing is a means to perform verification and validation• Requires

• Planning• Specification of test cases• Carrying out test cases• Reporting on the carried out tests

PR-101005-a-JGU, October 7th, 2010

J. Gutleber89

What is a Test Case?

• A test must be a specified procedure that can be repetitively executed without need for any additional knowledge about the system.

• A test always results in “passed” or “not passed”.• IMPORTANT: no other results must be emitted. Numerical data

obtained from tests must remain internal data. The procedure must specify the conditions on the numerical data to result in passed or not passed.

PR-101005-a-JGU, October 7th, 2010

J. Gutleber90

Verify and Validate

• Verify• Prove to be true by demonstration• Building the system right.• Ensuring that the system complies with requirements and conforms to

design

• Validate• Logically correctly derived. Conforming to law.• Building the right system.• Validated is what the customer wanted!

PR-101005-a-JGU, October 7th, 2010

J. Gutleber91

System Validation and Verification

• Importance is to write down WHAT IS MENT!• ISO-9000:

• Verify that a design meets the requirements AND validate that the product meets the requirements (“what the user wanted”)

• NASA:• Verification proves that system complies with requirements• Validation proves that system accomplishes its purpose

PR-101005-a-JGU, October 7th, 2010

J. Gutleber92

What Does Validation for MedAustron?• Validate requirements (and goals)

• All requirements must be correct, complete and consistent

• Validate design• All design components trace to 1) requirements and 2) tests• All requirements trace to tests

• Verify Components• All component tests pass in the operational environment

• Verify Requirements (and goals)• All tests specified on requirements pass in operational environment

• Validate system• Validation procedures exist for each phase of the process• All tests have been carried out in the operational environment• All particular validation tests have passed

PR-101005-a-JGU, October 7th, 2010

J. Gutleber93

Validation Procedures

• Validation occurs in each phase continuously• Requirements: Ensure correctness, completeness, consistency• Design: Review design and specify tests• Test: Review test plan and test procedures• Acceptance: Specify acceptance tests

• Validation defects can be identified by inspection• Requirements must cover all cases• Tests must cover all requirements (and therefore all cases)

PR-101005-a-JGU, October 7th, 2010

J. Gutleber94

Validation

• Is implicitly carried out by applying the iterative V-Model

PR-101005-a-JGU, October 7th, 2010

J. Gutleber95

Validation Test Plan

• Validation Test Plan must exist• WITHOUT PLAN there is NO VALIDATION!

• Includes the Systems Engineering Process• Includes which procedures are carried out in which phase in

the process to validate

PR-101005-a-JGU, October 7th, 2010

J. Gutleber96

Adding Test Diagrams

PR-101005-a-JGU, October 7th, 2010

• Right-click on Scenarios and select Add -> Package• Used for grouping common test cases

• Right-click on the package and select Add -> Add Diagram• Select Extended -> Requirements

97

Add Test Case

PR-101005-a-JGU, October 7th, 2010 J. Gutleber

• Drag and drop Test Case from Toolbox• Select in left sidebar More Tools … -> Use Case

98

Test Case Properties

PR-101005-a-JGU, October 7th, 2010 J. Gutleber

• Double click on test case and enter the following properties• Name = BRI-RT-0010 (Transport Vehicle)

• RT = Requirements Test• CT = Component Test

• Stereotype = testcase• Add scenario description with

• List of steps to perform• Condition if a test was successful

J. Gutleber99

Addition Test Case Properties (Manual)

PR-101005-a-JGU, October 7th, 2010

• Build Number – software version• Date tested – date when the last test was performed• Test Notes – any addition information• Test Status – untested, failed, conditional pass, pass• Tested By – person who carried out the test

These properties have to be entered manually

J. Gutleber100

Trace Requirements to Test Cases

PR-101005-a-JGU, October 7th, 2010

• Drag and drop Requirements into diagram from Project Browser• Connect requirements and test cases with a Dependency link

req Tests for Requirements

BRI-RT-0010 (transport v ehicle 1)

BRI-FU-0010 (Intended Use)

(from Functional requirements)

BRI-FU-0020 (Transport pedestrians)

(from Functional requirements)

BRI-FU-0030 (Bidirectional)

(from Functional requirements)

BRI-RT-0020 (transport v ehicle 2)

BR-SS-0030 (distance to cross)

(from System size)

BRI-TI-0010 (vehicle crossing time)

(from Time)

Looks like “Trace” link, but is different!

J. Gutleber101

Trace Design Components to Test Cases

PR-101005-a-JGU, October 7th, 2010

• Drag and drop Component into diagram from Project Browser• Connect Components and Test cases with a Dependency link

J. Gutleber102

Test Report

PR-101005-a-JGU, October 7th, 2010

• Select Test Plan & Report package• Press <F8> • Generate Report• Select Test Report template• Generate and View

J. Gutleber103

Test Plan

PR-101005-a-JGU, October 7th, 2010

• Select Test Plan & Report package• Press <F8>• Generate Report• Select Test Plan template• Generate and View

J. Gutleber104

Risk Management

PR-101005-a-JGU, October 7th, 2010

J. Gutleber105

Risk Case

PR-101005-a-JGU, October 7th, 2010

• Has one of three roots• Requirement,• Design Component or• Environmental Condition

• Consists of the following items which may be reused multiple times• Hazard is a potential source of a harm• Effect describes what the hazard causes• Harm is physical injury or damage to the health of people, or damage to

property or the environment• Risk is combination of

probability of occurrence of harm and severity of that harm

J. Gutleber106

Risk Mitigation

• Three mitigation types• Avoidance – a measure to reduce the risk• Transfer – share impact with third party• Acceptance – accept as is and assume consequences

• All three types can be realized by• A new requirement specifying the mitigation type• A new design component specifying the mitigation type

• For acceptance• Requirement: It is acceptable that …., The system may ….

PR-101005-a-JGU, October 7th, 2010

J. Gutleber107

Risk Management Model

PR-101005-a-JGU, October 7th, 2010

J. Gutleber108

Add Risk Group

PR-101005-a-JGU, October 7th, 2010

• Go under “Risk Management” -> “Risk Groups”• Right-click Add -> Add Package

• Create one group for related risks, e.g. users

• Right-click on the newly created risk group • Select Add -> Add Package

• Create three standard packages

• Effects, Hazards and Harms

J. Gutleber109

Adding Diagrams

PR-101005-a-JGU, October 7th, 2010

• Right click on the risk group and select Add -> Add Diagram• Select Extended -> Requirements• Select in left sidebar More Tools … -> Risk Modeling• Creation of multiple diagrams per risk group

J. Gutleber110

Add Hazard

PR-101005-a-JGU, October 7th, 2010

• Drag and drop Hazard from Toolbox• Move hazard to Hazards package• Double click on hazard and enter the following properties

• Name = BRI-HA-0010 (Icy Road)• Description = human readable description

J. Gutleber111

Additional Hazard Properties

PR-101005-a-JGU, October 7th, 2010

• Category – Function, Patient, Side-Effect, Accessory, Environment, Ecology

• Validity – Valid, Invalid• Life Cycle – Requirements, Design, Production, Use, Maintenance,

Retirement/Disposal• Occurs under - Normal or Single Fault

J. Gutleber112

Add Effects

PR-101005-a-JGU, October 7th, 2010

• Drag and drop Effect from Toolbox• Move effect to Effects package• Double click on effect and enter the following properties

• Name = BRI-EF-0010 (Driver looses control)• Description = human readable description

J. Gutleber113

Additional Effect Properties

PR-101005-a-JGU, October 7th, 2010

• Effect Category – Function, Patient, Side-Effect, Accessory, Environment, Ecology

• Other properties are inherited from the hazard

do not modify!

J. Gutleber114

Add Harms

PR-101005-a-JGU, October 7th, 2010

• Drag and drop Harm from Toolbox• Move harm to Harms package• Double click on harm and enter the following properties

• Name = BRI-HA-0010 (Driver looses control)• Description = human readable description

J. Gutleber115

Risk

PR-101005-a-JGU, October 7th, 2010

• Risk is stored with the harm• Original Risk = Risk before any mitigation action• Final Risk = residual risk after

appling all mitigation actions

• Determine risk by looking upRisk Matrix (will be automatized)

severity

small medium severe catastrophic

probabili

ty

always 1 1 1 1

often 2 1 1 1

occasional 2 2 1 1

seldom 3 2 2 1

unlikely 3 3 3 2

J. Gutleber116

Additional Harm Properties

PR-101005-a-JGU, October 7th, 2010

• Original Probability & Final Probability• Unlikely, Seldom, Occasional, Often, Always

• Original Severity & Final Severity• Small, Medium, Severe, Catastrophic

• Original Risk & Final Risk• 1, 2 or 3• manually calculated (risk matrix)

• Other properties are inheritedfrom the hazard and effect

do not modify!

J. Gutleber117

Risk Mitigation

PR-101005-a-JGU, October 7th, 2010

• Two possibilities to add a risk mitigation• Creating a new design component or requirement

from Risk Management toolbox in the diagram• Drag and drop existing design component or requirement

from Project Browser

Create New Risk Mitigation

PR-101005-a-JGU, October 7th, 2010 J. Gutleber118

• Select design component or requirement• Move risk mitigation to Requirements or Architecture & Design• Double click on risk mitigation and enter the following properties

• Name = BRI-HA-0010 (Driver looses control)• Description = human readable description

J. Gutleber119

Additional Risk Mitigation Properties

PR-101005-a-JGU, October 7th, 2010

• Mitigation Strategy – Acceptance, Avoidance, Reduction• Other properties are inherited

from the hazard, effect and harmdo not modify!

J. Gutleber120

Residual Risk Determination

PR-101005-a-JGU, October 7th, 2010

• Double-click associated Harm and modify• Final Probability• Final Severity• Final Risk

J. Gutleber121

Add Cause-Effect Links

PR-101005-a-JGU, October 7th, 2010

• Drag and drop risk roots into the diagram• Connect roots and hazards with Trace link• Connect hazards, effects, harms and risk mitigations

with Realization links

req Vehicle icy road

BRI-HA-0010 (Icy road)

(from Hazards)

BRI-EC-0010 (Low temperature)

(from Risk Management)

BRI-EF-0010 (User looses control)

(from Effects)

BRI-HM-0010 (Person injured, hitting truss)

(from Harms)

«Risk Mitigation»Safety Components::BRI-SC-0010 (crash

barriers)

«specification»Architecture::

Asphalt

«trace»

J. Gutleber122

Risk Analysis

PR-101005-a-JGU, October 7th, 2010

• Right-click on Risk Group package• Select Add in -> Cosylab -> Generate Risk Analysis• Click Browse to create new file for saving report• Click Generate Excel File

Make sure thatExisting .xls fileIs closed!

J. Gutleber123

Risk Analysis Report

PR-101005-a-JGU, October 7th, 2010

OriginalRisks

ResidualRisks

RiskCases

Report not final!(bugs, formatting missing)

J. Gutleber124

Generate Matrix Reports

PR-101005-a-JGU, October 7th, 2010

• Select package (Requirements, Architecture & Design)• Press <F8> to generate report

• Matrix_Req_To_Test• Matrix_Req_To_Design• Matrix_Design_To_Test

• Generate and View

J. Gutleber125

Relationship Matrix

PR-101005-a-JGU, October 7th, 2010

• View -> Relationship Matrix• Customized traceability reports• Can be saved for later use

J. Gutleber126

PR-101005-a-JGU, October 7th, 2010

Requirement doesNot trace to any test!

Choose what the matrix should display

Printing and

Exporting

J. Gutleber127

REMEMBER THE FOLLOWING SLIDE ALWAYS !

PR-101005-a-JGU, October 7th, 2010

J. Gutleber128

Saying & Hearing contain same Message

PR-101005-a-JGU, October 7th, 2010