Risk and opportunity Part 1 Tor Stålhane Torbjørn Skramstad.
-
Upload
julian-willis -
Category
Documents
-
view
214 -
download
0
Transcript of Risk and opportunity Part 1 Tor Stålhane Torbjørn Skramstad.
Risk and opportunityPart 1
Tor Stålhane
Torbjørn Skramstad
EuroSPI 2006 - Part 1 2
Time Topics
09:00 Risk and opportunity What is it and why do we need to manage it? Why is opportunity assessment important? Why should we worry about risk and opportunity in SPI?The SWIR model and how to use it.Exercise – Application of the SWIR model
09:50 Coffee break
10:00 Assessment and brainstorming The human bias Qualitative assessment Simple brainstorming techniques Some important diagrams for risk assessmentExercise – Build an event tree
10:50 Coffee break
11:00 Simple risk and opportunity assessmentRisk and opportunity management - barriers and enablers.The ROP - Risk and Opportunity PatternExercise – Application of the ROP in SPI
11:50 Coffee break
12:00 Leverage as a decision toolExtended risk and opportunity assessmentThe ALARP and GALE concepts – when is enough really enough?The CORAS model – quantitative and qualitative assessmentExercise – Application of the GALE conceptImportant things to remember – summing it up
12:30 Lunch
EuroSPI 2006 - Part 1 3
Contents of part 1
• What is risk and what is opportunity
• Why should we care
• Assessing risk and opportunity
• Risk and opportunity in SPI – the SWIR and the SWIRO models
• More on assessment
• Brainstorming techniques
EuroSPI 2006 - Part 1 4
Risk and opportunity
Risk and opportunity have three things in common:
• They are concerned with events that may – or may not – happen in the future.
• The events are identifiable but their effect are uncertain, although less uncertain than the probabilities.
• The outcome of the events can be influenced by our actions
EuroSPI 2006 - Part 1 5
What is risk
A risk is something that can be a problem in the future. It is defined by two parameters
• The consequences - C. What will happen if the risk becomes a problem?
• The probability - p. What is the probability that the risk will become a problem?
The risk – R – is defined as R = C*p
EuroSPI 2006 - Part 1 6
What is opportunity
An opportunity is something that can be beneficial in the future. It is defined by two parameters
• The value - V. What will happen if the opportunity becomes a reality?
• The probability - p. What is the probability that the opportunity will be realized?
The opportunity – O – is defined as O = V*p
EuroSPI 2006 - Part 1 7
Why should we care -1
Risks may turn into problems. We can reduce or avoid future problems by reducing their consequences or their probabilities. This can be done by
• Changing the way we work to– Replace a high risk activity with a low risk activity.– Remove the risk possibility
• Adding risk avoidance activities to the way we work
EuroSPI 2006 - Part 1 8
Why should we care - 2
Opportunities may turn into benefits. We can increase future benefits by increasing their probabilities. This can be done by
• Changing the way we work – replace a low opportunity activity with a high opportunity activity.
• Adding opportunity enabling activities to the way we work
EuroSPI 2006 - Part 1 9
Assessing risk and opportunity
Both risk and opportunity is defined by value and probability.
Experience and data are important for two reasons. They can:
• Be used to estimate values and probabilities.
• Serve as an anchor for assessment – e.g. “How bad can it get?”
EuroSPI 2006 - Part 1 10
Risk and improvement
All SPI activities implies change and all changes carries their own risks and opportunities.
We will present two relevant models called SWIR and SWIRO respectively.
The purpose of these models is to identify risks and opportunities in SPI work.
EuroSPI 2006 - Part 1 11
The SWIR model -1
The SWIR model is the SPI version of the SWOT model.
• SWOT – Strengths, Weaknesses, Opportunities and Threats.
• SWIR – Strengths, Weaknesses, Improvements and Risks.
EuroSPI 2006 - Part 1 12
The SWIR model - 2
StrengthsWere shall we win?
WeaknessesWhat are our weak sides?
Improvements Where shall we improve
ourselves?
Risks What can go wrong?Which opportunities can
we loose
EuroSPI 2006 - Part 1 13
The SWIR components - 1
• Strengths – we need to know and understand our strong sides so that we – do not destroy them in the SPI process– can build on them and improve them
• Weaknesses – must be known so that we understand what we are up against.
• Improvements – what we want to achieve. They must be discussed and understood together with our strengths and weaknesses.
EuroSPI 2006 - Part 1 14
The SWIR components - 2
• Risks – potential problems that we have to cope with. They can stem from:– Our weak sides– Changes that are a necessary part of the SPI
process.– Threats to our strong side – things that must
be kept the way they are.
EuroSPI 2006 - Part 1 15
The SWIRO model - 1
The SWOT model includes opportunities but ignore improvements
The SWIR model includes improvements but ignores opportunities.
It might be a good idea to merge these two models so that we have a unified presentation of strengths, weaknesses, risk, opportunities and improvements.
EuroSPI 2006 - Part 1 16
The SWIRO model - 2Strengths Where shall we win?
Weaknesses What are our weak sides?
Improvements Where shall we improve ourselves?
Risks What can go wrong?
Current opportunities Which opportunities do we have now?
New opportunities Which new opportunities will the change open up?
EuroSPI 2006 - Part 1 17
A caveat
None of the presented models – SWOT, SWIR or SWIRO – will help us to assess the risks and opportunities.
The models are just used to get a complete picture of the situation.
Assessment is the logical next step.
EuroSPI 2006 - Part 1 18
Exercise
You are considering the introduction of an ISO conform process into your company.
Fill in the SWIR or SWIRO diagram.
EuroSPI 2006 - Part 1 19
Assessment - 1
Even though assessment is a subjective activity it is not about throwing out any number that you like.
To be useful, an assessment must be
• Based on relevant experience.
• Anchored in real world data.
• The result of a documented and agreed-upon process.
EuroSPI 2006 - Part 1 20
Assessment - 2
Risk and opportunity assessment is critically dependent on the persons who participate, their experience and their knowledge.
Experiments have shown that people have some biases which implies that we need to be careful when we look at the identified risk events and their assessed consequences and probabilities.
EuroSPI 2006 - Part 1 21
The human bias
Two human biases are important:
• Omission bias - most persons prefer doing nothing instead of an action if the consequences have equal values.
• Status quo bias - people assign a larger risk to change than to maintaining status quo. This bias increases if the change action has the potential to create victims.
EuroSPI 2006 - Part 1 22
Qualitative assessment
We can assess consequences, probabilities and benefits qualitatively in two ways. We can use:
• Categories – e.g. High, Medium and Low
• Numbers – e.g. values from 1 to 10.
EuroSPI 2006 - Part 1 23
Categories – 1
When using categories, it is important to give a short description as to what each category implies. E.g. it is not enough to say “High consequences”. We must relate it to something already known, e.g.
• Project size
• Company turn-over
• Company profit
EuroSPI 2006 - Part 1 24
Categories – 2
Two simple examples:
• Consequences: we will use the category “High” if the consequence will gravely endanger the profitability of the project.
• Probability: we will use the category “Low” if the event can occur but only in extreme cases.
EuroSPI 2006 - Part 1 25
Impact and probability - 1
Impact
Probability H M L
H H H M
M H M L
L M L L
EuroSPI 2006 - Part 1 26
Impact and probability - 2
The multiplication table is used to rank risks and opportunities. It can not tell us how large they are.
We should only use resources on risks and opportunities that are above a certain, predefined level.
EuroSPI 2006 - Part 1 27
Numbers as categories -1
We can use numbers instead of names. This does not make the assessment more precise but will free us from the need to define a multiplication table in order to identify risks.
In principle we can use any numbers. The best solution is, however, to just assign number to the three aforementioned categories
EuroSPI 2006 - Part 1 28
Numbers as categories – 2
The following values are often used in practice, both for consequences, benefits and probabilities:
• 10 – high
• 4 – medium
• 1 – low
Thus, a medium consequence and a low probability will give a risk of 4*1 = 4.
EuroSPI 2006 - Part 1 29
Numbers as categories – 3
Impact
Probability H / 10 M / 3 L / 1
H / 10 H / 100 H / 30 M / 10
M / 3 H / 30 M / 9 L / 3
L / 1 M / 10 L / 3 L / 1
EuroSPI 2006 - Part 1 30
Simple brainstorming techniques
Brainstorming is an efficient way to use the creative abilities that each person have.
In its simplest form, people just generate ideas and a person registers the ideas on a whiteboard or a flip-over.
We can, however, use techniques to do better.
EuroSPI 2006 - Part 1 31
Brainstorming and risks - 1
We can use previous experiences to answer questions such as
• Can this really happen; e.g. has it happened before?
• Can we describe a possible cause - consequence chain for the event?
• How bad can it get?
• How often has this happened in the past?
EuroSPI 2006 - Part 1 32
Brainstorming and risks - 2
We can use techniques such as:
• Affinity diagrams – “post it notes”
• Cause – consequence diagrams, such as – Ishikawa diagrams – also called fishbone
diagrams– Event trees– Cause – consequence networks
EuroSPI 2006 - Part 1 33
Ishikawa diagram
Too latedelivery
Resources Planning
Requirements Development
Wrong personnel
Loose keypersonnel
Estimation
Follow-up
Changes Tool X is notworking
Reuse problems Misunderstandings
EuroSPI 2006 - Part 1 34
Event trees
Coding error
Found in unit test
Not found in unit test
Found in integration test
Not found in integration test
Found in systems test
Not found in systems test
Delivered to customer
EuroSPI 2006 - Part 1 35
Cause – consequence diagram
C1
C2
C3
C4
C5
C6
C7
Acc E2
E1
E3
E4
E5
E6
E6
E7
E8
EuroSPI 2006 - Part 1 36
Change and risk
Changes can introduce risks. The main reasons are that:
• Any effect of a change is related to the future and can thus not be certain
• It is difficult to completely understand the effect of changes in a complex, sociological system
EuroSPI 2006 - Part 1 37
Change and opportunities
Changes can create new opportunities. The opportunities are mostly
• Indirect effects of what we do to achieve our goals – e.g. a new tool that can be used in several ways
• Additional effects of having achieved the goals – e.g. less need for rework frees resources for developing a new product.
EuroSPI 2006 - Part 1 38
Risk and opportunity in SPI
Risk and opportunity are important in SPI. We need to consider:
• Cost related to the change.
• Benefit, which is its planned purpose
• Risk related to the change, since we are going to work in a new way.
• New opportunities that are opened up by the changes
EuroSPI 2006 - Part 1 39
Exercise
You want to study the effect of document inspection on the number of defects delivered to the customer.
Build an event tree for the starting event
“A defect has been introduced in high level design”
EuroSPI 2006 - Part 1 40
Next session
The next session will focus on
• How to do simple risk and opportunity assessment.
• The introduction of barriers and enablers into risk and opportunity assessment
• How to use leverage to prioritize our actions
Risk and opportunityPart 2
Tor Stålhane
Torbjørn Skramstad
EuroSPI 2006 - Part 1 42
Contents of part 2
• Simple risk assessment• Simple opportunity assessment• The total picture – risk and opportunity • The risk and opportunity pattern • Barriers, enablers and leverage • Extended risk analysis• Extended opportunity analysis• Risk and regret
EuroSPI 2006 - Part 1 43
Simple risk assessment
In order to a simple risk assessment we need to identify:
• Dangerous events
• Each event’s – consequence – C– probability – p
• Possible barriers – changes or controls
• Person responsible for each risk - Resp.
EuroSPI 2006 - Part 1 44
Simple risk table
Event C p R Barriers Resp
EuroSPI 2006 - Part 1 45
Events
We start by identifying dangerous events. The simple way to do this is to use brainstorming.
The process is simple – just sit down and envisage your worst nightmares related to the activities under consideration.
Be realistic – only consider things that you believe can happen.
EuroSPI 2006 - Part 1 46
Barriers
Barriers can be realized through:
• Prevention – we change our process so that the event cannot occur.
• Mitigation – we can– change the process in order to reduce the
event’s probability or consequences.– define activities that will reduce the problems
if the event occurs.
EuroSPI 2006 - Part 1 47
Bar
rier
1 Bar
rier
2 Bar
rier
3 Bar
rier
4 Bar
rier
5 Bar
rier
6
Risk Prob. Event
Prevention barriersPrevent risk from becoming a problem
Handling barriersPrevent event from having bad consequences
Reduction barriersReduce effect of event
EuroSPI 2006 - Part 1 48
Simple opportunity assessment
In order to assess opportunities, we need to identify:
• The event that opens up opportunities - enablers
• Each opportunity’s– realizable value – V– probability - p
• The activity needed to realize the value• Person responsible for each opportunity
EuroSPI 2006 - Part 1 49
Simple opportunity table
Enabler
Opportunity V p O Enabling activity
Resp.
EuroSPI 2006 - Part 1 50
Enablers
Any action – e.g. a change – can create an opportunity enabler. Each enabler opens up a set of opportunities.
Further actions are needed in order to realize value.
Both enablers, opportunities and enabling actions can be identified through brainstorming.
EuroSPI 2006 - Part 1 51
Opportunity and risk
Assessing consequences and value:• H – High. Will have large impact• M – Medium. Should not be ignored• L – Low. Can be ignoredAssessing Probability: • H – High. Will happen quite often• M – Medium. Will happen now and then• L – Low. Will almost never happen
EuroSPI 2006 - Part 1 52
The total picture - 1
The total picture of the situation shows the risks and the benefits that stem from a planned change.
This is not a mechanism that can be used to identify the best solution.
It is, however, an important input when we want to make a decision.
EuroSPI 2006 - Part 1 53
The total picture - 2
The total picture shows risks, benefits and opportunities. Risk can be shown in two ways:
1. Unmitigated risks2. Mitigated risks – include the effect of risk
reduction activities, e.g. barriers. This can be done by
– Modifying the risk assessment– Indicate how the risk will move in the
diagram
EuroSPI 2006 - Part 1 54
Costs and benefits
B
HReduced number of MMI-related defects
M
L
p L M H
C
LExtra work needed for MMI-specification
M
H
EuroSPI 2006 - Part 1 55
Unmitigated risks
B
HReduced number of MMI-related defects
M
L
p L M H
C
LExtra work needed for MMI-specification
M
HLarge disagreements between designers and MMI experts
Partnership does not work
EuroSPI 2006 - Part 1 56
The mitigation effect
B
HReduced number of MMI-related defects
M
L
p L M H
C
LExtra work needed for MMI-specification
M
HLarge disagreements between designers and MMI experts
Partnership does not work
1
2
EuroSPI 2006 - Part 1 57
Including opportunities
B
HReduced number of MMI-related defects
Better MMI for existing products
Better MMI requirements will reduce imp. costs
M
L
p L M H
C
L Extra work needed for MMI-specification
M
HLarge disagreements between designers and MMI experts
Partnership does not work
1
2
EuroSPI 2006 - Part 1 58
The tyranny of “either – or”
All too often we are confronted by the statement that we can get only get X if we are willing to suffer Y.
This is the wrong attitude. The right attitude is that we will
1. Do what is needed to get X
2. Perform activities that will remove or reduce the bad effects of Y.
EuroSPI 2006 - Part 1 59
The risk and opportunity pattern
A pattern is a description of a standard way to solve a common problem. The Risk and Opportunity Pattern – ROP – is a way to analyze and manage risk and opportunity.
ROP has two components:• A set of assessment and management
activities• A process that describe an activity
sequence
EuroSPI 2006 - Part 1 60
The ROP process
ROP consists of the following activities:
1. Define the job and its borders
2. Perform a risk assessment
3. Perform an opportunity assessment
4. Implement the identified barriers
5. Do the job while – controlling risks and preventing problems– searching for opportunity enables and harvesting
benefits
EuroSPI 2006 - Part 1 61
ROP activities – risk part
• Define the job and its borders. We cannot consider everything – only what is inside the defined borders.
• Perform a risk assessment.
• Implement the barriers identified in the previous step.
• Do the job - control risks and prevent problems.
EuroSPI 2006 - Part 1 62
Exercise
Your company consider buying a new test administration tool. Management is unsure whether this is a wise investment.
Use the risk part of ROP to help management in their decision.
EuroSPI 2006 - Part 1 63
Barriers and enablers
Barriers and enablers will define actions that will help us to
• Avoid problems – barriers
• Reap benefits – enablers
Identification of barriers and enablers is, however, not enough. We also need to assess how effective they are.
EuroSPI 2006 - Part 1 64
Leverage
Leverage is a prioritizing mechanism:
Leverage = (Benefit – Cost) / Cost
Leverage will prioritize activities with
• Large net benefits
• Small costs
EuroSPI 2006 - Part 1 65
Extended risk table -1
We can use cause – consequence chains or event trees for a risk to identify the best place to insert a barrier.
For each barrier, we need to assess:
• Cost - the cost of implementing it. We will use the scale H = 10, M = 3 and L = 1.
• E – how effective is the barrier? We will use the scale h = 1.0, m = 0.5 and l = 0.2
EuroSPI 2006 - Part 1 66
Extended risk table - 2
Event C p R Barrier Cost E L Resp.
EuroSPI 2006 - Part 1 67
Barrier leverage
Leverage = (C*p*E – Cost) / Cost
The leverage will prioritize barriers which:
• Have low costs – Cost is small
• Have high efficiency – E is large
• Attack important risks – C*p is high
EuroSPI 2006 - Part 1 68
Barrier – example Event Cons
.p R Mitigation E Cost
LResp
Partnership doesnot work – businessconflicts
10 3 30
Do a thorough researchon selected partner’sbusiness goals
0.5 10 0.5
John
Customers do notprioritize projectparticipation 10 3 30
State the conditions andconsequences of customerparticipation in thecontract
1.0 3 9.0
Pete
EuroSPI 2006 - Part 1 69
Some comments on barriers
It is important to remember that:• Each risk will usually need a different barrier – a
barrier that works against one risk can be valueless against another risk.
• It is important to consider the three main barrier strategies:– Prevent the risk from becoming a problem– Control the problem to avoid the consequences– Reduce the consequences
EuroSPI 2006 - Part 1 70
Extended opportunity table - 1
Even if an opportunity arises, nothing will really happen if we do not do something to realize it.
An enabler is an event that will help us to reap a benefit.
Just as barriers, the activities linked to an enabler have costs and effectiveness. Thus, we can compute the leverage and use this as a basis for our decisions.
EuroSPI 2006 - Part 1 71
Extended opportunity table - 2
Enabler
Opportunity
V p O Action Cost E L Resp.
EuroSPI 2006 - Part 1 72
Opportunity leverage
Leverage = (V*p*E – Cost) / Cost
The enabling activity leverage will prioritize activities which:
• Have low costs – Cost is small
• Have high efficiency – E is large
• Enable valuable opportunities – V*p is high
EuroSPI 2006 - Part 1 73
Enabler - exampleEnabler Better understanding of how MMI requirements are implemented
and adapted
Opportunity Value p O
Action E Cost L
Resp
Better MMI requirements, which will reduce imp. costs
10 10 100
Use new knowledge to make better MMI requirements spec
1 3 32
Peter
Use MMI more actively to create more popular products
10 3 30
Redesign user interface for products A and B
1 10 2
Brian
EuroSPI 2006 - Part 1 74
An alternative presentation - 1
We have earlier used the cost-benefit diagram to show benefits, opportunities, costs and risks.
By including the efficiency of barriers and enabling actions, we get a better picture of the overall situation.
Since we already have performed the necessary multiplications, we can use a one-dimensional representation.
EuroSPI 2006 - Part 1 75
An alternative presentation - 2
Benefits andopportunities
Costs andrisks
100100 30 301010
The alternative representation is just a representation. It is thus just one of several inputs to a decision.
EuroSPI 2006 - Part 1 76
A small example - 1We have the following assessed values:• Cost: C = medium, p = high, Cost = 30.0• Benefit: V = high, p = high, Benefit = 100.0• Risks
– R1: C = medium, p = low, barrier efficiency = medium, R1 = 1.5
– R2: C = high, p = low, barrier efficiency = low, R2 = 8.0
• Opportunities– O1: V = medium, p = high, enabling activity efficiency
= medium, O1 = 15.0– O2: V = high, p = high, enabling activity efficiency is
low, O2 = 20.0
EuroSPI 2006 - Part 1 77
A small example - 2
Benefits andopportunities
Costs andrisks
100100 30 301010
Cost Benefit R1R2 O1 O2
EuroSPI 2006 - Part 1 78
Regret and risk - 1
Instead of just looking at cost and value of an opportunity, we can include risk and regret in the leverage expression.
Regret is the, often indirect, cost of skipping or ignoring an opportunity.
Priority = (Value + Regret) / (Cost + Risk)
EuroSPI 2006 - Part 1 79
Regret and risk - 2
Just as cost, value and risk, regret has to bee assessed, for instance on a scale from 1 to 10 or just using three values such as 10, 3 and 1.
As should be expected• High regret and low risk will give high
priority. • Low regret and high risk will give low
priority
EuroSPI 2006 - Part 1 80
Next session
The next session will focus on
• Two risk assessment concepts – ALARP and GALE
• How to use the GALE method
• Quantitative assessment and the CORAS model
• Summing up - some important things to remember
Risk and opportunityPart 3
Tor Stålhane
Torbjørn Skramstad
EuroSPI 2006 - Part 1 82
Contents
• ALARP and GALE• Using GALE• How to do risk assessment with GALE• A small example• Quantitative assessment• The CORAS model• A small example• Important things to remember
EuroSPI 2006 - Part 1 83
ALARP and GALE
There are two competing principles in the assessment of risk:
• ALARP – As Low As reasonably Possible- We have done all that is reasonable to prevent problems and dangers.
• GALE – Globally At Least Equivalent. E.g. introducing a new process will not increase the risks compared to what it is today.
EuroSPI 2006 - Part 1 84
ALARP
ALARP requires that we analyze each risk separately and then implement mitigation activities.
A reasonable goal is to reduce each risk until the extra mitigation costs exceed the value of the risk reduction achieved.
All that we have seen up till now fits into an ALARP policy .
EuroSPI 2006 - Part 1 85
GALE
GALE requires us to look at the total risk of a change. In this way we can start by attacking the cheapest risk or the risk with the largest leverage.
The problem with the GALE principle is that we need to perform arithmetic on risks. E.g. we need to decide how many medium risks we need before we have a large risk
EuroSPI 2006 - Part 1 86
ALARP vs. GALE - 1
There is no such thing as the right risk principle. It is always a matter of company choice and company policy.
The two principles will lead to different prioritization of mitigation activities.
• ALARP – each risk is reduced as much as possible.
• GALE – we need to be below the present risk level.
EuroSPI 2006 - Part 1 87
ALARP vs. GALE - 2
The one important thing with using the GALE principle is that it forces us to ask “What is the current risk level?”
All too often we act as it the current way of doing things is risk free and all risk stems from changes.
This stance is enforced by the human tendency to underestimate the risk of status quo.
EuroSPI 2006 - Part 1 88
Using GALE
Important points
• GALE is a method for risk analysis. Benefits must be included elsewhere
• We need to look at both our current risk and the risk resulting from the proposed changes.
• Always perform a sensitivity analyses.
EuroSPI 2006 - Part 1 89
Risk – status quo vs. change
In many cases, maybe even in most of them, we do risk assessment because we want to compare two or more alternatives, e.g.:
• Status quo – no changes
• One or more changes - improvements
EuroSPI 2006 - Part 1 90
Event identification
• All significant dangerous events must have been identified.
• There must be a minimal overlap between the dangerous events .
• There must be a maximum of commonality between the dangerous events considered for the status quo and for the system after the proposed changes
EuroSPI 2006 - Part 1 91
The three event sets
The previous rules split the dangerous events into three sets – dangerous events that:
• Apply both to the status quo and to the new system.
• Are unique to the status quo
• Are unique to the new system
EuroSPI 2006 - Part 1 92
GALE and risk assessment - 1
GALE uses the following parameters for risk assessment:
• FE – the event frequency
• PE – the probability that the event will lead to an accident
• S – the severity score of an event
EuroSPI 2006 - Part 1 93
GALE and risk assessment - 2
We can compute individual and accumulated risk indices:
IE = FE + PE + S
IGR = log Sumi(10I)
IE is the risk index for a hazardous event
IGR is the global risk index
EuroSPI 2006 - Part 1 94
The GALE scoring scheme
The scoring scheme of GALE • Focuses on deviations from current
average. This is reasonable, given that it is mainly concerned with comparing status quo to a new situation.
• Must be tailored to each situation. The next slide shows an example from road safety. We need a scheme adapted to SPI.
EuroSPI 2006 - Part 1 95
Road safety - frequency score for event
Frequency classification
Occurrences / year on M42 ATM section FE
Very frequent 10000 Hourly 6
Frequent 1000 A few times a day 5
Probable 100 Every few days 4
Occasional 10 Monthly 3
Remote 1 Annually 2
Improbable 0.1 Every 10 years 1
Incredible 0.01 Every 100 years 0
EuroSPI 2006 - Part 1 96
SPI and GALE
We need a special scoring scheme for development projects. For events that can lead to problems we need to consider:
• How often does the event occur - FE?
• If the event occurs, what is the probability that it will cause a real problem - PE?
• If the problem occurs, how severe will the consequences be – S?
EuroSPI 2006 - Part 1 97
SPI goals
Based on the GALE parameters, we can also identify possible SPI goals:
• S: reduce the consequences – reduction and handling barriers
• FE: reduce the number of event occurrences – problem opportunities
• PE: reduce the probability that the event will cause a problem – prevention barriers
EuroSPI 2006 - Part 1 98
Frequency score for event Frequency
classOccurrences per project FE
Very frequent 200 Every project 6
Frequent 100 Every few projects 5
Probable 40 Every 10th project 4
Occasional 10 Every 100th project 3
Remote 1 A few times in the company’slifetime
2
Improbable 0.2 One or two times during thecompany’s lifetime
1
Incredible 0.01 Once in the company’slifetime
0
EuroSPI 2006 - Part 1 99
Probability score for event
Classification Interpretation PE
Probable It is probable that this event, if it occurs, will cause a problem 3
Occasional The event, if it occurs, will occasionally cause a problem 2
Remote There is a remote chance that this event, if it occurs, will cause a problem
1
Improbable It is improbable that this event, if it occurs, will cause a problem 0
EuroSPI 2006 - Part 1 100
Severity score for event
Severityclass
Interpretation S
Severe The portion of occurring problems thathave serious consequences is muchlarger than average
2
Average The portion of occurring problems thathave serious consequences is similarto our average
1
Minor The portion of occurring problems thathave serious consequences is muchlower than average
0
EuroSPI 2006 - Part 1 101
Sensitivity analysis
The global risk index is made of many indices. Each index will have a certain degree of uncertainty connected to it.
Usually, a few indices will have a large influence on the result while the rest will have but little influence.
Pareto’s rule applies - we need to identify the few important indices.
EuroSPI 2006 - Part 1 102
A small example - 1
Status quo After processimprovement
Event S FE PE S FE PE
Too late delivery – 1 1 5 3 1 4 3
Too high cost – 2 1 5 3 2 4 3
Low customer satisfaction – 3 1 4 3 0 3 2
Low developer satisfaction – 4 1 4 2 0 3 2
Too low product quality - 5 1 4 2 0 3 2
EuroSPI 2006 - Part 1 103
A small example - 2• Status quo:
I1 = 9, I2 = 9, I3 = 8, I4 = 7, I5 = 7• After SPI activity:
I1 = 8, I2 = 9, I3 = 5, I4 = 5, I5 = 5
IGR = log Sum(10I)
• Status quo: log Sum(10I) = 9.3• After SPI activity: log Sum(10I) = 9.0
EuroSPI 2006 - Part 1 104
A small example - 3
We see from the results that the risk reduction is small – from 9.3 to 9.0.
We also see that the main reason for this is that we have increased quality but increased the cost.
The main result from the GALE process is that we need to find ways to increase the quality without increasing our development cost.
EuroSPI 2006 - Part 1 105
Quantitative assessment -1
In some cases, we can use numerical values. This occurs if we can use
• Experience to identify the cost of a problem – e.g. correcting an error or loosing a customer.
• Old data to identify a probability – e.g. the probability of missing a defect during inspection.
EuroSPI 2006 - Part 1 106
Quantitative assessment - 2
Quantitative risks and opportunities give us real values.
The usefulness of this is, however, limited since it is difficult to find real values for all risks and opportunities.
It is not obvious how we can compare qualitative and quantitative risks or opportunities
EuroSPI 2006 - Part 1 107
The CORAS model
CORAS was developed as a framework for assessment of security risks.
What should concern us here, however, is how they related the qualitative risk categories, not to absolute values, but to the company’s turn-over.
EuroSPI 2006 - Part 1 108
The CORAS consequence table
Consequence values
Category Insignificant Minor Moderate Major Catastrophic
Measuredrelated toincome
0.0 – 0.1% 0.1 – 1.0% 1 – 5% 5 – 10% 10 – 100%
Measuredloss due toimpact onbusiness
No impact onbusiness. Minor delays
Lost profits
Reduce theresources of oneor moredepartmentsLoss of a coupleof customers
Close downdepartments orbusinesssectors
Out ofbusiness
EuroSPI 2006 - Part 1 109
The CORAS frequency table - 1
As we will see on the next slide, CORAS allows us to interpret frequency in two ways:
• The number of incidents per year• The failing portion of demandsWe will use the second interpretation but
instead of focusing on a system, we related it to the number of projects, e.g. SPI projects.
EuroSPI 2006 - Part 1 110
The CORAS frequency table - 2Frequency values
Category Rare Unlikely Possible LikelyAlmostcertain
Number ofUnwantedincidents perYear
1/100 1/100 – 1/50 1/50 - 1 1 - 12 > 12
Number ofUnwantedincidents perDemand
1/1000 (1/500) 1/50 (1/25) 1/1
Interpretationof number ofdemands
UnwantedincidentneverOccurs
Eachthousandtime thesystem isused
Each fivetimes thesystem isused
Each tenthtime thesystem isused
Everysecondtime thesystem isused
EuroSPI 2006 - Part 1 111
A small example
We have a company with 10 developers and an estimated yearly turnover of NOK 10 millions.
We decide that the consequences of a late delivery is “medium”, which gives a consequence of 1 – 5% or NOK 100 000 to 500 000.
We decide that the event is “likely” to occur, which gives us a p-value of 0.04
The expected loss is thus 4 000 to 20 000.
EuroSPI 2006 - Part 1 112
Exercise
Your company has decided to change development process.
• List all important events
• Find the risk index for each event for– Status quo– The new development process
EuroSPI 2006 - Part 1 113
Important things to remember - 1The most important things to remember:• Risk assessment is by its nature subjective. • Use group techniques and include all
stakeholders• Use simple techniques so that you do not
exclude one or more stakeholders• Anchor it in experience and available data will,
however, improve the quality• Subjective values like “High” must be anchored
in each company’s reality. One company’s “High” may be another company’s “Low”.
EuroSPI 2006 - Part 1 114
Important things to remember - 2
• Include the effect of choosing status quo in all SPI risk analyses.
• Always include opportunities• Consider the three barrier categories –
prevention, handling and reduction• Rank risks and opportunities according to
their leverage• The results from a risk assessment is just
one of several inputs to a decision