Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

53
Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb Problem Solving and Research Methodology Sanjay Goel, JIIT, 2014 Lecture Notes: Excerpts and Summary of References- Part I: Risk Engineering 1. 13.01.14 People who don’t take risks generally make about two big mistakes a year. People who do take risks generally make about two big mistakes a year. Peter Drucker Risk by Michelle Mckee There are no guarantees Life throws things at you You can catch or miss them But they will come, ready or not I always looked for the real thing Never trusting in the possibility Risk-taking not my forte Staying safe at all costs Even playing it safe is not certain Safe has hurt me Zero risk gets zero gain Sometimes playing it safe costs you more It has me, In not fighting the battle you may lose the war In not believing in a dream You may never sleep peacefully again So let go of the fear Reach out for the flame So what if you get burned Better that then numb for life Better to remember passion and joy Along with the pain and tears Then to have no memories worth Remembering So to hell with safe I am going to gamble and bet Until I win back everything I lost And my life is what it was meant to be

description

These are excerpts of the references discussed in the lectures of an elective course Problem Solving and research Methodology offered to MTech (CSE) and BTech (CSE/IT) students at JIIT. Part -I contains the references discussed in the class before first test, it mainly deals with the theme of Risk Engineering.

Transcript of Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Page 1: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Problem Solving and Research Methodology

Sanjay Goel, JIIT, 2014

Lecture Notes: Excerpts and Summary of References- Part I: Risk Engineering

1. 13.01.14

People who don’t take risks generally make about two big mistakes a year.

People who do take risks generally make about two big mistakes a year. —Peter Drucker Risk by Michelle Mckee

There are no guarantees

Life throws things at you

You can catch or miss them

But they will come, ready or not

I always looked for the real thing

Never trusting in the possibility

Risk-taking not my forte

Staying safe at all costs

Even playing it safe is not certain

Safe has hurt me

Zero risk gets zero gain

Sometimes playing it safe costs you more

It has me,

In not fighting the battle

you may lose the war

In not believing in a dream

You may never sleep peacefully again

So let go of the fear

Reach out for the flame

So what if you get burned

Better that then numb for life

Better to remember passion and joy

Along with the pain and tears

Then to have no memories worth

Remembering

So to hell with safe

I am going to gamble and bet

Until I win back everything I lost

And my life is what it was meant to be

Page 2: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Take Risks by William Arthur Ward

To laugh is to risk appearing the fool

To weep is to risk being called sentimental

To reach out to another is to risk involvement

To expose feelings is to risk showing your true self

To place your ideas and your dreams before the crowd is to risk being called naïve

To love is to risk not being loved in return

To live is to risk dying

To hope is to risk despair and,

To try is to risk failure

But risks must be taken

The greatest risk in life is to risk nothing

The person who risks nothing... does nothing, has nothing, and becomes nothing

He may avoid suffering and sorrow

But he simply cannot learn and feel and change and grow and love and live

Chained by his servitude, he is a slave

He has forfeited his freedom

Only the person who risks is truly free.

Problem Solving: Some Selected Pearls of Wisdom

1. Problem is a situation for which there isn’t an evident solution.-- Pérez et al.

2. Leaders are problem solvers by talent and temperament, and by choice.-- Harlan Cleveland

3. Creating something is all about problem-solving.--Philip Seymour Hoffman

4. The problems of the world cannot possibly be solved by skeptics or cynics whose horizons are

limited by obvious realities. We need men and women who can dream of things that never

were. - John F. Kennedy

5. We only think when we are confronted with problems. - John Dewey

6. No problem can withstand the assault of sustained thinking. – Voltaire

7. The problem is not that there are problems. The problem is expecting otherwise and thinking

that having problems is a problem. — Theodore Rubin

8. The most serious mistakes are not being made as a result of wrong answers. The truly

dangerous thing is asking the wrong questions. — Peter Drucker

9. If you only have a hammer, you tend to see every problem as a nail.---Abraham Maslow

10. The biggest problem in the world could have been solved when it was small. - Witter Bynner

11. Erroneous assumptions can be disastrous. — Peter Drucker

12. Drowning problems in an ocean of information is not the same as solving them. - Ray E.

Brown

13. The significant problems we face cannot be solved at the same level of thinking we were at

when we created them –Einstein

14. It's not that I'm so smart; it's just that I stay with problems longer. –Einstein

15. If I had an hour to solve a problem I'd spend 55 minutes thinking about the problem and

5 minutes thinking about solutions.― Einstein

16. The formulation of the problem is often more essential than its solution, which may be merely

a matter of mathematical or experimental skill.― Einstein

17. Every problem has in it the seeds of its own solution. If you don't have any problems, you don't

get any seeds. - Norman Vincent Peale

18. Few can really understand the problem, the answer will come out of it, because the

answer is not separate from the problem.--Krishnamurti

Page 3: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

19. The only difference between a problem and a solution is that people understand the solution. -

Dorothea Brande

20. When a problem comes along, study it until you are completely knowledgeable. Then find that

weak spot, break the problem apart, and the rest will be easy. - Norman Vincent Peale

21. A problem clearly stated is a problem half solved. - Dorothea Brande

22. To solve any problem, here are three questions to ask yourself: First, what could I do? Second,

what could I read? And third, who could I ask? - Jim Rohn

23. If you do not ask the right questions, you do not get the right answers. A question asked

in the right way often points to its own answer. Asking questions is the A-B-C of

diagnosis. Only the inquiring mind solves problems. - Edward Hodnett

24. No problem can be solved until it is reduced to some simple form. The changing of a vague

difficulty into a specific, concrete form is a very essential element in thinking. - J. P. Morgan

25. When I'm working on a problem, I never think about beauty. I think only how to solve

the problem. But when I have finished, if the solution is not beautiful, I know it is

wrong." - R. Buckminster Fuller

26. When the solution is simple, God is answering.--Albert Einstein

27. Science is always wrong, it never solves a problem without creating ten more. - George

Bernard Shaw

28. The solution to a problem changes the nature of the problem.--John Peers

29. If you get stuck, get away from your desk. Take a walk, take a bath, go to sleep, make a pie,

draw, listen to music, meditate, exercise; whatever you do, don't just stick there scowling at the

problem. But don't make telephone calls or go to a party; if you do, other people's words will

pour in where your lost words should be. Open a gap for them, create a space. Be patient.―

Hilary Mantel

30. It is well known that "problem avoidance" is an important part of problem solving. Instead of

solving the problem you go upstream and alter the system so that the problem does not occur in

the first place.--Edward de Bono

31. Our tendency to create heroes rarely jibes with the reality that most nontrivial problems

require collective solutions.--Warren Bennis

32. Recognizing a problem is the first step to solving it... Some problems cannot be solved but you

can make peace with them.--Sanya Friedman

Page 4: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

2,3. 14.01.14

Ref#1: Jonassen, David, Johannes Strobel, and Chwee Beng Lee. "Everyday problem

solving in engineering: Lessons for engineering educators." JOURNAL OF

ENGINEERING EDUCATION-WASHINGTON- 95, no. 2 (2006): 139.

Everyday problem solving in engineering workplace

1. Workplace problems are ill structured

2. Ill structured problems include aggregates of well structured problems.

3. Ill structured problems have multiple, often conflicting goals

4. Ill structured problems are solved in many different ways

5. Success is rarely measured by engineering standards: Time budget, legal, regulatory,

environmental etc.

6. Most constraints are non-engineering: budget, schedule, functionality, brand, jobs,

tasks, tools, , acceptability to citizens, political constraints, regulations, environmental,

permits, cultural, communication etc.

7. Problem solving knowledge is distributed among team members (and also institutional

knowledge found in several organisations, regulatory bodies, and support systems)

8. Most problems require extensive collaboration

9. Engineers primarily rely on experiential knowledge

10. Engineering problems often encounter unanticipated problems

11. Engineers use multiple forms of problem representation

12. Engineers recommend more communication skills in engineering curricula: more

instruction on client interaction, collaboration, making oral presentations, and writing, as

well as the ability to deal with ambiguity and complexity.

Page 5: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#2: David H. Jonassen, Toward a design theory of problem solving, Educational

Technology Research and Development, Volume 48, Number 4, Springer Boston, pp

63-85, December, 2000.

Ref#3: Linda S. Gottfredson, Dissecting practical intelligence theory: Its claims and

evidence Intelligence Volume 31, Issue 4, Elsevier, July-August 2003 , 2003.

Academic Problems Real life practical problems

1. Tend to be formulated by other people

2. Well-defined or well-structured

3. Tend to be complete. Presented with all the

parameters and constraints. Usually consist of a

well-defined initial state, a known goal state, and

a constrained set of logical operators.

4. Typically posses only a single answer

5. Tend to encourage single method of obtaining a

correct answer

6. Require application of a finite number of

concepts, rules, and principles

7. Divorced from ordinary experience

8. Tend to be of little or no intrinsic interest

1 Require (re)formulation.

2 Ill-defined or ill-structured

3 Require information seeking. One

or more elements of the ill-defined

problem are unknown or not known

with certainty. The goals of real-

life practical problems are usually

vaguely defined with unstated

constraints.

4 Usually possess multiple acceptable

solutions.

5 Allow multiple paths to solution.

6 Present uncertainty about useful and

usable concepts, rules, and principles as

well. Further, in case of ill-defined

problems, the relationships between

concepts, rules, and principles may be

inconsistent between cases.

7 Embedded in and require prior

experience. This requires the problem

solver of ill-structured problem to

distinguish important from irrelevant, and

construct a problem space for generating

solutions.

8 Require motivation and personal

involvement

Page 6: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#4: David H. Jonassen, Toward a design theory of problem solving, Educational

Technology Research and Development, Volume 48, Number 4, Springer Boston, pp

63-85, December, 2000

Finding the unknown is the process of problem solving.

any goal-directed sequence of cognitive operations

o Requires the mental representation of the situation- multimodal problem space

comprises of:

Structural knowledge (all the essential parts, states, or actions encountered in

the problem and the relationship between them at an appropriate level of

detail),

Procedural knowledge (how to perform procedures and test activities),

reflective knowledge, images and metaphors of the system, and executive/

strategic knowledge (e.g. search and replace, serial elimination, and space

splitting)

May be externalized through a variety of Knowledge representation and

modeling tools.

o Requires some activity-based manipulation of the problem space.

Problem type variations

o Structured-ness: well structured ill structured

o Complexity

how many, how clearly, and how reliably components (issues, functions,

variables, connections, and relations) are represented implicitly or explicitly

Most complex problems are dynamic – task environment and factors change

over time.

o Domain specificity: Abstract Situated

Representation variation – fidelity of representation

o Use of artificial symbol systems

o Is the problem represented in its natural complexity and modality, or is it filtered

when simulated? Should social pressures and time constraints be represented

faithfully? i.e., does the problem have to be solved in real time, or can it be solved

in leisure time? What levels of cooperation or competition are represented in the

problem?

4. 20.01.14:

Examples of non-engineering constraints: budget, schedule, functionality, brand, jobs,

tasks, tools, , acceptability to citizens, political constraints, regulations, environmental,

permits, cultural, communication etc.

Examples of unanticipated problems

Page 7: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

5,6. 21.01.14:

*Understanding failures is critical to engineering success*,

*Engineering – A profession for managing technical risks*,

*Engineering is Risk Engineering*

Ref#5: Gluch, David, "A Construct for Describing Software Development Risks"

(1994). Software Engineering Institute. Paper 118.

• A problem involves a value judgment made upon the merits of the current condition. It is

a condition that exists and is undesirable.

• A risk involves a value judgment made upon the potential implications of the current

condition. It suggests a possible, future undesirable condition (consequence).

Many problems are risks themselves in that they may lead to more serious symptoms or

other problems and diminished success (loss) for the program. A difference between a

problem and a risk is a matter of degree – the extent to which the program is being

adversely affected -- and time. The conditions of a problem are more noticeably affecting

the program now. A risk, which is also a problem, involves a condition that has a noticeable

adverse effect on the program now, but also is perceived to portend additional or more

serious problems in the future.

I. Impact Analysis (Ref#6: Mindtools.com): explore possible positive and negative

consequences of a change on different parts of a system or organization.

II. All Hazard Risk Assessment Methodology,

(Ref#7: Canada Govt., http://www.publicsafety.gc.ca/prg/em/emp/2012-

ahra/index-eng.aspx)

a. Adaptive/Malicious Threats: Intentional Threats;

Criminal: Terrorist Act, Extremist Act, Individual Criminal Act, Organised Crime,

Corporate/Insider Sabotage, Corporate Espionage.

Foreign State: State-Sponsored Terrorism, Espionage, Act of War.

b. Non-Malicious Threats/Hazards:

Social: Migration, Social Unrest/Civil Disobedience.

Technical/Accidental: Spill, Fire, Explosion, Structural Collapse, System Error(s)

Yielding Failure.

Health Threats/Hazards: Pandemics/Epidemics:, Human Health Related, Animal

Health Related

Large-Scale Contamination: Drugs and Health Products Contaminant,

Food/Water/Air Contaminant, Environment Contaminant.

Page 8: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Emerging Phenomena & Technologies: Biological Science & Technology, Health

Sciences, (Re) emerging Health Hazards, Chemical Compounds,

Emerging Natural Hazard(s), Material Science & Engineering,

Information Technologies

c. Natural Threats/Hazards:

Meteorological: Hurricane, Tornado/Wind Storm, Hail/Snow/ Ice Storm,

Flood/Storm Surge, Avalanche, Forest Fire, Drought, Extreme

Temperatures.

Geological: Tsunami, Earthquake, Volcanic Eruption, Land/Mudslide, Land

Subsidence, Glacier/Iceberg Effects, Space Weather.

Ecological/Global Phenomena: Infestations, Effects of Over-Exploitation, Effects

of Excessive Urbanisation, Global Warming, Extreme Climate Change

Conditions.

III. Failure Mode and Effects Analysis (FMEA) (Ref#8: Mindtools.com): ―What could go wrong‖ - reviewing existing processes or systems, so that

problems with these can be identified and eliminated.

a. Start by looking in detail at the proposed solution

b. Identify systematically all of the points where it could fail.

c. Rate the potential consequences of each according to:

a. Severity – how critical is the failure?

b. Occurrence – how likely is the failure to happen?

c. Detection – how easy will it be to detect the failure?

d. Using these rankings, identify the most serious threats

e. Alter the design to eliminate or minimize the likelihood of identified failure.

f. Repeat the process

IV. Risk Analysis Process (Ref#9: Mindtools.com): identify and manage potential problems

1st line of defense- Avoid or eliminate failure causes

2nd

line of defense – Detect and control failure early

3rd

line of defense – reduce impact/consequence of the failure

a. Identify Threats: identify the existing and possible threats: (understanding the limitations of

design)

Human - from illness, death, injury, or other loss of a key individual.

Operational - from disruption to supplies and operations, loss of access to essential assets, or

failures in distribution.

Reputational - from loss of customer or employee confidence, or damage to market

reputation.

Procedural - from failures of accountability, internal systems and controls; or from fraud.

Project - from going over budget, taking too long on key tasks, or experiencing issues with

product or service quality.

Financial - from business failure, stock market fluctuations, interest rate changes, or non-

availability of funding.

Technical - from advances in technology, or from technical failure.

Natural - from weather, natural disasters, or disease.

Political - from changes in tax, public opinion, government policy, or foreign influence.

Structural - from dangerous chemicals, poor lighting, falling boxes, or any situation where

staff/ products/ technology can be harmed.

Page 9: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

b. Estimate Risk

Risk Value = Probability of Event x Cost of Event

c. Manage Risk

Using existing assets - reusing or redeploying existing equipment, improving existing

methods and systems, changing people's responsibilities, improving accountability and

internal controls, choosing different materials, by improving safety procedures or safety

gear, or by adding a layer of security.

Developing a contingency plan - you accept a risk, but develop a plan to minimize its

effects if it happens. A good contingency plan will allow you to take action immediately,

and with the minimum of project control, if you find yourself in a crisis.

Investing in new resources - include insuring the risk

Procedural prevention plan - activities that need to take place every day, week, month, or

year to monitor or mitigate the risks you've identified. For example, daily backup of

computer files, yearly testing of your building's sprinkler system, or a monthly check on

your organization's security system.

(Ref#10: Williams, Ray C., George J. Pandelios, and Sandra G. Behrens. Software Risk

Evaluation (SRE) Method Description: Version 2.0. Carnegie Mellon University, Software

Engineering Institute, 1999

Ref#11: arr, Marvin J., Suresh L. Konda, Ira Monarch, F. Carol Ulrich, and Clay F.

Walker. Taxonomy-based risk identification. No. CMU/SEI-93-TR-06. Carnegie-Mellon

Univ Pittsburgh Pa Software Engineering Inst, 1993)

SEI Risk Management Paradigm)

- Balance the possible negative consequences of risk against the potential benefits of its associated

opportunity.

- Project risks change over time in both characteristics (probability, impact, time frame) and content

— i.e., the risk of yesterday could be the problem of today or cease to be a risk altogether, and

new risks could arise.

Element Purpose

Identify Make all known project risks explicit before they become problems

Analyze Transform risk data into decision-making information

Plan Translate risk information into decisions and mitigating actions (both present and

future) and implement those actions

Track Monitor risk indicators and mitigation actions

Control Correct for deviations from the risk mitigation plans

Communicate Enable the sharing of all information throughout the project and is the cornerstone of

effective risk management

A 2.5 hour session generates 15-40 risk statements. The risk statement is the product of the risk

interview step and consists of

• A condition: something that is true or accepted as true

• A separator: a semicolon, arrow, or linking phrase

• A consequence: something that may occur as a result of the condition

Page 10: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Page 11: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

SEI Software Development Risk Taxonomy (Ref#12: CMU/SEI-93-TR-6, 1993)

A. Product Engineering B. Development Environment C. Program

Constraints

1. Requirements Are there risks that may arise

from requirements being placed on the product?

a. Stability b. Completeness c. Clarity

d. Validity e. Feasibility

f. Precedent g. Scale

1. Development Process

a. Formality b. Suitability

c. Process Control d. Familiarity

e. Product Control

1. Resources

a. Schedule

b. Staff

c. Budget

d. Facilities

2. Design

a. Functionality b. Difficulty c. Interfaces

d. Performance e. Testability

f. Hardware Constraints

g. Non-Developmental Software

2. Development System

a. Capacity b. Suitability

c. Usability d. Familiarity

e. Reliability f. System Support

g. Deliverability

2. Contract

a. Type of

Contract

b. Restrictions

c. Dependencies

3. Code and Unit Test

a. Feasibility b. Testing

c. Coding/Implementation

3. Management Process

a. Planning b. Project Organization

c. Management Experience

d. Program Interfaces

3. Program

Interfaces

a. Customer

b. Associate

Contractors

c.

Subcontractors

d. Prime

Contractor

e. Corporate

Management

f. Vendors

g. Politics

4. Integration and Test

a. Environment b. Product c. System 4. Management Methods

a. Monitoring b. Personnel Management

c. Quality Assurance

d. Configuration Management

5. Engineering Specialties

a. Maintainability b. Reliability c. Safety

d. Security e. Human Factors

f. Specifications

5. Work Environment

a. Quality Attitude b. Cooperation

c. Communication d. Morale

Ref#13: Lawrence, Brian, Karl Wiegers, and Christof Ebert. "The top risk of requirements

engineering." Software, IEEE 18, no. 6 (2001): 62-63.

a. Overlooking a crucial requirement- functional or attribute (quality or performance)

b. Inadequate customer representation

c. Modelling only functional requirements – ignoring quality attributes (reliability, performance,

security, robustness, ease of use; scalability, innovation, coolness, or fun).

d. Not inspecting requirements

e. Attempting to perfect requirements before beginning construction

f. Representing requirements in the form of designs.

Page 12: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Taxonomy Based Questions (TBQ) wrt Requirements:

Quality of req. specs & implementation difficulty (Ref#12: CMU/SEI-93-TR-6, 1993)

a. Stability: Are requirements changing during product

development?

[1] Are the requirements stable?

(No) (1.a) What is the effect on the system - Quality/

Functionality/ Schedule/ Integration/ Design/

Testing?

[2] Are the external interfaces changing?

b. Completeness: Are requirements missing or incompletely

specified?

[3] Are there any TBDs in the specifications?

[4] Are there reqs. you know should be in the specification but

aren‘t?

(Yes) (4.a) Will you be able to get these requirements into the

system?

[5] Does the customer have unwritten

requirements/expectations?

(Yes) (5.a) Is there a way to capture these requirements?

[6] Are the external interfaces completely defined?

c. Clarity: Are requirements unclear or in need of

interpretation?

[7] Are you able to understand the requirements as written?

(No) (7.a) Are the ambiguities being resolved satisfactorily?

(Yes) (7.b) There are no ambiguities or problems of

interpretation?

d. Validity: Will the reqs. lead to the product the customer has

in mind?

[8] Are there any requirements that may not specify what the

customer really wants?

(Yes) (8.a) How are you resolving this?

[9] Do you and the customer understand the same thing by the

requirements?

(Yes) (9.a) Is there a process by which to determine this?

[10] How do you validate the requirements?

Prototyping/Analysis/ Simulations

e. Feasibility: Are requirements infeasible from an

analytical point of view?

[11] Are there any requirements that are technically

difficult to implement?

(Yes) (11.a) What are they?

(Yes) (11.b) Why are they difficult to implement?

(No) (11.c) Were feasibility studies done for these

requirements?

(Yes) (11.c.1) How confident are you of the

assumptions made in the

studies?

f. Precedent: Do requirements specify something

never done before, or that your company has not done

before?

[12] Are there any state-of-the-art requirements -

Technologies/ Methods/ Languages/ Hardware

(No) (12.a) Are any of these new to you?

(Yes) (12.b) Does the program have sufficient

knowledge in these areas?

(No) (12.b.1) Is there a plan for acquiring knowledge

in these areas?

g. Scale: Do requirements specify a product larger,

more complex, or requiring a larger organization than

in the experience of the company?

[13] Is the system size and complexity a concern?

(No) (13.a) Have you done something of this size and

complexity before?

[14] Does the size require a larger organization than

usual for your company?

Page 13: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Taxonomy Based Questions (TBQ) wrt Design:

Design & feasibility of algorithms/functions, performance requirements, and internal and external

product interfaces. Difficulty in testing begins here. (Ref#12: CMU/SEI-93-TR-6, 1993) a. Functionality: Are there any potential problems in

meeting functionality requirements?

[15] Are there any specified algorithms that may not satisfy

the requirements?

(No) (15.a) Are any of the algorithms or designs marginal

with respect to meeting requirements?

[16] How do you determine the feasibility of algorithms and

designs? - Prototyping/ Modeling/ Analysis/ Simulation

b. Difficulty: Will the design and/or implementation be

difficult to achieve?

[17] Does any of the design depend on unrealistic or

optimistic assumptions?

[18] Are there any requirements or functions that are difficult

to design?

(No) (18.a) Do you have solutions for all the requirements?

(Yes) (18.b) What are the requirements?

• Why are they difficult?

c. Interfaces: Are the internal interfaces (hardware and

software) well defined and controlled?

[19] Are the internal interfaces well defined?

- Software-to-software & Software-to-hardware

[20] Is there a process for defining internal interfaces?

(Yes) (20.a) Is there a change control process for internal

interfaces?

[21] Is hardware being developed in parallel with software?

(Yes) (21.a) Are the hardware specifications changing?

(Yes) (21.b) Have all the interfaces to software been defined?

(Yes) (21.c) Will there be engineering design models that can

be used to test the software?

d. Performance: Are there stringent response time or

throughput requirements?

[22] Are there any problems with performance? -

Throughput, Scheduling asynchronous real-time events,

Real-time response, Recovery timelines, Response time,

Database response, contention, or access

[23] Has a performance analysis been done?

(Yes) (23.a) What is your level of confidence in the

performance analysis?

(Yes) (23.b) Do you have a model to track performance

through design and implementation?

e. Testability: Is the product difficult or impossible to test?

[24] Is the software going to be easy to test?

[25] Does the design include features to aid testing?

[26] Do the testers get involved in analyzing requirements?

f. Hardware Constraints: Are there tight constraints on

the target hardware?

[27] Does the hardware limit your ability to meet any

requirements?

- Architecture, Memory capacity, Throughput, Real-time

response, Response time, Recovery timelines, Database

performance, Functionality, Reliability, Availability

g. Non-Developmental Software: Are there problems with

software used in the program but not developed by the

program?

If re-used or re-engineered software exists

[28] Are you reusing or re-engineering software not

developed on the program?

(Yes) (28.a) Do you foresee any problems?

- Documentation, Performance, Functionality, Timely

delivery, Customization

If COTS software is being used

[29] Are there any problems with using COTS (commercial

off-the-shelf) software?

• Insufficient documentation to determine interfaces, size,

or performance; Poor performance; Requires a large share

of memory or database storage; Difficult to interface with

application software; Not thoroughly tested; Not bug free;

Not maintained adequately; Slow vendor response

[30] Do you foresee any problem with integrating COTS

software updates or revisions?

Page 14: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

7. 27.01.14:

Ref#14: Karl E Wiegers; Joy Beatty, Ch 32: Software Requirements and Risk Management,

Software Requirements, 3rd Edition, Microsoft Press, 2013

Risk Item tracking template

Page 15: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Sample Risk item from the chemical tracking system

8,9. 28.01.14:

Ref#15: Westfall, Linda. "Software Risk Management." In Annual Quality Congress

Proceedings-American Society For Quality Control, pp. 32-39. ASQ; 1999, 2000.

Project Management Risk Management

Designed to address general or generic

risks

Designed to focus on risks unique to each

project

Looks at the big picture and plans for

details

Looks at potential problems and plans for

contingencies

Plans what should happen and looks for

ways to make it happen

Evaluates what could happen and looks for

ways to

minimize the damage

Plans for success Plans to manage and mitigate potential causes

of failure

Types of risks for software projects:

Technical risks problems with languages, project size, project functionality, platforms,

methods, standards, or processes. May result from excessive constraints, lack of

experience, poorly defined parameters, or dependencies on organizations outside the

direct control of the project team.

Page 16: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Management risks: lack of planning, lack of management experience and training,

communications problems, organizational issues, lack of authority, and control

problems.

Financial risks include cash flow, capital and budgetary issues, and return on investment

constraints.

Contractual and legal risks include changing requirements, market-driven schedules,

health & safety issues, government regulation, and product warranty issues.

Personnel risks staffing lags, experience/training problems, ethical and moral issues,

staff conflicts, productivity issues.

Other resource risks include unavailability or late delivery of equipment & supplies,

inadequate tools, inadequate facilities, distributed locations, unavailability of comp uter

resources, and slow response times.

Techniques for identifying risks:

Interviewing/Brainstorming: interviewing or brainstorming with project personnel,

customers, and vendors. Open-ended questions, e.g.,

What new or improved technologies does this project implement?

What interfaces issues still need to be defined?

What requirements exist that we aren‘t sure how to implement?

What concerns do we have about our ability to meet the required quality and

performance levels?

Voluntary Reporting: any individual who identifies a risk is encouraged and rewarded

for bringing that risk to management‘s attention. This requires the complete elimination

of the ―shoot the messenger‖ syndrome. It avoids the temptation to assign risk reduction

actions to the person who identified the risk. Risks can also be identified through

required reporting mechanisms such as status reports or project reviews.

Decomposition: As the product is being decomposed during the requirements and

design phases, another opportunity exists for risk identifications. Every TBD ("To Be

Done/Determined") is a potential risk. ―The most important thing about planning is

writing down what you don’t know, because what you don‘t know is what you must find

out‖ [Ould-90]. Decomposition in the form of work breakdown structures during project

planning can also help identify areas of uncertainty that may need to be recorded as

risks.

Assumption Analysis: Process and product assumptions must be analyzed. For

example, we might assume the hardware would be available by the system test date or

three additional experienced C++ programmers will be hired by the time coding starts. If

these assumptions prove to be false, we could have major problems.

Critical Path Analysis: As we perform critical path analysis for our project plan, we

must remain on the alert to identify risks. Any possibility of schedule slippage on the

critical path must be considered a risk because it directly impacts our ability to meet

schedule.

Risk Taxonomies: Risk taxonomies are lists of problems that have occurred on other

projects and can be used as checklists to help ensure all potential risks have been

considered, e.g., Software Engineering Institute‘s Taxonomy -Based Risk Identification

report that covers 13 major risk areas with about 200 questions.

Page 17: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Risk Analysis: each risk is assessed to determine:

Likelihood: the probability that the risk will result in a loss

Impact: the size or cost of that loss if the risk turns into a problem

Timeframe: when the risk needs to be addressed, i.e., risk associated with activities in

the near future would have a higher priority than similar risks in later activities.

Risk Exposure (RE) = Probability(of UO) * Loss (because UO), where UO =

Unexpected outcome

Avoiding: Avoiding risk may mean avoiding opportunities; Not all risks can be avoided;

Avoiding risk in one part of the project may create risks in other parts.

Getting more information: prototyping, modeling, simulation, research

Transfer: pay someone to take care of all the risks, subcontracting, outsourcing, build

penalties for contractors, pass extra charges/late deliveries for customers, insurance,

partnership, joint venture,

Risk reduction actions: actions to reduce the probability/impact, e.g., performance

simulation, liaison with customer, test bed to duplicate the operational environment, alpha

testing at contractor‘s site, periodic defect reports, reschedule some task, adjust the budget.

Take risk reduction action n, if RRL (Risk Reduction Leverage) = (REbefore – REafter) >

Risk Reduction cost

Contingency plan: implemented if risk actually turn into a problem, e.g., disaster recovery

plans, backup staff.

Risk must be assigned triggers - Early trigger, contingency plan trigger

Page 18: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#16: Fuller, Anne, Peter Croll, and Limei Di. "A new approach to teaching software risk

management with case studies." In Software Engineering Education and Training,

2002.(CSEE&T 2002). Proceedings. 15th Conference on, pp. 215-222. IEEE, 2002.

Page 19: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#17: Odzaly, Edzreena Edza, Des Greer, and Paul Sage. "Software risk management

barriers: An empirical study." In Empirical Software Engineering and Measurement, 2009.

ESEM 2009. 3rd International Symposium on, pp. 418-421. IEEE, 2009.

Page 20: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#18: Risk Management Guidelines Companion to AS/NZS 4360:2004, Standards

Australia/Standards New Zealand

Page 21: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Page 22: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

10. 03.02.14:

Ref#19: Risk Management Guidelines Companion to AS/NZS 4360:2004, Standards

Australia/Standards New Zealand

1. Consequence- outcome or impact of an event. (>=1, +ve/-ve, expressed as

quantitative/qualitative, considered in relation to achievement of objectives)

2. Control- an existing process, policy, device, practice or other action that acts to minimize -

ve risk or enhance positive opportunities.

3. Control assessment- systematic review of processes to ensure that controls are still

effective and appropriate

4. Event- occurrence of a particular set of circumstances, (certain/uncertain; single

occurrence/series of occurrence.)

5. Frequency- measure of the number of occurrences per unit of time.

6. Hazard- a source of potential harm

7. Likelihood- general description of probability or frequency, expressed qualitatively or

quantitatively.

8. Loss- any -ve consequence or adverse effect, financial or otherwise

9. Monitor- to check, supervise, observe critically or measure the progress of an activity,

action or system on a regular basis in order to identify change from the performance level

required or expected

10. Residual risk- risk remaining after implementation of risk treatment 11. Risk - chance of something happening that will have a -ve/+ve impact on objectives.

{events/ circumstances consequences}

Components of a risk a. A source of risk or hazard – the thing which has the intrinsic potential to harm or assist

e.g. a dangerous chemical, competitors, government.

b. An event or incident – something that occurs such that the source of risk has the impact

concerned e.g. a leak, competitor expands into or leaves your market area, new or

revised regulations, or some measure or observation reaching a particular trigger level.

c. A consequence, outcome or impact on a range of stakeholders and assets e.g.

environmental damage, loss or increase of market/profits, regulations increase or

decrease competitiveness.

d. A cause (what and why) (usually a string of direct and underlying causes) for the

presence of the hazard or the event occurring e.g. design, human intervention, funding,

prediction or failure to predict competitor activity, failure to or expansion of market

presence.

e. Controls and their level of effectiveness e.g. detection systems, clean up systems,

policies, security, training, market research and surveillance of market.

f. When could the risk occur and where could it occur.

12. Risk analysis- systematic process to understand the nature of and to deduce the level of risk

13. Risk assessment- risk identification+ risk analysis, + risk evaluation

14. Risk avoidance- a decision not to become involved in, or to withdraw from, a risk situation

15. Risk criteria- terms of reference by which the significance of risk is assessed, can include

associated cost and benefits, legal and statutory requirements, socioeconomic and

environmental aspects, the concerns of stakeholders, priorities and other inputs to the

assessment

16. Risk evaluation- process of comparing the level of risk against risk criteria, assists in

decisions about risk treatment.

17. Risk identification- process of determining what, where, when, why and how something

could happen

Page 23: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

18. Risk management framework- set of elements of an organization‘s management system

concerned with managing risk

19. Risk management process - the systematic application of management policies,

procedures and practices to the tasks of communicating, establishing the context,

identifying, and analysing, evaluating, treating, monitoring and reviewing risk

20. Risk management- the culture, processes and structures that are directed towards realizing

potential opportunities whilst managing adverse effects to identify change from the

performance level required or expected

21. Risk reduction- actions taken to lessen the likelihood, negative consequences, or both,

associated with a risk

22. Risk retention- acceptance of the burden of loss, or benefit of gain, from a particular Risk,

includes the acceptance of risks that have not been identified, level of risk retained may

depend on risk criteria

23. Risk sharing- sharing with another party the burden of loss, or benefit of gain from a

particular risk. Legal or statutory requirements can limit, prohibit or mandate the sharing of

some risks. Risk sharing can be carried out through insurance or other agreements. Risk

sharing can create new risks or modify an existing risk.

24. Risk treatment- process of selection and implementation of measures to modify risk,

sometimes used for the measures themselves, can include avoiding, modifying, sharing or

retaining risk.

25. Stakeholders- those people and organizations who may affect, be affected by, or perceive

themselves to be affected by a decision, activity or risk, may also include ‗interested

parties‘.

Page 24: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Decision making issues for selecting options for Risk treatment

Acceptability- Is the option likely to be accepted by relevant stakeholders?

Administrative efficiency- Is this option easy to implement or will it be neglected because of

difficulty of administration or lack of expertise?

Compatibility- How compatible is the treatment with others that may be adopted?

Continuity of effects- Will the effects be continuous or only short term? Will the effects of this

option be sustainable? At what cost?

Cost effectiveness- Is it cost-effective; could the same results be achieved at lower cost by other

means?

Economic and social effects- What will be the economic and social impacts of this option?

Effects on the environment- What will be the environmental impacts of this option?

Equity- Are risks and benefits distributed fairly e.g. do those responsible for creating the risk pay

for its reduction?

Individual freedom- Does the option deny any basic rights?

Jurisdictional authority- Does this level of organization or government have the authority to apply

this option? If not, can higher levels be encouraged to do so?

Leverage- Will the treatment options lead to additional benefits in other areas?

Objectives- Are organizational objectives advanced by this option?

Regulatory- Does the treatment (or lack of treatment) breach any regulatory requirements?

Political acceptability- Is it likely to be endorsed by the relevant government authority? Will it be

acceptable to communities?

Risk creation- Will this treatment introduce new risks?

Timing- Will the beneficial effects be realized quickly?

Contingency planning

Page 25: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Key Questions for Risk identification

(a) What is the source of each risk?

(b) What might happen that could:

- increase or decrease the effective achievement of objectives;

- make the achievement of the objectives more/ less efficient (financial, people, time);

- cause stakeholders to take action that may influence the achievement of objectives.

- produce additional benefits?

(c) What would the effect on objectives be?

(d) When, where, why, how are these risks (both +ve and -ve) likely to occur?

(e) Who might be involved or impacted?

(f) What controls presently exist to treat this risk?

(g) What could cause the control not to have the desired affect on the risk?

After reviewing each element, the following general questions should be considered:

• What is the reliability of the information?

• How confident are we that the list of risks is comprehensive?

• Is there a need for additional research into specific risks?

• Are the objectives and scope covered adequately?

• Have the right people been involved in the risk identification process?

Page 26: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Key Questions for Risk Analysis

a. What current systems may prevent, detect or lower the consequences or likelihoods of

undesirable risks or events?

b. What current systems may enhance or increase the consequences or likelihoods of

opportunities or beneficial events?

c. What are the consequences or range of consequences of the risks if they do occur?

d. What is the likelihood or range of likelihoods of the risks happening?

e. What factors might increase or decrease the likelihoods or the consequences?

f. What additional factors may need to be considered and modelled?

g. Are there limits of likelihood and consequence beyond which the analysis does not hold

true?

h. What are the limitations of the analysis and assumptions made?

i. How confident are you in your judgement or research specifically in relation to high

consequence and low likelihood risks?

j. What drives variability, volatility or uncertainty?

k. Is the logic behind the analysis methods sound?

l. For quantitative analysis, what if any statistical methods may be used to understand the

effect of uncertainty and variability?

Page 27: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Page 28: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

11,12. 04.02.14:

Ref#20: Kajko-Mattsson, Mira, and Jaana Nyfjord. "State of software risk management

practice." LAENG International Journal of Computer Science 35, no. 4 (2008): 1-12. &

Ref#19: Risk Management Guidelines Companion to AS/NZS 4360:2004, Standards

Australia/Standards New Zealand

A. Risk Identification

1 Study relevant historical risk information

1.1 Study the organizational taxonomy of risk types

1.2 Study lessons learnt

1.3 Study other relevant information, if needed

2 Study the domain that is subject to the risk exposure

3 Elicit Risks- personal & past organizational experience,

expert judgement, brainstorming, focus group,

interview, business/strategic plans, historical records,

post event reports, insurance claim/audit reports, what-

if and scenario analysis, system design review, flow

charting, prototyping, systems analysis (decomposition),

critical path analysis, assumption analysis, checklists,

system engg. Tech‘s, SWOT, survey/questionnaire, ...

3.1 Identify potential risks

For each risk

3.2 Identify risk consequences and effects

3.3 Identify risk sources

3.4 Analyse root cause of risk

3.5 Define risk categories/classes

3.6 Describe and record each identified risk

4 Create risk list

5 Circulate risk list

6 Update risks accordingly

7 Confirm risk list

B. Risk Analysis

1 Analyze risks

1.1 Analyse each risk independently

For each risk

1.1.1 Study the risk

1.1.2 Assess risk probability

1.1.3 Assess risk impact

1.1.4 Calculate risk exposure

1.1.5 Specify risk severity

1.1.6 Analyse risk – past records, literature, experience,

market research, public consultation, experiments,

prototype, experts, modelling & simulation,

qualitative, quantitative, semi-quantitative

techniques, sensitivity analysis// matrices,

decision/fault/event tree, influence diagram, life

cycle cost analysis, network analysis,

modelling/simulation, scenario analysis, test

marketing probability/statistical analysis,..

1.1.7 Specify preliminary risk threshold value

1.1.8 Assign priority to the risk

1.1.9 Record any assumptions made

1.2 Analyse risks in groups

1.2.1 Group risks according to chosen group criteria

1.2.1 Analyze risks in groups

1.3 Consolidate risk prioritization

2 Create top priority risk list

3 Create a list of risks requiring further attention

4 Suggest a preliminary plan for managing the risks

5 Circulate prioritized risk list & prelim. plan among

stakeholders

6 Update the prioritized risk list & the prelim. plan, if

needed

C. Risk Management Planning

1 Study the risk list, the analysis results, and the preliminary plan

2 Determine strategies for managing risks

For each risk group

2.1 Determine strategic procedure for managing the risk

2.2 Determine tolerance strategy (threshold value)

2.3 Determine values or events that may trigger contingency actions

3 Create a risk management plan implementing the risk strategies selected

For each risk or risk group

3.1 Create control and monitoring plan

3.1.1 Define relevant measures/metrics for monitoring and controlling the risk

3.1.2 Determine performance indicators for measuring action effectiveness

3.1.3 Document the control and monitoring plan

3.2 Create action plan

3.2.1 Define relevant measures for treating the risk

3.2.2 Develop a fallback action plan if preliminary plan does not prove adequate

3.2.3 Document the action plan

D. Risk Monitoring and Control

1 Ensure there are procedures to

monitor risks

2 Monitor continuously all risks

following the plan

2.1 Monitor risk status

2.2 Control the changes in risk status

2.3 Record the risk status

2.4 Take appropriate measures wrt

the changed status

2.4.1 Implement the planned risk

action, if needed

2.4.2 Implement the contingency

plan, if need arises

2.4.3 Implement other actions, if

needed

3 Monitor results to determine the

Page 29: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

3.3 Create contingency plan, if needed

3.3.1 Define relevant contingency actions

3.3.2 Document the contingency plan

3.4 Make schedule for implementing the plans

3.5 Identify constraints

3.6 Estimate efforts

3.7 Estimate resources

3.8 Assign budget

3.9 Assign roles/responsibilities for managing it

4 Combine all plans and put them into the risk management plan

5 Analyze the risk management plan (combined plan)

5.1 Conduct cross-plan analysis with regard to strategies chosen

5.2 Change to the plan according to the results of cross-plan analysis, if needed

6 Prepare risk related contractual agreements

7 Circulate the risk management plan to the stakeholders concerned

8 Update the risk management plan, if needed

9 Confirm risk management plan

10 Update and reconcile the documentation of the risk management plan

effectiveness of planned action

4 Seek out to identify new or

residual risks

4.1 Report the new risks accordingly

4.2 Start a new instance of the

process, if needed

5 Update the risk status and risk list

6 Record and update trends by

predefined performance

indicators

E. Risk Sign –off

1 Approve by formal sign-off

2 Update risk management progress

status

3 Eliminate risk from risk list

4 Update risk list

F. Risk Post-mortem Analysis

1 Evaluate the risk management

process

2 Create/update an organizational

risk taxonomy

3 Identify deficiencies and failures of

the process

4 Identify positive effects of the

process

5 Identify lessons learnt

6 Record the results

Page 30: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#21: Boehm, Barry W. "Software risk management: principles and practices." Software,

IEEE 8, no. 1 (1991): 32-41.

Top 10 Risk items Risk Management technique

Personnel shortfalls: skill and

knowledge levels, staff turnover,

team dynamics…

Staffing with top talent, job matching, team building,

key personnel agreements, cross training

Unrealistic schedules and

budgets: requirements demand

more time/ money...

Detailed multisource cost and schedule estimation,

design to cost, incremental development, software

reuse, requirement scrubbing

Developing the wrong functions

and properties: complexity,

imperfect understanding…

Organisational analysis, mission analysis, operations-

concept formulation, user survey, user participation,

prototyping, early user manual, off nominal

performance analysis

Developing the wrong user

interface: not user-friendly,

misleading…

Prototyping, scenarios, task analysis, user participation

Gold plating: adding

unnecessary ―nice‖ features

Requirements scrubbing, prototyping, cost benefit

analysis, designing to cost

Continuing stream of

requirements changes:

requirement volatility forces

rework

High change threshold, information hiding,

incremental development (deferring changes to later

developments)

Shortfalls in externally furnished

components: hardware or

supporting software is

inadequate

Benchmarking, inspection, reference checking,

compatibility analysis

Shortfalls in externally

performed tasks: subcontractors

or users don‘t do what‘s needed

Reference checking, pre-award audit, award-fee

contracts, competitive design or prototyping, team

building

Real-time performance

shortfalls: some or all of the

system causes bottlenecks…

Simulation, benchmarking, modelling, prototyping,

instrumentation, tuning

Straining computer-science

capabilities: unstable or

unfamiliar technology

Technical analysis, cost benefit analysis, prototyping,

reference checking

Page 31: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#22: KEIL, MARK, and PAUL CULE. "Identifying Software Project Risks: An

International Delphi Study." Journal of Management 17, no. 4 (2001): 5-36.

Page 32: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#23: David Johnson; Alexei White; Andre Charland, Chapter 10, Risks and best practices,

Enterprise AJAX: Strategies for Building High Performance Web Applications, Prentice Hall,

2007,

Technical Risks

Relate to the design, development, and maintenance of software, including security,

browser capabilities, timeline, cost of development and hardware, skills of the developers,

and other things of that nature.

Varied client browsers and operating systems

o Richness Vs Reach

o Varied browser capabilities – performance differences

o Unexpected & costly maintenance because of changes in the browser, which can be

exacerbated by hard-to-maintain spaghetti code (code with disorganized and tangled

control structures).

o Forward-compatibility with new browsers

Third-Party Tools Support and Obsolescence

Cultural/Political Risks

Focus around the experience of end users, their attitudes and expectations, and how all this

relates to software.

Perceived insufficiency of conventional visual cues (or affordances) can actually inhibit

usability for less-technologically expert users.

In a consumer-targeted application, switching costs are generally low, and users are

poorly motivated to acclimate to a new interface/ workflow

Legal- there have been efforts to sue private corporations with inaccessible web sites

under the Americans with Disabilities Act (ADA),

Marketing Risks

Relate to successful execution of the business model resulting in sales, donations, brand

recognition, new account registrations, and so on.

Search Engine Accessibility

Reach- because of disabled javascript on client browser

Monetization- if hyperlinks trigger an XHR instead of a full page load, the ad does not

register an impression, revenue loss for website.

Page 33: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#24: Luke Welling; Laura Thomson, PHP and MySQL® Web Development,

Fourth Edition, Addison-Wesley Professional, 2008,

Some important Risks faced by E-commerce companies:

Crackers- malicious computer users

Failure to attract sufficient business

Computer hardware failure

Power, communication, or network failures

Reliance on shipping services

Extensive competition

Software errors

Evolving governmental policies and taxes

System-capacity limits

Ref#25: Tom DeMarco and Tim Lister, Waltzing with Bears: Managing Risk on

Software Projects, Addison Wesley, 2013

Intrinsic Schedule Flaw (undoable estimates)

Specification Breakdown (stakeholder don‘t agree on what to build)

Scope Creep (additional requirements inflate the initially accepted set)

Personnel Loss

Productivity Variation (assumed <>actual performance)

Page 34: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#26: Robertson, Suzanne, and James Robertson, Mastering the Requirements

Process: Getting Requirements Right. 3rd Edition, Addison-Wesley, 2012.

Stakeholders

Truths of Requirements

1. Requirements are not really about requirements.

2. If we must build s/w, then it must be optimally valuable for its owner.

3. If your s/w does not have to satisfy a need, then you can build anything. However, if

it is meant to satisfy a need, then you have to know what that need is to build the

right s/w.

4. There is an important difference between building a piece of s/w and solving a

business problem. The former does not necessarily accomplish the latter.

5. The requirements do not have to be written, but they have to become known to the

builders.

6. Your customer won‘t always give you the right answer. Sometimes it is impossible

for the customer to know what is right, and sometimes he just doesn‘t know what he

needs.

7. Requirements do not come about by chance. There needs to be some kind of orderly

process for developing them.

8. You can be as iterative as you want, but you still need to understand what the

business needs.

9. There is no silver bullet. Methods and tools will not compensate for poor thought

and poor workmanship.

Page 35: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

10. Requirements, if they are to be implemented successfully, must be measurable and

testable.

11. You, the business analyst, will change the way the user thinks about his problem,

either now or later.

Ref#27: Karl E Wiegers; Joy Beatty, Ch 32: Software Requirements and Risk

Management, Software Requirements, 3rd Edition, Microsoft Press, 2013

Requirements Elicitation related risks:

1. Product vision and project scope- write a vision and scope document and use it to

guide decisions about requirements.

2. Time spent on requirements development -Tight proj. schedules often pressure

managers and customers into glossing over the reqs.

3. Customer engagement- Identify stakeholders, customers, and user classes early in the

project. Determine who will serve as the literal voice of the user for each user class.

Engage key stakeholders as product champions. Make sure product champions fulfill

their commitments to help you elicit the correct needs.

4. Completeness and correctness of requirements specifications-Elicit user

requirements that map to business requirements to ensure that the solution will deliver

what the customers really need. Devise usage scenarios, write tests from the

requirements, and have customers define their acceptance criteria. Create prototypes to

make the requirements more meaningful for users and to elicit specific feedback from

them. Enlist customer representatives to review the requirements and analysis models.

5. Requirements for innovative products -Emphasize market research, build prototypes,

and use focus groups to obtain early and frequent customer feedback.

6. Defining nonfunctional requirements- Precisely document nonfunctional

requirements, e.g., performance, usability, security, & reliability and their acceptance

criteria.

7. Customer agreement on requirements-Determine who the primary customers are.

Make sure you‘re relying on the right people for making decisions about requirements.

Have appropriate stakeholder representatives review the requirements.

8. Unstated requirements- Use open-ended questions to encourage customers to share

more of their thoughts, assumptions, wishes, ideas, information, and concerns than you

might otherwise hear. Asking customers what would make them reject the product might

reveal some topics that have not yet been explored.

9. Existing product used as the requirements- Reference requirements development

might not be deemed important on next-generation or replacement projects.

10. Solutions presented as needs- User-proposed solutions can mask the users‘ actual

needs, lead to automating ineffective business processes, and over-constrain the

developers‘ design options.

11. Distrust between the business and the development team-effective requirements

engineering demands close collaboration among various stakeholders, particularly

customer communities and developers.

Page 36: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Requirements Analysis related risks:

12. Requirements prioritization Ensure that every functional requirement, feature, or

user requirement is prioritized and allocated to a specific system release or iteration.

13. Technically difficult features- identify those reqs that might take longer than

anticipated to implement. Use status tracking to watch for requirements that are falling

behind their implementation schedule. Take corrective action as early as possible.

Prototype the novel or risky requirements to select effective approaches.

14. Unfamiliar technologies, methods, languages, tools, or hardware- Don‘t

underestimate the learning curve. Identify those high-risk requirements early on, and

work with the development team to allow sufficient time for false starts, learning, and

experimentation.

Requirements specification related risks:

15. Requirements understanding- Different interpretations of the requirements by

developers and customers lead to expectation gaps, in which the delivered product fails

to satisfy customer needs. Peer review of requirements by developers, testers, and

customers can mitigate this risk. Creating models and prototypes that represent the

requirements from multiple perspectives can reveal fuzzy, ambiguous requirements.

16. Time pressure to proceed despite open issues It is a good idea to mark areas of

the requirements that need further work with TBD (to be determined) or as issues, but

it‘s risky to proceed with construction if these haven‘t been resolved. Record who is

responsible for closing each open issue and the target date for resolution.

17. Ambiguous terminology Create a glossary to define business and technical terms

that might be interpreted differently by different readers. Requirements reviews can help

participants reach a common understanding of terms and concepts.

18. Design included in requirements Design elements that are included in the

requirements place constraints on the options available to developers. Unnecessary

constraints inhibit the creation of optimal designs. Review the requirements to make sure

they emphasize what needs to be done to solve the business problem, rather than

dictating the solution.

Requirements validation related risks:

19. Un-validated requirements- confirm the correctness and quality of each set of

requirements before their implementation, Include time and resources for these quality

activities in the project plan. Gain commitment from your customer representatives to

participate in requirements reviews. Perform incremental, informal reviews to find

problems as early and cheaply as possible.

20. Inspection proficiency- Train all team members who will participate in inspections

of requirements documents. Invite an experienced consultant to observe your early

inspections to coach the participants.

Requirements management related risks:

21. Changing requirements documented business requirements and scope definitions

as the benchmark for approving changes. Use collaborative elicitation process with

extensive user involvement. Design the system for easy modifiability, particularly when

you are following an iterative life cycle.

22. Requirements change process Risks related to how requirements changes are

handled include not having a defined change process, using ineffective change

mechanisms, failing to incorporate valuable changes efficiently, and incorporating

Page 37: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

changes that bypass the process. A requirements change process that includes impact

analysis, a change control board, and a tool to support the process is an important

starting point. Clear communication of changes to the affected stakeholders is essential.

23. Unimplemented requirements- Requirements tracing helps you avoid overlooking

any requirements during design, construction, or testing.

24. Expanding project scope- plan on a phased or incremental delivery life cycle.

Implement the top priority functionality in the early releases, and elaborate the system‘s

capabilities in later iterations.

Ref#28: Karl E Wiegers; Joy Beatty, Ch 15: Risk reduction through prototyping,

Software Requirements, 3rd Edition, Microsoft Press, 2013

Prototyping related Risks:

1. Pressure to release the prototype- Stakeholder will see a running throwaway prototype

and conclude that the product is nearly completed.

2. Distraction by details - With real-looking prototypes, users become fixated on details

about how the user interface will look and operate, it‘s easy for users to forget that they

should be primarily concerned with conceptual issues at the requirements stage. Limit

the prototype to the displays, functions, and navigation options that will let you clear up

uncertain requirements

3. Unrealistic performance expectations- users will infer the expected performance of the

final product from the prototype‘s performance

4. Investing excessive effort in prototypes- This can happen when you are prototyping

the whole solution rather than only the most uncertain, high-risk, or complex portions.

Treat a prototype as an experiment. You‘re testing the hypothesis that the requirements

are sufficiently defined and the key human-computer interface and architectural issues

are resolved so that design and construction can proceed. Do just enough prototyping to

test the hypothesis, answer the questions, and refine the requirements.

Page 38: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

13. 10.02.14:

Taxonomy Based Questions (TBQ) wrt Code and Unit Test (Ref#12: CMU/SEI-93-

TR-6, 1993)

a. Feasibility: Is the implementation of the

design difficult or impossible?]

[31] Are any parts of the product

implementation not completely defined by the

design specification?

[32] Are the selected algorithms and designs

easy to implement?

b. Testing: Are the specified level and time

for unit testing adequate?

[33] Do you begin unit testing before you

verify code with respect to the design?

[34] Has sufficient unit testing been specified?

[35] Is there sufficient time to perform all the

unit testing you think should be done?

[36] Will compromises be made regarding unit

testing if there are schedule problems?

c. Coding/Implementation: Are there any

problems with coding and implementation?

[37] Are the design specifications in

sufficient detail to write the code?

[38] Is the design changing while coding

is being done? [39] Are there system

constraints that make the code difficult to

write?

-- Timing, Memory, External storage

[40] Is the language suitable for

producing the software on this program?

[41] Are there multiple languages used

on the program?

(Yes) (41.a) Is there interface

compatibility between the code produced

by the different compilers?

[42] Is the development computer the

same as the target computer?

(No) (42.a) Are there compiler

differences between the two?

If developmental hardware is being

used

[43] Are the hardware specifications

adequate to code the software?

[44] Are the hardware specifications

changing while the code is being written?

Page 39: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Taxonomy Based Questions (TBQ) (Ref#12: CMU/SEI-93-TR-6, 1993)

Integration and Test Engineering Specialties

a. Environment: Is the integration and

test environment adequate?

[45] Will there be sufficient hardware to

do adequate integration and testing?

[46] Is there any problem with

developing realistic scenarios and test

data to demonstrate any requirements? -

Specified data traffic, Real-time

response, Asynchronous event handling,

Multi-user interaction

[47] Are you able to verify performance

in your facility?

[48] Does hardware and software

instrumentation facilitate testing?

(Yes) (48.a) Is it sufficient for all

testing?

b. Product: Is the interface definition

inadequate, facilities inadequate, time

insufficient?

[49] Will the target hardware be

available when needed?

[50] Have acceptance criteria been

agreed to for all requirements?

(Yes) (50.a) Is there a formal

agreement?

[51] Are the external interfaces defined,

documented, and base-lined?

[52] Are there any requirements that

will be difficult to test?

[53] Has sufficient product integration

been specified?

[54] Has adequate time been allocated

for product integration and test?

[55] Will vendor data be accepted in

verification of requirements allocated to

COTS products?

(Yes) (55.a) Is the contract clear on

that?

c. System: System integration

uncoordinated, poor interface definition,

or inadequate facilities?]

[56] Has sufficient system integration

been specified?

[57] Has adequate time been allocated

a. Maintainability: Will the implementation be

difficult to understand or maintain?

[61] Does the architecture, design, or code create

any maintenance difficulties?

[62] Are the maintenance people involved early

in the design?

[63] Is the product documentation adequate for

maintenance by an outside organization?

b. Reliability: Are the reliability or availability

requirements difficult to meet?

[64] Are reliability requirements allocated to the

software?

[65] Are availability requirements allocated to

the software?

(Yes) (65.a) Are recovery timelines any

problem?

c. Safety: Are the safety requirements infeasible

and not demonstrable?

[66] Are safety requirements allocated to the

software?

(Yes) (66.a) Do you see any difficulty in meeting

the safety requirements?

[67] Will it be difficult to verify satisfaction of

safety requirements?

d. Security: Are the security requirements more

stringent than the current state of the practice or

program experience?

[68] Are there unprecedented or state-of-the-art

security requirements?

[69] Is it an Orange Book system?

[70] Have you implemented this level of security

before?

e. Human Factors: Will the system will be

difficult to use because of poor human interface

definition?

[71] Do you see any difficulty in meeting the

Human Factors requirements?

(No) (71.a) How are you ensuring that you will

meet the human interface requirements?

If prototyping

(Yes) (71.a.1) Is it a throw-away prototype?

(No) (71.a.1a) Are you doing evolutionary

development?

(Yes) (71.a.1a.1) Are you experienced in this

Page 40: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

for system integration and test?

[58] Are all contractors part of the

integration team?

[59] Will the product be integrated into

an existing system?

(Yes) (59.a) Is there a parallel cutover

period with the existing system?

(No) (59.a.1) How will you guarantee

the product will work correctly when

integrated?

[60] Will system integration occur on

customer site?

type of development?

(Yes) (71.a.1a.2) Are interim versions

deliverable?

(Yes) (71.a.1a.3) Does this complicate change

control?

f. Specifications: Is the documentation adequate

to design, implement, and test the system?

[72] Is the software requirements specification

adequate to design the system?

[73] Are the hardware specifications adequate to

design and implement the software?

[74] Are the external interface requirements well

specified?

[75] Are the test specifications adequate to fully

test the system?

If in or past implementation phase

[76] Are the design specifications adequate to

implement the system?

Page 41: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

14,15. 10.02.14:

Ref#29: Cebula, James L., and Lisa R. Young. A Taxonomy of Operational Cyber

Security Risks. No. CMU/SEI-2010-TN-028. Carnegie-Mellon Univ Pittsburgh Pa

Software Engineering Inst, 2010.

Sources of Operational Cyber Security Risk to information and technology assets that

have consequences affecting the confidentiality, availability, or integrity of information

or information systems.

1. Actions of People

1.1 Inadvertent

mistake—individual with knowledge of

the correct procedure accidentally

taking incorrect action

error—individual without knowledge of

the correct procedure taking incorrect

action

omission—individual not taking a known

correct action often due to hasty

performance of a procedure

1.2 Deliberate

fraud—a deliberate action taken to

benefit oneself or a collaborator at the

expense of the organization

sabotage—a deliberate action taken to

cause a failure in an organizational

asset/process, generally carried out by

someone possessing or with access to

inside knowledge

theft—the intentional, unauthorized

taking of organizational assets, in

particular information assets

vandalism—the deliberate damaging of

organizational assets, often at random

1.3 Inaction

skills—an individual‘s lack of ability to

undertake the necessary action

knowledge—an individual‘s ignorance of

the need for action

guidance—a knowledgeable individual

lacking the proper guidance or

direction to act

availability—the unavailability or

nonexistence of the appropriate

resource needed to carry out the

action

2. Systems and Technology Failures

2.1 Hardware

capacity—inability to handle a given load or

volume of information

performance—inability to complete instructions

or process information within acceptable

parameters (speed, power consumption, heat

load, etc.)

maintenance—failure to perform required

upkeep of the equipment

obsolescence—operation of the equipment

beyond its supported service life

2.2 Software

compatibility—inability of >=2 pieces of s/w to

work together as expected

configuration management—improper

application and management of the

appropriate settings and parameters for the

intended use

change control—changes made to the

application or its configuration by a process

lacking appropriate authorization, review,

and rigor

security settings—improper application of

security settings, either too relaxed or too

restrictive, within the program or application

coding practices—failures due to programming

errors, including syntax and logic problems

and failure to follow secure coding practices

testing—inadequate testing of the software

application/configuration

2.3 Systems

design—improper fitness of the system for the

intended application/use

specifications—improper/inadequate definition

of requirements; failure to adhere to the

requirements during system construction

integration—failure of various components of

Page 42: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

the system to function together or interface

correctly; inadequate testing of the system

complexity—large number or interrelationships

between components

3. Failed Internal Processes

3.1 Process design or execution

process flow—poor design of the movement

of process outputs to their intended

consumers

process documentation—inadequate

documentation of the process inputs,

outputs, flow, and stakeholders

roles and responsibilities—insufficient

definition and understanding of process

stakeholder roles and responsibilities

notifications and alerts—inadequate

notification regarding a potential process

problem or issue

information flow—poor design of the

movement of process information to

interested parties and stakeholders

escalation of issues—the inadequate or

nonexistent ability to escalate abnormal or

unexpected conditions for action by

appropriate personnel

service level agreements—the lack of

agreement among process stakeholders on

service expectations that causes a failure to

complete expected actions

task hand-off—―dropping the ball‖ due to the

inefficient handing off of a task in progress

from one responsible party to another

3.2 Process controls

status monitoring—failure to review and

respond to routine information about the

operation of a process

metrics—failure to review process

measurements over time for the purpose of

determining performance trends

periodic review—failure to review the end-to-

end operation of the process on a periodic

basis and make any needed changes

process ownership—failure of a process to

deliver the expected outcome because of

poor definition of its ownership or poor

governance practices

4. External Events

4.1 Disasters

weather event—adverse weather situations

such as rain, snow, tornado, or

hurricane

fire—disruption caused by a fire within or

external to a facility

flood—disruption caused by a flood within

or external to a facility

earthquake—disruption of organizational

operations due to an earthquake

unrest—disruption of operations due to

civil-disorder/riot/terrorism

pandemic—widespread medical

conditions that disrupt organizational

operations

4.2 Legal issues

regulatory compliance—new

governmental regulation or failure to

comply with existing regulation

legislation—new legislation that impacts

the organization

litigation—legal action taken against the

organization by any stakeholder,

including employees and customers

4.3 Business issues

supplier failure—the temporary or

permanent inability of a supplier to

deliver needed products or services to

the organization

market conditions—the diminished ability

of the organization to sell its products

and services in the market

economic conditions—the inability of the

organization to obtain needed funding

for its operations

4.4 Service dependencies

utilities—failure of the organization‘s

electric power supply, water supply, or

telecommunications services

emergency services—dependencies on

public response services e.g., fire,

Page 43: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

3.3 Supporting processes

staffing—failure to provide appropriate human

resources to support its operations

funding—failure to provide appropriate

financial resources to support its operations

training and development—Failure to

maintain the appropriate skills within the

workforce

procurement—failure to provide the proper

purchased service and goods necessary to

support operations

police, and emergency medical services

fuel—failure of external fuel supplies, e.g.,

for a backup generator

transportation—failures in external

transportation systems, e.g.,, inability

of employees to report to work and

inability to make and receive deliveries

Page 44: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#30: Stoneburner, Gary, Alice Goguen, and Alexis Feringa. "Risk Management

Guide for Information Technology Systems: Recommendations of the National

Institute of Standards and Technology" Computer Security Division, NIST Special

Publication 800, no. 30 (2002): 800-30. Integration of Risk Management into the SDLC

SDLC Phases Phase Characteristics Support from Risk Management Activities

Initiation

The need for an IT system is expressed

and the purpose and scope of the IT

system is documented

Identified risks are used to support the

development of the system requirements,

including security requirements, and a security

concept of operations (strategy)

Development or

Acquisition

The IT system is designed, purchased,

programmed, developed, or otherwise

constructed

The risks identified during this phase can be

used to support the security analyses of the IT

system that may lead to architecture and

design tradeoffs during system Development.

Implementation

The system security features should be

configured, enabled, tested, and verified

The risk management process supports the

assessment of the system implementation

against its requirements and within its

modelled operational environment. Decisions

regarding risks identified must be made prior

to system operation

Operation or

Maintenance

The system performs its functions.

Typically the system is being modified

on an ongoing basis through the addition

of hardware and software and by changes

to organizational processes, policies, and

procedures

Risk management activities are performed for

periodic system reauthorization (or

reaccreditation) or whenever

major changes are made to an IT system in its

operational, production environment (e.g., new

system interfaces)

Disposal

This phase may involve the disposition

of information, hardware, and software.

Activities may include moving,

archiving, discarding, or destroying

information and sanitizing the hardware

and software

Risk management activities are performed for

system

components that will be disposed of or

replaced to

ensure that the hardware and software are

properly disposed of, that residual data is

appropriately handled, and that system

migration is conducted in a secure and

systematic manner

Sample Interview Questions to be asked during interviews with site personnel to gain an understanding of

the operational characteristics of an organization

1. Who are valid users?

2. What is the mission of the user organization?

3. What is the purpose of the system in relation to the mission?

4. How important is the system to the user organization‘s mission?

5. What is the system-availability requirement?

6. What information (both incoming and outgoing) is required by the organization?

7. What information is generated by, consumed by, processed on, stored in, and retrieved by the

system?

8. How important is the information to the user organization‘s mission?

9. What are the paths of information flow?

10. What types of information are processed by and stored on the system (e.g., financial, personnel,

research and development, medical, command and control)?

11. What is the sensitivity (or classification) level of the information?

12. What information handled by or about the system should not be disclosed and to whom?

13. Where specifically is the information processed and stored?

Page 45: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

14. What are the types of information storage?

15. What is the potential impact on the organization if the information is disclosed to unauthorized

personnel?

16. What are the requirements for information availability and integrity?

17. What is the effect on the organization‘s mission if the system or information is not reliable?

18. How much system downtime can the organization tolerate? How does this downtime compare with

the mean repair/recovery time? What other processing or communications options can the user

access?

19. Could a system or security malfunction or unavailability result in injury or death?

Ref#31: Dey, Prasanta Kumar, Jason Kinch, and Stephen O. Ogunlana. "Managing risk in

software development projects: a case study." Industrial Management & Data Systems 107, no.

2 (2007): 284-303.

Some questions for risk identification

1. Whether the developed software fulfils the customers‘ demand/requirement?

2. How much competition it is likely to face?

3. Whether benefits from the software surpass the cost of development?

4. Is the project technically feasible?

5. Will hardware, software, and networks function properly?

6. Will the technology be available in time to meet project objectives?

7. Is there any chance of the technology becoming obsolete before use?

8. Will security system work throughout its life?

Page 46: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Ref#32: Iacovou, Charalambos L., and Robbie Nakatsu. "A risk profile of offshore-

outsourced development projects." Communications of the ACM 51, no. 6 (2008): 89-

94. Most important Risk Factor in Offshore-outsourced projects

1. Lack of top management commitment

2. Original set of requirements is mis-communicated

3. Language barriers in project communications

4. Inadequate user involvement

5. Lack of offshore project management know-how

by client

6. Failure to manage end user expectations

7. Poor change controls

8. Lack of business know-how by offshore team

9. Lack of required technical know-how by offshore

team

10. Failure to consider all costs

11. Telecommunications and infrastructure issues

12. Vendor viability

13. Difficulties in ongoing support and maintenance

14. Low visibility of project process

15. Cross-cultural differences

16. High turnover of vendor employees

17. Constraints due to time-zone differences

18. Lack of continuous, face-to-face interactions

across team members

19. Threats to the security of information resources

20. Negative impact on employee morale

21. Unfamiliarity with international and foreign

contract law

22. Differences in development

methodology/processes

23. Political instability in offshore destinations

24. Negative impact on image of client organization

25. Currency fluctuations

Ref#33: Persson, John Stouby, and Lars Mathiassen. "A process for managing risks in

distributed teams." Software, IEEE 27, no. 1 (2010): 20-29. Identifying and analyzing distributed-team risks

Area Risk factor and risk

question

Low risk Medium risk High risk

Task

distri-

bution

Task uncertainty. Do team

members posses the knowledge

and capabilities needed?

Team members know the

task, and it fits well with their

capabilities.

Team members have

reasonable task knowledge,

and their capabilities cover

most challenges.

There are serious

gaps in team

members‘ task

knowledge and

required capabili-

ties.

Task equivocality. Do team

members understand the

specification of the task?

The task is well specified and

understood by team members.

Most aspects of the specifica-

tion are clear, and key team

members understand the task.

The specification

lacks clarity on

major points, and

many team

members have

limited task

understanding.

Task coupling. Is the task

divided into distinct subtasks

across sites?

There is minor need to coor-

dinate development work

across sites.

There is some need to coordi-

nate development work across

sites.

There is major need

to coordinate

development work

across sites.

Know-

ledge

manage-

ment

Knowledge creation. How is

task knowledge created across

sites?

All sites contribute well to the

creation of required task

knowledge.

Most sites contribute reason-

ably well to the creation of

required task knowledge.

There are major

problems related to

the creation of

required task

knowledge.

Knowledge capture. How is

task knowledge captured

across sites?

Task knowledge is captured

effectively across sites.

Task knowledge is, with

some exceptions, captured

effectively across sites.

There are major

problems related to

capturing task

knowledge across

sites.

Knowledge integration. How

is task knowledge integrated

and shared across sites?

Task knowledge is integrated

and shared well across sites.

Task knowledge is, with some

exceptions, well integrated

and shared across sites.

There is limited

task knowledge

integration and

sharing across sites.

Geog-

raphical

Spatial distribution. How

many sites are involved, and

There are few sites collabo-

rating over limited distance.

There are several sites

collaborating over some

There are many

sites collaborating

Page 47: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

distri-

bution

what‘s the distance between

them?

distance. over considerable

distance.

Temporal distribution. How

do time zone differences

impact development work?

Time zone differences cause

no or only minor problems.

Time zone differences require

some ad hoc coordination

across sites.

Time zone

differences cause

major problems and

require constant

attention across

sites.

Goal distribution. How

diverse are goals across sites?

Team members share major

goals across sites.

There is some variation in

goals across sites.

There are major

goal conflicts

across sites.

Colla-

boration

structure

Collaboration capability. Can

team members collaborate

across sites?

Team members collaborate

across sites as needed.

In most cases, team members

collaborate across sites as

needed.

Breakdowns in

collaboration across

sites are common.

Coordination mechanisms.

Are coordination mechanisms

appropriate across sites?

Coordination mechanisms are

shared across sites and well

adapted to the distributed

context.

Coordination mechanisms are

shared by most team members

and reasonably well adapted

to the distributed context.

Coordination

mechanisms are not

shared across sites

and poorly adapted

to the distributed

context.

Process alignment. Are

processes aligned across sites?

Processes (including methods,

templates, and guidelines) are

shared across sites.

Processes (incl. methods,

templates & guidelines) vary

but are reasonably well

aligned across sites.

Processes

(including methods,

templates, and

guidelines) are

different across

sites.

Cultural

distri-

bution

Language barriers. Do

language and communication

norms vary across sites?

Team members share lan-

guage and communication

norms across sites.

Team members use a

common language with minor

differences in comm‘ norms.

Team members

don‘t share a

common language

and have different

comm‘ norms.

Work culture. Does work

culture differ between sites?

Team members share work

culture (including authority

and team behavior) across

sites.

Team members understand

variations in work culture

(including authority and team

behavior) across sites.

Team members

don‘t understand

variations in work

culture (including

authority and team

behavior) across

sites.

Cultural bias. Does cultural

bias impact communication

and cooperation across sites?

There are no major variations

in cultural values across sites.

Team members communicate

and collaborate based on

appreciation of cultural

variations across sites.

Team members

lack knowledge of

variations in

cultural values

across sites.

Stake-

holder

relations

Stakeholder commitment. Are

stakeholders committed to the

project?

Key stakeholders are com-

mitted and share a common

project identity across sites.

Most stakeholders are com-

mitted to the project and

know about its distributed

organization.

Stakeholder

commitment varies,

and there is no

shared project

identity.

Mutual trust. Is there trust

between stakeholders across

sites?

There is appropriate mutual

trust across sites.

There are instances of insuf-

ficient trust across sites.

Stakeholders don‘t

trust each other

across sites.

Relationship building. Can the

project integrate stakeholders

across sites?

Existing and new stake-

holders are well integrated

across sites.

Existing and new stakehold-

ers are mostly integrated well

across sites.

There are several

cases of

stakeholders not

being well

integrated.

Comm-

unication

infra-

structure

Personal communication.

What‘s the level of personal

communication and social

interaction across sites?

The level of personal com-

munication and social

interaction across sites is

appropriate.

There is some personal

communication and social

interaction across sites.

There is limited

personal

communication and

social interaction

across sites.

Page 48: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Interaction media. How well

is communication across sites

supported by media?

Communication needs across

sites are well supported by

media.

Communication across sites

is for many purposes well

supported by media.

There are severe

shortcomings in

media support of

communication

across sites.

Teleconference management.

How well is teleconferencing

managed across sites?

Teleconferencing is used

appropriately and managed

effectively across sites.

Teleconferencing is used to

some extent across sites and

reasonably well managed.

There is limited use

of teleconferencing

across sites and

they aren‘t

managed well.

Techno-

logy

setup

Network capability. Are

communication networks

reliable?

Networks aren‘t causing

delays in development work

and communication.

Networks are causing some

delays in development work

and communication.

Networks are

causing serious

delays in

development work

and

communication.

Tool compatibility. Are tools

compatible across sites?

There are no compatibility

issues between tools across

sites.

Compatibility issues between

tools create some

collaboration barriers across

sites.

Compatibility

issues between

tools create serious

collaboration

barriers across

sites.

Configuration management.

How are configurations

managed across sites?

There‘s appropriate configu-

ration management across

sites.

Configuration management

is, with some exceptions,

appropriate across sites.

There is limited

configuration

management across

sites.

Resolution techniques for mitigating distributed team software development risks

SWOT analysis (Ref#34: Mindtools.com): Identify the key internal and external factors

that are important to achieving the objective Strengths: characteristics of the business or project that give it an

advantage over others- What advantages does your organization have?

What do you do better than anyone else? What unique or lowest-cost

resources can you draw upon that others can't? What do people in your

market see as your strengths? What is your organization‘s USP? What

factors mean that you "get the sale"?

-We are able to respond very quickly as we have no red

tape, and no need for higher management approval.

-We are able to give really good customer care, as the

current small amount of work means we have plenty of

time to devote to customers.

-Our lead consultant has strong reputation in the market.

-We can change direction quickly if we find that our

marketing is not working.

-We have low overheads, so we can offer good value to

Planning

1. Acquire

complementary skills

2. Adjust meetings to

distributed context

3. Divide tasks

systematically

between sites

4. Reduce coupling

between sites

5. Create shared

collaboration platform

6. Establish shared goals

7. Establish

communication norms

8. Define roles and

responsibilities

Control

9. Focus on deliverables

10. Establish task coordination

between sites

11. Maintain site autonomy

12. Establish shared control

mechanisms

13. Establish temporal

coordination mechanisms

14. Maintain project

organization overview

15. Maintain task overview

within and across sites

16. Monitor and improve

communication

17. Maintain a supportive

environment

18. Analyze and manage errors

Social integration

19. Improve capability to

manage cultural

differences

20. Improve distributed

collaboration skills

21. Improve language skills

22. Emphasize early

teambuilding activities

23. Promote humor and

openness

24. Use mentors to integrate

new members

25. Use face-to-face

meetings appropriately

26. Develop liaisons

between sites

27. Adopt shared reward

systems

Technical integration

28. Increase technical

compatibility between

sites

29. Standardize and train

in methods across sites

30. Adopt appropriate

communication

technologies

31. Improve collaboration

and communication

technology skills

32. Improve development

technology skills

33. Handle differences in

methods between sites

34. Combine waterfall

model and prototyping

Page 49: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

customers.

Weaknesses: characteristics that place the team at a disadvantage

relative to others- What could you improve? What should you avoid?

What are people in your market likely to see as weaknesses?

What factors lose you sales?

-Our company has little market presence or reputation.

-We have a small staff, with a shallow skills base in many

areas.

-We are vulnerable to vital staff being sick, and leaving.

-Our cash flow will be unreliable in the early stages.

Opportunities: elements that the project could exploit to its advantage- What good

opportunities can you spot? What interesting trends are you aware of? Useful

opportunities can come from such things as:--

Changes in - technology/markets on broad/narrow scale; government policy related to your

field; social patterns, population profiles, lifestyle changes; Local events.

-Our business sector is expanding, with

many future opportunities for success.

-Local government wants to encourage

local businesses.

-Our competitors may be slow to adopt

new technologies.

Threats: elements in the environment that could cause trouble for the business or

project- What obstacles do you face? What are your competitors doing? Are quality

standards or specifications for your job, products or services changing? Is changing

technology threatening your position? Do you have bad debt or cash-flow

problems? Could any of your weaknesses seriously threaten your business?

-Developments in technology may change this

market beyond our ability to adapt.

-A small change in the focus of a large

competitor might wipe out any market position

we achieve.

Ref#35: Manual of Accreditation, National Board of Accreditation, India, 2013,

Graduate Attributes UG Engineering

1. Engineering knowledge: Apply the knowledge of mathematics, science, engineering

fundamentals, and an engineering specialisation to the solution of complex engineering

problems.

2. Problem analysis: Identify, formulate, research literature, and analyse complex

engineering problems reaching substantiated conclusions using first principles of

mathematics, natural sciences, and engineering sciences.

3. Design/development of solutions: Design solutions for complex engineering problems

and design system components or processes that meet the specified needs with

appropriate consideration for the public health and safety, and the cultural, societal,

and environmental considerations.

4. Conduct investigations of complex problems: Use research-based knowledge and

research methods including design of experiments, analysis and interpretation of data,

and synthesis of the information to provide valid conclusions.

5. Modern tool usage: Create, select, and apply appropriate techniques, resources, and

modern engineering and IT tools including prediction and modelling to complex

engineering activities with an understanding of the limitations.

6. The engineer and society: Apply reasoning informed by the contextual knowledge to

assess societal, health, safety, legal, and cultural issues and the consequent

responsibilities relevant to the professional engineering practice.

7. Environment and sustainability: Understand the impact of the professional

engineering solutions in societal and environmental contexts, and demonstrate the

knowledge of, and need for sustainable development.

8. Ethics: Apply ethical principles and commit to professional ethics and responsibilities

and norms of the engineering practice.

9. Individual and Team work: Function effectively as an individual, and as a member or

leader in diverse teams, and in multidisciplinary settings.

10. Communication: Communicate effectively on complex engineering activities with the

engineering community and with society at large, such as, being able to comprehend and

write effective reports and design documentation, make effective presentations, and give

and receive clear instructions.

Page 50: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

11. Project management and finance: Demonstrate knowledge and understanding of the

engineering and management principles and apply these to one's own work, as a member

and leader in a team, to manage projects and in multidisciplinary environments.

12. Life-long learning: Recognise the need for, and have the preparation and ability to

engage in independent and life-long learning in the broadest context of technological

change.

Graduate Attributes PG Engineering

1. Scholarship of Knowledge: Acquire in-depth knowledge of specific discipline or

professional area, including wider and global perspective, with an ability to discriminate,

evaluate, analyse and synthesise existing and new knowledge, and integration of the

same for enhancement of knowledge.

2. Critical Thinking: Analyse complex engineering problems critically, apply independent

judgement for synthesising information to make intellectual and/or creative advances for

conducting research in a wider theoretical, practical and policy context.

3. Problem Solving: Think laterally and originally, conceptualise and solve engineering

problems, evaluate a wide range of potential solutions for those problems and arrive at

feasible, optimal solutions after considering public health and safety, cultural, societal

and environmental factors in the core areas of expertise.

4. Research Skill: Extract information pertinent to unfamiliar problems through literature

survey and experiments, apply appropriate research methodologies, techniques and tools,

design, conduct experiments, analyse and interpret data, demonstrate higher order skill

and view things in a broader perspective, contribute individually/in group(s) to the

development of scientific/technological knowledge in one or more domains of

engineering.

5. Usage of modern tools: Create, select, learn and apply appropriate techniques,

resources, and modern engineering and IT tools, including prediction and modelling, to

complex engineering activities with an understanding of the limitations.

6. Collaborative and Multidisciplinary work: Possess knowledge and understanding of

group dynamics, recognise opportunities and contribute positively to collaborative-

multidisciplinary scientific research, demonstrate a capacity for self-management and

teamwork, decision-making based on open-mindedness, objectivity and rational analysis

in order to achieve common goals and further the learning of themselves as well as

others.

7. Project Management and Finance: Demonstrate knowledge and understanding of

engineering and management principles and apply the same to one‘s own work, as a

member and leader in a team, manage projects efficiently in respective disciplines and

multidisciplinary environments after consideration of economical and financial factors.

8. Communication: Communicate with the engineering community, and with society at

large, regarding complex engineering activities confidently and effectively, such as,

being able to comprehend and write effective reports and design documentation by

adhering to appropriate standards, make effective presentations, and give and receive

clear instructions.

9. Life-long Learning: Recognise the need for, and have the preparation and ability to

engage in life-long learning independently, with a high level of enthusiasm and

commitment to improve knowledge and competence continuously.

Page 51: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

10. Ethical Practices and Social Responsibility: Acquire professional and intellectual

integrity, professional code of conduct, ethics of research and scholarship, consideration

of the impact of research outcomes on professional practices and an understanding of

responsibility to contribute to the community for sustainable development of society.

11. Independent and Reflective Learning: Observe and examine critically the

outcomes of one‘s actions and make corrective measures subsequently, and learn from

mistakes without depending on external feedback.

Page 52: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

Graduate Attributes for UG Engineering Graduate Attributes for PG Engineering Engineering knowledge: Apply the knowledge of

mathematics, science, engineering fundamentals,

and an engineering specialisation to the solution of

complex engineering problems.

Scholarship of Knowledge: Acquire in-depth knowledge of

specific discipline or professional area, including wider and global

perspective, with an ability to discriminate, evaluate, analyse and

synthesise existing and new knowledge, and integration of the same

for enhancement of knowledge.

Problem analysis: Identify, formulate, research

literature, and analyse complex engineering

problems reaching substantiated conclusions using

first principles of mathematics, natural sciences,

and engineering sciences.

Critical Thinking: Analyse complex engineering problems

critically, apply independent judgement for synthesising

information to make intellectual and/or creative advances for

conducting research in a wider theoretical, practical and policy

context.

Design/development of solutions: Design

solutions for complex engineering problems and

design system components or processes that meet

the specified needs with appropriate consideration

for the public health and safety, and the cultural,

societal, and environmental considerations.

Problem Solving: Think laterally and originally, conceptualise and

solve engineering problems, evaluate a wide range of potential

solutions for those problems and arrive at feasible, optimal

solutions after considering public health and safety, cultural,

societal and environmental factors in the core areas of expertise.

Conduct investigations of complex problems: Use research-based knowledge and research

methods including design of experiments, analysis

and interpretation of data, and synthesis of the

information to provide valid conclusions.

Research Skill: Extract information pertinent to unfamiliar

problems through literature survey and experiments, apply

appropriate research methodologies, techniques and tools, design,

conduct experiments, analyse and interpret data, demonstrate

higher order skill and view things in a broader perspective,

contribute individually/in group(s) to the development of scientific/

technological knowledge in one or more domains of engineering.

Modern tool usage: Create, select, and apply

appropriate techniques, resources, and modern

engineering and IT tools including prediction and

modelling to complex engineering activities with

an understanding of the limitations.

Usage of modern tools: Create, select, learn and apply appropriate

techniques, resources, and modern engineering and IT tools,

including prediction and modelling, to complex engineering

activities with an understanding of the limitations.

The engineer and society: Apply reasoning

informed by the contextual knowledge to assess

societal, health, safety, legal, and cultural issues

and the consequent responsibilities relevant to the

professional engineering practice.

Ethical Practices and Social Responsibility: Acquire professional

and intellectual integrity, professional code of conduct, ethics of

research and scholarship, consideration of the impact of research

outcomes on professional practices and an understanding of

responsibility to contribute to the community for sustainable

development of society.

Environment and sustainability: Understand the

impact of the professional engineering solutions in

societal and environmental contexts, and

demonstrate the knowledge of, and need for

sustainable development.

Ethics: Apply ethical principles and commit to

professional ethics and responsibilities and norms

of the engineering practice.

Individual and Team work: Function effectively

as an individual, and as a member or leader in

diverse teams, and in multidisciplinary settings.

Collaborative and Multidisciplinary work: Possess knowledge

and understanding of group dynamics, recognise opportunities and

contribute positively to collaborative-multidisciplinary scientific

research, demonstrate a capacity for self-management and

teamwork, decision-making based on open-mindedness, objectivity

and rational analysis in order to achieve common goals and further

the learning of themselves as well as others.

Communication: Communicate effectively on

complex engineering activities with the engineering

community and with society at large, such as, being

able to comprehend and write effective reports and

design documentation, make effective

presentations, and give and receive clear

instructions.

Communication: Communicate with the engineering community,

and with society at large, regarding complex engineering activities

confidently and effectively, such as, being able to comprehend and

write effective reports and design documentation by adhering to

appropriate standards, make effective presentations, and give and

receive clear instructions.

Project management and finance: Demonstrate

knowledge and understanding of the engineering

and management principles and apply these to

one's own work, as a member and leader in a team,

to manage projects and in multidisciplinary

Project Management and Finance: Demonstrate knowledge and

understanding of engineering and management principles and apply

the same to one‘s own work, as a member and leader in a team,

manage projects efficiently in respective disciplines and

multidisciplinary environments after consideration of economical

Page 53: Problem Solving and Research Methodology: Part-I- Risk Engineering - Excerpts of references:

Lecture Notes, PSRM-I: Risk Engg. Sanjay Goel, JIIT, 2014, Jan-Feb

environments. and financial factors.

Life-long learning: Recognise the need for, and

have the preparation and ability to engage in

independent and life-long learning in the broadest

context of technological change.

Life-long Learning: Recognise the need for, and have the

preparation and ability to engage in life-long learning

independently, with a high level of enthusiasm and commitment to

improve knowledge and competence continuously.

Independent and Reflective Learning: Observe and examine

critically the outcomes of one‘s actions and make corrective

measures subsequently, and learn from mistakes without depending

on external feedback.

Ref#36: Software Engineering Code of Ethics and Professional Practice for Ver 5.2,

IEEE and ACM, 1999

Software engineers shall commit themselves to making the analysis, specification, design,

development, testing and maintenance of software a beneficial and respected profession. In

accordance with their commitment to the health, safety and welfare of the public, software

engineers shall adhere to the following Eight Principles:

1. PUBLIC - Software engineers shall with act consistently the public interest.

2. CLIENT AND EMPLOYER - Software engineers shall act in a manner that is in the best

interests of their client and employer consistent with the public interest.

3. PRODUCT - Software engineers shall ensure that their products and related modifications meet

the highest professional standards possible.

4. JUDGMENT - Software engineers shall maintain integrity and independence in their

professional judgment.

5. MANAGEMENT - Software engineering managers and leaders shall subscribe to and promote an

ethical approach to the management of software development and maintenance.

6. PROFESSION - Software engineers shall advance the integrity and reputation of the profession

consistent with the public interest.

7. COLLEAGUES - Software engineers shall be fair to and supportive of their colleagues.

8. SELF - Software engineers shall participate in lifelong learning regarding the practice of their

profession and shall promote an ethical approach to the practice of the profession.

##################################### First Test #############################