Information Systems Overview Organizing Complex Projects Around Risks Arising From System Dynamic...
-
date post
21-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Information Systems Overview Organizing Complex Projects Around Risks Arising From System Dynamic...
Information Systems Overview
Organizing Complex Projects Around Risks Arising From System Dynamic Behavior
Neil SiegelSector Vice-President & Chief Engineer
Northrop Grumman
9 March 2011
Scope of proposed study
This study aims to assess the efficacy of one particular method of improving the outcomes on
complicated, large-scale, software-intensive system development projects
3
“Bottom-line up front”
4
Systems are important to society . . .
Hypothesis: centralizing control of
system dynamic behavior while applying critical skills will lead to superior results
. . . but system development efforts often
fail
Assess in 1 domain
Assess in 4 additional domains
Formulate conclusions
Select scope / characteristics of systems of
interest
Systems are important to society,but many system development efforts fail
5
• What do we mean by “fail”:– Do not produce a product that meets the original specifications– Produce such a product only after taking significantly more time
and/or money than originally expected. – In the extreme, many such projects are cancelled before
completion• Failure is apparently common:
– (Glass 2001) cites data indicating that only about 16% of the system development projects that he surveyed were listed as successful by their own developers
– (Johnson 2006) cites data from the Standish Group, which describes results from a 2004 survey: just 29% of development projects succeeded
Scope of the systems of interest
• Complex emergent behavior, as described by (Rechtin 1991)
• Interactions with physical devices (physically-moving mechanisms, other time-sensitive mechanical devices, etc.)
• Stressing asynchronous stimuli (such as extraordinary high data-ingest rates, or highly-stressed communications structures)
• Extraordinarily high availability / reliability requirements• Development efforts of large size • Systems that need to display much early progress
through prototyping and re-use
6
Hypotheses
During the development phase of a large-scale, complex computer-based system, the use of a specific design-based technique that
centralizes the control of the dynamic behavior of a system will lower the density of
those defects that are attributable to unplanned adverse dynamic system behavior
7
Partitioning the work
• I propose using the design process to partition the work into different “skill bins”, so as to provide better ways to assign people to tasks.
– This allows a particular difficult task (control and management of the system’s dynamic behavior) to be partitioned away from most of the development team.
• Under current methods, the “hard” parts of the work can be disseminated into a large portion of the tasks
8
Describe system’s desired dynamic behavior, to the level of every independently-schedulable entity
Take a design decision to centralize control of system dynamic behavior into a small portion of the design and implementation (“control kernel”)
Recognize every mechanism available to a developer that could create concurrency and other forms of dynamic behavior
Designate a small team with suitable expertise as the implementers of the control kernel
Train the developers outside of those implementing the control kernel that they are not to use these mechanisms, and create ways to enforce that proscription
Provide a mechanism for specification & implementation of the threads and allowable interconnects / interactions amongst the components
Provide a mechanism that allows for a small portion of the implementation to specify and control the dynamic behavior of the system, e.g., implement the control kernel
Provide for a mechanism that supports the instrumented exercising and analysis of the threads under realistic stimuli, and do so in a way that allows the dynamic behavior to be observed and adjusted
separately from the specific functionality of the system, and can do so even prior to the availability of actual system components
Allow for the entities within the system that are designed and implemented without reference to dynamic behavior in some fashion to “inherit” a set of behavioral controls in accordance with the
parameters and algorithms implemented within the control kernel
Automatically translate the specification of the threads and allowable interconnects / interactions amongst the components (whether hardware or software or people) into an executable mechanism
How I accomplish that partitioningThe System Architecture Skeleton
9
All 6 periods / all 4 cases
11
0
5
10
15
20
25
30
1 2 3 4 5 6 7 8 9 10 11
Project YYYY period II
Project AAAA
Project BBBB
Project YYYY period I
Project YYYY period IIII
Project ZZZZ
Did not use the design-based
technique
Did use the design-based
technique
Defect density“Use of the design-based technique will reduce the density of defects attributable to errors in managing system dynamic
12
Periods I-III contractor test on a single plot – FBCB2 – problem reports attributable to unplanned dynamic system behavior – opened per month
(adjusted for peer-review and unit-test procedure changes)(time is discontinuous between periods)
Initial version 31 December 2010This version 31 December 2010
1998 2007 2009Den
sity
–op
ened
thi
s m
onth
/ 1
M S
LOC
-5
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Monthly results
UNPL
LNPL
2-s upper
2-s lower
1-s upper
1-s lower
average
0
1
2
3
4
5
6
7
8
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
MR
MR limit
Period II behavior is materially different – o(4x) more problem reports attributable to this cause
Project YYYY, periods I-III contractor tests, attributable problem reports by month.
behavior”
13
Periods I-III contractor test on a single plot – FBCB2 –actual cost per month divided by budgeted cost per month
(time is discontinuous between periods)
1998 2007 2009
Actu
al c
ost
per
mon
th /
bud
get
per
mon
th
0.85
0.9
0.95
1
1.05
1.1
1.15
1.2
1.25
1.3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Monthly results
UNPL
LNPL
2-s upper
2-s lower
1-s upper
1-s lower
average
Initial version 2 February 2011This version 2 February 2011
Cost performance also tracksthe dependent variable
15
• The data indicate that the proposed design-based technique may in fact lead to better outcomes.
• Indicated by a materially-lower density of defects (on the order of four times lower) that were attributable to errors in controlling system dynamic behavior
Summary & interpretations
• The design-based technique considered herein only applies to projects where such centralization is possible
– The study did demonstrate that this set of projects is not vanishingly small
• This specific technique, however, is only one possible technique for creating a partitioning of a project into easy and difficult portions
– Future studies could propose and assess other such techniques.
16
Summary & interpretations
Implications for practice
• Control structures may be more important than generally recognized
• New goal for the design process
17