Presentation from September 12, 2006 Dinner Meeting

8/9/2019 Presentation from September 12, 2006 Dinner Meeting

1/75

Enabling Systems Thinking toEnabling Systems Thinking to

Accelerate the Development ofAccelerate the Development of

Senior Systems EngineersSenior Systems Engineers

INCOSE PresentationINCOSE PresentationSeptember 2006September 2006

Heidi L. DavidzHeidi L. Davidz


2/75

September 12, 2006 Massachusetts Institute of Technology Slide 2

Heidi Davidz, [email protected]

* Introduction *MethodsResults

Implications

Conclusion

Acknowledgment ofAcknowledgment of

Research SupportResearch Support Doctoral Committee

Professor Deborah Nightingale (chair)

Professor Tom Allen Dr. Joel Cutcher-Gershenfeld

Dr. Eric Rebentisch

Dr. Donna Rhodes

Research Sponsored by the Lean AerospaceInitiative (LAI) Additional Reader: Professor John Carroll


3/75



AgendaAgenda IntroductionIntroduction

Research MethodsResearch Methods ResultsResults

ImplicationsImplications

ConclusionConclusion Conclusions on My ResearchConclusions on My Research

Reflections on Enabling SE Research RigorReflections on Enabling SE Research Rigor


Implications

Conclusion


4/75

Enabling Systems Thinking to Accelerate theEnabling Systems Thinking to Accelerate the

Development of Senior Systems EngineersDevelopment of Senior Systems EngineersHeidi DavidzHeidi Davidz

Advisor: Professor Deborah NightingaleAdvisor: Professor Deborah Nightingale

Results

Even though systems thinking definitionsdiverge, there is consensus on primarymechanisms that enable or obstructsystems thinking development in engineers

Enabling mechanisms include experientiallearning, certain individual characteristics,supportive environment

Developed a framework and conceptualillustration for systems thinking

Implications

Identified implications for government,industry, and academia

Highlighted inconsistencies between policy& effective mechanisms

Need to evolve intervention maturity Government should set enabling policy Industry should utilize primary mechanisms Academia should continue studying how

systems thinking actually develops

Methods

1. Literature Review2. Pilot Interviews3. Field Study with Interviews & Surveys

205 Participants, 10 Companies Expert Panelists, Sr. Systems Engineers,

Sr. Technical Specialists & Jr. Systems

Engineers4. Blue Chip Interviews5. Data Analysis6. Theory Synthesis

Motivation

Increasing interest in systems thinking

Data needed on systems thinking development

InnatetraitsJob

rotations

Systemswork roles

Training

classes

What are themechanisms thatdevelop systems

thinking in engineers?

Universityprograms

Exploratory

Inductive

How do senior systemsengineers develop?


5/75



MotivationMotivation

Increasing complexity of engineering systems and theIncreasing complexity of engineering systems and the

corresponding need for systems professionalscorresponding need for systems professionals

Importance of systems engineering, demonstrated inImportance of systems engineering, demonstrated in

policy mandatespolicy mandates

Importance of systems engineering workforce issues,Importance of systems engineering workforce issues,

also shown in policy documentsalso shown in policy documents

DataData needed on systems thinking development inneeded on systems thinking development in

order to know which methods are most effective inorder to know which methods are most effective indeveloping systems thinking in engineersdeveloping systems thinking in engineers


Implications

Conclusion

Need for DATA on Systems Thinking Development


6/75



Key ResearchKey Research

QuestionsQuestions1.1. What are enablers, barriers, andWhat are enablers, barriers, and

precursors to the development ofprecursors to the development of

systems thinking in engineers?systems thinking in engineers?

2.2. How do senior systems engineersHow do senior systems engineers

develop?develop?

3.3. What are the mechanisms that developWhat are the mechanisms that developsystems thinking in engineers?systems thinking in engineers?


Implications

Conclusion


7/75



Broad literature found on systems thinkingBroad literature found on systems thinking

Lack of a central, ongoing discussionLack of a central, ongoing discussion

Systems thinking literature found in disparate fields andSystems thinking literature found in disparate fields and

journals, from systems dynamics to systems engineeringjournals, from systems dynamics to systems engineering

to general philosophyto general philosophy

Very limited literature on systems thinkingVery limited literature on systems thinking

development and mechanisms for developmentdevelopment and mechanisms for development

Heavy dependence on heuristics of how systemsHeavy dependence on heuristics of how systemsthinking developsthinking develops

Introduction

* Methods *Results

Implications

Conclusion

Literature ReviewLiterature Review

and Existing Theoryand Existing Theory

Scant Literature on Systems Thinking Development


8/75



ResearchResearch

MethodsMethodsLiterature

review

Inductive

Exploratory

PilotInterviews

(N=12)

AdditionalInterviewswith Blue

ChipProvenExperts(N=2)

(c) ExpertPanelists(N=37)

Completedsurvey andinterview

Identifiedsubjects for3 follow-ongroups

(d) Follow-On Subjects Completed interview

Completed survey

(a) Contacted

Company 10 companies

participated

Primarily U.S.aerospace companies

(b) Point-of-contactIdentified POC to workwith others to identify

Expert Panelists

1. SeniorSystemsEngineers(N=62)

2. SeniorTechnicalSpecialists(N=53)

3. JuniorSystemsEngineers(N=53)

(Total of 205 interviews and 188 surveys)

FieldStudy

Data AnalysisUsing QSR N6,SPSS,MS Excel

TheorySynthesis


9/75



Company Site System Context

The Aerospace

Corporation

Systems Engineering in Chantilly, VA &

Los Angeles, CA

FFRDC - Global Positioning System (GPS), Air Force Satellite

Communications (AFSATCOM) System, etc.1

BMW Systems Architects at BMW Group inMunich, Germany

Commercial - Manufacturer of premium automobiles and

motorcycles2

Boeing Boeing Commercial Airplanes,Engineering Liaison group in Renton

and Everett, Washington

Contractor - Commercial jetliner manufacturer3

Booz Allen

Hamilton

Systems group, multiple locations,

referred by a systems partner at

headquarters in McLean, VA

Consultant - Strategic management and technology consulting

firm to industry and government4

General

Dynamics

Sites 1 & 2

SE at General Dynamics Advanced

Information Systems in Bloomington,

MN and in Pittsfield, MA

Contractor - Provider of transformational mission solutions in

command, control, communications, computers, intelligence,

surveillance and reconnaissance (i.e. Future Combat Systems)5

MITRE Systems Engineering in Bedford, MA &

McLean, VA

FFRDC - Global Information Grid, IRS enterprise modernization

program, etc.6

Northrop

Grumman

Airborne Ground Surveillance & Battle

Management Systems, Integrated

Systems, Melbourne, FL, SE

Contractor - E-8C Joint Surveillance Targeting Attack Radar

System (Joint STARS), Cyber Warfare Integration Network

(CWIN), etc.7

Pratt &

Whitney

SE in East Hartford, CT Contractor - Design, manufacture, and support of turbine engines8

Sikorsky SE in Stratford, CT Contractor - Design and build advanced helicopters for

commercial, industrial and military use9

Participating CompaniesParticipating Companies


10/75



Coding in QSR N6Coding in QSR N6

Q: How to make sense of 205 interviews each with a 4Q: How to make sense of 205 interviews each with a 4--55page transcript?page transcript?

A: Use content analysis to categorize key ideas andA: Use content analysis to categorize key ideas and

thoughts from the interviewthoughts from the interview This categorization process is called codingThis categorization process is called coding

The resulting categories are called nodesThe resulting categories are called nodes

The nodes were recorded and organized in aThe nodes were recorded and organized in a

qualitative data management tool calledQ

SR N6qualitative data management tool calledQ

SR N6

1010

Nodes were organized in hierarchies, with Level 2Nodes were organized in hierarchies, with Level 2as a subas a sub--node of Level 1node of Level 1

Content Analysis Performed Using QSR N6 Tool

Introduction

* Methods *Results

Implications

Conclusion


11/75



Introduction

* Methods *Results

Implications

Conclusion

Content Analysis:Content Analysis:

From Raw Data to NodesFrom Raw Data to NodesQuestion Response

Node Hierarchies Organize Conceptual Patterns

ScreenShot

Coded As:Coded As: Level 1 NodeLevel 1 Node -- ExperienceExperience

Level 2 NodeLevel 2 Node -- Job/opportunity to seeJob/opportunity to see

systems viewsystems view

Individual lines of ~1000 pages ofIndividual lines of ~1000 pages of

transcripts coded in this waytranscripts coded in this way

Yield of 908 nodesYield of 908 nodes


12/75



AdditionalAdditional

Data AnalysisData Analysis Interview data exported fromQSR N6 to MS ExcelInterview data exported fromQSR N6 to MS Excel

to determine top interview responsesto determine top interview responses

Interview data exported fromQSR N6 to SPSS toInterview data exported fromQSR N6 to SPSS to

run statistical testsrun statistical tests Results reported at both Level 1 and Level 2 ofResults reported at both Level 1 and Level 2 of

the node hierarchy to address aggregation biasthe node hierarchy to address aggregation bias

SPSS used to analyze survey dataSPSS used to analyze survey data

Manual content analysis performed on pilotManual content analysis performed on pilotinterviews and blue chip interviewsinterviews and blue chip interviews

UtilizedMultiple Data Exploration Methods

Introduction

* Methods *Results

Implications

Conclusion


13/75



UnderlyingUnderlying

Research ResultResearch Result

Even though systems

thinking definitions diverge,there is consensus on primary

mechanisms that enable orobstruct systems thinking

development in engineers

IntroductionMethods

* Results *Implications

Conclusion


14/75



Consensus on PrimaryConsensus on Primary

Enabling MechanismsEnabling Mechanisms There is consensus on primaryThere is consensus on primary

mechanisms that enable systemsmechanisms that enable systems

thinking development in engineersthinking development in engineers1.1. Experiential learningExperiential learning

2.2. Individual characteristicsIndividual characteristics

3.3. Supportive environmentSupportive environment

Data Show Consensus

IntroductionMethods


Conclusion


15/75



Q: How can people agree on mechanisms that enableQ: How can people agree on mechanisms that enable

systems thinking when their definitions of systemssystems thinking when their definitions of systems

thinking do not agree?thinking do not agree?

A: Though the articulation of the systems thinkingA: Though the articulation of the systems thinking

definitions diverge, there are common themes:definitions diverge, there are common themes:

(a) Functions and behaviors at the(a) Functions and behaviors at the contextual edgecontextual edge

(b)(b) InteractionsInteractions of elements and how large scale things relateof elements and how large scale things relate

The primary mechanisms cited enable and encourageThe primary mechanisms cited enable and encourage(a) Translation across contextual edges(a) Translation across contextual edges

(b) Consideration of interactions(b) Consideration of interactions

(c) Higher impact learning(c) Higher impact learning

Solving the PuzzleSolving the PuzzleIntroduction

Methods


Conclusion


16/75



Expert Panelists and followExpert Panelists and follow--on subjectson subjects

were asked:were asked: How do you define systems thinking?How do you define systems thinking?

Considering a given systems thinking definition, whatConsidering a given systems thinking definition, what

aspects do you agree or disagree with and whyaspects do you agree or disagree with and why

205 interviews, 205 unique definitions205 interviews, 205 unique definitions

Data show that when people refer to theData show that when people refer to the

phrase systems thinking they are oftenphrase systems thinking they are often

not articulating the same conceptnot articulating the same concept

Divergent SystemsDivergent Systems

Thinking DefinitionsThinking Definitions

Systems Thinking Definitions Diverge

IntroductionMethods


Conclusion

InterviewQuestions


17/75



Example SystemsExample Systems

Thinking DefinitionsThinking Definitions Big picture

Interactions

Worrying about everything

System thinking is the ability to think about a system or system architecture

holistically, considering the design elements, complexities, the ilities, thecontext that product or system will be used in, etc.

You have to think extremely broadly. You cant focus on a specific aspect.

Think from the application of what a product is. Think from what the customer

wants explicitly. Be able to think in all the areas that are related to that device.

Its broad and deep thinking. If you cant do both, then you shouldnt do

systems stuff. You must be organized. Think without boundaries at the start.If you think that your job is the requirements, then you are a clerk, not a

systems engineer.

Connecting lots of dissimilar disciplines and weighing trade offs between

them

IntroductionMethods


Conclusion

MoreDefinitions


18/75



Reconciling SystemsReconciling Systems

Thinking DefinitionsThinking Definitions Synthesis of the definitions from the field study and the definitionsSynthesis of the definitions from the field study and the definitions

in the literature yielded an original framework of systems thinkingin the literature yielded an original framework of systems thinking

Five foundational elements:Five foundational elements:

1.1. COMPONENTIALCOMPONENTIAL -- What types of things are consideredWhat types of things are considered2.2. RELATIONALRELATIONAL -- Interconnections, interactions, and interdependenciesInterconnections, interactions, and interdependencies

both within the system of interest and between the system of interestboth within the system of interest and between the system of interest

and other systemsand other systems

3.3. CONTEXTUALCONTEXTUAL The nested and embedded nature of systemsThe nested and embedded nature of systems

4.4. DYNAMICDYNAMIC Links system in time to future and past, includes feedback,Links system in time to future and past, includes feedback,

uncertainty, risk, and the ilitiesuncertainty, risk, and the ilities5.5. MODALMODAL Aids to understand and comprehend systemAids to understand and comprehend system

Systems thinking is utilizing modal elements to consider theSystems thinking is utilizing modal elements to consider the

componential, relational, contextual, and dynamic elements of thecomponential, relational, contextual, and dynamic elements of the

system of interest.system of interest.

IntroductionMethods


Conclusion


19/75



IntroductionMethods


Conclusion

Conceptual IllustrationConceptual Illustration

of Systems Thinkingof Systems Thinking

2005 Andreas Davidz, Elizabeth Davidz,Heidi Davidz. All rights reserved. Used with permission.

Tools &

Methods

Types of

Thinking

Models &

Simulations

Processes &

Frameworks


20/75



Coding ResultsCoding Results

Systems Thinking MindsetSystems Thinking Mindset This MUST be decomposed, since understandings can beThis MUST be decomposed, since understandings can be

contradictorycontradictory

Before designing an intervention, know what you are trying toBefore designing an intervention, know what you are trying toproduceproduce

SystemSystem--ofof--Systems SE TraitsSystems SE Traits

Not detail focusedNot detail focused

Thinks outThinks out--ofof--thethe--boxbox

CreativeCreative

Abstract thinkingAbstract thinking

ProcessProcess--Centered SE TraitsCentered SE Traits

Detail orientedDetail oriented

StructuredStructuredMethodicalMethodical

AnalyticalAnalytical

Define the Goal then Design the Intervention

IntroductionMethods


Conclusion


21/75



Difficulties withDifficulties with

Determining Strength ofDetermining Strength of

Systems ThinkingSystems Thinking

Systems Thinking Definitions Diverge

IntroductionMethods


Conclusion

DivergentDefinitions

Divergent systems thinking definitions areDivergent systems thinking definitions are

problematic since strength of systemsproblematic since strength of systems

thinking is determined by observation andthinking is determined by observation andsubjective measuresubjective measure

In addition, many of the respondents do notIn addition, many of the respondents do not

know how strength of systems thinking isknow how strength of systems thinking is

determined in their organizationdetermined in their organization


22/75



IntroductionMethods


Conclusion

How does your company determine if anHow does your company determine if an

employee displays strong systems thinking?employee displays strong systems thinking?

Difficulty

Observation & Subjective Measure

Level

Determination of StrengthDetermination of Strength

of Systems Thinkingof Systems Thinking


23/75



Subjective Determination ofSubjective Determination of

Strength of Systems ThinkingStrength of Systems Thinking

IntroductionMethods


Conclusion

How does your company determine if anHow does your company determine if an

employee displays strong systems thinking?employee displays strong systems thinking?

Do not

know

Observation

& Subjective

Measures


24/75



Consensus onConsensus on

EnablersEnablers

IntroductionMethods


Conclusion

Consensus on primary mechanisms that enable orobstruct systems thinking development in engineers

1. Experiential learning2. Individual characteristics3. Supportiveenvironment

Even though systemsthinking definitions diverge,

there is consensus onprimary mechanisms that

enable or obstruct systemsthinking development in

engineers


25/75



IntroductionMethods


Conclusion

Experiential LearningExperiential Learning

Develops Systems ThinkingDevelops Systems Thinking

Remarkable Consensus for Data Solicitation Format

Q: What were key steps in your life that developed your systems thinking abilities?Q: What were key steps in your life that developed your systems thinking abilities?


26/75



Top Ranked Categories Are All Experiential Learning

Experiential LearningExperiential Learning Inside and Outside WorkInside and Outside Work


27/75



IntroductionMethods


Conclusion


Develops Systems ThinkingDevelops Systems ThinkingQ: In your experience, what enablers or barriers have you seen to theQ: In your experience, what enablers or barriers have you seen to the

development of systems thinking in engineers?development of systems thinking in engineers?

Top Node Category for Enablers is Experiential Learning


28/75



IntroductionMethods


Conclusion



3 of 4 Top Node Categories Are Experiential Learning


29/75



Blue chip interviewees also support experiential learningBlue chip interviewees also support experiential learning

When I was involved in the midWhen I was involved in the mid--60s, programs went from concept to operation in 360s, programs went from concept to operation in 3--5 years. In5 years. In

a period of15 years of experience, an engineer would work on 3a period of15 years of experience, an engineer would work on 3--5 programs. They would work5 programs. They would work

up progressively to larger and larger responsibilities.up progressively to larger and larger responsibilities. There was a whittling down process soThere was a whittling down process so

that we could pick the systems engineer. There would be 3that we could pick the systems engineer. There would be 3--5 programs with 45 programs with 4--5 segments5 segments

each, so we could pick the systems engineers for the new programs from this pool.each, so we could pick the systems engineers for the new programs from this pool. We wouldWe would

have 3 to 5 to 8 people to pick from, and we could pick the best.have 3 to 5 to 8 people to pick from, and we could pick the best.

We never had a problem with training, since this was provided by onWe never had a problem with training, since this was provided by on--thethe--job training andjob training and

experience.experience. We never thought about setting up training until the 2001 timeframe when weWe never thought about setting up training until the 2001 timeframe when we

thought about how to fix the problems in space acquisitionthought about how to fix the problems in space acquisition

The training was all onThe training was all on--thethe--job. We would have young guys work on a section of the program,job. We would have young guys work on a section of the program,

then they would move up to be in charge of a particular element, then they would work there forthen they would move up to be in charge of a particular element, then they would work there for

44--5 years, then they would move to a subsystem level, then they would move up to be5 years, then they would move to a subsystem level, then they would move up to be

responsible for a segment of the programresponsible for a segment of the program. Each time, we could pick from 5. Each time, we could pick from 5--8 engineers to8 engineers to

move up to the position at that higher level.move up to the position at that higher level.

IntroductionMethods


Conclusion




30/75



Consensus onConsensus on

EnablersEnablers

IntroductionMethods


Conclusion

Consensus on primary mechanisms that enable orobstruct systems thinking development in engineers

1. Experiential learning2. Individual characteristics3. Supportiveenvironment

Even though systemsthinking definitions diverge,

there is consensus onprimary mechanisms that

enable or obstruct systemsthinking development in

engineers


31/75



IntroductionMethods


Conclusion

Q: Are there certain individual characteristics or innate traits that seem toQ: Are there certain individual characteristics or innate traits that seem to

predict the development of systems thinking? If so, what are they?predict the development of systems thinking? If so, what are they?

Individual CharacteristicsIndividual Characteristics

Enable Systems ThinkingEnable Systems Thinking

Personality is Top Node Category for Individual Characteristics


32/75



IntroductionMethods


Conclusion

Individual CharacteristicsIndividual Characteristics

Enable Systems ThinkingEnable Systems Thinking


33/75



IntroductionMethods


Conclusion

Note: Junior SystemsNote: Junior SystemsEngineers add AnxietyEngineers add Anxiety

High in:High in:

Openness toOpenness to

IdeasIdeas

CompetenceCompetence

Low In:Low In:

SelfSelf--

ConsciousnessConsciousness

TenderTender--

MindednessMindedness

Different SampleDifferent Sample

Explanation ofOpenness to Ideas

Background onNEO PI-R

Results of NEO PIResults of NEO PI--RR

Personality TestPersonality Test1212 fromfrom

One CompanyOne Company


34/75



IntroductionMethods


Conclusion

Results correlate to findings by the Interdisciplinary Studies Project at

Project Zero, Harvard Graduate School of Education led by Howard

Gardner and Veronica Boix-Mansilla13,14

At the individual intellectual level, the paper characterizes exemplary

interdisciplinary workers as embodying a disposition toward curiosity, risk-

taking, open mindedness and humility.

Curiosity in multiple areas of knowledge was a mobilizing force for the

interdisciplinary workers in our study. Curiosity emerged implicitly in their

accounts of professional growth as well as explicitly as a driving force of

interdisciplinary work. Open-mindedness is the second trait repeatedly attributed to

interdisciplinary workers and collaborators.

Outside Study Also Emphasizes Curiosity and Open-Mindedness

Link to InterdisciplinaryLink to Interdisciplinary

Studies ProjectStudies Project


35/75


36/75



IntroductionMethods


Conclusion

3 of 5 Top Barriers Are Environmental

Q: In your experience, what enablers or barriers have you seen to theQ: In your experience, what enablers or barriers have you seen to the

development of systems thinking in engineers?development of systems thinking in engineers?

Environment AffectsEnvironment Affects

Systems ThinkingSystems Thinking


37/75


38/75



IntroductionMethods


Conclusion

Differences Are Not SignificantMost of the Time

Statistical TestsStatistical Tests Multiple statistical tests run to compare differences between groups

Comparison of all classifications

Comparison of Senior Systems Engineers to:

The Expert Panelists

The control group of Senior Technical Specialists

The control group of Junior Systems Engineers

Comparison of all companies

Comparison of two opposing companies

Results show that the differences between groups are not

significant most of the time The Senior Systems Engineers do not differ from the other

classifications for the majority of the top-ranked node categories


39/75



Need for SystemsNeed for Systems

OpportunitiesOpportunities

IntroductionMethodsResults

*Implications*

Conclusion

New United

States

Military

Aircraft

Programs byDecade and

Career

Lengths of a

Typical

Engineer(FromMurman, Walton etal. 2003, citing

Hernandez)15

Declining Opportunities for Experiential Learning


40/75




*Implications*

Conclusion

Inappropriate EmphasisInappropriate Emphasis

on Trainingon Training1616

Emphasis is on Training Not Experiential Learning


41/75




*Implications*

Conclusion

InterventionMaturityInterventionMaturity


42/75




*Implications*

Conclusion

Systems Thinking Interventions Should Be Based

on Knowledge and Include Feedback Mechanisms

InterventionMaturityInterventionMaturity


43/75




for Governmentfor Government


*Implications*

Conclusion

Applications of Research for GovernmentApplications of Research for Government1.1. INCENTIVESINCENTIVES -- Provide incentives to promote strong systemsProvide incentives to promote strong systems

thinkingthinking

2.2. POLICYPOLICY -- Adjust policies to emphasize experiential learning forAdjust policies to emphasize experiential learning for

systems thinking developmentsystems thinking development

3.3. ACQUISITION STRATEGYACQUISITION STRATEGY -- Change acquisition strategy toChange acquisition strategy to

provide more programs and opportunities for engineers toprovide more programs and opportunities for engineers to

develop systems thinkingdevelop systems thinking

4.4. RESEARCHRESEARCH -- Promote research on the mechanisms for effectivePromote research on the mechanisms for effective

systems thinking developmentsystems thinking development5.5. SYSTEMS PROGRAMSSYSTEMS PROGRAMS -- Encourage systems programs thatEncourage systems programs that

teach systems skills and systems thinkingteach systems skills and systems thinking

Set Policy Environment to Enable Systems Thinking Development


44/75




for Industryfor Industry


*Implications*

Conclusion

Applications of Research for IndustryApplications of Research for Industry1.1. INTERVENTION STRUCTUREINTERVENTION STRUCTURE -- Structure systems thinkingStructure systems thinking

interventions to emphasize experiential learninginterventions to emphasize experiential learning

2.2. FILTER AND FOSTERFILTER AND FOSTER -- Filter and foster identified individualFilter and foster identified individual

characteristics in systems organizationscharacteristics in systems organizations

3.3. SUPPORTIVE ENVIRONMENTSUPPORTIVE ENVIRONMENT -- Provide an environmentProvide an environment

supportive to the development of systems thinkingsupportive to the development of systems thinking

4.4. COMMUNICATE ASSESSMENTCOMMUNICATE ASSESSMENT -- Clearly communicate howClearly communicate how

strength of systems thinking is assessedstrength of systems thinking is assessed

5.5. SYSTEMS PROGRAMSSYSTEMS PROGRAMS -- Offer systems programs to teachOffer systems programs to teachsystems skills and systems thinkingsystems skills and systems thinking

Utilize the Primary Mechanisms That Enable Systems Thinking


45/75




for Academiafor Academia


*Implications*

Conclusion

Applications of Research for AcademiaApplications of Research for Academia1.1. SYSTEMS PROGRAMSSYSTEMS PROGRAMS -- Offer systems programs to teachOffer systems programs to teach

systems skills and systems thinkingsystems skills and systems thinking

2.2. FEEDBACKFEEDBACK -- Use feedback mechanisms to continually improveUse feedback mechanisms to continually improve

systems programs and systems coursessystems programs and systems courses

3.3. EMPHASIZE EXPERIENCEEMPHASIZE EXPERIENCE -- Structure programs and courses toStructure programs and courses to

emphasize experiential learningemphasize experiential learning

4.4. COURSE STRUCTURECOURSE STRUCTURE -- Structure courses and programs toStructure courses and programs to

promote systems thinking by emphasizing context and knowledgepromote systems thinking by emphasizing context and knowledge

integrationintegration5.5. RESEARCHRESEARCH -- Continue research on the mechanisms for effectiveContinue research on the mechanisms for effective


Continue Studying How Systems Thinking Actually Develops


46/75



Key ResearchKey Research

QuestionsQuestions1.1. What are enablers, barriers andWhat are enablers, barriers and

precursors to the development ofprecursors to the development of

systems thinking in engineers?systems thinking in engineers?2.2. How do senior systems engineersHow do senior systems engineers

develop?develop?

3.3. What are the mechanisms that developWhat are the mechanisms that developsystems thinking in engineers?systems thinking in engineers?


Implications

* Conclusion *


47/75



IntellectualIntellectual

ContributionsContributions Organized analysis of existing literature on enablers to systems

thinking development at multiple levels of analysis

Assembled extensive data set on systems thinking development

Developed a framework and conceptual illustration for reconcilingdivergent systems thinking definitions

Revealed dearth of measures for strength of systems thinking

Uncovered primary mechanisms that enable systems thinking

development in engineers

Highlighted the importance of experiential learning Discovered individual traits that enable systems thinking

Highlighted inconsistencies between policy & effective mechanisms

Identified implications for government, industry, and academia


Implications

* Conclusion *


48/75



SummarySummaryIntroduction

MethodsResults

Implications

* Conclusion *

Exploratory and inductive study

Field study with auxiliary interviews

Result: Even though systems thinking definitions diverge, there

is consensus on primary mechanisms that enable or obstructsystems thinking development in engineers Divergent systems thinking definitions reconciled with a systems thinking

framework, illustration and definition

Highlights importance of experiential learning

Development is enabled by individual characteristics such as openness to

ideas, curiosity, questioning, strong communication and interpersonal skills

A supporting environment also enables development

Implications for government, industry, and academia given

Rigorous Exploration of an Extensive Data Set to Discover

EffectiveMechanisms to Develop Systems Thinking in Engineers


49/75



Enabling RigorEnabling Rigor

in SE Researchin SE Research


Implications

* Conclusion *

Many engineers are not familiar withMany engineers are not familiar with

research methods applicable to studyingresearch methods applicable to studying

systems problemssystems problems Ideas for enhancing academic rigor inIdeas for enhancing academic rigor in

systems engineering researchsystems engineering research1.1. SE Research Methods TutorialsSE Research Methods Tutorials

2.2. SE Research Methods Task ForceSE Research Methods Task Force

3.3. SE Research CritsSE Research Crits

Ideas for Enabling SE Research Rigor


50/75



SE ResearchSE Research

M

ethods TutorialsM

ethods Tutorials


Implications

* Conclusion *

After SEANET panel, multiple inquiries about research methodsAfter SEANET panel, multiple inquiries about research methods

At SEANET/CSER, there could be a series of tutorials onAt SEANET/CSER, there could be a series of tutorials on

"Systems Engineering Research Methods""Systems Engineering Research Methods" Morning tutorial on "Quantitative Research Methods for SE Research"Morning tutorial on "Quantitative Research Methods for SE Research"

Afternoon tutorial on "Qualitative Research Methods for SE Research"Afternoon tutorial on "Qualitative Research Methods for SE Research"

Or, have "Systems Engineering Research Methods" tutorials atOr, have "Systems Engineering Research Methods" tutorials at

other INCOSE conferences and at the local chapter meetingsother INCOSE conferences and at the local chapter meetings

One cannot properly cover research methods in short sessionsOne cannot properly cover research methods in short sessions

like this, but these sessions could be beginning guidance to:like this, but these sessions could be beginning guidance to:

a)a) Make people aware that rigorous methods exist andMake people aware that rigorous methods exist andb)b) Guide people to proper followGuide people to proper follow--up resourcesup resources

Highlight Possible SE ResearchMethods Using Tutorials


51/75



SE Research MethodsSE Research Methods

Task ForceTask Force


Implications

* Conclusion *

The INCOSE Academic Forum could form a "Systems EngineeringThe INCOSE Academic Forum could form a "Systems Engineering

Research Methods Task Force"Research Methods Task Force" to work with the INCOSE Head ofto work with the INCOSE Head of

Research and Education to emphasize rigorous SE researchResearch and Education to emphasize rigorous SE research

The task force could:The task force could:

a)a) Develop a research methods section on the INCOSE website to guide studentsDevelop a research methods section on the INCOSE website to guide studentsand researchers to available books, experts, and other resourcesand researchers to available books, experts, and other resources

b)b) Coordinate the INCOSE offerings of tutorials and courses in SE researchCoordinate the INCOSE offerings of tutorials and courses in SE research

methodsmethods

c)c) Provide critical feedback to doctoral students during SEANET sessionsProvide critical feedback to doctoral students during SEANET sessions

d)d) Monitor the quality of the research methods in INCOSE papers/presentationsMonitor the quality of the research methods in INCOSE papers/presentations

e)e) Certify SE doctoral programsCertify SE doctoral programs Helps to raise the benchmark for rigor in SE researchHelps to raise the benchmark for rigor in SE research

Assists individuals with the often overwhelming process of SE researchAssists individuals with the often overwhelming process of SE research

SE ResearchMethods Task Force Would Address This Issue


52/75



SE Research CritsSE Research CritsIntroduction

MethodsResults

Implications

* Conclusion *

In architecture school, "crits" are critical design reviews whereIn architecture school, "crits" are critical design reviews where

students pinstudents pin--up their work for professors and students to provideup their work for professors and students to provide

critical feedbackcritical feedback This review process enhances the design and the endThis review process enhances the design and the end--productproduct

NonNon--traditional students might not have a peer group to work withtraditional students might not have a peer group to work with

INCOSE could sponsor a series of regional crits where doctoralINCOSE could sponsor a series of regional crits where doctoral

students present their work at different stages and willingstudents present their work at different stages and willing

volunteers throw darts at it.volunteers throw darts at it. This could possibly be done:This could possibly be done:a)a) At SEANETAt SEANET

b)b) Before or after INCOSE conferencesBefore or after INCOSE conferences

c)c) Before or after INCOSE regional meetingsBefore or after INCOSE regional meetings

d)d) By WebExBy WebEx

SE Research Crits Provide Critical Feedback For SE Research


53/75



Enabling RigorEnabling Rigor

in SE Researchin SE Research For the quality of SE research to be enhanced, it isFor the quality of SE research to be enhanced, it is

important for wellimportant for well--meaning students to have access tomeaning students to have access to

proper training, methods resources, and supportproper training, methods resources, and support

Better research yields better understanding of the fieldBetter research yields better understanding of the field

For an emerging academic field like SE, it is importantFor an emerging academic field like SE, it is important

for INCOSE to take leadership on enabling rigor in SEfor INCOSE to take leadership on enabling rigor in SE

researchresearch

Use INCOSE Leadership for Enabling SE Research Rigor


Implications

* Conclusion *


54/75



1)1) http://www.aero.orghttp://www.aero.org2)2) http://www.bmw.comhttp://www.bmw.com3)3) http://www.boeing.comhttp://www.boeing.com4)4) http://www.boozallenhamilton.comhttp://www.boozallenhamilton.com5)5) http://www.generaldynamics.comhttp://www.generaldynamics.com6)6) http://www.mitre.orghttp://www.mitre.org7)7) http://www.northgrum.comhttp://www.northgrum.com8)8) http://www.pratthttp://www.pratt--whitney.comwhitney.com9)9) http://www.sikorsky.comhttp://www.sikorsky.com10)10) QSR N6 Student MiniQSR N6 Student Mini--Manual, copyright by QSR International Pty. Ltd. Melbourne, Australia, MarchManual, copyright by QSR International Pty. Ltd. Melbourne, Australia, March

2002.2002.11)11) Maier, M. W. and E. Rechtin, The Art of Systems Architecting, CRC Press LLC, 2002.Maier, M. W. and E. Rechtin, The Art of Systems Architecting, CRC Press LLC, 2002.12)12) Costa, J., Paul T. and R. R. McCrae, NEO PICosta, J., Paul T. and R. R. McCrae, NEO PI--R Professional Manual, Revised NEO PersonalityR Professional Manual, Revised NEO Personality

Inventory (NEO PIInventory (NEO PI--R) and NEO FiveR) and NEO Five--Factor Inventory (NEOFactor Inventory (NEO--FFI), Psychological AssessmentFFI), Psychological AssessmentResources, Inc., 1992.Resources, Inc., 1992.

13)13) http://www.pz.harvard.edu/interdisciplinary/research.html, 2006.http://www.pz.harvard.edu/interdisciplinary/research.html, 2006.

14)14) Mansilla, Veronica Boix, Dan Dillon, and Kaley Middlebrooks, Building Bridges Across Disciplines:Mansilla, Veronica Boix, Dan Dillon, and Kaley Middlebrooks, Building Bridges Across Disciplines:Organizational and Individual Qualities of Exemplary Interdisciplinary Work, Interdisciplinary StudiesOrganizational and Individual Qualities of Exemplary Interdisciplinary Work, Interdisciplinary StudiesProject, Project Zero, Harvard Graduate School of Education, 2000.Project, Project Zero, Harvard Graduate School of Education, 2000.

15)15) Murman, E., M. Walton, et al. "Challenges in the Better, Faster, Cheaper Era of Aeronautical Design,Murman, E., M. Walton, et al. "Challenges in the Better, Faster, Cheaper Era of Aeronautical Design,Engineering and Manufacturing." Massachusetts Institute of Technology Engineering Systems DivisionEngineering and Manufacturing." Massachusetts Institute of Technology Engineering Systems DivisionWhite Paper, paper to appear in The Aeronautical Journal, 2003, citing HernandezWhite Paper, paper to appear in The Aeronautical Journal, 2003, citing Hernandez..

16)16) Skalamera, R. J., Implementing OSD Systems Engineering Policy, 2004.Skalamera, R. J., Implementing OSD Systems Engineering Policy, 2004.

ReferencesReferences


55/75

Thank You!Thank You!

Questions or Comments?Questions or Comments?


56/75

Additional SlidesAdditional Slides

D t C ll tiD t C ll ti


57/75



Data Collection:Data Collection:

Interview QuestionsInterview Questions

DefinitionsDefinitions

Enablers & barriersEnablers & barriers

Innate traitsInnate traits InterventionIntervention

effectivenesseffectiveness

Key stepsKey steps

StrengthsStrengths

Indicators of qualityIndicators of quality


58/75



Data Collection:Data Collection:

Survey FormsSurvey Forms DemographicsDemographics

Education & training historyEducation & training history

Work historyWork history

Processed in SPSSProcessed in SPSS


59/75



Each interview firstEach interview first

segmented bysegmented by

questionquestion

During contentDuring content

analysis, each lineanalysis, each line

was highlighted andwas highlighted and

coded in thecoded in the

appropriate nodeappropriate node

All node categoriesAll node categories

emergedemerged -- no nodeno nodecategories existedcategories existed

at the beginning ofat the beginning of

the coding processthe coding process

QSR N6QSR N688

Coding ScreensCoding Screens


60/75



Divergent DefinitionsDivergent Definitions

in Same Companyin Same CompanyExamples of divergent definitions from one systems organizationExamples of divergent definitions from one systems organization

Senior Systems Engineer1Senior Systems Engineer1 -- focused on thefocused on the decompositiondecomposition process,process,Deliberate decomposition process where you have requirements defined into functionsDeliberate decomposition process where you have requirements defined into functions

and then those functions are broken down into design; feedback loops betweenand then those functions are broken down into design; feedback loops between

requirements and functions; process of decomposing a complex problemrequirements and functions; process of decomposing a complex problem

Senior Systems Engineer 2Senior Systems Engineer 2 -- focused on thefocused on the domaindomain,, YouYou MUSTMUSTUNDERSTAND THE DOMAIN. You have to understand how the system could be usedUNDERSTAND THE DOMAIN. You have to understand how the system could be used

Senior Systems Engineer 3Senior Systems Engineer 3 -- focused on thefocused on the types of systemtypes of system

elementselements,, I used to think about it as technical. NOW, as I get into monetary items, itsI used to think about it as technical. NOW, as I get into monetary items, itsDOTMILPF people, technology, money, etc. (Note that DOTMILPF is the acronym forDOTMILPF people, technology, money, etc. (Note that DOTMILPF is the acronym for

Doctrine, Organization, Training, Material, Leadership and Education, Personnel, andDoctrine, Organization, Training, Material, Leadership and Education, Personnel, and

Facilities.)Facilities.) Senior Systems Engineer 4Senior Systems Engineer 4 -- focused onfocused on interactionsinteractions,, Similar to howSimilar to how

Ackoff does, look at the system function in the larger system, that sets the context, theAckoff does, look at the system function in the larger system, that sets the context, the

larger system. Then, disaggregate that bigger system, which is not the same aslarger system. Then, disaggregate that bigger system, which is not the same as

decomposition. Look at it in the INTERACTIONS, not the internal functions, look at it indecomposition. Look at it in the INTERACTIONS, not the internal functions, look at it in

the context of the higher level system.the context of the higher level system.


61/75



Validated NEO PIValidated NEO PI--RR

Personality InventoryPersonality Inventory Revised NEO Personality InventoryRevised NEO Personality Inventory

(NEO PI(NEO PI--R) designed by PaulR) designed by Paul

Costa, Jr. & Robert McCraeCosta, Jr. & Robert McCrae

Inventory is based on the fiveInventory is based on the five--factorfactor

model of personalitymodel of personality

Measures the five major dimensionsMeasures the five major dimensions

of personality, along with sixof personality, along with six

important traits or facets of eachimportant traits or facets of each

Validated in the personality literatureValidated in the personality literature99

The NEO PIThe NEO PI--R embodies a conceptual model thatR embodies a conceptual model that

distills decades of factor analytic research on thedistills decades of factor analytic research on thestructure of personality. The scales themselves werestructure of personality. The scales themselves weredeveloped and refined by a combination of rationaldeveloped and refined by a combination of rationaland factor analytic methods and have been theand factor analytic methods and have been thesubject of intensive research conducted for15 yearssubject of intensive research conducted for15 yearson both clinical and normal adult sampleson both clinical and normal adult samples

DomainsN:NeuroticismE:ExtraversionO: OpennessA:AgreeablenessC: Conscientiousness

Neuroticism FacetsN1:AnxietyN2:Angry HostilityN3:DepressionN4: Self-ConsciousnessN5: ImpulsivenessN6: Vulnerability

Extraversion facets

E1: WarmthE2:GregariousnessE3:AssertivenessE4:ActivityE5:Excitement-SeekingE6: Positive Emotions

Openness facetsO1: FantasyO2:AestheticsO3: FeelingsO4:ActionsO5: IdeasO6: Values

Agreeableness facetsA1: TrustA2: StraightforwardnessA3:AltruismA4: Compliance

A5: ModestyA6: Tender-Mindedness

Conscientiousness facetsC1: CompetenceC2: OrderC3:DutifulnessC4:Achievement StrivingC5: SelfDiscipline

C6:Deliberation


62/75



Personality Facet ofPersonality Facet of

Openness to IdeasOpenness to Ideas The personality test authors stateThe personality test authors state99,, a high scorer on the Ideasa high scorer on the Ideas

facet enjoys rich, varied, and novel experiences in his or herfacet enjoys rich, varied, and novel experiences in his or her

intellectual life.intellectual life.

They go on to say that:They go on to say that:Intellectual curiosity is an aspect of Openness that has long beenIntellectual curiosity is an aspect of Openness that has long beenrecognized (Fiske, 1949).recognized (Fiske, 1949). This trait is seen not only in an active pursuitThis trait is seen not only in an active pursuit

of intellectual interests for their own sake, but also in openof intellectual interests for their own sake, but also in open--mindednessmindedness

and a willingness to consider new, perhaps unconventional ideas.and a willingness to consider new, perhaps unconventional ideas. HighHigh

scorers enjoy both philosophical arguments and brainscorers enjoy both philosophical arguments and brain--teasers.teasers.

Openness to ideas does not necessarily imply high intelligence,Openness to ideas does not necessarily imply high intelligence,although it can contribute to the development of intellectual potential.although it can contribute to the development of intellectual potential.

Low scorers on the scale have limited curiosity and, if highly intelligent,Low scorers on the scale have limited curiosity and, if highly intelligent,

narrowly focus their resources on limited topics.narrowly focus their resources on limited topics.


63/75





vs. Expert Panelistsvs. Expert Panelists


64/75




vs. Senior Technical Specialistsvs. Senior Technical Specialists



65/75




vs. Junior Systems Engineersvs. Junior Systems Engineers



66/75



Comparison ofComparison of

All ClassificationsAll Classifications



67/75



Comparison ofComparison of

All CompaniesAll Companies



68/75



Two companiesTwo companies

with opposingwith opposing

systemssystems

contextscontexts

comparedcompared One companyOne company

is a productis a product--

centric systemcentric system

Other companyOther company

has an interesthas an interest

in systemin system--ofof--systems issuessystems issues

Comparison of TwoComparison of Two

Opposing CompaniesOpposing Companies



69/75

Development of Systems ThinkingDevelopment of Systems Thinking

in Systems Engineers: The Role ofin Systems Engineers: The Role of

Experience in LearningExperience in Learning

INCOSE SymposiumINCOSE Symposium

July 10, 2006July 10, 2006

Heidi L. DavidzHeidi L. Davidz


70/75



AgendaAgenda

IntroductionIntroduction

Research MethodsResearch Methods

ResultsResults


ConclusionConclusion Conclusions on My ResearchConclusions on My Research

Reflections on Enabling SE Research RigorReflections on Enabling SE Research Rigor


ImplicationsConclusion


71/75



ResearchResearch

MethodsMethods

Introduction

* Methods *Results


Literaturereview

AdditionalInterviews

with BlueChip ProvenExperts (N=2)Data Analysis

Using QSR N6,SPSS, MS Excel

Field Study of10 Companies

and 205Subjects

Using

Interviewsand Surveys

PilotInterviews

(N=12)

Inductive

Exploratory

TheorySynthesis


72/75



Field StudyField Study

(a) ContactedCompany

10 companiesparticipated

Primarily U.S.aerospacecompanies

Introduction

* Methods *Results


(b) Point-of-contact

worked withothers toidentifyExpertPanelists

(c) ExpertPanelists(N=37)

Completedsurvey andinterview

Identifiedsubjects forthree follow-ongroups

(d) Follow-On Subjects Completed interview Completed survey

1. SeniorSystemsEngineers(N=62)

2. Senior

TechnicalSpecialists(N=53)

3. JuniorSystemsEngineers(N=53)

(Total of 205 interviews and 188 surveys)


73/75




for Governmentfor Government


*Implications*

Conclusion

Applications of Research for GovernmentApplications of Research for Government1.1. Provide incentives to promote strong systems thinkingProvide incentives to promote strong systems thinking

2.2. Adjust policies to emphasize experiential learning forAdjust policies to emphasize experiential learning for

systems thinking developmentsystems thinking development3.3. Change acquisition strategy to provide more programs andChange acquisition strategy to provide more programs and

opportunities for engineers to develop systems thinkingopportunities for engineers to develop systems thinking

4.4. Promote research on the mechanisms for effectivePromote research on the mechanisms for effective


5.5. Encourage systems programs that teach systems skillsEncourage systems programs that teach systems skillsand systems thinkingand systems thinking

Set Policy Environment to Enable Systems Thinking Development


74/75




for Industryfor Industry


*Implications*

Conclusion

Applications of Research for IndustryApplications of Research for Industry1.1. Structure systems thinking interventions to emphasizeStructure systems thinking interventions to emphasize

experiential learningexperiential learning

2.2. Filter and foster identified individual characteristics inFilter and foster identified individual characteristics insystems organizationssystems organizations

3.3. Provide an environment supportive to the development ofProvide an environment supportive to the development of

systems thinkingsystems thinking

4.4. Clearly communicate how strength of systems thinking isClearly communicate how strength of systems thinking is

assessedassessed5.5. Offer systems programs to teach systems skills andOffer systems programs to teach systems skills and


Utilize the Primary Mechanisms That Enable Systems Thinking


75/75



for Academiafor Academia


*Implications*

Conclusion

Applications of Research for AcademiaApplications of Research for Academia1.1. Offer systems programs to teach systems skills andOffer systems programs to teach systems skills and


2.2. Use feedback mechanisms to continually improve systemsUse feedback mechanisms to continually improve systemsprograms and systems coursesprograms and systems courses

3.3. Structure programs and courses to emphasize experientialStructure programs and courses to emphasize experiential

learninglearning

4.4. Structure courses and programs to promote systemsStructure courses and programs to promote systems

thinking by emphasizing context and knowledge integrationthinking by emphasizing context and knowledge integration5.5. Continue research on the mechanisms for effective systemsContinue research on the mechanisms for effective systems

thinking developmentthinking development

Continue Studying How Systems Thinking Actually Develops

Presentation from September 12, 2006 Dinner Meeting

Documents

Transcript of Presentation from September 12, 2006 Dinner Meeting