Raytheon (Powell) - Behaviour Trees

8/12/2019 Raytheon (Powell) - Behaviour Trees

1/44

Dan Powell24 August 2011

Behaviour

TreesGrowing Understanding,

Taming Complexity,

Minimising Risk

Copyright 2011Raytheon Company. All rights reserved.Customer Success Is Our Mission is a registered trademark of Raytheon Company.


2/44

Page

Topics in this Presentation

The Case for Behaviour Engineering

What is Behaviour Engineering?

The Behaviour Engineering ProcessThe Behaviour Modelling Language

Outcomes so far

Applications of Behaviour Engineering


3/44

Page

The Case for Behaviour EngineeringThe Case for Behaviour Engineering

What is Behaviour Engineering?


Outcomes so far



4/44

Page

Large Projects/ProgramsGreatest Risks

Cost Overruns

Schedule Overruns

Ending up with a system that does not meet the needs andexpectations of stakeholders.

Ending up with a system that requires a lot of maintenance(defect fixing)

Project failure or abandonment


5/44

Page

Early Life Cycle the Biggest Issue

Upto6

0%ofalld

efects

inasys

tems

lifetime

origina

tefromde

ficient

requ

irement

s

Reworkingrequirementscostsbetween

40%and80%oftotalprojectcosts


6/44Page

The Problem

Critical Defect

Implied/Assumed KnowledgeOK

Industry average specification defectiveness metrics (from analysis of 15000+ requirementsfrom mature sectors of the industry on large specifications used for tender or development)

Vocabulary - Inconsistent, undefined

Missing conditional cases, events

Unstated preconditions for system Behaviour

Multiple possible interpretations

Inconsistent requirements

Over-specification

Requirements not independently verifiable

Interpretation dependent on subject matter expertise


7/44Page

The Communication Problem

Have we got this right?

Can this be built and how longwill it take / how much will itcost?

Can we verify and validate this

when it is built?

About 46% of the pieces are missing, dont fit, belong to a different puzzle or require significant interpretation

(glue and scissors) to fit into the puzzle

Existing system and software engineering methods dont let us find these problems early enough - they donteven provide a way to assemble the puzzle!

That is, existing methods dont assist us in understanding the huge volume of information presented.


8/44Page

Critical Success Factor

Achieved by:

Identification/removal of ambiguity,incompleteness and inconsistency inrequirements

Provision of human readable modelenabling understanding (and hencevalidation) of the integrated set of allrequirements.

Customer


9/44Page

The Case for Behaviour Engineering

What is Behaviour Engineering?What is Behaviour Engineering?


Outcomes so far



10/44Page

Customer

What is Behaviour Tree Analysis

A Discipline (Behaviour Engineering)

An Associated Modeling Language (BML)

Translate

Integrate

Resolve

Refine


11/44Page

Behaviour Engineering

Used in industry to:

provide early life-cycle evaluation and correction of specifications; enable shared understanding of a specification;

explore concepts and architectures;

analyse models (FMECA, Reliability Analysis, ...); incrementally grow and validate specifications/scenarios/stories/use

cases;

incrementally refine, implement and test models.


12/44Page

Behaviour Tree Analysis

Inherently Manages Complexity

Improves Project Outcomes

Reduces Risk

Customer


13/44Page

The Behaviour Engineering Processfor Evaluating Requirements


14/44Page

Step 1- Requirements Translation

Translate each requirement 1 at a time to a Behaviour tree.

Provides a means to manage complexity by localising complexity -can translate one requirement at a time Translation of natural language to a formal language (while trying to

maintain as much original language as possible)

Formal languages do not tolerate ambiguity Need to use common vocabulary during translation - produce anintegrated composition tree at same time as translation - aids inunderstanding


15/44Page

Formalise One Requirement at a TimeRetain vocabulary of requirement

Step 1- Requirements Translation


16/44Page

Order is not important BUT position where placed is Information about f is spread across THREE pieces Only see there are missing pieces when integrate pieces

Only see some pieces dont fit when integrate pieces

Step 2- Integration


17/44Page

Integration EnablerThe Precondition Axiom

Each and every functionalrequirement expressed as aBehaviour tree, BTi, has associated

with it a precondition, Pi, that needsto be satisfied in order for theBehaviour encapsulated within it tobe exhibitable


18/44Page

Why Integrate

When we can analyse the whole, emergent properties becomeapparent - we can reason about them.

Reviews/inspections of large specifications are ineffective at findingdefects that can only be found by understanding the wholespecification - the requirement on page 3 that is inconsistent withthe requirement on page 253; the business requirement that should

constrain the user story that was pasted on the wall 3 weeks ago.

Again, this is due to the limitations of human short-term memory

The tool supported integration of, and analysis of integratedBehaviour Engineering models facil itates finding these types ofissues.


19/44Page

Example Behaviour Tree


20/44

Page

Why BE Works

What differentiates Behaviour Trees from any other modellinglanguage


21/44

Page

Why BE Works - Representation

7

+ 5

-----

This simple sum is easy to do in our head.For simple problems representation maters little.

This is harder to do in our head (for most of us). This is due to thelimitations of human short-term memory.

However, on paper, this is a trivial (although lengthy) sum - therepresentation allows us to localise complexity. We simply repeat aunit level operation that does not tax short term memory.

15476735386

+ 37932914987

+ 79109026544

+ 33987110011

+ 44488730928

-------------------------The Behaviour Modelling Language has this property

Contrast this to a representation that does not permit unit leveloperation

MMDCCLI

+ CCCLXIX

------------------


22/44

Page

Why BE Works - Representation

The Behaviour Modelling Language supports the use, managementand resolution of the language used by the stakeholders (it even

supports managing aliases where different stakeholders usedifferent terms)

This helps to preserve intent

This enables validation by model walkthrough

The Behaviour Modelling Language is not large but is veryexpressive


23/44

Page

Behaviour Modelling Language (BML)A Lesson from Chemistry

Ethane MoleculeRepresentations


24/44

Page

Why BE Works - BE is Scalable

The basic unit-level operations of requirement translation andrequirement integration scale (900+ requirements, 200+ pages)


25/44

Page

The Case for Behaviour Engineering What is Behaviour Engineering?


Outcomes so farOutcomes so far



26/44

ROI on a


27/44

Page

ROI on aLarge SW Development Project(cost data from an industry pro ject (non-Raytheon) applied to ave. size and defectiveness metrics)

Estimated Cost per Requirements Defect = $3K For defects not detected in traditional reviews and found at System Test

Based on the organisations own and accessible industry data

Number of requirements defects detected by BTA on Requirements = 307 These were major requirements defects found afterclose out of traditional review process That is, these issues would have been missed at the requirements stage

Cost of fixing Requirement Defects from BTA = $300 / requirement

Note from this data the cost of defect at System Test is 10 times the cost at Requirements

Some studies report 60-80 times!!!

Payback from BTA on Average Project = ($3000 - $300) x 307 = $828,900

Cost of Behaviour Engineering on Average Project = $30,000

ROI = ((828,900 - 30,000) / 30,000)*100 = 2663%


28/44

Page

Results - Behaviour Engineering

Defect Detection Rate

2 to 3 times higher than the defect detection rates reported for scenario-based and checklist-based inspectionaccording to the most often cited data


29/44

Page

CommunicationEngineering PM

Engineering Users (validation of requirements)

Engineering (SYS) other Engineering DisciplinesProject Customer

Across Integrated Product Teams

Common vocabulary through Composition Trees Real increase in understanding across the project

Relatively small investment for the return

Risk reduced to Project, Customer and Stakeholders

BE Other BenefitsImprovement in Behaviour?


30/44

Page

Validations with the User / Capability.Cannot have requirements that are vague, unconnected etc

beyond Systems Requirements Review defects can be

contained early in the development life cycle Improve Estimating (System Engineering Test andAcceptance)

Changes (new / changed requirements or cannot achievespecification) fully analysed to determine the impact on theSystem

Non-domain experts can detect material errors inrequirements

low cost, high rate of detection

BE Other BenefitsImprovement in Behaviour?


31/44

Page

The Case for Behaviour Engineering What is Behaviour Engineering?

The Behaviour Engineering Process

The Behaviour Modelling Language

Outcomes so far

Applications of Behaviour EngineeringApplications of Behaviour Engineering


32/44

Page

Behaviour Engineering Centric Life Cycle


33/44

Page

Representing Architecture Artefacts


34/44

Page

Supporting FMEAProduce Models from Specifications / Design Descriptions


35/44

Page

Supporting FMEAIntroduce Failure Into Models

Introduce Failure:Engine 1 Lost


36/44

Page

Supporting FMEAAnalyse Consequences of Introduced Fai lure

Loss of Engine 1:Hydraulic system, Yellow and Blue Distribution

lines still providing pressure to hydraulic systemdirectly, If PTU operating, Green Distribution Lineproviding pressure through PTU from Yellow EDP


37/44

Page


Loss of Engine 2:Hydraulic system, Green and Blue Distribution

lines still providing pressure to hydraulic systemdirectly, If PTU operating, Yellow Distribution Lineproviding pressure through PTU from Green EDP


38/44

Page


Loss of Engine 1 and 2:Green and Yellow Distribution lines not providing

pressure to hydraulic systemPTU wont be operating

Blue Channel has pressure driven by RAT if flyingand doing > 100 kts


39/44

Page


3 Independent Failures Example:Engine 1 Lost & PTU Failed & Blue Dist Line Cut

Hydraulic system able to control surfaces throughYellow Channel


40/44

Page

Using Behaviour Engineering to Understand the Problem Space

Growing Requirements

Before

After

Initialdescrip

tionofp

roblem

Descrip

tionofPro

blemafte

r

queries

andiss

uesresolve

d.

Resolving Requirements Defects


41/44

Page

Resolving Requirements DefectsCurrent Situation

Requirements Quality

Initial BTA ReviewRequirements Quality

Subsequent BTA Review

Have we hit a wall with our current methods?

Resolving Requirements Defects


42/44

Page

Resolving Requirements DefectsBESituation

Requirements Quality

Initial BTA ReviewRequirements Quality

Subsequent BTA Review

Use BT to Resolve

Summary


43/44

Page

SummaryBehaviour Engineering Benefits

Higher Yield than current early lifecycle defect detection...

...at 2-3 times defect detection rate.

Systematic and Scalable

Results in a model that facilitates shared understanding...

...and understanding of emergent properties

Results in a model that supports validationResults in a model that supports further engineering/development

effort

IS A KEY RISK MITIGATION STRATEGY

Significant saving throughout the development life cycle


44/44

You cant do things differently,

Until you see things differently.Unknown

Raytheon (Powell) - Behaviour Trees

Documents

Transcript of Raytheon (Powell) - Behaviour Trees