Error-Proofing of the Product Development Process

MML/Stanford University Seminar Series at ICAPError-Proofing the Product Development Process

Larry Chao, October 10, 2001Manufacturing Modeling Laboratory 1

Manufacturing Modeling LabStanford University

ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

1

Manufacturing Modeling LaboratoryStanford University

Error-Proofing of theProduct Development Process

MML/Stanford University Seminar Series at ICAPOctober 10, 2001

Larry ChaoResearch Assistant / Ph.D. Candidate

[email protected]

Professor Kos IshiiDepartment of Mechanical Engineering

Design DivisionStanford, CA 94305-4022http://mml.stanford.edu


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

2

A few words about Larry...

• School– MIT, BS in ME (1998)– Stanford University, MS in ME (1999)– Currently pursuing PhD in dfM at Stanford

• Work– GE Aircraft Engines (Cincinnati, OH)– MITI’s Mechanical Engineering Laboratory

(Tsukuba, Japan)– U.S. Department of Transportation (Boston, MA)– General Motors (Warren, MI)

• Fun– tennis, movies, travel




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

3

Outline

1) Motivation2) Background

• common design process errors• international industry survey results

3) Current Techniques• tools and techniques currently used in industry to remedy

errors

4) Proposed Research Roadmap• Prediction: design process FMEA• Prevention: design process error-proofing

5) Conclusions

MotivationBackgroundTechniques

ProposalConclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

4

“Design Error Benchmarking”MotivationBackgroundTechniques

ProposalConclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

5

Sources of Quality LossMotivationBackgroundTechniques

ProposalConclusions

Production5%

Manufacture10%

Design process72%

Part design1%

Die design9%

Subcontracted design3%

(Japanese manufacturing equipment company 2000)


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

6

Objectives

• Evaluate error and error managementtechniques and tools in the design process? Gather and analyze common error modes in the

design process? Develop design strategies and tools to predict

potential errors and problems in tasks during thedesign phase of a project

? Determine error prevention strategies andmethods for the design phase and suggest changesto the process to incorporate them


ProposalConclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

7

Survey Method

• Reviewed detailed reports of errors at anairline engine manufacturer– TOPS-8D reports

• incidents reported to the Federal AviationAdministration (FAA)

• Surveyed companies with a two-pagequestionnaire on the design process– includes general questions on common errors and

managing error in the design process– survey on the design practices at the organization– interviewed design engineers and managers

MotivationBackground

TechniquesProposal

Conclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

8

TOPS-8D example:turbine blade shroud cracks

• Process breakdowns:– inaccurate treatment of heat transfer

(analysis techniques predicted lowerrunning temperatures)

– operating environment was notconsistent with pre-test predictions(resulted in inadequate materialselection)

– inadequate or incompleteobservations and documentation ofthe post test condition of hardware(resulted in inadequate assessment ofthe capability of the component)

• Corrective Actions:– lessons learned

incorporated intodesign best practices


TechniquesProposal

Conclusions

Problems could be traced to deficiencies in the design process.




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

9

0

2

4

6

8

10

12

14

16

18

incomplete inadequate omission misperformed incorrect unclear0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

What types of errors are there?

Incomplete: task specifiedbut not performedcomprehensively

Omission: tasknot performed

Inadequate: tasknot specified

comprehensively

Incorrect:task as specifiedresults in errors

Misperformed:task performed

incorrectly


TechniquesProposal

Conclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

10

Causes of errors42%

33% 33% 33%

25% 25%

17% 17% 17% 17%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

sched

ule pr

essure

sov

ersigh

t

lack o

f testi

ng

chan

ging r

equir

emen

ts

lack o

f stru

cture

miscom

munica

tions

lack o

f proto

typing

incom

plete C

R

depe

nden

ce on

expe

rienc

e

dimen

sion e

rror

Cause of error

Per

cen

tag

e o

f re

spo

nd

ents

ob

serv

ing


TechniquesProposal

Conclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

11

Response Averages

(1.5)

(1.0)

(0.5)

0.0

0.5

1.0

1.5

2.0

2.5

A structured designprocess strategy is in

place.

Your organizationdocuments previous

designs andprocesses and

makes themaccessible to design

teams.

Your organizationuses guidelines,

checklists, ordocumentation of

best practices

Your design teamsexplicitly record

observations andassumptions

throughout the designprocess.

There are regularlyscheduled design

reviews.

The design managersuse performance

metrics to evaluatethe design process.

Average

stronglyagree

disagree

agree

neither agree nor disagree


TechniquesProposal

Conclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

12

Results

• Range of scores: -1.0 to 2.0– No negative scores (“disagree”/”strongly disagree”) for

“design review”

• Mode:– 2 for structure, documentation, design review– 1 for checklists, observations, metrics

• Average standard deviation of ~0.9– highest variation for “structure”– lowest variation for “design review”


ProposalConclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

13

0

5

10

15

20

25

30

35

guidelines review documentation0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Traditional error remediesMotivation

BackgroundTechniques

ProposalConclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

14

Benchmark of design toolsMotivationBackgroundTechniques

ProposalConclusions

Design reviews are “like 100% inspection with aso-so gage.”




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

15

MotivationBackgroundTechniquesProposal

ConclusionsResearch Opportunity

• Adapt and develop failure modes and effectsanalysis (FMEA) for the tasks of the designprocess to predict errors that may commonlyoccur at an organization

• Establish error-proofing for the designprocess and develop specific poka-yokeexamples as well as develop additionaltechniques to prevent design errors


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

16

Definition: FMEA

• failure modes and effects analysisengineering technique used to define,identify, and eliminate known and/orpotential failures, problems, and errorsfrom the system, design, or process beforethey reach the customer

(Stamatis 1995)

Function or Requirement

Potential Failure Modes

Potential Causes of Failure

Occ

urre

nce

Local EffectsEnd Effects on Product, User, Other Systems S

ever

ity Detection Method/ Current Controls

Det

ectio

n RPN

Actions Recommended to Reduce RPN

Responsibility and Target Completion

Date


ProposalConclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

17

Types of FMEA

Type of FMEA How it works/what it does

System FMEA Use VOC's to assign risks to thefailure of a system function

Design FMEA Looks at components of thesystem

Assembly ProcessFMEA

Looks at impact of failures of themanufacturing and assemblyprocess on the final system

Human Error FMEA Narrows process FMEA to lookat human mistakes and omissionsin manufacturing


ProposalConclusions

Current FMEA’s are focused more onmanufacturing and operation errors


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

18

• Design has longer process interval (weeks toyears) versus manufacturing (hours or days).

? Must analyze design process in general ratherthan specific product or process.

• Greater variation from one developmentproject to the next.

? Harder to foresee all problems that may occur.

• Different value system where “creativefreedom” is emphasized.

? Engineers often don’t want to be managed.

Why is process FMEA for designharder than for manufacturing?


Conclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

19

Design Process FMEA

• Similar to Assembly Process FMEA– question-based analysis– quicker analysis

• Analyze and improve the organization’s design ordevelopment process ratherthan a specific product

? Process can be continuously improved to optimizeperformance for many products

• Decompose problem into design tasks instead ofsubassemblies


Conclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

20

QFD Matrix: Phase 1

• Quality Function Deployment:a disciplined approach

• Customer Requirements vs.Engineering Metrics:“9” Strongly Correlated“3” Correlated“1” Somewhat Correlated

“0” Not Correlated

BrightnessWeightGirth + width +Time/Tasks required to start Distortion - -Distance from presenter + + -- Time to insert/pull-out slide + +Attractive product

Preferredup dwn dwn dwndwn dwn dwn nomEngineering Metrics

Customer Requirements Cus

tom

er W

eigh

ts

Brig

htne

ss

Wei

ght

Girt

h +

wid

th

Tim

e/Ta

sks r

equi

red

to st

art

Dis

torti

on

Dis

tanc

e fr

om p

rese

nter

Tim

e to

inse

rt/pu

ll-ou

t slid

e

Attr

activ

e pro

duct

Good image 9 9 9Easy to transport 9 9 9Device sets up quickly 9 3 1 9 9 3Works well for short present. 9 1 1 9 3 3Keeps present. flowing 1 9 3 9Image visible in bad conditions 3 9 3Minimizes unplanned interruptions 1 3 1 9Design makes the product attractive 3 3 3 9

Technical Targets

lum

ens

< 6

poun

ds

< 22

inch

es

< 5

seco

nds

VT

TF

< 3

feet

< 1

seco

nd

N/A

Raw score 108

126

108

174

90 112

72 27

Relative Weight 13%

15%

13%

21%

11%

14%

9% 3%


ProposalConclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

21

Quantifying Design ProcessFMEA

EngineeringMetrics

Design ProcessTasks

• Perform a QFD to determinecustomer requirements toengineering metricsrelationship

• Determine the engineeringmetrics affected in eachdesign process task

• Use the relative weightsdetermined for each EM inQFD I to rank “importance”

EM

#1

EM

#2

EM

#3

Task #1 1 9Task #2 9 3Task #3 9


Conclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

22

Parts vs. Engineering Metrics

• Good correlation overall of normalizedimportance scores between using parts andengineering metrics– Correlation coefficient of 0.665– More than half of the tasks have a difference in

score of less than 1

• Using parts emphasizes the importance oftasks involving areas like industrial design,layout, or assembly/production


Conclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

23

• Design of a force sensor comprised of acantilever beam and a strain gage

Example: Force SensorMotivationBackgroundTechniquesProposal

Conclusions

L±?Lb±?b

h±?h

Material: AluminumE = 1.25x10 7 psi

StrainGauge


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

24

Design task Potential Failure Modes

Occ

urre

nce

Sev

erity

Det

ectio

n

RPN

Calculate cost function Arithmetic error 6 9 1 54Calculate cost function Calculus error 4 9 1 36Calculate cost function Didn't use calculus 2 9 1 18Calculate cost function Excel solver is slightly off 1 9 1 9Calculate cost function Forget to recalculate 1 9 1 9Calculate cost function Forget to specify answer 1 9 1 9Calculate cost function Mistake cost function for 1 9 1 9

Force Sensor Design:QFD+FMEA


Conclusions

Engineering Metrics Phas

e I R

elat

ive

Wei

ghts

De

term

ine

be

am

re

qu

ire

me

nts

Sel

ect m

ater

ial

Se

lect

no

min

al d

ime

nsi

on

s

De

term

ine

ma

nu

fact

uri

ng

tole

ran

ces

Cal

cula

te c

ost f

unct

ion

De

term

ine

op

tim

al d

ime

nsi

on

s

Rev

iew

Sel

ect s

trai

n ga

uge

Cal

cula

te p

rodu

ctio

n co

st

Sys

tem

Out

put

Height 6 % 1 9 1 3 9 9 1 1

Width 6 % 1 9 1 3 9 9 1

Length 6 % 1 9 1 3 9 9 1Yield Strength 17% 9 9 1

Stiffness 8 % 3 9 1 1 9 3 1

Weight 4 % 1 9 1 3

Range 6 % 1 3 3 9 1 3

Resolution 14% 3 3 3 1 9Drift 20% 3 1 3

Error 13% 9 9 3 3 9

5 Calculate cost function 96 Determine optimal dimensions 8.4431148 Select strain gauge 8.2814372 Select material 6.6287437 Review 5.559881 Determine beam requirements 5.1736534 Determine manufacturing tolerances 5.0209583 Select nominal dimensions 4.5628749 Calculate production cost 0.754491

Preferred dwn dwn dwn up up dwn dwn dwnEngineering Metrics

Customer Requirements Cus

tom

er W

eigh

ts

Hei

ght

Wid

th

Leng

th

Yie

ld S

treng

th

Stif

fnes

s

Wei

ght

Mea

sure

men

t err

or

Dev

iatio

n of

mea

sure

men

ts

Compact size 1 9 9 9Durable 9 1 1 1 9 3Portable 1 3 3 3 9Accuracy 9 1 1 1 3 3 9Precision 9 1 1 1 1 3 9Can measure a sufficient range 1 1 1 1 9 9 1

Technical Targets

met

ers

met

ers

met

ers

Pasc

als

Pasc

als

kilo

gram

s

Pasc

als

Pasc

als

Raw score 40 40 40 126

81 9 90 82

Relative Weight 8

%

8%

8%

25

%

16

%

2%

18

%

16

%




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

25

Definition: Error-Proofing

• error-proofingtechnique for eliminating errors such that itis impossible to make mistakes

? Shigeo Shingo started the concept in Japan -poka-yoke where “poka” means an inadvertentmistake and “yoke” means to prevent.

? Many poka-yoke devices are used formanufacturing and operation.


ProposalConclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

26

Error-Proofing Strategies

• Eliminate the chance of making the mistake• Provide automatic feedback to sense and fix

the error• Make incorrect actions correct• Make wrong actions more difficult• Make it easier to discover the errors that occur• Make it possible to reverse actions - to “undo”

them - or make it harder to do what cannot bereversed


ProposalConclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

27

preventionmakes it impossible tomake a mistake at all

Categories of poka-yokedevices

detectionsignals the user when amistake has been madeso that the user canquickly correct theproblem

teh

the


ProposalConclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

28

Some Error-Proofing ResourcesMotivationBackgroundTechniques

ProposalConclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

29

Approach for Error-Proofingthe Design Process

• Start with categorizing design process errors• Find analogies in manufacturing/assembly poka-

yoke• Active prevention rather than rely on detection

– Prevent mistakes in communication and performance ofanalysis, verification

• Try to build into design process rather thanadding “patches”

Development of design process poka-yoke is moredifficult due to the lack of a known desired outcome


Conclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

30

“Five Best Poka-yoke”for manufacturing

1. Guide pins of

of different sizes

2. Error detection

and alarms

3. Limit switches

4. Counters

5. Checklists

(Nikkan Kogyo Shimbun 1987)

0 5 3 4 1


Conclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

31

3. Limit switches

“Design Process Poka-yoke”

1. Guide pins of

of different sizes

2. Error detection

and alarms

4. Counters

5. Checklists

0 5 3 4 1

1. Uniform design environment

(units, software, language/terminology)

2. Design reviews

5. Design process templates and guidelines

4. Receipts, checksums, regression testing

3. Double check against

specifications, experience, intuition


Conclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

32

EP Web PagesMotivationBackgroundTechniquesProposal

Conclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

33

Causes of Mistakes

• 1. Mental– Memory– Decision– Distraction

• 2. Perception– Misunderstand– Misread– Misidentify

• 3. Communication– Ambiguous– Incorrect– Incomplete

• 4. Speed/skill– Inexperience– Inadequate training– Inadequate skill– Too fast a pace– Lack of standards

• 5. Coordination– Incomplete motion– Adjustment error

• 6. Intentional– Shortcut– Sabotage– Crime

(Hinckley 2001)


Conclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

34

Error Commonality Index

• Determine the accountabilityof each of the 19 causes foran error on a 0-9 scale tocalculate each score si

• Determine the errorcommonality index (ECI) byfinding the averagedifference for two errors foreach of the 19 causes

199

919

1

2,1,??

??

? c

cc ss

ECI

)10( ?? CI


Conclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

35

Error Commonality Index

• “Index” search– characterize errors and

error-proofs byfundamental attributessuch as

• memory

• training

– facilitates intelligent andflexible searching

Mis

sing

par

ts (

mfg

.)

Mis

sing

inf

orm

atio

n (d

esig

n)

Com

mon

alit

y

1. Mental Errors Memory 9 3 0.33333Decision 1Distraction 3 0.66667

2. Perception Errors Misunderstand 1Misread 1Misidentify 1

3. Communication Ambiguous 3 3 1Incorrect 3 0.66667Incomplete 9 9 1

4. Lack of speed/skill Inexperience 1Inadequate training 1Inadequate skill 1Too fast a pace 3 0.66667Lack of standards 1

5. Coordination Errors Incomplete Motion 1Adjustment error 1

6. Intentional Errors Shortcut 3 0.66667Sabotage 3 0.66667Crime 3 0.66667

Commonality Index: 0.86


Conclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

36

Mapping the Matrices

[ CW x CR ]x[ CR x EM ]x[ EM x Task ]=[ CW x Task ]

[ Error x Attributes ]x[ Attributes x Task ]=[ Error x Task ]

• Occurrence - task and error attributes– map type of error with type of task

• Severity - task importance– use QFD results to determine important customer

requirements


Conclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

37

Design Structure Matrix(DSM)

• a square matrix which mapsout the information linksamong individual design tasks

• a systematic mapping that iseasy to read

• offers compactness inrepresentation

• can be used to analyzeprecedence relationshipsamong various design tasks

(Mori 1999)

Test

Com

pone

nts

Dev

elop

Ret

urn

Pro

duct

Log

istic

sS

peci

fy P

arts

Dis

asse

mbl

e S

yste

mS

peci

fy S

ubsy

stem

sE

stim

ate

Ser

vice

Par

t Qua

ntiti

esD

eliv

er C

ompo

nent

sR

ecyc

le P

arts

Cre

ate

Ser

vice

Pla

nP

rocu

re C

ompo

nent

sC

reat

e W

arra

nty

Pla

nS

elec

t Con

cept

sId

entif

y S

ervi

ce P

rovi

ders

Spe

cify

Pro

duct

ion

Pla

nA

ssem

ble

Com

pone

nts

Bra

in S

torm

Gen

erat

e P

rodu

ct R

etire

men

t Pla

n

Test Components \ 9 3 9 3 1

Develop Return Product Logistics \ 3 1 3 9 9 1 9 9 9 3 9

Specify Parts 1 1 \ 9 3 3 9 3 1

Disassemble System \ 1 1 9 9

Specify Subsystems 1 1 3 \ 3 3 9 3 1

Estimate Service Part Quantities1 3 \ 1 9 1 9 9 3

Deliver Components 3 1 1 \ 3 9 3 1 3 9 1

Recycle Parts 9 9 \ 1 1 9

Create Service Plan 9 3 3 3 1 \ 1 9 3 3 1 3

Procure Components 3 3 9 3 3 \ 3

Create Warranty Plan 9 3 3 3 1 9 1 \ 3 3 1 3

Select Concepts \ 9

Identify Service Providers 1 3 9 1 9 1 9 \ 3

Specify Production Plan 1 1 9 9 3 3 3 3 \ 1

Assemble Components 3 9 1 9 1 3 1 9 \

Brain Storm \

Generate Product Retirement Plan1 3 9 1 9 9 1 9 1 9 9 \

Forward / Delivers information to

Feedback / Relies on information from


Conclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

38

Conclusions

• Current research on prediction and prevention oferrors in the design process is fairly limited– interest from industry is high– tools for predicting and preventing errors in other

areas, such as manufacturing and assembly process,exist

• In addition to creating design process poka-yoke,it is necessary to establish the mentality of error-proofing the design process– design process error-proofing training and education


ProposalConclusions




ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

39

Future Work

• Error Categorization and Strategies– Refinement of categorization of errors and error-proofs– Build towards question-based analysis to identify type of

error and strategies• Assist Root Cause Analysis to design process problems

– Quantifying errors: RPN vs. expected cost

• Identification and Development of EP Tools– Knowledge-Based Engineering (KBE) - e.g. CAD add-ons– Knowledge Management (KM) - e.g. error-proofing, best

practice, and/or corrective actions web sites


ProposalConclusions


ResearchResearch

Application

Application

Educ

atio

n

Educ

atio

n

dfM

40

Questions?

Questions?

Error-Proofing of the Product Development Process

Documents

Transcript of Error-Proofing of the Product Development Process