Pitfalls of Measurement System Analysis

MSA+DFSS

Tom Early, GE Global Research

QPRC2013

2

GE Public

June 7, 2013

Abstract

Measurement system analysis (MSA) will be

discussed in an industrial environment. MSA is

more than just gauge R&R. Using a “top down”

approach, MSA will be discussed at a system-

level view of the measurement process. Often

misunderstood sub-systems like gauge metrics

and gauge error analysis will be discussed.

3

GE Public

June 7, 2013

Gauge Requirements

Accurate calibration

Consistent stable, reproducible

Sensitive S/N, repeatable

Linear resolution, discrimination

Useful JIT, traceable to the customer

Inexpensive non-destructive, alpha vs. beta errors

4

GE Public

June 7, 2013

Gauge Performance Required: measurement context

Is a part within spec? If we ship it, will it work?

What is our process capability? Is the gauge good enough to understand sub-grouping?

Is the process changing or drifting? Can we do SPC to monitor the factory?

Can the process be improved?

Is the gauge good enough to use in an improvement project?

5

GE Public

June 7, 2013

A Process to be Measured…

http://www.islandone.org/MMSG/aasm/AASM53.html

Inputs Process Outputs

6

GE Public

June 7, 2013

Where Does the Measurement Begin?

Measurement

System

7

GE Public

June 7, 2013

Where Does the Measurement Begin?

Measurement

System

Right here!

8

GE Public

June 7, 2013

The Measurement System is a Process!

Inputs

Measurement

Process

Measurement Data

9

GE Public

June 7, 2013

The Measurement System as Process

Inputs

Measurement

Process

Measurement Data

Selection Preparation Mount Read

22222 ... ReadingMountingnPreperatioSelectiontSystemMeasuremen

10

GE Public

June 7, 2013

Measurement Sub-systems

• Measurement subsystems can be very diverse.

• Different failure mechanisms.

• Subsystems may have their own

specifications.

• Can be destructive, even when the

“measurement” is not.

• Might involve different operators.

• Ideally, variability between subsystems does

not correlate.

11

GE Public

June 7, 2013

Complications from Destructive Tests

• If the sample is destroyed in the measurement

process, sample variability exists in all

estimable characteristics of the study.

• Therefore, use hand-selected identical parts!

• You can’t get at operator*part interactions

in a nested experiment anyway!

12

GE Public

June 7, 2013

One problem, two kinds of errors.

LSL USL

T

Process

Gauge

Failed Good Units

Gauge

Passed Bad Units

It’s a good idea to track PB and FG separately, even though the exact same mechanism is responsible for both.

13

GE Public

June 7, 2013

Measurement Error

Input Test Pass

Fail

Repair/Scrap/Retest

Submitdp PBEscape dpdp

FGSubmitPBObserve dpdpdpdp

Observedp

14

GE Public

June 7, 2013

Gauge Noise Performance Metric A: against a tolerance

Some inflated value of the gauge noise needs to be a small fraction of the process tolerance.

Good for the “end-game” but…

%𝐺𝑅𝑅 = 100% ×𝑓 × 𝜎

𝑈𝑆𝐿 − 𝐿𝑆𝐿

USL LSL

Gauge Error Distribution

f

15

GE Public

June 7, 2013

Gauge Noise Performance Metric B: against process noise.

A 60% GRR used to measure a Z = 3 process has the same relative contribution to the observed process

variability that a 30% gauge contributes to the observed process variability when measuring a Z = 6 process.

MeasureObserve

Process

cGaugeMetriMeasure

2

Process 2

For a gauge to be useful, it just needs to see the process noise.

16

GE Public

June 7, 2013

Gauge Noise Performance

Gage R&R

%Contribution

Source VarComp (of VarComp)

Total Gage R&R 12.226 5.62

Repeatability 9.332 4.29

Reproducibility 2.895 1.33

Oper(i) 2.895 1.33

Part-To-Part 205.483 94.38

Total Variation 217.709 100.00

Process tolerance = 20

Study Var %Study Var %Tolerance

Source StdDev (SD) (5.15 * SD) (%SV) (SV/Toler)

Total Gage R&R 3.4966 18.0077 23.70 90.04

Repeatability 3.0548 15.7321 20.70 78.66

Reproducibility 1.7014 8.7624 11.53 43.81

Oper(i) 1.7014 8.7624 11.53 43.81

Part-To-Part 14.3347 73.8236 97.15 369.12

Total Variation 14.7550 75.9881 100.00 379.94

Number of Distinct Categories = 5

A

B

Here, a measurement system that is noisy compared to the tolerance (A) is much less noisy than the process being measured (B).

4966.3

3347.142floor

17

GE Public

June 7, 2013

Product Process Signal Attenuation

GaugeStudy

Parts

2

2

1

Study

PartsPPSI

D.J. Wheeler, “An Honest Gauge R&R Study”, 2006 ASQ/ASA Fall

Technical Conference.

Interclass Correlation Coefficient

18

GE Public

June 7, 2013

The Point of a GR&R Study (Random Effects!)

Measure-

ments …

…

Operators P

art

s

But sometimes we

aren’t interested in

the distribution!

19

GE Public

June 7, 2013

Single, or Limited Operator(s)

If all operators are in the GR&R Study:

• The GR&R experiment is no longer a full

random effect model.

• Each operator can have his own calibration

curve, potential eliminating all operator

contributions to the gauge error.

20

GE Public

June 7, 2013

Using Calibration Regression Effectively

xy

m N M

x x

x xi

1 1 2

2

( )

( )

Where: dx = Error associated with back calculating the most likely x value from y measured

dy = Standard error of the regression fit

N = Number of replicates of unknown samples

M = Number of calibration samples

m = Calibration slope

x = Back calculated (predicted) value of the unknown

= Average value of calibration samples

xi = Individual calibration samples (1 to M)

x

0

10

20

30

40

50

60

0 20 40 60 80 100 120

Input

Re

spo

nse

Fit

Standards

Error

2 Unknowns

Prediction intervals provide error estimates from a “single” measurement!

21

GE Public

June 7, 2013

Calibration Pitfalls

• Frequently, calibration standards and process

samples are measured using a different

process.

• Typically, calibration standards require less

preparation.

• As a result, calibration regression residual

error doesn’t capture all sources of sample

measurement variability.

22

GE Public

June 7, 2013

Summary

• Be sure to treat a measurement process as a complete system.

• Know the measurement process boundaries.

• “Hand select” pseudo-random parts for a gauge R&R study.

• Use more operators than parts.

• Calibration regression is an effective tool to control gauge variability, but, again, know the measurement process boundaries.