1. MSA Training1

1

Click to edit Master title styleClick to edit Master title style

w w w . t u v . c o mTÜV Rheinland (India) Pvt. Ltd.,BF – 1.8 System Certification

WELCOME TO ALL WELCOME TO ALL

2



PURPOSE

• To UNDERSTAND THE ROLE AND FUNCTION OF MSA IN To UNDERSTAND THE ROLE AND FUNCTION OF MSA IN PRODUCT DEVELOPMENT PRODUCT DEVELOPMENT

• PREPARE THE TEAM FOR CONDUCTING THE PREPARE THE TEAM FOR CONDUCTING THE MEASUREMENT SYSTEMS ANALYSISMEASUREMENT SYSTEMS ANALYSIS

3



ContentsContents

• Introduction

• Objectives

• QS9000/TS16949 Requirements

• Measurement System Analysis

• Selection of Instrument ( Discrimination )

• Statistical Properties of Measurement System

• Bias , Linearity,Stability & Gauge Repeatability & Reproducibility

• Graphical Analysis

• Attribute data Study – Short & Long Method, & Conclusion

4



The process of assigning a number is called measurement The process of assigning a number is called measurement

process the assigned number is called measurement value process the assigned number is called measurement value

Measurement is defined as Measurement is defined as "" the assignment of numbers the assignment of numbers

( values ) to material things to represent the relations among ( values ) to material things to represent the relations among

them with respect to particular properties them with respect to particular properties ""

DEFINE MEASUREMENTDEFINE MEASUREMENT

5



The quality of measurement data is defined by the statistical The quality of measurement data is defined by the statistical

properties of multiple measurements obtained from a properties of multiple measurements obtained from a

measurement system operating under stable conditionsmeasurement system operating under stable conditions

If the measurements are If the measurements are closeclose to the master value for the to the master value for the

characteristic, then the quality of the data is said to be characteristic, then the quality of the data is said to be highhigh

If all or some of the measurements are If all or some of the measurements are far awayfar away from the from the

master value, then the quality of the data is said to be master value, then the quality of the data is said to be lowlow

QUALITY OF MEASUREMENT DATA QUALITY OF MEASUREMENT DATA

6



Production process is controlled using a measurement process. Production process is controlled using a measurement process.

A decision to adjust a manufacturing process or not is A decision to adjust a manufacturing process or not is commonly based on measurement datacommonly based on measurement data

Since measurement is a Process, variation do exist in the Since measurement is a Process, variation do exist in the measurement process itself. If variation in measurement measurement process itself. If variation in measurement process is high, the quality of measurement data will be Poor. process is high, the quality of measurement data will be Poor.

Using this measurement process having high variation, the data Using this measurement process having high variation, the data obtained will not represent the actual variation present in the obtained will not represent the actual variation present in the Manufacturing processManufacturing process

This may lead to wrong decision – which will in turn lead to high This may lead to wrong decision – which will in turn lead to high variation in the manufacturing process – causing high rejection variation in the manufacturing process – causing high rejection & rework.& rework.

IMORTANCE OF QUALITY OF MEASUREMENTIMORTANCE OF QUALITY OF MEASUREMENT

7



8



STANDARD:STANDARD:Accepted basis for comparison.

Criteria for acceptance

Known value, within stated limits of uncertainty,accepted as a true value

A standard which will yield the same results when applied by supplier / customer with the same meaning yesterday, today, and tomorrow

9



GAGE: Is any device used to obtain measurements frequently used to refer specifically to the devices used on the shop floor

MEASUREMENT SYSTEM: Is the collection of instruments or gages, standards, operations, methods, fixtures, software personnel, environment and assumptions used to quantify a unit of measurement

10



DISCRIMINATION, READABILITY, RESOLUTION Smallest readable unit, measurement resolution, scale

limit, detection limit An inherent property fixed by design Always reported as a unit of measure 10 to 1 rule of thumb

BASIC EQUIPMENT BASIC EQUIPMENT

11



REFERENCE VALUE Accepted value of an artifact Determined by averaging several measurements with a

higher level of measuring equipment Used as the surrogate for the true value.

TRUE VALUE Actual value of an artifact Unknown and unknowable

BASIC EQUIPMENT BASIC EQUIPMENT

12



TYPES OF VARIATION Location variation Width variation

Location Accuracy Bias Stability Linearity

Width Repeatability Reproducibility GR & R Measurement system

performance & capability

13



LOCATION VARIATION :

Accuracy is closeness to the true value or to an accepted reference value

Bias difference between the observed average of the measurements and the reference value

1 2 3 4 5 6

123456

LOCATION

MS AVERAGE REF VALUE

BIAS

14



LOCATION VARIATION :

Stability Change in bias over time A stable measurement process is in statistical control w.r.t

the location Alias drift

Reference ValueReference Value

TimeTime

15



OPERATING RANGE

LOCATION VARIATION : Linearity The change in bias over the normal operating range The change in bias over the normal operating range

BiasBias

BIAS

Size 1Size 1 Size NSize N

16



WIDTH VARIATION : Precision Precision Closeness of repeated readings to each other

Repeatability :

• Variation in measurement obtained with one measuring instrument when used several times by an appraiser while measuring the identical characteristics on the same part

• Commonly referred as Equipment Variation (EV)

• Instrument / gage capability or potential

APPRAISER

INSTRUMENT

CHARACTERISTICS

BUT N TRIALSSAME

RepeatabilityRepeatability

17



Reproducibility:Reproducibility:

• Variation in the average of the measurements made by different appraisers using the same gage when measuring a characteristic on one part

• Referred as appraiser variation

WIDTH VARIATION :

ReproducibilityReproducibility

AA CC BBAppraisersAppraisers

18



GRR OR GAGE R & R:

• Gage Repeatability and Reproducibility : combined estimate of measurement system repeatability and reproducibility

MEASUREMENT SYSTEM CAPABILITY :

• Short term estimate of measurement system variation Example : GRR

GR&RGR&R

CC BBAA

19



Measurement Process :Measurement Process : Measurement is a process which measures the process quality Measurement is a process which measures the process quality and provides feedback. Following is a typical measurement and provides feedback. Following is a typical measurement process modelprocess model

Process to be managed

Decision

Value

Analysis Measurement

Measurement – generally viewed as a black box. No real effort Measurement – generally viewed as a black box. No real effort made to improve the quality of measurement.made to improve the quality of measurement.

Calibration is performed to assess the Instrument alone. But Calibration is performed to assess the Instrument alone. But Instrument is only one part of the process. Other sources of Instrument is only one part of the process. Other sources of variation are Appraiser, Part, Environment, Method.variation are Appraiser, Part, Environment, Method.

MSA supports in assessing the quality of measurement process.MSA supports in assessing the quality of measurement process.

20



Variation :Variation : It is the difference between two things or an entity . It is the difference between two things or an entity .

Measurement System Variation :Measurement System Variation :Variation present in the measurement system which affects the Variation present in the measurement system which affects the process resultsprocess results

Total variation :Total variation : It is the total variation observed in the process outputsIt is the total variation observed in the process outputs

A)A) There can be variation due to There can be variation due to ProcessProcess

A)A) Part to Part Variation (PV)Part to Part Variation (PV)

B)B) Within Part Variation (WPV)Within Part Variation (WPV)

B)B) There can be variation due to There can be variation due to Measurement systemMeasurement system

A)A) Variation due to Appraiser (AV)Variation due to Appraiser (AV)

B)B) Variation due to Equipment (EV)Variation due to Equipment (EV)

C)C) Variation due to InteractionVariation due to Interaction

21



1. Adequate discrimination - The instrument discrimination should divide the tolerance into ten parts

2. For product control, variability of the MS must be small compared to the specification limits

3. The MS ought to be in statistical control. This means the variations are all due to common cause and not due to special cause

4. For process control, the variability of the MS ought to demonstrate effective resolution and be small compared to The manufacturing process variation assess the ms to the 6σ Process &/or total variation from MSA Study

FUNDAMENTALS PROPERTIES THAT DEFINE A GOOD MS :

22



Objectives of MSA Study :Objectives of MSA Study :• To Quantify the variation present in the measurement

system

• Ensure the stability of the measurement system

• Initiate appropriate actions to minimize the contamination of measurement system variation in total process variation

23



Effects of Measurement System Variability

Effects on product decisions :

A wrong decision will sometimes be made whenever any part of the above measurement distribution overlaps a specification limit.

For example a good part classified as bad this is type I error otherwise called as producer’s risk / false alarm

A bad part will sometimes be called good type II error consumer’s risk or miss rate

False rate + Miss rate = error rate

24



Effects of Measurement System Variability

LSLLSL USLUSL

IIII IIIIIIII

Effects on product decisions :

Where Where

I - Bad parts will always be called badI - Bad parts will always be called bad

II – Potential wrong decisions can be madeII – Potential wrong decisions can be made

III – Good parts will always be called good.III – Good parts will always be called good.

25



Effects on process decisions : With a process the following needs to be established

• Statistical control

• On target

• Acceptable variability

The impact of the process decisions would be

• Calling a common cause - a special cause

• Calling a special cause - a common cause

σ2obs =observed process variance

σ2actual =actual process variance

σ2msa =variance of the measurement system

Cp = TOLERANCE RANGE / 6 σ

(Cp)2obs = (Cp)2

actual + (Cp)2msa

26



Process setup / control - Tampering : • Often – manufacturing process - qualified - based on

setup part at the beginning of the day - to verify whether that process is targeted

• If the part measured is off the target, then the process is adjusted. Later in some cases another part is measured and again the process may be adjusted.

• Dr. Deming referred this phenomena of measurement and decision making as tampering

27



Four rules for Process Control

1. Make no adjustment or take no action unless the process is unstable

2. Adjust the process in an equal amount and in an opposite direction from where the process was last measured to be

3. Reset the process to the target. Then adjust the process in an equal amount and in an opposite direction from the target

4. Adjust the process to the point of the last measurement

Rule 2, 3 4 add progressively more variation

28



DISCRIMINATION:

Amount of change from the reference value that an instrument can detect and faithfully indicate.This is also referred to as the readability or resolution or Least Count

The discrimination is unacceptable for analysis if it cannot detect the variation of the process , and unacceptable if it cannot detect the special cause variation

29



# Of data category

Control Analysis

1 Can be used to control only if The process variation is small when compared to the specification

Un acceptable for estimating process parameters and indicesOnly indicates whether the process is producing conforming or non conforming parts

1 Data Category1 Data Category

30



# Of data category

Control Analysis

2-4 Can be used with semi variable control techniques based on the process distribution Can produce insensitive variable control charts

Generally unacceptable for estimating process parameters and indices since it only provides coarse estimates

2-4 Data Category2-4 Data Category

31



# Of data category

Control Analysis

5 or more data categories

Can be used with variable control chart

Recommended

5 Data Category5 Data Category

32



ISO/TS 16949-2002 ISO/TS 16949-2002 Requirements :Requirements :

This is referred in the clause 7.6 of ISO/TS 16949:2002This is referred in the clause 7.6 of ISO/TS 16949:2002

Requirement :Requirement :

1.1. Appropriate statistical studies to be conductedAppropriate statistical studies to be conducted

2.2. Applies to all measurement systems referred in Applies to all measurement systems referred in Control planControl plan

3.3. Mandatory before conducting SPC StudiesMandatory before conducting SPC Studies

33



Measurement System

It is the process by which we assign a number to a characteristics of a product/service.

Types of Measurement System• Attribute • Variable

Measurement System Analysis

Is the process to identify, Quantify and analyze the variation

present in the measurement system

34



When MSA Studies to be conducted:• Introduction of a new MS• During selection of new instrument• Before & After calibration• Changes in the method or environment

Types of StudiesVariableR&R, Bias, Linearity, Stability,

AttributeHypothesis Test Analyses – Cross Tab Method

35



36



R&R Study:-

The variable gage study can be performed using a number of differing techniques . they are

• Range method• Average and range method (including the control chart method)• ANOVA method

Except for the range method, the study data design is very similar for each of these methods. As presented , all methods ignore the within part variation example roundness, diametric taper, flatness.

37



• This will give a quick approximation of measurement variability. This method will provide only the overall picture of the MS. It does not decompose the variability into repeatability & reproducibility

• It typically uses 2 appraiser and five parts for the study

• In this study both appraisers measure each part once.

• The range for each part is the absolute difference between the measurement obtained by app A and the measurement obtained by app B

• The average of range is calculated (R)

GR&R - RANGE METHODGR&R - RANGE METHOD

38



The total measurement variability is calculated by multiplying the average range by 1 / d2

*

d2* is obtained from Appendix “C” with m = 2, g = Number

of parts.

Parts Appraiser A Appraiser A Range ( A,B)

1 0.85 0.80 0.05

2 0.75 0.70 0.05

3 1.00 0.95 0.05

4 0.45 0.55 0.10

5 0.50 0.60 0.10

GR&R- RANGE METHODGR&R- RANGE METHOD

39



AVERAGE RANGE ( R ) = ∑ R i = 0.35

g 5GRR = R = 0.07 = 0.0588 d2

* 1.19

Process standard deviation from previous study = 0.0777

% GRR = 100 * GRR / process standard deviation

% GRR = 100 * 0.0588 / 0.0777 = 75.7 %

Conclusion the MS is in need of improvement

GR&R- RANGE METHODGR&R- RANGE METHOD

40



This will give an estimate of both repeatability & reproducibility for a MS

This method will allow the MS variation’s to be decomposed into 2 separate components repeatability & reproducibility , but not their interactions

Conducting the study 1. Obtain a sample of n > 5 that represent the actual or

expected range of process variation

2. Refer to the Appraisers A,B,C etc., And number the parts 1 through n So that the numbers are not visible to the Appraiser.

GRR - AVERAGE & RANGE METHODGRR - AVERAGE & RANGE METHOD

41



3. Calibrate the gage if this is part of the normal ms procedure. Let appraiser measure n parts in a random order and enter the results in row 1.

4. Let appraiser B & C measure the same n parts without seeing each other reading, then enter the results in row 6 & 11, respectively.

5. Repeat the cycle using a different random order of measurement. Enter data in row 2, 7 and 12 . record the data in the appropriate column. For example if the 1st piece measured is part 7 then record the results in the column labeled 7. If three trials are needed repeat the cycle data in rows 3, 8 & 13.

6. Calculate using the Data analysis sheet – AV, EV, PV, TV, R&R & R&R%


42



Analysis of GRR studies

If Repeatability is large compared to Reproducibility, the reasons may be :• The instrument needs maintenance • The gage may need to be redesigned to be more rigid • The clamping or location for gaging needs to be improved

• There is excessive within part variation


43



Analysis of GRR studies

If reproducibility is large compared to reproducibility, the possible causes may be :

• The appraiser needs to be better trained in how to use and read the gage instrument

• Calibrations on the gage dial are not clear improved


44



CONCLUSION• GRR% < Less than 10 % - acceptable, more than 10 % and

less than 30 % conditionally accepted evaluate the economies of the measurement

• If it is more than 30 % reject - Analyze the causes of the high GRR.

• Ndc less than 5 – Not acceptable – Increase the discrimination of the instrument/gage.

• Ndc more than 5 – Acceptable.


45



Graphical Analysis It is used to analyze the results of a measurement system. It provides insight in to the variation model and inter- relationship.

Following charts are used to analyze the data.

Range chart : Assists in determining statistical control w.r.t repeatability & consistency of measurement process between appraisers for each part.

Error chart : Assists in determining consistency of measurement process between appraisers for each part.

Average chart : Assists in determining consistency between appraisers and the usability of Measurement Process.

46



Graphical AnalysisNormalized Individuals Chart : Assists in determining Reproducibility , consistency between appraisers , existence of outliers( Abnormal reading ) & Part appraiser interaction

Whiskers chart : Assists in determining consistency between appraisers , existence of outliers( Abnormal reading ) & Part appraiser interaction

X-Y Plot : Assists in determining Linearity ( If reference value is used ), consistency in linearity between appraisers

Scatter plot : Assists in determining consistency between appraisers , existence of outliers( Abnormal reading ) & Part appraiser interaction

47



48



StabilityStability (or drift) is the total variation in the measurements obtained with a measurement system on the same master or parts when measuring a single characteristic over an extended time period.

Stability study:Stability study:

Procedure for Stability study:• Decide the part and characteristics for the study.• Identify the location of measurement on the part.

• Make this part as a master part and ensure that the part is preserved till the study is completed.

• Decide the subgroup size (3 to 5), no. Of subgroup (20) & frequency (every day) of data collection.

• As per the frequency, measure the master part 3 to 5 times and record the details in the Xbar-R chart.

49



Procedure for Stability study ( Contd..) :• After completion of data collection, calculate each subgroup avg

and range.

• Calculate Xbar & Rbar.

• PLOT Xbar-R Control Chart

• Analyze the chart as per the normal interpretation of control chart.

• Calculate s = Rbar / d2

• Calculate stability % as follows:

• Stability % = s/Tolerance Or Process Variation * 100.

50



Procedure for Stability study ( Contd..) :• Interpret the results based on the acceptance

criteria given belowStability% below 10% - M.S.Is acceptable.

Stability% between 10 & 30% - M.S. Can be acceptable Under concession, based on the cost of repair,importance on Product

quality, etc..Stability% greater than 30% - Not acceptable. M.S.

Needs improvement.

51



52



Bias is the difference between the true value (ref value) and the observed average of measurements on the same characteristics

Possible causes for excessive bias

Instrument needs calibrationWorn instrument, equipment or fixtureImproper calibration or use of the setting standardPoor quality instrumentLinearity errorWrong gage for application

Different measurement method, setup loading, clamping, techniqueMeasuring the wrong characteristicDistortionEnvironmentViolation of an assumptionApplication error

Bias Study

53



Procedure for Bias Study Obtain a sample and establish its reference value relative

to a traceable standard. If not select a production part that falls in the mid range of the production measurement and designate it as master sample for bias analysis

Measure the Part n≥ 10 times in the tool Room, and compute the average of the n Readings. Use this average as the reference value.

Have a Single Appraiser measure the sample n ≥ 10 times in the normal manner.

Plot the data as a histogram relative to the reference value review the histogram ( see if there are any special causes or any anomalies present

Bias Study

54



Analysis of results

5. Compute the average of n readings N Σ xi

X = i = 1

N

6. Compute the repeatability standard deviation σ repeatability = max(xi) – min(xi)

d2*

Where value of d2 is taken from Appendix “C”, with g=1 & m=n g = number of sub groups m = sub group size

Bias Study

55



7. Determine the t estimate for the Bias. BIAS = OBSERVED AVERAGE - REFERENCE VALUE

σb = σr / √n t = bias/ σb

8. Bias is acceptable at the α level if zero falls within the 1- α confidence bounds around the bias value.

BIAS - [ σb(tv, 1- α/ 2) ] ≤ zero ≤ BIAS + [ σb(tv, 1- α / 2)]

Where the value of v is found from Appendix C with g=1, m=n &Tv, 1- α /2 : is found using standard t table.

α : Customer agreement should be obtained if the confidence level is other than 95 %

Bias Study

56



0.06.015

-0.45.614

0.26.213

0.16.112

0.06.011

0.36.310

0.46.49

0.16.18

0.06.07

0.16.16

0.06.05

-0.15.94

-0.15.93

-0.35.72

-0.25.81

BIASREFERENCE VALUE 6.00

TRIALS

Bias Study

57



0.058130.225146.006715MEASURED VALUE

STANDARD ERROR OF MEAN σb

STANDARD DEVIATION σr

MEANn(m)

.1349

HIGHER

-.1215

LOWER

.00672.20610.8.1153MEASURED VALUE

95 % CONFIDENCE LEVEL OF THE BIAS

BIASSIGNNIFICANT t VALUE (2

TAILED)

dft STATISTIC

REFERENCE VALUE = 6.00, α = .05, g=1, d2* = 3.55

Conclusion:Conclusion:

Bias is acceptable at the α level if zero falls within the 1- α confidence bounds around the bias value

Bias Study

58



59



Linearity can be calculated using the following procedure 1. Select g ≥ 5 parts whose measurements, due to process

variation cover the operating range of the gage. 2. Have each part measured by the layout inspection to

determine the ref value and confirm that the operating range of the subject gage is encompassed.

3. Have each part measured m ≥ 10 times on the subject gage by one operator. Select the parts at random to minimize appraiser recall in the measurement.

4. Calculate the part bias for each measurement and bias average for each part bias = X i,j- ( reference value)

M Σ bias i,j

Bias = j = 1

M

Linearity Study

60



5. Plot the individual biases and the bias averages w.r.t ref value on a linear graph

6. Calculate and plot the best fit and the confidence band of the line.

7. Use regression analysis method for best fit & Estimation method for arriving at the confidence band.

Linearity Study

61



-0.80000

-0.60000

-0.40000

-0.20000

0.00000

0.20000

0.40000

0.60000

0.80000

2 4 6 8 10

Reference Values

Bia

s

Fitted Line

95% CI

Zero Bias

Linearity Study

62



63



Attribute Gage Study Attribute Gage Study ::

Miss:Miss: Part is bad. Decision given is good.Part is bad. Decision given is good.

False Alarm:False Alarm: Part is good. Decision given is bad. Part is good. Decision given is bad.

Procedure :Procedure :

• Decide the no. Parts to be selected (generally 50 parts), no. Of Decide the no. Parts to be selected (generally 50 parts), no. Of appraisers (3 appraisers) & no. Of trials(3 trials) for conducting appraisers (3 appraisers) & no. Of trials(3 trials) for conducting the study.the study.

• Identify all parts by a unique number and inspect the part to Identify all parts by a unique number and inspect the part to decide the master value / reference value. This has to be decide the master value / reference value. This has to be approved by a expert.approved by a expert.

• Have each appraiser measure the parts randomly for the no. Of Have each appraiser measure the parts randomly for the no. Of trials decided.trials decided.

• Record the results as “good / bad” or “go/no go” or “accept / Record the results as “good / bad” or “go/no go” or “accept / reject”, “ 1” – for ok, “0” – for not ok. etc..reject”, “ 1” – for ok, “0” – for not ok. etc..

64



Part A - 1 A - 2 A - 3 B - 1 B - 2 B - 3 C - 1 C - 2 C - 3 Ref. Ref. value

1 P P P P P P P P P P 0.476901

2 P P P P P P P P P P 0.509015

3 NEG NEG NEG NEG NEG NEG NEG NEG NEG NEG 0.576459



6 P P NEG P P NEG P NEG NEG P 0.544951

7 P P P P P P P NEG P P 0.465454

8 P P P P P P P P P P 0.502295


10 P P P P P P P P P P 0.515573

11 P P P P P P P P P P 0.488905

12 NEG NEG NEG NEG NEG NEG NEG P NEG NEG 0.559918

13 P P P P P P P P P P 0.542704

14 P P NEG P P P P NEG NEG P 0.454518

15 P P P P P P P P P P 0.517377

16 P P P P P P P P P P 0.531939

17 P P P P P P P P P P 0.519694

18 P P P P P P P P P P 0.484167

19 P P P P P P P P P P 0.520496

20 P P P P P P P P P P 0.477236

21 P P NEG P NEG P NEG P NEG P 0.452310

22 NEG NEG P NEG P NEG P P NEG NEG 0.545604

23 P P P P P P P P P P 0.529065

24 P P P P P P P P P P 0.514192


26 NEG P NEG NEG NEG NEG NEG NEG P NEG 0.547204

27 P P P P P P P P P P 0.502436

28 P P P P P P P P P P 0.521642

29 P P P P P P P P P P 0.523754

30 NEG NEG NEG NEG NEG P NEG NEG NEG NEG 0.561457

31 P P P P P P P P P P 0.503091

32 P P P P P P P P P P 0.505850

33 P P P P P P P P P P 0.487613

34 NEG NEG P NEG NEG P NEG P P NEG 0.449696

35 P P P P P P P P P P 0.498698

36 P P NEG P P P P NEG P P 0.543077


38 P P P P P P P P P P 0.488184


40 P P P P P P P P P P 0.501132

41 P P P P P P P P P P 0.513779


43 P NEG P P P P P P NEG P 0.462410

44 P P P P P P P P P P 0.470832


46 P P P P P P P P P P 0.493441

47 P P P P P P P P P P 0.486379


49 P P P P P P P P P P 0.483803


Where,Where,

P – passP – pass

NEG – Not goodNEG – Not good

Data Collection sheetData Collection sheet


65



Since reference decision is not know, develop cross tabulations Since reference decision is not know, develop cross tabulations comparing each appraisers to the othercomparing each appraisers to the other

CROSS TABULATION – BETWEEN APPRAISERSCROSS TABULATION – BETWEEN APPRAISERS

A to B crosstabulation

B0 1 Total

A 0 Count 44 6 50Expected count 15.7 34.3 50.0

1 Count 3 97 100Expected count 31.3 68.7 100.0

Total Count 47 103 150Expected count 47.0 103.0 150.0

A to REF crosstabulation

REF0 1 Total

A 0 Count x y a=x+yExpected count e=(a*c)/(150) f=(a*d)/150 j=e+f

1 Count p q b=p+qExpected count g=(b*c)/150 h=(b*d)/150 k=g+h

Total Count c=x+p d=y+q o = a+bExpected count m=e+g n=f+h r=j+k


66



Similarly tabulate for A to C & B to C cross tabulationsSimilarly tabulate for A to C & B to C cross tabulations

B to C crosstabulation

C0 1 Total

B 0 Count 42 5 47Expected count 16.0 31.0 47.0



A to C crosstabulation

C0 1 Total





67



Purpose of these tables is to determine the extent of agreement among the appraisers

To determine the level of this agreement, kappa which measures the agreement between the evaluation of two raters when both are rating the same object

Kappa is a measure of interrater agreement that tests if the counts in the diagonal cells (the parts that receive the same rating) differ from those expected by chance alone.

A value of Kappa > 0.75 indicates good to excellent agreement

A value of kappa < 0.45 indicates poor agreement


68



Kappa = (Po – Pe ) / (1- Pe )

Where Po = the sum of the observed proportions in the diagonal cells

= (x+q ) / o for each Cross tabulation (A-B, A-C, B-C)

Pe = the sum of the expected proportions in the diagonal cells

= (e+h )/r for each Cross tabulation (A-B, A-C, B-C)


69



Prepare the Calculate Kappa for the appraisers

-- 0.790.78B

-- 0.86B

0.780.86-- A

CBAKappa

This analysis indicates that all the appraisers show good agreement each other.

This analysis is necessary to determine if there are any differences


70



This analysis is necessary to determine if there are any differences among the appraisers.

But to understand the differences between the appraisers decision and the reference decision, a separate cross tabulation to be developed for appraiser and reference decision.

Cross Tabulation – Appraiser – Reference Decision


71





REF0 1 Total

A 0 Count x y a=x+yExpected count e=(a*c)/(150) f=(a*d)/150 j=e+f

1 Count p q b=p+qExpected count g=(b*c)/150 h=(b*d)/150 k=g+h

Total Count c=x+p d=y+q o = a+bExpected count m=e+g n=f+h r=j+k


REF0 1 Total





72




B to REF crosstabulation

REF0 1 Total

B 0 Count 45 2 47Expected count 15.0 32.0 47.0



C to REF crosstabulation

REF0 1 Total

C 0 Count 42 9 51Expected count 16.3 34.7 51.0




73



Prepare the Calculate Kappa measure to determine the agreement of each appraiser to the reference decision.

A B CKappa 0.88 0.92 0.77


74



Calculate the effectiveness of the measurement

Effectiveness = number of correct decisionEffectiveness = number of correct decision

total opportunities for a decisiontotal opportunities for a decision


75



Appraiser Number Good

correct

Number bad

correct

Number

correct

Number false alarm

Number

missed

Number

totalA 97 45 142 5 3 150B 100 45 145 2 3 150C 93 42 135 9 6 150

Appraiser

A

B

C

Effectiveness Pf alse alarm Pmissed

84.00 6.25

90.00

80.00

4.90

1.96

8.82

6.25

12.50


76



> 90 % < 5 % < 2 %

> 80 -

< 90 %

> 5 -

< 10 %

> 2 -

< 5 %

< 80 % > 10 % > 5 %

Effective-ness

False alarm rate

Miss rate

Decision on Measurment system

Acceptable for the appraiser

Marginally acceptable for the appraiser - may need

improvement

Unacceptable for the appraiser - needs improvement

Acceptance criteria for Effectiveness, False alarm rate & Miss rate.


77



MSA PlanningMSA Planning

1. Include all measurement systems referred in Control plan.

2. Prepare a MSA plan as per below

3. For all existing MS, conduct appropriate studies

4. Applicable study can be done. Min. R&R is must. Others optional.

Bias R & R Linearity StabilityOutside Micrometer

0.01 L.C , ( 0 - 25 mm )

MSA Plan

1 Mitutoyo 15 - 21ThicknessTurning

Sr.No Part CharacteristicsMake Of

InstrumentMeasuring Inst with Least

count & Range Working Range

Variable DataAttribute Data

78



Conclusion :MSA study has become important since it can be applied

under the following areas• Criterion to accept new instrument• Comparison of one measuring device against another• Basis for evaluating a gage suspected of being deficient• Comparison of equipment before and after repair• Required element for calculating process variation

Limitation of standard are it is Difficult to use in destructive testing , Some Processes ,Product Characteristics & Tests have no defined industry or National standards

79



THANK YOU

1. MSA Training1

Documents

Transcript of 1. MSA Training1