6 Sigma Final Draft

Assessment of the Analytical Phase of Selected Biochemical Tests at the UP-PGH Central Laboratory: Application of Six Sigma Metrics

to Internal and External Quality Control Data

Amado O. Tandoc III, MD Principal Author

Ariel M. Vergel De Dios, MD, FPSP Nelson T. Geraldino, MD, MSPH, FPSP Ma. Carmen L. Cagampan, MD, FPSP

Joselyn A. Mendoza, RMT Co-Authors

UNIVERSITY OF THE PHILIPPINES-PHILIPPINE GENERAL HOSPITAL DEPARTMENT OF LABORATORIES

1

Assessment of the Analytical Phase of Selected Biochemical Tests at the UP-PGH Central Laboratory: Application of Six Sigma Metrics to

Internal and External Quality Control DataA.O.Tandoc III, MD; A.M.Vergel De Dios,MD,FPSP; N.T.Geraldino,MD,MSPH,FPSP; M.C.L.Cagampan,MD,FPSP; J.A.Mendoza,RMT

UNIVERSITY OF THE PHILIPPINES-PHILIPPINE GENERAL HOSPITAL DEPARTMENT OF LABORATORIES

ABSTRACT. Ten (10) common biochemical tests (Glucose, BUN, Creatinine, Uric Acid, Total Protein, Albumin, Cholesterol, Sodium, Potassium and Chloride) performed at the Chemistry Section of the UP-PGH Central Laboratory were analyzed by reviewing available paired Internal and External QC (EQAS, AQAS, PCQACL & NRLB) data in 2007 & 2008 and subjecting these data to Six Sigma principles. With a benchmark for world class quality of 6 sigma and acceptable process quality of 4-5 Sigma, only Uric Acid testing achieved an acceptable average Sigma metric at 4.69 using EQAS data. Creatinine, Total protein, Albumin and Potassium testing were within 3-4 Sigma. Glucose, Cholesterol, BUN, Sodium and Chloride testing all fall below 3-Sigma which is unacceptable quality. Using AQAS data, no analyte tested above 4-Sigma. Using PCQACL and NRLB data, only Potassium was within 4-5 Sigma.

INTRODUCTION

Medical laboratory work is composed of the technical activities that produce laboratory

results for patient care. Preanalytic (blood sample collection, receiving, accessioning), analytic

(testing, examinations, interpretation) and postanalytic activities (reporting results, archiving samples,

charging) transform a clinician’s order for a laboratory examination into the results used by the

clinician to diagnose and treat patients.

Central to every laboratory is its obligation to assure that the whole testing process is

accurate, reliable and ultimately, useful to the clinician. It is mandated by the Department of Health

(DOH) Administrative Order No. 2007-0027 Revised Rules and Regulations Governing the Licensure

and Regulation of Clinical Laboratories in the Philippines that every clinical laboratory shall be

organized to ensure the quality of laboratory results.1

Quality control mechanisms, both internal and external, are put in place just for this purpose.

Internal QC procedures are performed regularly through assaying QC materials for which the correct

result is known in advance in order to verify that a measurement procedure is performing as

expected.2 External quality control programs, in the form of proficiency testing, is one way by which

laboratory performance is evaluated against those of its peers that use the same instruments and

principles within the country and between countries. Such assessments allow a laboratory to verify

that its results are consistent with those of other laboratories using the same or similar methods for an

analyte.2

2

FIG. 1. THE SIX SIGMA PERFORMANCE GOAL. Six standard deviations of process variation should fit within the tolerance limits for the process.

FIG. 2. Two ways of deriving the Sigma Metric.

As will be demonstrated in the present study, a step further towards a more meaningful

interpretation of the data gleaned from both internal and external QC data may be achieved through

the use of the principles of Six Sigma.

Six Sigma is a process improvement

strategy that has been hailed as an “evolution

in quality management.”3 It is the universal

and objective way of quality measurement as

it measures the degree to which any process

deviates from its goal. How well the process

works is analyzed and measured as the first

step towards improving quality. It has been

implemented initially–and successfully–in

business and industry starting in the early

1990’s by companies such as Motorola and General Electric and espouses a performance goal that

“Six sigmas or 6 standard deviations of process variation should fit within the tolerance limits for the

process.”4

As a universal measure of process performance, any process can be evaluated in terms of a

Sigma Metric that describes how many “sigmas” fit within the tolerance limits. Of the two ways of

assessing performance, the measurement of variation is applicable in the laboratory setting. It makes

use of the information on precision and accuracy that laboratories derive from internal and external

quality control programs.5

The sigma-capability of the process tells us where and what level of improvement is

necessary. A high quality process strives to achieve a sigma value of 6. Processes of acceptable

quality have a sigma value of 4-5. Most processes are sigma 3, which connotes great opportunity for

improvement. As Sigma increases, process reliability improves, operating costs go down and

customer satisfaction increases.6,7,8

3

MEASURE OUTCOME MEASURE VARIATION

Inspect Outcomes & Count Defects

Calculate Defect Per Million (DPM)

Convert DPM to Sigma Metric

Measure Variation of a Process

Calculate SD and Process Capability

Convert Capability to Sigma Metric

In 2000, Nevalainen et al evaluated laboratory performance of quality indicators with the Six

Sigma Scale.8 This is the first application of Sigma metrics to laboratory data. In this study, the

preanalytic, analytic and postanalytic phase of laboratory operations in 3 clinical laboratories in the

U.S. were analyzed. Laboratory performance was expressed in parts-per-million which when

converted to Six Sigma using a standard table, demonstrated opportunities for significant

improvements across the total testing process encompassing all the phases. In 2006, Westgard and

Westgard, utilized Six Sigma principles in assessing the quality of the analytic phase of laboratory

testing of selected analytes in the US using data from proficiency testing surveys.9 For both studies,

the highest Sigma of US laboratories was between 4-5 sigma with most methods and processes

falling below this level.

Internet search revealed that no local study has been done on the application of Six Sigma in

clinical laboratories in the Philippines.

OBJECTIVES

The study aims to use the principles of Six Sigma to determine the quality of ten (10) selected

biochemical tests [Glucose, BUN, Creatinine, Uric Acid, Total Protein, Albumin, Cholesterol,

Sodium, Potassium and Chloride] performed at the UP-PGH Central Laboratory Chemistry Section

using internal and external QC data. Specifically, the study aims to:

1. Review the performance of the UP-PGH Central Laboratory in terms of External Quality

Assurance Programs (EQAS, AQuAS, PCQACL, NRLB) that the laboratory has

participated in for the last two years;

2. Determine the quality of ten (10) selected tests [Glucose, BUN, Creatinine, Uric Acid,

Total Protein, Albumin, Cholesterol, Sodium, Potassium and Chloride] at the UP-PGH

Central Laboratory Chemistry Section by applying Six Sigma metrics to data gleaned

from the aforementioned proficiency testing programs and the laboratory’s own internal

QC data;

3. Make recommendations on how to improve the present quality of laboratory testing at the

UP-PGH; and to

4. Make recommendations on possible directions of further investigations using Six Sigma

principles.

4

FIG. 3. Formula for deriving the Sigma Metric for an analytic test.

Sigma = (TEa – bias)/CV

Where TEa = Total Allowable Error for the analytic test based on CLIA criteria

Bias = [(Laboratory mean – Group Mean)/Group Mean] x 100 from External QC

CV = Coefficient of Variation of the UP-PGH Central Laboratory Internal QC

METHODOLOGY

Data from Internal QC procedures and External Quality Assurance Programs* for the years

2007-2008 from the Section of Biochemistry of the UP-PGH Central Laboratory pertaining to ten

common (10) biochemical tests were collected. The analytes included are Glucose, Blood Urea

Nitrogen (BUN), Creatinine, Uric Acid, Total Protein, Albumin, Cholesterol, Sodium, Potassium and

Chloride. Values of these analytes were expressed in SI units.

From the External QC results, data grouped according to same principle-and-machine are

chosen over data grouped only according to same principle. This is in order to decrease the bias that

may be imparted by a difference in the chemistry analyzer being used. Other sources of variability

such as environmental factors due to geographic location and climate, patient load per annum,

regularity of maintenance and calibration (under which conditions the analyzer is operated) were

beyond the scope of this study.

Where available, Internal and External Quality control data were matched as to month of

actual execution of QC procedures. This is in order to arrive at a true picture of QC performance at a

particular time. Only those months that have both Internal and External QC data were included in this

study. As this is a retrospective approach, Internal QC was performed by different medical

technologists over the study period.

The Sigma metric for each of the ten (10) analytes were computed based on the following

formula (Fig. 3) which takes into account both internal and external QC data.

__________________

*The UP-PGH Central Laboratory has been enrolled in two (2) international proficiency testing programs: 1) EQAS

(worldwide) and AQUAS (Asia). EQAS proficiency testing was performed every 2 weeks from November 2007 to April

2008. AQUAS has been performed twice yearly from 2001 to the present. The Central Laboratory has also participated in

two (2) local proficiency testing programs: 1) PCQACL (Philippine Council for Quality Assurance of Clinical Laboratories)

and 2) NRLB (National Reference Laboratory for Biochemistry). Both were done once during the year 2007.

5

TABLE 1. CLIA Criteria for Chemistry Analytes. Highlighted are the ones included in this study.

The most recent CLIA (Clinical Laboratory

Improvement Ammendments) Criteria for

Acceptable Performance in Proficiency Testing**

(Table 1) were used to determine the value of the

allowable total errors (TEa) for each analytic test.

The TEa represents the sum of all allowable

imprecision and bias of a test, i.e., the tolerance

specifications within which 6 Sigmas of standard

deviation must fall, in order to make the results of a

test measurement reliable.

The calculated value is the Sigma Metric

which will deem if the analytic process is of superior

or inferior quality. As earlier stated, the target goal

is a Sigma value of 6. Acceptable performance

ranges from Sigma 4 to 5.

RESULTS

External QC results (EQAS) from November

2007 to April 2008 were matched per month with

Internal QC data from the UP-PGH Central

Laboratory gathered from the same period of time.

Internal QC data were matched for June and

September 2007 AQuAS results. Results from local External QC surveys (PCQACL, NRLB)

conducted once in September 2007 were likewise matched with September 2007 Internal QC data.

Tables 2 through 11 show that the external

and

internal QC data are used to determine the Bias and

Coefficient of Variation (CV), respectively, in order

to arrive at the Sigma metric through the formula,

Sigma metric = (TEa – Bias)/CV. The TEa for

each analyte is taken from the CLIA requirements

(see Table 1).

6

Analyte PT Limit

Alanine aminotransferase (ALT, SGPT)

+/- 20%

Albumin +/- 10%

Alkaline phosphatase +/- 30%

Amylase +/- 30%

Aspartate aminotransferase (AST, SGOT)

+/- 20%

Bilirubin, total +/- 0.4 mg/dL or +/- 20% (greater)

Blood gas p02 +/- 3 SD

Blood gas pCO2 +/- 5 mm Hg or +/- 8% (greater)

Blood gas pH +/- 0.04

Calcium, total +/- 1.0 mg/dL

Chloride +/- 5%

Cholesterol, high dens. lipoprotein

+/- 30%

Cholesterol, total +/- 10%

Creatine kinase +/- 30%

Creatine kinase isoenzymes

MB elevated (present or absent) or Target value +/- 3 SD

Creatinine +/- 0.3 mg/dL or +/- 15% (greater)

Glucose +/- 6 mg/dL or +/- 10% (greater)

Iron, total +/- 20%

Lactate dehydrogenase (LDH)

+/- 20%

LDH isoenzymes LDH1/LDH2 (+ or -) or Target value +/- 30%

Magnesium +/- 25%

Potassium +/- 0.5 mmol/L (+/- 12%)

Sodium +/- 4 mmol/L (+/- 2.6%)

Total protein +/- 10%

Triglycerides +/- 25%

Urea Nitrogen +/- 2 mg/dL or +/- 9% (greater)

Uric acid +/- 17%

** There is no local equivalent as yet for the U.S. CLIA Criteria for Acceptable Performance in Proficiency Testing. Other sources of values for the TEa include the European Biologic Goals and Calculated Biologic Allowable Total Errors, Global Quality Requirements Consensus Draft, Desirable Specifications for Total Error, Imprecision and Bias Derived from Biologic Variation, et cetera.

TABLE 2. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR GLUCOSE.

GLUCOSE with TEa = + 10% EXTERNAL QC DATA INTERNAL QC DATA

SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV

(NOVEMBER 2007)NO. 1 17.65 18.35 3.87 4.56 0.15 3.29 1.86 14.13 0.34 2.41 2.55NO. 2 19.45 23.61 17.58 4.56 0.15 3.29 NO SIGMA 14.13 0.34 2.41 NO SIGMA

(DECEMBER 2007) NO. 3 2.57 2.54 1.18 4.70 0.16 3.40 4.96 14.61 0.26 1.78 3.54NO. 4 12.50 12.98 3.70 4.70 0.16 3.40 1.85 14.61 0.26 1.78 3.54(JANUARY 2008) NO. 5 18.14 18.46 1.73 4.73 0.16 3.38 2.44 14.73 0.41 2.78 2.97NO. 6 12.68 13.13 3.43 4.73 0.16 3.38 1.94 14.73 0.41 2.78 2.36(FEBRUARY 2008) NO. 7 2.64 2.55 3.53 5.11 0.09 1.76 3.67 16.41 0.36 2.19 2.95 NO. 8 22.66 23.62 4.06 5.11 0.09 1.76 3.37 16.41 0.36 2.19 2.71(MARCH 2008) NO. 9 22.53 23.41 3.76 4.80 0.15 3.13 2.00 15.36 0.60 3.91 1.60 NO. 10 2.46 2.53 2.77 4.80 0.15 3.13 2.31 15.36 0.60 3.91 1.85 (APRIL 2008) NO. 11 18.2 18.31 0.60 4.75 0.28 5.89 1.59 15.21 0.79 5.19 1.81 NO. 12 13.21 12.90 2.40 4.75 0.28 5.89 1.29 15.21 0.79 5.19 1.46

AQUAS (JUNE 2007)LEVEL 1 5.34 5.61 4.90 5.07 0.16 3.16 1.62LEVEL 2 13.7 13.97 1.97 15.41 0.76 4.93 1.63(SEPTEMBER 2007) LEVEL 1 4.90 5.39 9.18 4.68 0.14 2.99 0.27LEVEL 2 12.93 13.97 7.48 14.31 0.33 2.31 1.09

PCQACL (SEPTEMBER 2007)NORMAL 5.88 5.70 3.16 4.68 0.14 2.99 2.29ABNORMAL 14.52 14.4 0.83 14.31 0.33 2.31 3.98

NRLB (SEPTEMBER 2007)

VIAL 1 4.42 4.48 7.92 4.68 0.14 2.99 0.70VIAL 2 13.86 14.70 5.71 14.31 0.33 2.31 1.86

For glucose, 6-sigma values range from as low as 0.27 (which is substandard) to 4.96 (which is acceptable). Two instances during the 2nd cycle of November 2007 for the EQAS have no calculated sigma value. This is because the bias calculated is greater than the TEa set by the CLIA for glucose. When this happens, a negative value is calculated for the sigma metric. Because the bias is greater than the total allowable error for the analyte, we cannot consider the calculated sigma metric as “within 6 standard deviations of the tolerance limits.” Out of 22 observations where the sigma metric can be calculated, 5 are above 3.00, which is not impressive by 6-sigma standards. If we average the sigma metric for glucose, we get 2.48 (EQAS), 1.15 (AQUAS), 3.14 (PCQACL) and 1.63 (NRLB) – all of which are unimpressive and below the acceptable standards.

7

TABLE 3. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR BLOOD UREA NITROGEN.

BLOOD UREA NITROGEN with TEa = + 9% EXTERNAL QC DATA INTERNAL QC DATA

SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV

(NOVEMBER 2007)NO. 1 27.62 27.50 0.44 6.26 0.58 9.27 0.92 18.15 1.25 6.89 1.24NO. 2 30.71 36.00 14.69 6.26 0.58 9.27 NO SIGMA 18.15 1.25 6.89 NO SIGMA(DECEMBER 2007) NO. 3 4.13 3.62 14.09 5.37 0.67 12.48 NO SIGMA 16.51 1.11 6.72 NO SIGMANO. 4 18.32 19.60 6.53 5.37 0.67 12.48 0.20 16.51 1.11 6.72 0.37(JANUARY 2008) NO. 5 28.39 27.80 2.12 5.28 0.41 7.77 0.89 17.1 0.67 3.92 1.76NO. 6 19.08 19.40 1.65 5.28 0.41 7.77 0.95 17.1 0.67 3.92 1.88(FEBRUARY 2008) NO. 7 3.43 3.53 2.83 5.60 0.48 8.57 0.72 17.30 0.79 4.57 1.35NO. 8 34.75 35.20 1.28 5.60 0.48 8.57 0.90 17.30 0.79 4.57 1.69(MARCH 2008) NO. 9 35.75 35.40 0.99 5.47 0.66 12.07 0.66 16.56 1.17 7.07 1.13NO. 10 4.11 3.55 15.77 5.47 0.66 12.07 NO SIGMA 16.56 1.17 7.07 NO SIGMA(APRIL 2008) NO. 11 26.70 27.30 2.20 4.97 0.59 11.87 0.57 16.1 1.19 7.39 0.92NO. 12 18.95 18.90 0.26 4.97 0.59 11.87 0.74 16.1 1.19 7.39 1.18

AQUAS (JUNE 2007)LEVEL 1 6.07 7.14 15.00 6.01 0.56 9.32 NO SIGMALEVEL 2 21.78 20.35 7.02 18.18 1.01 5.56 0.36(SEPTEMBER 2007) LEVEL 1 6.07 7.14 15.00 5.57 0.78 14.00 NO SIGMALEVEL 2 18.21 19.99 8.93 17.12 1.24 7.24 0.01

PCQACL (SEPTEMBER 2007)NORMAL 7.99 7.29 9.6 5.57 0.78 14.00 NO SIGMAABNORMAL 23.25 20 16.25 17.12 1.24 7.24 NO SIGMA

NRLB (SEPTEMBER 2007)VIAL 1 5.46 6.00 9.00 5.57 0.78 14.00 0.00VIAL 2 17.08 16.70 2.28 17.12 1.24 7.24 0.93

For BUN, 6-sigma values range from as low as 0.00 to 1.88. In nine instances there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa. Moreover, most of the CV’s of the internal QC data are very high (6-14), which will likewise decrease the calculated sigma metric. No sigma values are above 3. The average sigma metric for BUN is 1 (EQAS), 0.37 (AQUAS) and 0.47 (NRLB).

8

TABLE 4. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR CREATININE.

CREATININE with TEa = + 15% EXTERNAL QC DATA INTERNAL QC DATA

SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 256.00 260.00 1.54 189.36 8.75 4.62 2.91 540.96 24.93 4.61 2.92NO. 2 57.00 58.40 2.40 189.36 8.75 4.62 2.73 540.96 24.93 4.61 2.73(DECEMBER 2007) NO. 3 874.00 884.00 1.13 187.20 7.48 4.00 3.47 559.10 14.93 2.67 5.19NO. 4 434.00 484.00 10.35 187.20 7.48 4.00 1.17 559.10 14.93 2.67 1.75(JANUARY 2008) NO. 5 263.00 267.00 1.50 187.48 5.41 2.89 4.68 567.97 8.80 1.55 8.71NO. 6 484.00 465.00 4.09 187.48 5.41 2.89 3.78 567.97 8.80 1.55 7.04(FEBRUARY 2008) NO. 7 880.00 880.00 0.00 199.71 7.27 3.64 4.12 574.14 18.34 3.19 4.70NO. 8 53.00 59.10 10.32 199.71 7.27 3.64 1.29 574.14 18.34 3.19 1.46(MARCH 2008) NO. 9 53.00 57.10 7.18 200.23 11.21 5.60 1.40 563.93 20.11 3.57 2.19NO. 10 889.00 873.00 1.83 200.23 11.21 5.60 2.35 563.93 20.11 3.57 3.69(APRIL 2008) NO. 11 265.00 265.00 0.00 199.80 12.32 6.17 2.43 583.80 22.69 3.89 3.86NO. 12 476.00 460.00 0.83 199.80 12.32 6.17 2.30 583.80 22.69 3.89 3.64



NRLB (SEPTEMBER 2007)VIAL 1 197.00 153.70 28.17 200.11 7.55 3.77 NO SIGMAVIAL 2 572.00 508.00 12.60 578.54 32.31 5.58 0.43

For Creatinine, 6-sigma values range from as low as 0.43 to 8.71 (even higher than 6-sigma). In two instances the sigma metric is greater than 6-sigma, in four instances between 4-5 sigma, and in five instances between 3-4 sigma. In only one instance (NRLB vial 1) is there no calculated sigma value because the bias calculated from the external QC data is greater than the TEa. The findings suggest an excellent method performance for Creatinine since the bias and CV are relatively low to yield higher sigma values. However, it must be considered that the CLIA requirement for Creatinine is very high at + 15% total allowable error leaving

9

TABLE 5. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR URIC ACID.

much room for bias and inaccuracy. The average sigma metric for Creatinine is 3.35 (EQAS), 1.56 (AQUAS), and 2.49 (PCQACL).

URIC ACID with TEa = + 17% EXTERNAL QC DATA INTERNAL QC DATA

SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 0.260 0.276 5.80 0.25 0.01 4.00 2.80 0.53 0.02 3.77 2.97NO. 2 0.160 0.193 17.10 0.25 0.01 4.00 NO SIGMA 0.53 0.02 3.77 NO SIGMA(DECEMBER 2007) NO. 3 0.560 0.553 1.27 0.28 0.01 3.57 4.41 0.56 0.01 1.79 8.81NO. 4 0.380 0.374 1.60 0.28 0.01 3.57 4.31 0.56 0.01 1.79 8.62(JANUARY 2008) NO. 5 0.280 0.277 1.08 0.28 0.02 7.14 2.23 0.56 0.01 1.79 8.91NO. 6 0.380 0.368 3.26 0.28 0.02 7.14 1.92 0.56 0.01 1.79 7.69(FEBRUARY 2008) NO. 7 0.560 0.549 2.00 0.29 0.01 3.45 4.35 0.56 0.10 18.04 0.83NO. 8 0.190 0.193 1.55 0.29 0.01 3.45 4.48 0.56 0.10 18.04 0.86(MARCH 2008) NO. 9 0.200 0.194 3.09 0.35 0.02 5.71 2.43 0.55 0.02 3.64 3.82NO. 10 0.570 0.552 3.26 0.35 0.02 5.71 2.40 0.55 0.02 3.64 3.78(APRIL 2008) NO. 11 0.280 0.281 0.36 0.28 0.01 3.57 4.66 0.55 0.01 1.82 9.15NO. 12 0.370 0.372 0.54 0.28 0.01 3.57 4.61 0.55 0.01 1.82 9.05




For Uric Acid, 6-sigma values range from as 0.72 to 9.15 (greater than 6-sigma). In two instances there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa. Again, the TEa set by CLIA for Uric Acid is generous at + 17%, accounting for

10

TABLE 6. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR TOTAL PROTEIN.

the high sigma values in six instances (7.69-9.15). In six instances the sigma metric is between 4-5-sigma. The average sigma metric for Uric Acid is 4.69 (EQAS), 1.93 (AQUAS), 2.40 (PCQACL) and 1.56 (NRLB).

TOTAL PROTEIN with TEa = + 10% EXTERNAL QC DATA INTERNAL QC DATA




NRLB (SEPTEMBER 2007)VIAL 1 67.00 62.60 7.029 67.89 1.69 2.49 1.19VIAL 2 46.00 41.70 10.31 45.46 1.10 2.42 NO SIGMA

For Total Protein, 6-sigma values range from 1.19 to 5.81. In three instances there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa.

11

TABLE 7. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR ALBUMIN.

In two instances, the sigma value is greater than 5, in four instances, between 4-5 sigma, and in six instances, between 3-4 sigma. The average sigma metric for Total Protein is 3.31 (EQAS), 2.24 (AQUAS) and 2.81 (PCQACL).

ALBUMIN with TEa = + 10% EXTERNAL QC DATA INTERNAL QC DATA

SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 41.00 41.15 0.36 38.08 1.22 3.20 3.01 25.56 0.71 2.78 3.47NO. 2 41.00 46.25 11.35 38.08 1.22 3.20 NO SIGMA 25.56 0.71 2.78 NO SIGMA(DECEMBER 2007) NO. 3 25.00 28.20 11.35 38.73 1.20 3.10 NO SIGMA 26.10 0.55 2.11 NO SIGMANO. 4 35.00 35.29 0.82 38.73 1.20 3.10 2.96 26.10 0.55 2.11 4.36(JANUARY 2008) NO. 5 41.00 41.22 0.53 38.38 0.86 2.24 4.22 25.79 0.62 2.40 3.94NO. 6 35.00 35.20 0.57 38.38 0.86 2.24 4.21 25.79 0.62 2.40 3.92(FEBRUARY 2008) NO. 7 46.00 46.50 1.08 38.60 1.30 3.37 2.65 25.90 0.84 3.24 2.75NO. 8 24.00 23.70 1.27 38.60 1.30 3.37 2.59 25.90 0.84 3.24 2.69(MARCH 2008) NO. 9 24.00 23.80 0.84 39.34 0.77 1.96 4.68 26.31 0.54 2.05 4.46NO. 10 46.00 46.41 0.88 39.34 0.77 1.96 4.66 26.31 0.54 2.05 4.44(APRIL 2008) NO. 11 43.00 41.54 3.51 37.07 0.59 1.59 4.07 24.57 0.82 3.32 1.95NO. 12 37.00 35.09 5.44 37.07 0.59 1.59 2.86 24.57 0.82 3.32 1.37




For Albumin, 6-sigma values range from as low as 0.67 to 4.68. In four instances there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa.

12

TABLE 8. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR CHOLESTEROL.

In nine instances, the sigma value is between 4-5, and in four instances, between 3-4 sigma. The average sigma metric for Albumin is 3.23 (EQAS), 1.10 (AQUAS), 1.52 (PCQACL) and 1.55 (NRLB).

CHOLESTEROL with TEa = + 10% EXTERNAL QC DATA INTERNAL QC DATA




*No values were submitted for Cholesterol for the PCQACL External Quality Control Survey for September 2007.

For Cholesterol, 6-sigma values range from as low as 0.12 to 5.67. In two instances there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa. In four instances, the sigma value is between 4-5, and in seven instances, between 3-4 sigma. The average sigma metric for Cholesterol is 2.98 (EQAS), 1.25 (AQUAS), and 2.28 (NRLB).

13

TABLE 9. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR SODIUM.

SODIUM with TEa = + 2.6% EXTERNAL QC DATA INTERNAL QC DATA

SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 155.00 147.40 5.16 148.80 4.50 3.02 NO SIGMA 126.68 3.39 2.68 NO SIGMANO. 2 143.00 157.40 9.15 148.80 4.50 3.02 NO SIGMA 126.68 3.39 2.68 NO SIGMA(DECEMBER 2007) NO. 3 110.00 110.80 0.72 148.87 2.79 1.87 1.00 126.00 2.07 1.64 1.14NO. 4 139.00 135.90 2.28 148.87 2.79 1.87 0.17 126.00 2.07 1.64 0.19(JANUARY 2008) NO. 5 157.00 149.70 4.88 149.55 2.98 1.99 NO SIGMA 127.10 1.78 1.40 NO SIGMANO. 6 140.00 136.40 2.64 149.55 2.98 1.99 NO SIGMA 127.10 1.78 1.40 NO SIGMA(FEBRUARY 2008) NO. 7 113.00 110.90 1.89 149.76 1.83 1.22 0.58 129.03 1.92 1.49 0.47NO. 8 174.00 160.50 8.41 149.76 1.83 1.22 NO SIGMA 129.03 1.92 1.49 NO SIGMA(MARCH 2008) NO. 9 168.00 158.90 5.73 148.60 2.55 1.72 NO SIGMA 129.03 3.59 2.78 NO SIGMANO. 10 110.00 110.70 0.63 148.60 2.55 1.72 1.15 129.03 3.59 2.78 0.71(APRIL 2008) NO. 11 156.00 149.20 4.56 149.00 2.42 1.62 NO SIGMA 128.57 2.97 2.31 NO SIGMANO. 12 142.00 138.00 2.90 149.00 2.42 1.62 NO SIGMA 128.57 2.97 2.31 NO SIGMA


PCQACL (SEPTEMBER 2007)NORMAL 142.00 137.00 3.65 148.86 3.42 2.30 NO SIGMAABNORMAL 159.00 158.00 0.63 126.50 6.01 4.75 0.41


For Sodium, 6-sigma values range from 0.17 to 1.15. Due to a very stringent CLIA requirement of + 2.6%, no sigma value was calculable in seventeen instances. The rest of the values

14

TABLE 10. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR POTASSIUM.

are between slightly above 0 to slightly above 1. No values reach at least 3-sigma. Taking into consideration the strict CLIA criterion, the findings for Sodium are suggestive of a very difficult quality assurance for this analyte, a very important chemical test in many a patient’s diagnostic work up. The average sigma metric is 0.68 (EQAS), 0.32 (AQUAS), 0.41 (PCQACL) and 0.77 (NRLB).

POTASSIUM with TEa = + 12% EXTERNAL QC DATA INTERNAL QC DATA

SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 2.50 2.39 4.60 4.03 0.11 2.73 2.71 6.02 0.18 2.99 2.47NO. 2 1.20 1.24 3.23 4.03 0.11 2.73 3.21 6.02 0.18 2.99 2.93(DECEMBER 2007) NO. 3 6.00 5.96 0.67 4.00 0.08 2.00 5.66 5.97 0.10 1.68 6.76NO. 4 3.80 3.63 4.68 4.00 0.08 2.00 3.66 5.97 0.10 1.68 4.37(JANUARY 2008) NO. 5 2.60 2.39 8.79 4.04 0.07 1.73 1.85 6.31 0.08 1.27 2.53NO. 6 3.80 3.62 4.97 4.04 0.07 1.73 4.06 6.31 0.08 1.27 5.54(FEBRUARY 2008) NO. 7 6.20 5.56 11.51 4.08 0.08 1.96 0.25 5.98 0.10 1.67 0.29NO. 8 1.40 1.24 12.90 4.08 0.08 1.96 NO SIGMA 5.98 0.10 1.67 NO SIGMA(MARCH 2008) NO. 9 1.30 1.22 6.56 4.93 0.08 1.99 2.74 5.99 0.16 2.67 2.04NO. 10 6.00 5.91 1.52 4.93 0.08 1.99 5.28 5.99 0.16 2.67 3.92(APRIL 2008) NO. 11 2.50 2.36 5.93 4.07 0.05 1.23 4.94 5.99 0.08 1.34 4.54NO. 12 3.90 3.64 7.14 4.07 0.05 1.23 3.95 5.99 0.08 1.34 3.64




15

TABLE 11. SIX SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR CHLORIDE.

Potassium fares better with sigma values ranging from 0.25 to 6.76. In only two instances there is no calculated sigma because the bias is greater than the total allowable error. In one instance, the sigma is greater than 6 (6.76), in four instances, the sigma is between 5-6, in eight instances, the sigma is between 4-5 and in seven instances, between 3-4. The average sigma metric for Potassium is 3.52 (EQAS), 3.59 (AQUAS), 4.74 (PCQACL) and 4.80 (NRLB). Is this acceptable quality? We must consider that the CLIA requirement for Potassium is + 12%.

CHLORIDE with TEa = + 5% EXTERNAL QC DATA INTERNAL QC DATA

SIGMA METRICEQAS PGH MEAN GROUP MEAN BIAS MEAN SD CV (NOVEMBER 2007)NO. 1 119.00 109.30 8.87 110.60 4.26 3.85 NO SIGMA 93.96 3.12 3.32 NO SIGMANO. 2 111.00 109.30 1.56 110.60 4.26 3.85 0.89 93.96 3.12 3.32 1.04(DECEMBER 2007) NO. 3 92.00 88.45 4.01 110.30 2.25 2.04 0.48 93.40 1.89 2.02 0.49NO. 4 109.00 103.40 5.42 110.30 2.25 2.04 NO SIGMA 93.40 1.89 2.02 NO SIGMA(JANUARY 2008) NO. 5 120.00 110.40 8.70 108.79 2.21 2.03 NO SIGMA 92.83 1.56 1.68 NO SIGMANO. 6 110.00 104.00 5.77 108.79 2.21 2.03 NO SIGMA 92.83 1.56 1.68 NO SIGMA(FEBRUARY 2008) NO. 7 127.00 117.90 7.72 110.03 2.98 2.71 NO SIGMA 92.53 2.97 3.21 NO SIGMANO. 8 91.00 88.46 2.87 110.03 2.98 2.71 0.79 92.53 2.97 3.21 0.66(MARCH 2008) NO. 9 94.00 88.50 6.21 109.71 2.71 2.47 NO SIGMA 92.17 1.63 1.77 NO SIGMANO. 10 131.00 119.70 9.44 109.71 2.71 2.47 NO SIGMA 92.17 1.63 1.77 NO SIGMA(APRIL 2008) NO. 11 120.00 110.80 8.30 111.53 2.00 1.79 NO SIGMA 93.33 1.33 1.43 NO SIGMANO. 12 111.00 103.50 7.25 111.53 2.00 1.79 NO SIGMA 93.33 1.33 1.43 NO SIGMA



NRLB (SEPTEMBER 2007)VIAL 1 106.00 101.10 4.85 110.75 2.89 2.61 0.06VIAL 2 93.00 87.30 6.53 94.39 2.50 2.65 NO SIGMA

16

For Chloride, 6-sigma values range from 0.06 to 1.89. In nineteen instances, there is no calculated sigma value because the bias calculated from the external QC data is greater than the TEa. No sigma values are beyond 1.50, suggesting a poor method performance for Chloride. The average sigma value for Chloride is 0.73 (EQAS), 1.17 (AQUAS), 1.53 (PCQACL) and 0.06 (NRLB).

17

DISCUSSION

Physicians use laboratory work-ups in aid of diagnosis and management of patients.

Accuracy of laboratory results is therefore important in assuring and improving the quality of patient

care. The Philippine government is mandated to ensure accuracy and precision of laboratory

examinations to safeguard public health and safety.

Laboratories are thus obliged to regularly perform Internal Quality assurance procedures that

include use of control materials, calibration, result verification, application of Westgard Multirules

and statistical analysis, and equipment maintenance.

At the same time, laboratories are enjoined to participate in External Quality Assurance

Programs in which the quality of laboratory performance is assessed through the closeness of its

results to the pre-determined value or reference value generated by the other participants through peer

group analysis.

If the lab control results for a particular run are “in-control” and do not violate the Westgard

Multirules, is the laboratory performing as well as it should? If the lab satisfactorily passes External

Quality Assurance Programs, is the lab truly consistent and dependable? Corollary to these two

questions: are the results generated by an “in-control” lab reliable and, therefore, truly useful to the

clinician?

Six Sigma Metrics, as an objective and universal tool in measuring process performance,

redefines the concept of quality. Novel–and more stringent–standards are set for methods and

processes to ensure results that are clinically relevant and reliable. A process is deemed to be of

superior or inferior quality by calculating the “Sigma metric” with a standard formula that utilizes

both Internal and External QC data. The once separate (but essentially complimentary) concepts and

methods of Quality Assurance are combined, to yield data that is more reflective of true lab

performance at a particular time.

Table 12 shows a summary of the 6-sigma values calculated for each analyte using the EQAS

data to calculate the bias.

At 4.69 sigma, Uric Acid is at an acceptable Sigma process capability. Creatinine, Total

protein, Albumin and Potassium are within 3-4 sigma. Glucose, Cholesterol, BUN, Sodium and

Chloride all fall below 3-sigma which is unacceptable quality. We must consider of course

differences in the TEa set by the CLIA for each analyte (i.e., the TEa of Uric Acid is at a generous +

17% while the TEa of sodium is strict at + 2.6%).

18

TABLE 12. SUMMARY OF SIX-SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR THE 10 ANALYTES USING EQAS DATA.

SIX SIGMA PERFORMANCE USING EQAS DATA

EQAS GLU BUN CREA UA TP ALB CHOLE NA K CLNOV.2007

NO.1 1.86 0.92 2.91 2.8 2.29 3.01 3.19 NO SIGMA 2.71 NO SIGMA

2.55 1.24 2.92 2.97 2.31 3.47 3.69 NO SIGMA 2.47 NO SIGMA

NO.2 NO SIGMA NO SIGMA 2.73 NO SIGMA NO SIGMA NO SIGMA NO SIGMA NO SIGMA 3.21 0.89NO SIGMA NO SIGMA 2.73 NO SIGMA NO SIGMA NO SIGMA NO SIGMA NO SIGMA 2.93 1.04

DEC.2007NO.3 4.96 NO SIGMA 3.47 4.41 2.75 NO SIGMA 2.78 1 5.66 0.48

3.54 NO SIGMA 5.19 8.81 2.81 NO SIGMA 3.64 1.14 6.76 0.49NO.4 1.85 0.2 1.17 4.31 1.97 2.96 0.54 0.17 3.66 NO SIGMA

3.54 0.37 1.75 8.62 2.01 4.36 0.71 0.19 4.37 NO SIGMA

JAN.2008NO.5 2.44 0.89 4.68 2.23 4.73 4.22 3.82 NO SIGMA 1.85 NO SIGMA

2.97 1.76 8.71 8.91 3.6 3.94 4.72 NO SIGMA 2.53 NO SIGMA


2.36 1.88 7.04 7.69 4.14 3.92 5.67 NO SIGMA 5.54 NO SIGMA

FEB.2008NO.7 3.67 0.72 4.12 4.35 4.32 2.65 3.65 0.58 0.25 NO SIGMA

2.95 1.35 4.7 0.83 3.18 2.75 3.09 0.47 0.29 NO SIGMA

NO.8 3.37 0.9 1.29 4.48 5.81 2.59 1.84 NO SIGMA NO SIGMA 0.792.71 1.69 1.46 0.86 4.28 2.69 1.55 NO SIGMA NO SIGMA 0.66

MAR.2008NO.9 2.00 0.66 1.4 2.43 3.75 4.68 2.92 NO SIGMA 2.74 NO SIGMA

1.60 1.13 2.19 3.82 2.62 4.46 2.91 NO SIGMA 2.04 NO SIGMA

NO.10 2.31 NO SIGMA 2.35 2.4 2.54 4.66 2.62 1.15 5.28 NO SIGMA

1.85 NO SIGMA 3.69 3.78 1.78 4.44 2.61 0.71 3.92 NO SIGMA

APR.2008NO.11 1.59 0.57 2.43 4.66 3.72 4.07 3.89 NO SIGMA 4.94 NO SIGMA

1.81 0.92 3.86 9.15 2.93 1.95 1.35 NO SIGMA 4.54 NO SIGMA


1.46 1.18 3.64 9.05 2.57 1.37 1.50 NO SIGMA 3.64 NO SIGMA

AVERAGE 2.48 1.00 3.35 4.69 3.31 3.23 2.98 0.68 3.52 0.73

19

TABLE 13. SUMMARY OF SIX-SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR THE 10 ANALYTES USING AQAS DATA.

TABLE 14. SUMMARY OF SIX-SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR THE 10 ANALYTES USING PCQACL DATA.

TABLE 15. SUMMARY OF SIX-SIGMA PERFORMANCE OF UP-PGH CENTRAL LABORATORY FOR THE 10 ANALYTES USING NRLB DATA.

Table 13 shows a summary of the 6-sigma values calculated for each analyte using the

AQUAS data to calculate the bias.

SIX SIGMA PERFORMANCE USING AQUAS DATA

AQUAS GLU BUN CREA UA TP ALB CHOLE NA K CLJUN.2007 LEVEL 1 1.62 NO SIGMA 0.71 0.72 2.08 1.85 0.12 0.33 3.23 1.43LEVEL 2 1.63 0.36 1.37 2.35 2.44 0.56 0.98 0.16 2.58 1.18

SEP.2007 LEVEL 1 0.27 NO SIGMA 1.94 2.93 2.07 1.13 2.33 0.51 3.79 1.16LEVEL 2 1.09 0.01 2.23 1.71 2.37 0.84 1.58 0.28 4.77 0.92

AVERAGE 1.15 0.37 1.56 1.93 2.24 1.10 1.25 0.32 3.59 1.17

The UP-PGH Central Laboratory’s performance using AQUAS data is unimpressive by the Sigma

standards with Sigma values below 1 (BUN and Sodium), the highest at 3.59 (Potassium) and the rest

between 1-3 sigma.

Tables 14 and 15 show a summary of the 6-sigma values calculated for each analyte using the

PCQACL and NRLB data, respectively to calculate the bias.

SIX SIGMA PERFORMANCE USING PCQACL DATA

PCQACL GLU BUN CREA UA TP ALB CHOLE NA K CLSEP.2007 NORMAL 2.29 NO SIGMA 2.59 3.22 3.1 0.88 NO RESULT NO SIGMA 4.31 1.16

ABNORMAL 3.98 NO SIGMA 2.38 1.57 2.51 2.15 NO RESULT 0.41 5.17 1.89AVERAGE 3.14 ---- 2.49 2.40 2.81 1.52 ---- 0.41 4.74 1.53

SIX SIGMA PERFORMANCE USING NRLB DATA

NRLB GLU BUN CREA UA TP ALB CHOLE NA K CLSEP.2007

VIAL 1 0.7 0 NO SIGMA 2.43 1.19 2.43 2.84 1.04 4.82 0.06VIAL 2 1.86 0.93 0.43 0.69 NO SIGMA 0.67 1.71 0.5 4.77 NO SIGMA

AVERAGE 1.63 0.47 0.43 1.56 1.19 1.55 2.28 0.77 4.80 0.06

Again, we note that the average performance of the analytes (except for Potassium) falls

below of the acceptable quality of 4-5 sigma and world class quality of 6-sigma using the two local

proficiency testing survey results.

20

Internal QC data generated by the UP-PGH laboratory itself in its daily operations is a major

contributor to its poor sigma performance. A review of Tables 2 to 11 will show that the Coefficients

of Variation for the analytes at various points in time and at various levels of analyte testing are high

and far from the ideal CV of 1.

Short of changing analyzers and/or reagents/control materials, which entails substantial cost

to the hospital, are there any realistic solutions to such a low Sigma performance in the UP-PGH

Central Laboratory?

Westgard urges that for a low quality performance such as that measured in our institution,

Internal QC procedures should be maximized, utilizing all of the Westgard Multirules in order to

detect laboratory errors and a minimum of 2-4 control measurements per run when using two control

materials is recommended. Our findings thus strengthen the obvious need to emphasize the

importance of regular and properly implemented QC practices. Participation in External Quality

Assurance Programs is likewise a relevant practice that is necessary in order to provide the laboratory

with an assessment of its performance compared with its peers.

Conscious efforts must be made to reduce the CV (taken from Internal QC data) and the Bias

(taken from External QC data) through more frequent instrument function checks, calibrations and

preventive maintenance. Other “non-statistical” Quality Control strategies should be employed and

these include: selection of appropriate standards, improving calibration procedures, standardizing

operator techniques, mechanizing manual steps in the process, and increasing operator training and

expertise. Moreover, with frequent assessment of laboratory performance using Six Sigma,

information on the performance of different methods and instruments will allow both laboratory

managers and manufacturers more informed selection and marketing decisions.

Local QC groups (e.g., PCQACL) should focus on formulating and promulgating a US CLIA

88 equivalent which will be adhered to by accrediting and licensing agencies in our country. This

may give even more meaningful interpretation of performance as the tolerance specifications that

shall be set for all analytes shall be applicable to and relevant in our setting. Future local studies on

Six Sigma principles may be made on the pre-analytic and post-analytic phases of laboratory testing.

Application of Six Sigma may be done with data gathered from other sections of the laboratory (e.g.,

Hematology, Immunology, Microbiology, et cetera). Prior to purchase and commercial use, laboratory

analyzers may be subjected to Six Sigma in order to determine its true analytic performance.

21

CONCLUSION

By telling us the level of quality of our laboratory performance, calculating the Sigma Metric is

the first step towards improving quality. Moreover, it emphasizes the need for more Quality Control

procedures and the importance of participating in External Quality Assurance Programs.

The analytic quality of ten (10) common biochemical tests (Glucose, BUN, creatinine, uric acid,

total protein, albumin, cholesterol, sodium, potassium and chloride) performed at the Chemistry

Section of the UP-PGH Central Laboratory was determined using Six Sigma principles. With a

benchmark for world class quality of 6-Sigma and acceptable process quality of 4-5 Sigma, only Uric

Acid testing achieved an acceptable Sigma metric at 4.69 using EQAS data. Creatinine, Total

protein, Albumin and Potassium testing are within 3-4 Sigma. Glucose, Cholesterol, BUN, Sodium

and Chloride testing all fall below 3-Sigma which is unacceptable quality. Using AQAS data, no

analyte tested above 4-Sigma. Using PCQACL and NRLB data, only Potassium is within 4-5 Sigma.

While the eventual Sigma value derived from the equation is dependent on how high or low the

tolerance specifications are (which are actually standards set by the US for their own use), the results

of this study do suggest significant room for improvement.

Several recommendations on how to improve the present quality of laboratory testing at the UP-

PGH were made as well as possible directions of further investigations using Six Sigma principles.

As a preliminary study, it is hoped that the conclusions from this research will draw more attention to

the issues of quality assurance in the country because the reliability and accuracy of the laboratory’s

output will impact heavily on patient diagnosis and management.

22

BIBLIOGRAPHY

1. Department of Health. Administrative Order No. 2007-0027. Revised Rules and Regulations Governing the Licensure and Regulation of Clinical Laboratories in the Philippines. Http://www.doh.ph.gov. 2007.

2. Westgard, James O., Ph.D.; Klee, George G., M.D., Ph.D. Quality Management. Henry’s Clinical Diagnosis and Management by Laboratory Methods, 21st Edition. 2007; 19: 485-523.

3. Westgard, J.O. Six Sigma Basics Process Improvement, Goals, and Measurements. Http://www.westgard.com.

4. Westgard, J.O. Six Sigma Quality Management and Desirable Laboratory Imprecision. Http://www.westgard.com.

5. Westgard, J.O. Six Sigma Basics: Outcome Measurement of Process Performance. Http://www .westgard.com.

6. Westgard, J.O. Six Sigma Quality Management and Requisite Laboratory QC. Http://www.westgard.com .

7. Westgard, J.O. AACC Expert Access: A Primer for Six Sigma. Http://www.westgard.com

8. Nevalainen, D.; Berte, L; Kraft, C; et al. Evaluating Laboratory Performance on Quality Indicators with the Six Sigma Scale. Arch Pathol Lab Med. 2000; 124:516-519.

9. Westgard, James O., Ph.D. and Westgard, Sten A., MS. The Quality of Laboratory Testing Today: An Assessment of Sigma Metrics for Analytic Quality Using Performance Data from Proficiency Testing Surveys and the CLIA Criteria for Acceptable Performance. American Journal for Clinical Pathology. 2006; 125.

10. Riebling, Nancy, M.B.B., M.S., M.T. (ASCP) and Tria, Laurel, M.S., S.C. (ASCP). Six Sigma Project Reduces Analytical Errors in Automated Lab. Http://www.mlo-online.com. 2005: 20-23.

11. Westgard, JO.; Basic Method Validation. Chapter 12: The Decision on Method Performance in Basic Method Validation. Madison, WI: Westgard QC, Inc., 1999: 125-134.

12. Westgard, JO, and Burnett, R.W.; Precision requirements for cost-effective operation of analytical processes. Clin Chem 1990;36:1629-1632.

13. Koch, D.D.; Oryall, J.J., Quam, E.F.; Feldbruegge, D.H.; Dowd, D.E.; Barry, P.L.; Westgard, J.O. Selection of medically useful quality-control procedures for individual tests done in a multitest analytical system. Clin Chem 1990;36:230-233.

14. Westgard, J.O.; Sechafer, J.j.; Barry, Patricia L. Allowable Imprecision for Laboratory Tests Based on Clinical and Analytical Test Outcome Criteria. Clin Chem 1984;40:1909-1914.

15. Rej, R.; Jenny, R.W. How Good Are Clinical Laboratories? An Assessment of Current Performance. Clin Chem 1992;36:1210-1217.

23

http://www.mlo-online.com/

http://www.westgard.com/

http://www/

http://www.doh.ph.gov/

6 Sigma Final Draft

Documents

Transcript of 6 Sigma Final Draft