INTERNAL QUALITY CONTROL

Shewhart or Levey- Jennings control charts

In a stable test environment the distribution of the results of the same sample analyzed a number of times is a Gaussian distribution

Deviation indicates a systematic error The levey-Jennings control chart is derived from the Gaussian

distribution indicating the mean and the one, two, and three standard deviation ranges on both sides of the mean

The chart illustrates the relationship between the levey-Jennings control chart and the Gaussian distribution from which it is derived

This figure also shows the expected percentage of results that should fall within each of the standard deviation ranges

Shewhart or Levey- Jennings control charts

Run Number/ Date

Shewhart or Levey- Jennings control charts

On the control chart used to evaluate the result of the control runso the dates or number of analyses are listed along the X-axis o and the values of the control along the Y- axis

The mean and the 1,2 and 3 standard deviation (s) limits of the control analysis to date are marked

As data is obtained it is plotted one point at a time along the chart

Shewhart or Levey- Jennings control charts

oIn a random distribution and in a correctly operating test system approximately: o 65% of the values will be between the

± 1s ranges and will be evenly distributed on either side of the mean

o 95% of the values should fall between the ± 2s ranges

o and 99% between the ± 3s limits

This means that one data point in 20 should be placed between either of the 2s and 3s limits

One data point placed outside of the 3s limits will occur once in 100 analyses

Shewhart or Levey- Jennings control charts

The ± 2s limits are considered as warning limits A value between the 2s and 3s limit indicates the analysis

should be repeated The ± 3s limits are rejection limits. Analysis should stop,

patient results held and the test system investigated

Run Number/ Date

Inspecting the pattern of plotted points can lead to the detection of increased random error and shifts and/or trends in calibration or in the analysis process

Example of a shift in calibration can be failure in recalibration when lot numbers of reagents changed during analytical run

A trend can start on one side of the mean and move across it or it can occur entirely on one side of the mean

Trends can be caused by the deterioration of reagents, pump tubing or light sources in instruments

Occurrence of shifts and trends is the result of either proportional or constant systematic error

9

Westgard Multi-rules Analysis

Two levels of control at different concentrations will be more efficient in monitoring the method when evaluated statistically

By running and evaluating the results of two controls together, trends and shifts can be detected much earlier

Westgard and associates have formulated a series of so called “multi-rules” to help evaluate results from Gaussian distribution both within a level of control and between control levels

Violation of any of these rules individually or in any combination indicates an error

Modified shewhart control charts for use with multi-rules

10

11

Westgard Multi-rules

•12S rule

•13S rule

•22S rule

•R4S rule

•41S rule

•10X rule

12

Westgard Multi-rules, 12S Rule

One of two control results falls outside ±2s Warning Rule – does not cause rejection of a run Alerts technologist to possible method or instrument

malfunction

13

Westgard Multi-rules, 13s Rule When the control limits are set as the mean ± 3s A run is rejected when a single control measurement exceeds

the mean plus 3s or the mean minus 3s control limit

14

Westgard Multi-rules, 22S Rule

22s - reject when 2 consecutive control measurements exceed the same mean plus 2s or the same mean minus 2s control limit

Level 1

Level 2

15

Westgard Multi-rules, R4S Rule

R4s- reject when 1 control measurement in a group exceeds the mean plus 2s and another exceeds the mean minus 2s

Level 1

Level 2

16

Westgard Multi-rules, 41S Rule

41s- reject when 4 consecutive control measurements exceed the same mean plus 1s or the same mean minus 1s control limit

Level 1

Level 2

17

Westgard Multi-rules, 10x Rule

10x- reject when 10 consecutive control measurements fall on one side of the mean

Level 1

Level 2

18

Random or Systematic

13S and R4S usually associated with random error

22S, 41S, and 10X most often associated with systematic errors

M

1SD

2SD

3SD

M

1SD

2SD

3SD

-1SD

-2SD

-3SD

-1SD

-2SD

-3SD

Level 2 Glucose

Level 1 Glucose

21

Use of Patient Samples as Controls

Using of patient samples for quality control is

acceptable because:

1. The physical and chemical differences between control

material and patient samples are absent

2. The data being generated by the analysis of the patient

samples require no extra effort to obtain for use as

quality control data

3. Inexpensive method of quality control

22

Absurd Values Values that are beyond reference intervals or are not

consistent with the survival of life. Results should be checked by repeat analysis on a fresh

aliquot of the same sample or by an alternate method. The values are usually the result of a random error or

mistake such as short sampling, clot formation. Comparison of absurd values to the clinical picture of the

patient provides validity of the result.

EXTERNAL QUALITY CONTROL TESTING:

PROFICIENCY SURVEYS

24

1- Proficiency Surveys

In a proficiency survey, a laboratory is compared to other laboratories with the same or similar methodology by the analysis of identical samples sent to each of the participants.

Upon receiving the survey specimen, the participant prepares it according to the provided instructions and analyzes it as a routine sample

The results and information concerning the method of analysis are recorded on a provided report form and mailed back to the supplier of the survey.

25

1- Proficiency Surveys For the evaluation, results for each sample are grouped by

analyte and methodology The performance of the group is determined either by

calculating a mean result and standard deviation for each specimen or by determining the consensus or majority opinion

The performance of the in dividual laboratories that comprise the group is evaluated by comparison to the group performance

The mean or consensus result for a survey specimen is considered the accurate and true value for that sample

The closer the in dividual laboratory result is to the survey mean or consensus, the more ac curate it is

26

1- Proficiency Surveys

CAP and similar surveys are designed to accomplish the following goals:

1. To assess the current state of the art of laboratory medicine

2. To provide information to help laboratory directors select the best meth ods and reagents

3. To satisfy the accreditation requirements

4. To provide a voluntary educational peer comparison program that allows a laboratory to compare its performance with its peers

27

A- Survey Samples The proficiency survey samples sent to participating

laboratories are very different from patient samples The rigors of shipping materials great dis tances and the need

for a large batch of sample to send to all of the partici pants does not permit unstabilized or unfixed specimens to be used

Profi ciency samples include: o lyophilized serum, plasma, urine, spinal fluid, stabilized blood

cells, ampules of whole blood for blood gas analysis, lyophi lized cultures, photomicrograph transparencies, and stained blood films for identification

Therefore, survey specimens may be different physically and chemically from patient samples, and are not subjected to collection and processing procedures

28

A- Survey Samples These differences prevent proficiency surveys from accurately

reflecting the overall performance of the laboratory. The preparation of survey material by the laboratory is critical The speci mens must be prepared according to instructions to

reduce interlaboratory variation Exact details for the reconstitution of lyophilized samples are

given, and sometimes the diluent is provided Once the sample is ready for analysis it should be treated as a

routine sample The questionnaire or report form must also be filled out correctly Mistakes made in filling out these forms would be interpreted as

analytical errors and may lead to time wasted in trying to investigate a bad result

29

B- Evaluation of Data from Quantitative Procedures Quantitative procedures are evaluated statistically by

comparing the individual laboratory's results to its peer group

A peer group is defined as a minimum of 20 laboratories having the same or similar methodology or instrument in common

When fewer than 20 laboratories are in a peer group, the group is evaluated against a chosen comparative method

A comparative method is not always a reference method but one that has historically performed well and has good agreement with most of the common methods.

30

B- Evaluation of Data from Quantitative Procedures The mean and standard deviation(s) of the peer group is

calculated Outliers beyond three standard deviations are excluded and the

mean and standard deviation is calculated a second time The second mean and standard devia tion is used to calculate

the Standard Deviation Index Mistakes such as reporting the wrong values for the wrong

samples and the misplacing of decimal points are common transcription errors that can exert undue influence on the calculation of the mean and standard deviation statistics

Their elimination will result in a more accurate picture of the peer group.

31

B- Evaluation of Data from Quantitative Procedures

The Standard Deviation Index (SDI) describes how far from the mean an individual laboratory's result falls in a peer group survey

The closer to the mean or the smaller an individual laboratory's SDI, the more accurate the result

The mean is accepted as the true value of the specimen All of the results for a survey are grouped by sample,

analyte and methodology The mean and standard deviation is calculated for each

peer group.

32

B- Evaluation of Data from Quantitative Procedures The SDI is calculated as follows:

A chart showing the SDI of successive survey specimens will graphically display the laboratory's performance over several separate shipments or sur vey samples

A cumulative SDI chart similar to the one illustrated in Figures 8-1 and 8-2 demonstrates random and systematic analytical error by the pat tern of the SDI values for successive surveys

SDI = Individual Lab Result — Group mean

Group Standard Deviation

33

B- Evaluation of Data from Quantitative Procedures

If a laboratory is consistently below the mean, the SDI will be a series of negative numbers

If the labora tory is consistently above the mean, the SDI will be a series of positive values

All SDIs beyond the ±2 SDI are unacceptable and need to be investigated

Values falling in the ± 1.5 to ±2.0 SDI range, though acceptable, are flagged as a warning of the beginning of a possible trend or shift.

34

35

36

C- The Youden Plot In 1964 Dr. W. J. Youden proposed a method of evaluating precision

as well as accuracy in chemistry and hematology analyzers and manual analytical methods using paired survey samples

The technique involves sending two samples to participating laboratories

One sample would have a low concentration, the other a high concentration

The results are grouped by analyte and methodology and a mean and standard deviation is calculated for each sample

A control chart is prepared so that one sample is on the x-axis and the other on y-axis

Each axis is scaled so that 1 standard deviation is the same size on each coordinate and the mean value for each sample is at the midpoint

37

C- The Youden Plot The ± 3s (from the mean) limit for each sample is marked A line perpendicular to the axis is drawn from the mean for

each sample A circle with a radius = 3s is drawn where the two

perpendicular lines meet A 45o line is drawn from the X-Y intercept, through the center

of the circle, and beyond (Figure 8.3) After the Youden Plot is constructed, the individual

laboratories are plotted, using the results of the paired samples This shows the distribution of the participants of that survey

38

39

C- The Youden Plot The closer an individual lies to the junction of the two

sample means, the more accurate its performance Interpretation of the plot is as follows:o Laboratories falling near the intersection of the means and

within the circle are considered accurate. The closer to the center, the more accurate the result

o Laboratories that fall either high or low along the 45o line or between lines parallel to it and tangent to both sides of the circle are indicative of systematic error, that is, the method is precise but not necessarily accurate

o Laboratories outside of the 45o parallel tangent lines are of random error

40

C- The Youden Plot An easier to construct, modified form of the Youden Plot

can be made by using uniform scales on both axes and drawing rectangle that encompasses the area within the 3s limits of both samples (Figure 8.4 and 8.5)

The 45o line is drawn in but not the parallel tangent lines. Interpretation is similar to the traditional form

42

43

44

D- Qualitative Procedures For qualitative procedures a "consensus method" or referees are

used for evaluating individual results. A consensus method is one that has been chosen by either the

majority of the participants or the survey committee as the best standardized and easily performed method for the analysis.

The use of referees is most common in the identification of blood cells, urine sediment, and microorganisms.

These individuals or laboratories receive samples, as do the other participants, and submit their results.

The consensus of a group of referees is designated as the acceptable identification.

Accurate and acceptable performance is in agreement with the referees or the consensus method.

45

E- Survey Evaluation by the Participants

The survey summaries are returned to the participants for evaluation of their performance.

The laboratory director or his designee, preferably the section head, should review the survey report and initiate an investigation of all unacceptable results.

The findings of the investigation are to be documented, signed by the laboratory director, and retained with the survey report as part of a permanent record of the laboratory's participation

46

Possible causes of unacceptable proficiency survey performance1. Transcription errors made when filling out the report form.

2. Improper or incomplete reconstitution of lyophilized survey samples.

3. Inaccurate identification of cells, microorganisms, or formed urine sediment or transparen cies or stained slides.

4. Inaccurately calibrated analyzer.

5. Analytical test procedure error, i.e., short sampling, timing error, loss of precision due to sample clot, or poorly maintained instrument.

6. Failure to provide a suitable environment for the growth of microorganism.

7. The use of reagents and solutions that have lost reactivity and sensitivity.

8. Mixing up of paired or multiple samples during analysis.

9. Damage to survey material during shipping, i.e.. exposure to excessive heat, or rough han dling resulting in physical damage

47

F- Regional Quality Control Programs

The manufacturers of commercial control material offer a form of external quality control through the use of regional quality control programs.

A labora tory buys sufficient control product to last for a year or longer.

Each month the laboratory submits to the manufacturer's regional computer center either the results of all of its control runs grouped by control lot number and analyte or a summary of the control runs consisting of the monthly mean, standard deviation and the number of control runs for each analyte.

The computer center calculates a monthly mean, standard deviation, and coefficient of vari ation (CV) for the peer group that consists of all laboratories with the same method and lot number of control for each analyte.

48

Regional Quality Control Programs

Comparison among the members of the peer group may be either by SDI or Youden Plots.

The SDI or the Youden Plots indicate where the laboratory stands with respect to the other members of the group and the group mean.

Assuming that the mean value of each group is the accurate value for that control material the accu racy of each individual laboratory can be inferred by its position with respect to the mean.

For laboratories who submit all of their data to the computer center a monthly mean s, and CV of each analyte is calculated.

All participating laboratories receive the lot-to-date cumulative mean, s, and CV for each analyte in that lot of control. This summary data is returned to the participat ing laboratory for evaluation of its precision by the laboratory director.

49

Regional Quality Control Programs

Regional quality control programs do not have the number of participants that the larger CAP and other survey programs are able to attract

The vendors of commercial controls offer regional programs and statistical evaluation as an enhancement to purchase their product.

Because of fewer participants each group may or may not be classified by methodology or instrumentation.

An advantage of the regional programs is that the data is evaluated more frequently, on a monthly basis, and not quarterly, as with most surveys.

50

Fallacies Of External Quality Control

Good performance on proficiency surveys and in regional quality control programs does not necessarily indicate a proficient laboratory

If proficiency survey samples receive special treatment, a true picture of the laboratory's accuracy and analytical error is not shown

Nor do these samples reflect any problems with collection, sample processing, instrument maintenance programs, transcription errors, or the rapport the laboratory has with the users of its services.

The use of the results of the external quality control samples should be placed in proper perspective as an important but partial picture of the laboratory's performance.