Analyzing & Presenting Performance Improvement (PI) Data

ObjectivesDemonstrate an exercise that uncovers how leaders make managerial decisions based upon data

Identify barriers to effective analysis and reporting of PI data

Share 2 data analysis/reporting educational tools targeted for staff

Provide sample PowerPoint slides for staff training re: data analysis and process variability

Discuss PI information needs of leadership

CSR ©2011

1 2 3

CSR ©2011

Why aggregate and analyze?Transform data into information

Identify current performance levels, patterns, or trends

Determine

Whether or not improvement is needed

If a strategy to stabilize or improve performance was effective

If design specifications met

Judge a particular process’s stability or a particular outcome’s predictability in relation to performance expectations

Problem #1Lumping data together is usually not

enough! Aggregate #’s do not show any

“unusual” circumstances. If leaders take action based on data

assumptions without taking into account unusual circumstances – they can muck up a perfectly good process!

Time to work each day

Minutes

Should I change the route to work each day?

September’s Rates –Minutes to work

0

50

100

150

200

250

300

350

400

450

Problem #2Before and after measures aren’t enough! Two aggregate measures taken before and

after a change do not in themselves demonstrate that a process has improved.

One needs to know the stability of the processes that produced these aggregate measures.

To determine process stability, it is necessary to look at data over time i.e., in a time series design.

Staff Turnover

Intervention Begins 10/2009

Staff Turnover – the same data using the # of staff over time!

Intervention Begins 10/2009

Sheward & Deming Points

Variation exists in all we do Processes that exhibit common causes of variation

are predictable within statistical limits Processes often have both common and special

cause variation How we respond to special causes is different than

our response to common cause variation Attempting to improve processes that contain special

causes will increase variation and waste resources Once special causes have been “eliminated”, it is

appropriate to consider changing the process

Common vs. Special Cause Variation

Common Cause Is inherent in the design of the process. Is due to regular, natural, or ordinary causes. Results in a stable process. The variation is predictable. Also known as random or unassignable causes.

Special Cause Is due to causes not inherent in a process. Results in an unstable process, because the variation is not

predictable. Also know as non-random or assignable causes. Might be described as a “signal” that the process has changed.

CSR ©2011

Neither type of variation is “good” or “bad” in

itself!Common Cause Only tells you that a process is stable and predictable within

certain limits However, it may be functioning at an unacceptable level!

Special Cause Usually undesirable when you did not plan for it. Can also be a “signal” that a planned change was effective.

When people do not understand variation

See trends where there are no trends Blame and give credit to others for things

over which they have little or no control Build barriers, decrease morale, and create

an atmosphere of fear Never be able to fully understand past

performance, make predictions about the future and make significant improvements in processes

How Do We Analyze Variation?

Run charts and control charts are the tools used to determine whether variation is:Common cause, orSpecial cause

They tell us what the process is actually doing –Not what we would like it to do!

Bar Graphs

A preliminary exploration of datamay be time-

ordered

Are a common graphical display format

Can be difficult for trend detection

20

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

0

200

400

600

800

1000

1200

1400

2009

2010

Same bar graph data displayed in a simple Line Graph

Offers a preliminary view of time ordered data

Stock market trends are viewed in line graphs

Common mistake is to see trends where they statistically don’t exist

21

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY JUN

400

500

600

700

800

900

1000

1100

1200

1300LINE GRAPH

Run Chart

Used to detect common cause vs. special cause variation

Easy to construct and evaluate

Less sensitive than control charts for identifying extreme data points as a special cause

22

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY JUN

400

500

600

700

800

900

1000

1100

1200

1300RUN CHART: Monthly Calls Received

1999 2000

MEDIAN

"COMMON CAUSE VARIATION"

Run Chart = Line Graph + Center Line*

23

*The center line in a run chart is typically the median point for the data, but some use the process average or mean as the center line

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY JUN

400

500

600

700

800

900

1000

1100

1200

1300

RUN CHART: Monthly Requests for Services

1999 2000

MEDIAN

"COMMON CAUSE VARIATION"

Run Chart Terminology

RUNDefined as one or more consecutive data

points occurring on the same side of the center line

TRENDDefined as an unusually long series of data

points steadily increasing or decreasing

24

EASY Run Chart Tests for Special Causes

TREND of 6 consecutive data points steadily increasing or decreasing

RUN of 8 consecutive data points on one side of the center line (median or mean)

OUTLIER POINTS – use your judgment whether to expend resources to investigate further to understand cause and determine if improvement is needed

25CSR ©2011

1

10

Feb March April May June July Aug Sept Oct Nov Dec

2Median

Test #1 TREND of 6 consecutive data points steadily increasing or decreasing

Sequential Run of only 5 –

Common Cause Variation

1

10

Feb March April May June July Aug Sept Oct Nov Dec

2Median

Test #2 RUN of 8 consecutive data points on one side of the center

line (median or mean)

Run of 9 –

Special Cause Variation

median

Test #3OUTLIER POINTS – use your

judgment whether to expend resources to investigate further

to understand cause and determine if improvement is

needed

Is May’s result a special cause????

Improvement Strategies: After making a run or control chart, what’s next?

29

The type of variation determines your approach:

Special cause variation?If negative, eliminate it.If positive, emulate it.But don’t change the process!

Common cause variation?If process is functioning at an unacceptable level, change the process!Don’t “tamper” with individual data points!

How will you know your intervention is a success?

30

A Special cause in the desired direction will signal that the old process is changed for the better.

A Special cause in the wrong direction will indicate that your intervention was counterproductive.

Continued common cause variation will indicate that your intervention did not help – but did not hurt either.CSR ©2011

Improvement Strategy

31

Time 1 Time 2 Time 3 Time 4

Conduct Initial

Investigation

Standardize

The

Process

Introduce Improvement - 1

Introduce Improvement - 2

Targeting Your MessageHospital boards should hold accountable and require full and complete explanations from management when safety and quality performance levels differ significantly from national benchmarks or fall below expectations, with specific attention devoted to the organization’s plan for improvement (e.g., its development, performance expectations, and the basis on which expectations are established).

  Hospital Governing Boards and Quality of Care: A Call to Responsibility. Washington, DC: National Quality Forum; 2004.

Leadership should….• Create alignment between organizational strategy, measures, and improvement projects• Unify Quality Improvement Plans, Strategic Plans, and Financial Plans within the organization• Ensure that the daily work of employees is organized to support deployment of strategies and improvement projects chosen because of their direct impact on system-level measures or direct support of strategic objectives. Leaders should then implement, monitor, and revise the strategy as needed if the desired changes are not occurring.

Botwinick L, Bisognano M, Haraden C. Leadership Guide to Patient Safety. IHI Innovation Series white paper. Cambridge, Massachusetts: Institute for Healthcare Improvement; 2006. (Available on www.IHI.org)

Use of Lean/6 SigmaPotential Topics to Report:• Voice of the customer, suppliers and process workers• Key critical customer requirements• Outputs which are “Critical to Quality” (CTQ)• Rating of relationship between the process steps [inputs] to the customer requirements• Current Process Controls Prevention• Current Process Controls Detection• FMEAs – findings about severity, occurrence and detection• Description of standard work – current and future states (i.e., value stream map)

Questions?

Richard Scalenghe, CPQHrscalenghe@jcrinc.com

630-740-7914