1 The Accreditation Process Performance Measurement and the
ORYX Initiative (PM)
Slide 2
2 Speaker Information Frank S. Zibrat Associate Director,
Accreditation Systems Integration and ORYX, Division of
Accreditation and Certification Operations The Joint
Commission
Slide 3
3 Performance Measurement and the ORYX Initiative (PM)
Slide 4
4 Overview The use of performance measures and performance
measure data are essential to the credibility of any modern
evaluation activity. The Joint Commissions ORYX initiative
integrates the use of performance measurement data into the
standards-based survey and accreditation process. The use of ORYX
performance measure data in the survey process supplements and
helps guide that process by providing a more targeted basis for the
regular accreditation survey. The Joint Commissions ORYX
performance measurement requirements also are intended to support
Joint Commission accredited hospitals in their quality assessment
and improvement efforts through the continuous monitoring of actual
performance, and by helping guide and stimulate continuous
improvement.
Slide 5
5 The Continued Role of ORYX Hospitals and The Joint Commission
use ORYX performance measure data to continuously assess key
performance areas ORYX data reported by hospitals provide surveyors
with information on hospital performance in important care,
treatment, and services areas and the hospitals ability to effect
change in clinical processes. ORYX core measure data are: One of
the key elements included in the Priority Focus Process (PFP) (see
The Accreditation Process [ACC] chapter, page ACC-31) Incorporated
into the hospitals quarterly Strategic Surveillance System (S3)
report Displayed on The Joint Commissions Quality Report (see The
Joint Commission Quality Report [QR] chapter) ORYX core measure
data also are applied in the accreditation process (see Use of
Performance Measure Data on page PM-8.)
Slide 6
6 The Continued Role of ORYX (Continued) Core measure
information displayed on The Joint Commissions Quality Check
Provides a wide array of audiences with valuable information on
hospital performance on the core measures Provides Joint
Commissionaccredited hospitals with the opportunity to distinguish
themselves among other hospitals based upon their performance on
the core measures and measure sets. A hospital can also use ORYX
data in intra-cycle, continuous performance improvement activities
to proactively identify potential opportunities for
improvement.
Slide 7
7 The Continued Role of ORYX (Continued) To help hospitals
prepare for performance measurement in the new health care
environment, The Joint Commission will continue to expand the scope
of ORYX core measures: Focus on accountability measures Quality
measures that meet four criteria designed to identify measures that
produce the greatest positive impact on patient outcomes when
hospitals demonstrate improvement. Increased emphasis on
organization performance on accountability measures and the
integration of performance expectations on accountability measures
into accreditation standards; Hospitals will not be required to
directly address ORYX accountability measure performance in
intracycle activities Hospital review of its ORYX accountability
measure performance can help identify potential opportunities for
improvement related to standards. To learn more about
accountability measures, go to the Joint Commissions website at:
http://www.jointcommission.org/accountability_measures
http://www.jointcommission.org/accountability_measures
Slide 8
8 Current Requirements for Hospitals* Accredited hospitals with
an average daily census of greater than 10 inpatients: Must select
the currently required minimum number of core measure sets and/or
noncore measures from Joint Commission listed vendors Notify The
Joint Commission of their chosen core measure set(s) and/or noncore
measures and the associated vendor For initial surveys, data
collection must begin on the first day of the first calendar
quarter following survey. * For current ORYX requirements for
critical access hospitals, please refer to the Performance
Measurement and the ORYX Initiative chapter in the Comprehensive
Accreditation Manual for Critical Access Hospitals.
Slide 9
9 Current Requirements for Hospitals* Hospitals must select one
of the following: If the hospital serves patient populations with
conditions that correspond with four or more core measure sets, the
hospital must select a minimum of four of the available measure
sets and submit data for all the applicable measures in the measure
set via its selected vendor. If the hospital serves patient
populations with conditions that correspond with only three core
measure sets, the hospital must collect data on all of the
applicable measures in the three core measure sets along with data
on three noncore measures. Core and noncore measure data must be
submitted via the selected vendor.
Slide 10
10 Current Requirements for Hospitals* (Continued) Hospitals
must select one of the following: If the hospital serves patient
populations with conditions that correspond with only two core
measure sets, the hospital must collect data on all of the
applicable measures in the two core measure sets along with six
noncore measures. Core and noncore measure data must be submitted
via the selected vendor. If the hospital can only identify one core
measure set related to its patient population, it must collect data
on all the applicable measures in that core measure set along with
nine noncore measures. Core and noncore data must still be
submitted via the selected vendor.
Slide 11
11 Current Requirements for Hospitals* (Continued) Hospitals
must select one of the following: A hospital that cannot identify
any applicable core measure sets must collect and transmit data on
nine noncore measures via its selected vendor. To find a complete
list of vendors, go to the Joint Commissions website at:
http://www.jointcommission.org/Core_Systems_List
Slide 12
12 Requirements for Small Hospitals A number of modifications
apply to small hospitals: A hospital with an average daily census
of 10 or fewer inpatients is exempt from the requirement to
transmit data via a listed vendor to The Joint Commission. Must
select the currently required minimum number of core measure sets
and/or non-core measures Notify The Joint Commission of their
chosen core measure set(s) and/or non-core measures For initial
surveys, data collection must begin on the first day of the first
calendar quarter following survey.
Slide 13
13 Requirements for Small Hospitals Qualifying small hospitals
are required to select one of the following: If four of the core
measure sets are relevant to the small hospitals patient population
mix, the hospital is required to collect data internally on all of
the measures in the four core measure sets, generate either run
charts or control charts on each measure at least quarterly for use
in internal performance improvement activities, and share data and
conclusions with surveyors at the time of survey. If only three
core measure sets are relevant to the small hospitals patient
population mix, the hospital must collect data internally on all of
the applicable measures in the three core measure sets along with
three noncore measures, generate either run charts or control
charts on each measure at least quarterly for use in internal
performance improvement activities, and share data and conclusions
with surveyors at the time of survey.
Slide 14
14 Requirements for Small Hospitals (Continued) Qualifying
small hospitals are required to select one of the following: If
only two core measure sets are relevant to the small hospitals
patient population mix, the hospital must collect data internally
on all of the applicable measures in the two core measure sets
along with six noncore measures, generate either run charts or
control charts on each measure at least quarterly for use in
internal performance improvement activities, and share data and
conclusions with surveyors at the time of survey. If only one core
measure set is appropriate to a small hospitals patient population
mix, the hospital must collect data internally on all applicable
measures in that set, along with data on nine other noncore
measures, generate either run charts or control charts on each
measure at least quarterly for use in internal performance
improvement activities, and share data and conclusions with
surveyors at the time of survey
Slide 15
15 Requirements for Small Hospitals (Continued) Qualifying
small hospitals are required to select one of the following: If
none of the core measure sets is appropriate, the hospital must
collect data internally on nine non-core measures and generate
either run charts or control charts on each measure at least
quarterly for use in internal performance improvement activities,
and share data and conclusions with surveyors at the time of
survey.
Slide 16
16 Requirements for Small Hospitals (Continued) Qualifying
small hospitals continue to have the option of participating with a
Joint Commission listed vendor and submitting data to The Joint
Commission. For additional information on performance measurement
and ORYX, go to the Joint Commissions website at:
http://www.jointcommission.org/accreditation/
performance_measurementoryx.aspx
http://www.jointcommission.org/accreditation/
performance_measurementoryx.aspx
Slide 17
17 Requirements for Psychiatric Hospitals The following
requirements apply to Joint Commissionaccredited psychiatric
hospitals that are: Surveyed under the Comprehensive Accreditation
Manual for Hospitals, and that are not otherwise accredited as a
site under the accreditation of a general medical/surgical
hospital, i.e.: Participate with a Joint Commissionlisted vendor
Submit aggregate monthly data on a quarterly basis to The Joint
Commission on all applicable measures and related age strata that
comprise The Joint Commissions Hospital Based Inpatient Psychiatric
Services (HBIPS) core measure set
Slide 18
18 Requirements for Psychiatric Hospitals (Continued)
Psychiatric hospitals with an average daily census of 10 or fewer
inpatients: Are exempt from the requirement to transmit data to The
Joint Commission via a Joint Commission-listed vendor Must meet the
following requirements: Collect data internally on all applicable
measures and related age strata that comprise The Joint Commissions
Hospital Based Inpatient Psychiatric Services (HBIPS) measure set
Generate either run charts or control charts on each measure, at
least quarterly, for use in internal quality improvement activities
(no data are required to be submitted to The Joint Commission)
Slide 19
19 Requirements for Psychiatric Hospitals (Continued)
Psychiatric hospitals with an average daily census of 10 or fewer
inpatients: Make data reports available for review by surveyors
during on-site surveys and produce them upon request of The Joint
Commission At the time of survey, discuss how the data were used in
identifying priorities for performance improvement activities
Qualifying small psychiatric hospitals continue to have the option
of participating with a Joint Commission- listed vendor and
submitting data to The Joint Commission
Slide 20
20 Performance Expectations for Accountability Measures
Effective January 1, 2012: Hospitals must meet a specific level of
performance on ORYX accountability measures transmitted quarterly
to The Joint Commission. Performance Improvement (PI) Standard
PI.02.01.03 establishes a composite measure rate to assess hospital
performance on ORYX accountability measures. Compliance with the
new requirement, identified as a direct impact requirement, is
based on performance on a single composite measure rate derived
from all reported accountability measures for which the hospital
submits data to The Joint Commission. Accountability measures are
performance measures linked directly to a positive impact on
patient outcomes when hospitals perform well on them. The hospitals
most recent composite rate is available on its quarterly ORYX
Performance Measure Report available on its secure Joint Commission
Connect extranet site. The report also identifies the
accountability measures used to calculate the composite measure
rate.
Slide 21
21 Performance Expectations for Accountability Measures
(Continued) A hospitals performance is: Assessed using a single
composite measure Calculation of the composite score is based on
the sum of all the numerator counts from a hospitals reported
accountability measures across all measure sets divided by the sum
of all the denominator counts from across the same accountability
measures. The composite rate is calculated following each
submission of quarterly data to The Joint Commission using the most
recent four consecutive quarters of data available for all of the
accountability measures that a hospital reports to The Joint
Commission. Data on newly designated accountability measures or
accountability measures newly added by the hospital will be
collected for 12 months before they are included in the composite
measure.
Slide 22
22 ORYX Performance Measure Report ORYX Performance Measure
Reports: Are available to each accredited hospital and critical
access hospital through their secure Joint Commission Connect
extranet site. ORYX Performance Measure Reports are updated after
each submission of quarterly measure data to The Joint Commission
Currently data are due at The Joint Commission January 31, April
30, July 31, and October 31 of each year. Updated reports are
available approximately four weeks after each submission of
quarterly data Hospitals receive an electronic message notifying
them that their reports are available.
Slide 23
23 ORYX Performance Measure Report (Continued) ORYX Performance
Measure Reports: Accredited hospitals that have the option of
meeting their ORYX performance measurement requirements without
submitting data to The Joint Commission will continue to receive a
report four times per year. The reports are provided to: Remind
hospitals of their ORYX requirements for the internal collection
and analysis of performance measure data for the appropriate
required number of measures Better guide the survey process by
informing surveyors to explore how the hospital is using the
information derived from the internal collection and analysis of
its ORYX performance measure data in their ongoing performance
improvement activities
Slide 24
24 ORYX Performance Measure Report (Continued) ORYX Performance
Measure Reports: Are designed to better support and help guide
accredited hospitals in their performance assessment and
improvement activities. Help surveyors better assess: Hospital use
of performance measure data in performance improvement activities
and, where applicable, compliance with the performance expectations
for accountability measures as established under Performance
Improvement Standard PI.02.01.03. (See Figures 15 for examples of
selected sections of the ORYX Performance Measure Report.)
Slide 25
25 ORYX Performance Measure Report (Continued) The ORYX
Performance Measure Report: Provides a user-friendly format with
summary dashboards comprehensive measure details automated links to
specific sections, and selective printing capabilities Highlights
compliance with Standard PI.02.01.03 Highlights desirable and
undesirable data trends
Slide 26
26 ORYX Performance Measure Report (Continued) The ORYX
Performance Measure Report: Key features include: A cover page that
displays the hospitals accountability measure composite rate based
upon the most recent four quarters of accountability measure data
displayed in the report. (See Figure 1) A dashboard of color-coded
symbols at both the measure set and individual measure level
Provides a quick and easy graphical summary of a hospitals
performance on its measures Each measure set and individual measure
on the dashboard provides a hyperlink that the user can click on to
access more detailed information.
Slide 27
27 ORYX Performance Measure Report (Continued) The ORYX
Performance Measure Report: Key features include: At the measure
set level, the dashboard displays the Total number of measures
within the set used by the hospital Number of accountability
measures within the measure set Highlights the set(s) where there
may be desirable or undesirable data trends, and/or statistical
process control issues. (See Figure 2) An easy-to-use legend that
defines each of the color- coded symbols (See Figure 3)
Slide 28
28 ORYX Performance Measure Report (Continued) The ORYX
Performance Measure Report: Key features include: At the individual
measure level, the dashboard identifies: Measures included in the
calculation of the accountability composite rate Desirable and
undesirable data trends, and/or statistical process control issues
(See Figure 4) Control charts, target charts (core measures),
and/or comparison charts (noncore measures) (See Figure 5)
Individual measures that require a hospitals attention are
highlighted when potential standards compliance issues and/or
undesirable trends are identified
Slide 29
29 Analyzing ORYX Data The Joint Commission uses a combination
of control charts, target charts, and comparison charts to evaluate
ORYX data. Control chart analysis is based on a hospitals own
historical (longitudinal) data and is used to assess internal
process stability. Target chart and comparison chart analysis is
used to evaluate a hospitals relative performance level. The use of
target chart and comparison chart analysis in addition to control
chart analysis is a key feature of the Joint Commissions analytic
methods in the ORYX initiative. These types of analyses evaluate
hospital performance from two distinct perspectives and thus can
provide a more comprehensive framework to assess a hospitals
overall performance level.
Slide 30
30 Analyzing ORYX Data (Continued) Control chart, target chart,
and comparison chart analyses may portray different interpretations
of performance, e.g.: A control chart may show a desirable pattern
(one that is statistically in control), but the target and/or
comparison chart may illustrate undesirable outliers (e.g., a high
rate of infections relative to others in the comparison group or
the target range). Perhaps the hospitals performance has been
consistently less desirable than that of other hospitals using the
same measure or below the target range. In such a case, the
hospital needs to think about changing its process for the measure
concerned in order to improve its performance. On the other hand, a
hospital without outliers in the target and/or comparison chart
analysis may have special cause variation (i.e., a statistically
out-of-control pattern) detected in the control chart. In such a
case, the hospital needs to investigate the special cause variation
in its process before making any conclusions about performance
level. In general, a hospital should perform control chart analysis
before target and/or comparison chart analysis to ensure that a
given process is stable before attempting to evaluate relative
performance level.
Slide 31
31 Analyzing ORYX Data (Continued) For more information on ORYX
data analysis, including use of control charts, target charts, and
comparison charts, log on to the secure Joint Commission Connect
extranet site and, under Performance Measurement (ORYX), select
Documentation and Related Links.
Slide 32
32 Performance Measurement
Slide 33
33 Use of Performance Measure Data As part of the Joint
Commissions accreditation process, during the on-site survey, Joint
Commission surveyors assess the following: The hospitals
integration and use of ORYX data into internal performance
improvement activities The hospitals data collection processes
(such as data accuracy, reliability, and security) The hospitals
data analysis methodologies and related training The dissemination
of findings
Slide 34
34 Integrating Accountability Measure Data into the On-site
Survey Process If at the time of survey, based upon the most recent
four consecutive quarters of accountability measure data available,
the hospitals performance on the composite measure falls below the
85% threshold required under Standard PI.02.01.03, the hospital
will automatically receive a Requirement for Improvement (RFI) on
its Summary of Survey Findings Report. The hospital can only clear
an RFI through the submission of quarterly data to The Joint
Commission via the hospitals selected ORYX vendor. The hospital
will not have an opportunity to submit more recent data or clarify
the finding.
Slide 35
35 Integrating Accountability Measure Data into the On-site
Survey Process (Continued) An RFI for PI.02.01.03 will be resolved
using a modified Evidence of Standards Compliance (ESC) process: A
hospital that does not meet the 85% threshold established by the
standard will be required to: Submit an ESC within 45 days in the
form of a Plan of Correction that outlines the actions to be taken
to meet the 85% threshold. To clear the RFI, the hospital must
demonstrate an accountability measure composite rate that meets or
exceeds the 85% threshold for any two consecutive quarters of
regularly submitted data. Failure to resolve the RFI after 18
months following the on-site survey may result in a recommendation
for Contingent Accreditation. For general information on ORYX
requirements, contact the ORYX Information Line at 630-792-5085, or
e-mail [email protected].