Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March...
-
Upload
shanon-whitehead -
Category
Documents
-
view
217 -
download
2
Transcript of Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March...
![Page 1: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/1.jpg)
Performance Improvement Projects Technical Assistance
Prepaid Mental Health Plans
Thursday, March 29, 200710:30 a.m. – 12:15 p.m.
Cheryl L. Neel, RN, MPH, CPHQ
Manager, Performance Improvement Projects
David Mabb, MS
Sr. Director, Statistical Evaluation
![Page 2: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/2.jpg)
Presentation Outline
• PIP Overall Comments• Aggregate MCO PIP Findings• Aggregate PMHP Specific Findings• Technical Assistance with Group Activities
– Study Design– Study Implementation– Quality Outcomes Achieved
• Questions and Answers
![Page 3: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/3.jpg)
Key PIP StrategiesKey PIP Strategies
1. Conduct outcome-oriented projects
2. Achieve demonstrable improvement
3. Sustain improvement
4. Correct systemic problems
![Page 4: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/4.jpg)
Validity and Reliability of PIP Results
• Activity 3 of the CMS Validating Protocol: Evaluating overall validity and reliability of PIP results:– Met = Confidence/High confidence in reported
PIP results– Partially Met = Low confidence in reported PIP
results– Not Met = Reported PIP results not credible
![Page 5: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/5.jpg)
Summary of PIP Validation Scores
Percentage Score of Evaluation Elements Met HMO NHDP PMHP Total 90% to 100% 2 0 0 2 80% to 89% 13 0 6 19 70% to 79% 13 1 0 14 60% to 69% 8 3 1 12 Less than 60% 17 16 1 34
Total 53 20 8 81
![Page 6: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/6.jpg)
Proportion of PIPs Meeting the Requirements for Each Activity
![Page 7: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/7.jpg)
Aggregate Valid Percent Met
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53
Evaluation Element Number
I II III IV V VI VII VIII IX X
![Page 8: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/8.jpg)
PMHP Specific Findings
• 8 PIPs submitted• Scores ranged from 37% to 89%• Average score was 77%• Assessed evaluation elements were scored
as Met 78% of the time
![Page 9: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/9.jpg)
Summary of PMHP Validation Scores
Percentage Score of Evaluation Elements Met PMHP 90% to 100% 0 80% to 89% 6 70% to 79% 0 60% to 69% 1 Less than 60% 1
Total 8
![Page 10: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/10.jpg)
Study Design
Four Components:
1. Activity I. Selecting an Appropriate Study Topic
2. Activity II. Presenting Clearly Defined, Answerable Study Question(s)
3. Activity III. Documenting Clearly Defined Study Indicator(s)
4. Activity IV. Stating a Correctly Identified Study Population
![Page 11: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/11.jpg)
Activity I. Selecting an Appropriate Study Topic - PMHP Overall Score
88% 88% 88%
100% 100%
88%
80%
82%
84%
86%
88%
90%
92%
94%
96%
98%
100%
1 2 3 4 5 6
Evaluation Element Number
![Page 12: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/12.jpg)
Activity I. Selecting an Appropriate Study Topic
Results:
• 92 percent of the six evaluation elements were Met• 8 percent were Partially Met or Not Met• None of the evaluation elements were Not Applicable
or Not Assessed
![Page 13: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/13.jpg)
Activity I: Review the Selected Study Topic
HSAG Evaluation Elements:• Reflects high-volume or high-risk conditions (or was selected
by the State).• Is selected following collection and analysis of data (or was
selected by the State). • Addresses a broad spectrum of care and services (or was
selected by the State).• Includes all eligible populations that meet the study criteria.• Does not exclude members with special health care needs.• Has the potential to affect member health, functional status,
or satisfaction.
Bolded evaluation elements show areas for improvement
![Page 14: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/14.jpg)
Activity II. Presenting Clearly Defined, Answerable Study Question(s) - PMHP
Overall Score
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2
Evaluation Element Number
No PIP studies received a Met for both evaluation elements
![Page 15: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/15.jpg)
Activity II. Presenting Clearly Defined, Answerable Study Question(s)
Results:
• 0 percent of the two evaluation elements were Met
• 100 percent were Partially Met or Not Met• None of the evaluation elements were Not
Applicable or Not Assessed
![Page 16: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/16.jpg)
Activity II: Review the Study Question(s)
HSAG Evaluation Elements:
• States the problem to be studied in simple terms.
• Is answerable.
Bolded evaluation elements show areas for improvement
![Page 17: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/17.jpg)
Activity III. Documenting Clearly Defined Study Indicator(s) - PMHP Overall Score
88%
75%
0%
88%
100%
0%
100%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6 7
Evaluation Element Number
![Page 18: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/18.jpg)
Activity III. Documenting Clearly Defined Study Indicator(s)
Results:
• 59 percent of the seven evaluation elements were Met
• 21 percent were Partially Met or Not Met• 20 percent of the evaluation elements were
Not Applicable or Not Assessed
![Page 19: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/19.jpg)
Activity III: Review Selected Study Indicator(s)
HSAG Evaluation Elements:
• Is well defined, objective, and measurable.• Are based on practice guidelines, with sources
identified. • Allows for the study question to be answered. • Measures changes (outcomes) in health or functional
status, member satisfaction, or valid process alternatives. • Have available data that can be collected on each
indicator.• Are nationally recognized measure such as HEDIS®, when
appropriate.• Includes the basis on which each indicator was adopted, if
internally developed.
Bolded evaluation elements show areas for improvement
![Page 20: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/20.jpg)
Activity IV. Stating a Correctly Identified Study Population - PMHP Overall Score
88%
0%
88%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3
Evaluation Element Number
![Page 21: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/21.jpg)
Activity IV. Stating a Correctly Identified Study Population
Results:
• 58 percent of the three evaluation elements were Met
• 13 percent were Partially Met or Not Met• 29 percent of the evaluation elements were
Not Applicable or Not Assessed
![Page 22: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/22.jpg)
Activity IV: Review the Identified Study Population
HSAG Evaluation Elements:
• Is accurately and completely defined. • Includes requirements for the length of a
member’s enrollment in the managed care plan.
• Captures all members to whom the study question applies.
Bolded evaluation elements show areas for improvement
![Page 23: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/23.jpg)
Group Activity
![Page 24: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/24.jpg)
Study Implementation
Three Components:
1. Activity V. Valid Sampling Techniques
2. Activity VI. Accurate/Complete Data Collection
3. Activity VII. Appropriate Improvement Strategies
![Page 25: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/25.jpg)
Activity V. Presenting a Valid Sampling Technique - PMHP Overall Score
100% 100% 100% 100% 100% 100%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6
Evaluation Element Number
![Page 26: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/26.jpg)
Activity V. Presenting a Valid Sampling Technique
Results:
• 3 out of the 8 PIP studies used sampling.• 38 percent of the six evaluation elements were
Met. • 0 percent were Partially Met or Not Met. • 63 percent of the evaluation elements were Not
Applicable or Not Assessed.
![Page 27: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/27.jpg)
Activity V: Review Sampling Methods
* This section is only validated if sampling is used.
HSAG Evaluation Elements:
• Consider and specify the true or estimated frequency of occurrence. (N=8)
• Identify the sample size. (N=8)• Specify the confidence level to be used. (N=8)• Specify the acceptable margin of error. (N=8)• Ensure a representative sample of the eligible population.
(N=8)• Ensure that the sampling techniques are in accordance
with generally accepted principles of research design and statistical analysis. (N=8)
![Page 28: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/28.jpg)
Populations or Samples?
Generally,Generally,
– Administrative data uses populations– Hybrid (chart abstraction) method uses
samples identified through administrative data
![Page 29: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/29.jpg)
Activity VI. Specifying Accurate/Complete Data Collection - PMHP Overall Score
100% 100%
86%
100% 100% 100% 100% 100%
0%
86%
43%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6 7 8 9 10 11
Evaluation Element Number
![Page 30: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/30.jpg)
Activity VI. Specifying Accurate/Complete Data Collection
Results:
• 55 percent of the eleven evaluation elements were Met
• 10 percent were Partially Met or Not Met• 35 percent of the evaluation elements were
Not Applicable or Not Assessed
![Page 31: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/31.jpg)
Activity VI: Review Data Collection Procedures
HSAG Evaluation Elements:
• Clearly defined data elements to be collected. (55 percent Met)
• Clearly identified sources of data. (77 percent Met)• A clearly defined and systematic process for collecting
data that includes how baseline and remeasurement data will be collected. (62 percent Met)
• A timeline for the collection of baseline and remeasurement data. (57 percent Met)
• Qualified staff and personnel to collect manual data. (13 percent Met; 77 percent N/A)
• A manual data collection tool that ensures consistent and accurate collection of data according to indicator specifications. (13 percent Met; 77 percent N/A)
Bolded evaluation elements show areas for improvement
![Page 32: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/32.jpg)
Activity VI: Review Data Collection Procedures (cont.)
HSAG Evaluation Elements:
• A manual data collection tool that supports interrater reliability. (13 percent Met; 77 percent N/A)
• Clear and concise written instructions for completing the manual data collection tool. (13 percent Met; 77 percent N/A)
• An overview of the study in the written instructions. • Administrative data collection algorithms that show steps
in the production of indicators• An estimated degree of automated data completeness
(important if using the administrative method).
Bolded evaluation elements show areas for improvement
![Page 33: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/33.jpg)
Where do we look for our sources of data?
![Page 34: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/34.jpg)
Baseline Data Sources
• Medical Records• Administrative claims/encounter data• Hybrid• HEDIS• Survey Data• MCO program data• Other
![Page 35: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/35.jpg)
Activity VII. Documenting the Appropriate Improvement Strategies - PMHP Overall
Score
100% 100%
0% 0%0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4
Evaluation Element Number
![Page 36: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/36.jpg)
Activity VII. Documenting the Appropriate Improvement Strategies
Results:
• 44 percent of the four evaluation elements were Met
• 3 percent were Partially Met or Not Met• 53 percent of the evaluation elements were
Not Applicable or Not Assessed
![Page 37: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/37.jpg)
Activity Seven: Assess Improvement Strategies
HSAG Evaluation Elements:
• Related to causes/barriers identified through data analysis and Quality Improvement (QI) processes.
• System changes that are likely to induce permanent change.
• Revised if original interventions are not successful.
• Standardized and monitored if interventions are successful.
Bolded evaluation elements show areas for improvement
![Page 38: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/38.jpg)
Determining Interventions
Once you know how you are doing at baseline, what interventions will produce meaningful improvement in the target population?
![Page 39: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/39.jpg)
First Do A Barrier Analysis
What did an analysis of baseline results show ?
How can we relate it to system improvement?
• Opportunities for improvement• Determine intervention(s) • Identify barriers to reaching improvement
![Page 40: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/40.jpg)
How was intervention(s) chosen?
• By reviewing the literature– Evidence-based– Pros & cons– Benefits & costs
• Develop list of potential interventions -- what is most effective?
![Page 41: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/41.jpg)
Types of Interventions
• Education • Provider performance feedback• Reminders & tracking systems• Organizational changes• Community level interventions• Mass media
![Page 42: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/42.jpg)
Choosing Interventions
• Balance – potential for success with ease of use– acceptability to providers & collaborators– cost considerations (direct and indirect)
• Feasibility – adequate resources – adequate staff and training to ensure a
sustainable effort
![Page 43: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/43.jpg)
Physician Interventions: Multifaceted Most Effective
• Most effective: – real-time
reminders– outreach/detailing – opinion leaders – provider profiles
• Less effective: – educational
materials (alone)– formal CME
program without enabling or reinforcing strategies
![Page 44: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/44.jpg)
Patient Interventions
• Educational programs– Disease-specific education booklets– Lists of questions to ask your physician– Organizing materials: flowsheets, charts,
reminder cards– Screening instruments to detect
complications– Direct mailing, media ads, websites
![Page 45: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/45.jpg)
Evaluating Interventions
• Does it target a specific quality indicator? • Is it aimed at appropriate stakeholders?• Is it directed at a specific process/outcome of
care or service?• Did the intervention begin after baseline
measurement period?
![Page 46: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/46.jpg)
Interventions Checklist
Analyze barriers (root causes) Choose & understand target audience Select interventions based on cost-benefit Track intermediate results Evaluate effectiveness Modify interventions as needed Re-Measure
![Page 47: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/47.jpg)
Group Activity
![Page 48: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/48.jpg)
Quality Outcomes Achieved
Three Components:
1. Activity VIII. Presentation of Sufficient Data Analysis and Interpretation
2. Activity IX. Evidence of Real Improvement Achieved
3. Activity X. Data Supporting Sustained Improvement Achieved
![Page 49: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/49.jpg)
Activity VIII. Presentation of Sufficient Data Analysis and Interpretation - PMHP Overall
Score
50%
0%
50%
100%
50%
100%
50% 50% 50%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6 7 8 9
Evaluation Element Number
![Page 50: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/50.jpg)
Activity VIII. Presentation of Sufficient Data Analysis and Interpretation
Results:
• 14 percent of the nine evaluation elements were Met
• 8 percent of the evaluation elements Partially Met or Not Met
• 78 percent of the evaluation elements were Not Applicable or Not Assessed
![Page 51: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/51.jpg)
Activity VIII: Review Data Analysis and Interpretation of Study Results
HSAG Evaluation Elements:
• Is conducted according to the data analysis plan in the study design.
• Allows for generalization of the results to the study population if a sample was selected.
• Identifies factors that threaten internal or external validity of findings.
• Includes an interpretation of findings. • Is presented in a way that provides accurate,
clear, and easily understood information.
Bolded evaluation elements show areas for improvement
![Page 52: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/52.jpg)
Activity VIII: Review Data Analysis and Interpretation of Study Results (cont.)
HSAG Evaluation Elements:
• Identifies initial measurement and remeasurement of study indicators.
• Identifies statistical differences between initial measurement and remeasurement.
• Identifies factors that affect the ability to compare initial measurement with remeasurement.
• Includes the extent to which the study was successful.
Bolded evaluation elements show areas for improvement
![Page 53: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/53.jpg)
Changes in Study Design?
Study design should be same as baselineStudy design should be same as baselineData sourceData collection methodsData analysisTarget population or sample sizeSampling methodology
If change:If change: rationale must be specified & appropriaterationale must be specified & appropriate
![Page 54: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/54.jpg)
Activity IX. Evidence of Real Improvement - PMHP Overall Score
100%
50% 50% 50%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4
Evaluation Element Number
![Page 55: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/55.jpg)
Activity IX. Evidence of Real Improvement
Results:
• 16 percent of the four evaluation elements were Met
• 9 percent were Partially Met or Not Met• 75 percent of the evaluation elements were
Not Applicable or Not Assessed
![Page 56: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/56.jpg)
Activity IX: Assess the Likelihood that Reported Improvement is “Real” Improvement
HSAG Evaluation Elements:
• The remeasurement methodology is the same as the baseline methodology.
• There is documented improvement in processes or outcomes of care.
• The improvement appears to be the result of intervention(s).
• There is statistical evidence that observed improvement is true improvement.
Bolded evaluation elements show areas for improvement
![Page 57: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/57.jpg)
Statistical Significance Testing
Time Periods
Measurement Periods
Numerator Denominator Rate or Results
Industry Benchmark
Statistical Testing and Significance
CY 2003 Baseline 201 411 48.9% 60% N/A
CY 2004 Re-measurement 1 225 411 54.7% 60% Chi-square = 2.8
P-value = 0.09387
NOT SIGNIFICANT AT THE 95% CONFIDENCE LEVEL
![Page 58: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/58.jpg)
Activity X. Data Supporting Sustained Improvement Achieved - PMHP Overall Score
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1
Evaluation Element Number
No PMHP reached this Activity
![Page 59: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/59.jpg)
Activity X. Data Supporting Sustained Improvement Achieved
Results:
• 0 percent of the one evaluation element was Met
• 0 percent was Partially Met or Not Met• 100 percent of the evaluation element was
Not Applicable or Not Assessed
![Page 60: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/60.jpg)
Activity X: Assess Whether Improvement is Sustained
HSAG Evaluation Elements:
• Repeated measurements over comparable time periods demonstrate sustained improvement, or that a decline in improvement is not statistically significant.
Bolded evaluation elements show areas for improvement
![Page 61: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/61.jpg)
Quality Outcomes Achieved
BaselineBaseline
1st Yr1st Yr
Demonstrable Demonstrable ImprovementImprovement
Sustained Sustained ImprovementImprovement
![Page 62: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/62.jpg)
Sustained Improvement
• Modifications in interventions• Changes in study design• Improvement sustained for 1 year
![Page 63: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/63.jpg)
HSAG Contact Information
Cheryl Neel, RN, MPH,CPHQ
Manager, Performance Improvement Projects
602.745.6201
Denise Driscoll
Administrative Assistant
602.745.6260
![Page 64: Performance Improvement Projects Technical Assistance Prepaid Mental Health Plans Thursday, March 29, 2007 10:30 a.m. – 12:15 p.m. Cheryl L. Neel, RN,](https://reader036.fdocuments.us/reader036/viewer/2022062520/5697bf9d1a28abf838c938cb/html5/thumbnails/64.jpg)
Questions and Answers