Quality Monitoring in New York State: ACT Quality Monitoring in New York State: ACT Licensing, Client Outcomes, and DIGS Licensing, Client Outcomes, and DIGS

Quality IndicatorsQuality Indicators

Anthony Mancini, PhDAnthony Mancini, PhD

Bureau of Adult Services Evaluation ResearchBureau of Adult Services Evaluation Research

New York State Office of Mental HealthNew York State Office of Mental Health

2

OverviewOverview

1. NYS ACT Licensing Initiative

2. Development of Fidelity-based Licensing Protocol for ACT

3. Outcomes Monitoring (CAIRS)

4. Evaluation of ACT Licensing

5. Focus on DIGS: Deriving Quality Indicators for Adult EBPs from Administrative Data

3

1. NYS ACT Licensing Initiative1. NYS ACT Licensing Initiative

Background Rationale Challenges

4

Background for ACT LicensingBackground for ACT Licensing

Part of the state’s “Winds of Change” campaign to Part of the state’s “Winds of Change” campaign to increase access to evidence-based practices (EBPs)increase access to evidence-based practices (EBPs)

Switch to Medicaid billingSwitch to Medicaid billing Expand ACT teams from 27 to 60 in first phaseExpand ACT teams from 27 to 60 in first phase Make fidelity to ACT & EBPs linchpin of Make fidelity to ACT & EBPs linchpin of

implementationimplementation Make ACT a platform for all EBPSMake ACT a platform for all EBPS Training in ACT model and all EBPsTraining in ACT model and all EBPs

5

Rationale for Fidelity-Based LicensingRationale for Fidelity-Based Licensing

Promote a well-validated treatment for high-Promote a well-validated treatment for high-need clientsneed clients

Through licensing, can promote quality of care, Through licensing, can promote quality of care, i.e., high fidelityi.e., high fidelity

Fidelity provides index of success for MHAFidelity provides index of success for MHA Clarify requirements for participating programsClarify requirements for participating programs Allows for focused technical assistanceAllows for focused technical assistance Identifies additional training needsIdentifies additional training needs

6

Challenges for ACT Licensing InitiativeChallenges for ACT Licensing Initiative

Can licensing and certification staff reliably use Can licensing and certification staff reliably use the DACTS?the DACTS?

Is the DACTS too time consuming to administer Is the DACTS too time consuming to administer in the context of a licensing visit?in the context of a licensing visit?

Will licensing staff accept a new methodology Will licensing staff accept a new methodology for evaluating programs? for evaluating programs?

How do we integrate assessment of fidelity and How do we integrate assessment of fidelity and traditional state standards?traditional state standards?

Should fidelity items be adapted to match NYS Should fidelity items be adapted to match NYS standards, or is it better to use the DACTS as is standards, or is it better to use the DACTS as is to allow for comparison with national data?to allow for comparison with national data?

7

2. Development of a Fidelity-Based 2. Development of a Fidelity-Based Licensing ProtocolLicensing Protocol

Process Outcome Example

8

Development Process for Development Process for ACT Licensing ProtocolACT Licensing Protocol

Working group of various stakeholdersWorking group of various stakeholders Lengthy iterative process taking over 1 yearLengthy iterative process taking over 1 year Developed content, language, methodology, Developed content, language, methodology,

anchors for each itemanchors for each item Reviewed by senior OMH staff, ACT providers, Reviewed by senior OMH staff, ACT providers,

and county representativesand county representatives Trained licensing and certification staffTrained licensing and certification staff Revisions based on two years of useRevisions based on two years of use Cut points for scoringCut points for scoring

9

Outcome: ACT Licensing Outcome: ACT Licensing InstrumentInstrument 56-item survey (example to follow) 56-item survey (example to follow) Modeled on Dartmouth ACT scale; that is, uses Modeled on Dartmouth ACT scale; that is, uses

5-point 5-point behaviorallybehaviorally-anchored items -anchored items Integrates into one survey traditional outpatient Integrates into one survey traditional outpatient

and fidelity standardsand fidelity standards Includes standards that address Includes standards that address recovery-recovery-

oriented practice, oriented practice, an increasing focus of an increasing focus of concernconcern

10

Outcome: ACT Licensing Instrument (cont’d)

Domains included: Domains included: DACTSDACTS (excluding 2 items; (excluding 2 items; αα 0.68 0.68)) NYS Staffing requirementsNYS Staffing requirements (e.g., % of professional staff) (e.g., % of professional staff) EBP fidelityEBP fidelity (e.g., family psychoed., wellness management; (e.g., family psychoed., wellness management;

αα 0.41 0.41)) Recipients’ rightsRecipients’ rights (e.g., cultural comp., grievance procedure, (e.g., cultural comp., grievance procedure,

recipients’ rights, recipient choice; recipients’ rights, recipient choice; αα 0.78 0.78)) Case recordsCase records (e.g., assessment & treatment planning; (e.g., assessment & treatment planning; αα 0.83)0.83) Program operationsProgram operations (e.g., policies & procedures related to (e.g., policies & procedures related to

incident review, quality improvement; incident review, quality improvement; αα 0.81 )0.81 ) Program siteProgram site (e.g., furniture, appropriate storage of medications (e.g., furniture, appropriate storage of medications

& case records; & case records; αα 0.64)0.64)

11

Example of Fidelity-Based Licensing Indicator ACT 4.14: RECIPIENT INPUT: Program should have policy and formal mechanism for

incorporating ongoing recipient input into program planning, service development, and program review activities.

Survey Guidelines: A systematic method and formal policy for incorporating recipient input into program operations is necessary to ensure that service design adapts to recipient preferences and needs. Approaches to incorporating recipient input may include having a recipient representative on the program advisory board or governing body, satisfaction surveys, recipient meetings, and recipient committees; optimally, programs will have a recipient on the program/ agency’s governing body and use a range of approaches to solicit input. There should be evidence that program responds to recipient complaints and institutes changes when needed. (PG: 4.16) Sources of Information: Team leader interview; recipient interview; program policies; minutes of meetings.

(1) (2) (3) (4) (5)

No policy or formal mechanism for

incorporating recipient input into program

planning.

Program has policy but no formal mechanism for incorporating recipient

input into program planning.

Program has a policy and a formal mechanism for incorporating recipient

input into program planning.

In addition to (3), program uses range of formal and informal

strategies for soliciting recipient input and

makes concerted effort to respond to input

using different approaches.

In addition to (4), program/ agency has

recipient on governing body.

12

3. Outcomes Reporting & 3. Outcomes Reporting & Monitoring (CAIRS)Monitoring (CAIRS)

Background Features Domains Reports

13

Background on Outcomes Reporting System (CAIRS) Child and Adult Integrated Reporting

System (CAIRS)System evolved out of need to track

child servicesNeed to standardize reporting across

different program typesNow collects data for Child Services,

ACT, AOT, Case Management, and Housing Programs

Will be extended to other programs, including Personalized Recovery-Oriented Services (PROS)

14

Features of CAIRS

Secure and confidential HIPAA compliant Web-based Collects client outcomes data from

program staff Data submission required at baseline

and every 6 months thereafter for every client

15

Domains Reported on in CAIRS CAIRS collects data on a variety of outcomes, CAIRS collects data on a variety of outcomes,

including: including: Psychosocial functioningPsychosocial functioning

Housing status (e.g., type of residence) Housing status (e.g., type of residence) Employment (e.g., current and past, Employment (e.g., current and past,

competitive vs. sheltered) competitive vs. sheltered) Educational achievement (and current Educational achievement (and current

activity)activity)Quality of relationships with othersQuality of relationships with othersSelf-care (e.g., ADLs)Self-care (e.g., ADLs)High risk (e.g., violence toward self or others)High risk (e.g., violence toward self or others)Substance abuse (e.g., type and level of use)Substance abuse (e.g., type and level of use)

Medication regimen, side effects, and adherenceMedication regimen, side effects, and adherence

16

Domains (cont’d)

Type of services in treatment planType of services in treatment plan Medical diagnosesMedical diagnoses Entitlements and income sourcesEntitlements and income sources Relationship status, child custody statusRelationship status, child custody status Inpatient psychiatric utilization history (up Inpatient psychiatric utilization history (up

to past 24 months)to past 24 months) Criminal justice involvementCriminal justice involvement Advanced directive or health care proxyAdvanced directive or health care proxy

17

Reports Generated by CAIRS

Reports generated via web-based Reports generated via web-based applicationapplication

Aggregated at different levels (e.g., Aggregated at different levels (e.g., consumer, program, county, region, state)consumer, program, county, region, state)

Flexibility in report generationFlexibility in report generation

Instant feedback to clinicians and Instant feedback to clinicians and administrators on individual clients and administrators on individual clients and across clients across clients

18

4. Evaluation of ACT Licensing4. Evaluation of ACT Licensing

a) Fidelity

b) Clients

c) Outcomes

19

4a. Fidelity4a. Fidelity

Questions: Questions: What is the impact of licensing on fidelity? Where do programs excel and where do they have

difficulty? What correlates with higher fidelity?

20

Fidelity

DACTS SubscalesDACTS Subscales

Statewide MeanStatewide Mean

(N = 50)(N = 50)

State PCState PC

(N=5)(N=5)

All Other ProgramsAll Other Programs

(N=45)(N=45)

Human Resources Human Resources 4.38 (.37)4.38 (.37) 4.80 (.36) 4.80 (.36) aa 4.34 (.41) 4.34 (.41) aa

Organizational Organizational Boundaries Boundaries

4.49 (.48)4.49 (.48) 4.20 (.36)4.20 (.36) 4.52 (.48)4.52 (.48)

Services Services 3.29 (.49) 3.29 (.49) 3.53 (1.06)3.53 (1.06) 3.27 (.39)3.27 (.39)

Total ACT FidelityTotal ACT Fidelity 4.04 (.33)4.04 (.33) 4.22 (.52)4.22 (.52) 4.02 (.37)4.02 (.37)

NoteNote: Standard deviations are in parentheses. Cells that share subscripts are : Standard deviations are in parentheses. Cells that share subscripts are significantly different at p<.05. significantly different at p<.05. All indicators use 5-point scales.All indicators use 5-point scales.

ACT fidelity was very good overall: high mean fidelity scores and higher in state PCs

21

Fidelity

DACTS Item > 4.6DACTS Item > 4.6

Item Item MM SDSD

Caseload/clinician ratio of 10:1Caseload/clinician ratio of 10:1 4.96 (.20)4.96 (.20)

Slow intake rateSlow intake rate 4.90 (.30)4.90 (.30)

Practicing team leaderPracticing team leader 4.78 (.46)4.78 (.46)

Responsibility for crisis servicesResponsibility for crisis services 4.72 (.73)4.72 (.73)

No dropout policyNo dropout policy 4.67 (.59)4.67 (.59)

Explicit Admission criteriaExplicit Admission criteria 4.60 (.83) 4.60 (.83)

Key ACT fidelity items showed high levels of implementation

22

Fidelity

DACTS Item < 3.5DACTS Item < 3.5

Item Item MM SDSD

Work with support systemWork with support system 1.58 (.91)1.58 (.91)

Dual disorders treatment groupsDual disorders treatment groups 2.33 (1.75)2.33 (1.75)

Frequency of contactFrequency of contact 2.34 (.56)2.34 (.56)

Employment specialist on teamEmployment specialist on team 3.08 (1.97)3.08 (1.97)

Intensity of serviceIntensity of service 3.36 (.92)3.36 (.92)

However, teams struggled in areas related to service intensity and specific services

23

Fidelity

VariableVariable BetaBeta RR22

Team morale Team morale -.11-.11 .21*.21*

Perceived Lack of Skill in EBPs Perceived Lack of Skill in EBPs -.16-.16

Perceived impact of ACT fidelity Perceived impact of ACT fidelity -.18-.18 .14.14**

Perceived difficulty of ACT fidelityPerceived difficulty of ACT fidelity -.05-.05

Intention to implement ACT fidelityIntention to implement ACT fidelity .48*.48*

Endorse dual disorders treatment modelEndorse dual disorders treatment model .23.23 .30***.30***

Use contingenciesUse contingencies -.34**-.34**

Invoke authoritiesInvoke authorities -.33**-.33**

PL tenure with agencyPL tenure with agency .25*.25* .05*.05*

NoteNote: All values are standardized beta coefficients. Variables : All values are standardized beta coefficients. Variables were entered in four steps. Beta weights are for final model. were entered in four steps. Beta weights are for final model.

PL=program leaderPL=program leader

***=p.<.001;**=p.<.01; *=p.<.05; ***=p.<.001;**=p.<.01; *=p.<.05; =p<.10=p<.10

Total Total RR2 2 .70***.70***

A number of program leader characteristic predict ACT fidelity

24

Fidelity

Licensing DomainLicensing Domain

DACTS ScoreDACTS Score

rr

EBP IndicatorsEBP Indicators .37**.37**

Recipients’ RightsRecipients’ Rights .50**.50**

Case RecordsCase Records .51**.51**

Program. Operations Program. Operations .47**.47**

Program SiteProgram Site .38**.38**

**=p<.01**=p<.01

Fidelity scores were highly correlated with other licensing domains

25

4b. Client Characteristics & 4b. Client Characteristics & AppropriatenessAppropriateness Questions: Questions:

1. Who gets ACT in NY?

2. Are appropriate clients being enrolled?

26

Who Gets ACT in NY? Demographics & Diagnosis

CharacteristicCharacteristic % or M% or M Frequency/ RangeFrequency/ Range

AgeAge 43.42 (12.48)43.42 (12.48) 18-8818-88

WomenWomen 42.5 %42.5 % 748748

White (non-Hispanic)White (non-Hispanic) 51.2%51.2% 901901

Black (non-Hispanic)Black (non-Hispanic) 31.0 %31.0 % 546546

Hispanic/LatinoHispanic/Latino 11.2 %11.2 % 198198

Other ethnicity (Asian, multi-racial) Other ethnicity (Asian, multi-racial) 4.0 %4.0 % 6767

Schizo. Spectrum DiagnosisSchizo. Spectrum Diagnosis 70.2 %70.2 % 11151115

Mood DisorderMood Disorder 25.425.4 403403

Anxiety DisorderAnxiety Disorder 1.9%1.9% 3030

Alcohol S/A DisorderAlcohol S/A Disorder 1.3 %1.3 % 2020

Personality & OtherPersonality & Other 1.3 %1.3 % 2121

27

Are ACT Clients Appropriate?

CriteriaCriteria %%

AOT StatusAOT Status 10.410.4

Any Criminal Justice InvolvementAny Criminal Justice Involvement 16.516.5

Homeless in Last 6 MonthsHomeless in Last 6 Months 16.116.1

Diminished Self-CareDiminished Self-Care 47.147.1

High Use of Psych ER (>2 visits)High Use of Psych ER (>2 visits) 30.630.6

High Inpatient Psych (>2 hospitalizations)High Inpatient Psych (>2 hospitalizations) 27.927.9

Suicidal/parasuicidal behavior (< 6 months) Suicidal/parasuicidal behavior (< 6 months) 5.95.9

Co-occurring Substance Abuse DiagnosisCo-occurring Substance Abuse Diagnosis 47.747.7

No Competitive EmploymentNo Competitive Employment 91.691.6

Low Levels of Med. Adherence & EngagementLow Levels of Med. Adherence & Engagement 78.478.4

High Risk (AOT | High Utilization | Jail |Homeless)High Risk (AOT | High Utilization | Jail |Homeless) 50.550.5

% that Meet Any of the Above Criteria % that Meet Any of the Above Criteria 99.699.6

28

4c. Client Outcomes4c. Client Outcomes

Questions: Questions: Does enrollment in ACT reduce inpatient and

ER utilization? Does enrollment in ACT reduce inpatient and

ER cost?

29

Client OutcomesMCD Inpatient/ER Utilization & Costs for New Admissions (N=438)

Pre-ACT Pre-ACT EnrollmentEnrollment

(6 Months)(6 Months)

Post- ACT Post- ACT EnrollmentEnrollment

(6 Months)(6 Months)

Mean Mean DifferenceDifference

(Post-Pre)(Post-Pre)

Inpatient Days Inpatient Days 19.96 (30.32)19.96 (30.32) 11.06 (23.11)11.06 (23.11) -8.90*** -8.90***

Inpatient Cost ($)Inpatient Cost ($) 9,126 (17,908) 9,126 (17,908) 5,461 (13,873) 5,461 (13,873) -3,665*** -3,665***

ER DaysER Days .23 (.81).23 (.81) .10 (.59).10 (.59) -.13**-.13**

ER Cost ($)ER Cost ($) 59.05 (274.63)59.05 (274.63) 36.47 (262.74)36.47 (262.74) -22.58 ns-22.58 ns

Note: Standard deviations are in parentheses. Sample includes new Note: Standard deviations are in parentheses. Sample includes new admissions with at least two months of ACT billing between 12/02 – 10/03. admissions with at least two months of ACT billing between 12/02 – 10/03.

**=p<.01; ***=p<.001**=p<.01; ***=p<.001

30

Client OutcomesMCD Inpatient/ER Utilization & Costs for New Admissions (N=438)

Pre-ACT Pre-ACT EnrollmentEnrollment

(2 Years-(2 Years-

6 Months Aver.) 6 Months Aver.)

Post- ACT Post- ACT EnrollmentEnrollment

(6 Months)(6 Months)

Mean Mean Difference Difference (Post-Pre)(Post-Pre)

Inpatient DaysInpatient Days 13.65 (18.48)13.65 (18.48) 11.06 (23.11)11.06 (23.11) -2.6* -2.6*

Inpatient Cost ($)Inpatient Cost ($) 6,610 (11,290) 6,610 (11,290) 5,461(13,873) 5,461(13,873) -1,150 ns -1,150 ns

ER ServicesER Services .16 (.59).16 (.59) .10 (.59).10 (.59) -.06*-.06*

ER Cost ($)ER Cost ($) 66.50 (233.62)66.50 (233.62) 36.47 (262.74)36.47 (262.74) -30.03*-30.03*

Note: Standard deviations are in parentheses. Sample includes new Note: Standard deviations are in parentheses. Sample includes new admissions with at least two months of ACT Medicaid billing between 12/02 – admissions with at least two months of ACT Medicaid billing between 12/02 – 10/03. 10/03.

*=p<.05*=p<.05

31

5. Focus on DIGS: Deriving 5. Focus on DIGS: Deriving Adult EBP Quality Indicators from Adult EBP Quality Indicators from

Administrative DataAdministrative Data ContextContext GoalsGoals DataData ConceptualizationConceptualization IndicatorsIndicators Next stepsNext steps

32

New York State Context for DIGS

New York’s Winds of Change initiative to New York’s Winds of Change initiative to support EBPssupport EBPs

Roll-out and licensure of ACT teams with focus Roll-out and licensure of ACT teams with focus on EBP fidelityon EBP fidelity

Data systems for assessing outcomes, service Data systems for assessing outcomes, service delivery, and treatment planning (CAIRS)delivery, and treatment planning (CAIRS)

Personalized Recovery-Oriented Services Personalized Recovery-Oriented Services (PROS) programs (PROS) programs

33

Goals for DIGS

Overarching goal:Overarching goal:

To meet reporting requirements for Federal Data To meet reporting requirements for Federal Data Infrastructure Grants (focus on Tables 16 & 17)Infrastructure Grants (focus on Tables 16 & 17)

To achieve that goal, we have: To achieve that goal, we have: Identified potential data sourcesIdentified potential data sourcesAssessed their relevance to EBP reportingAssessed their relevance to EBP reportingConceptualized reasonable standards for EBP Conceptualized reasonable standards for EBP

conformanceconformance

34

Data Sources for DIGS

1.1. Licensing data on ACTLicensing data on ACT Program levelProgram level Fidelity to ACT, IDDT, SE Fidelity to ACT, IDDT, SE

2.2. EBP Implementation data (toolkit, FPE projects)EBP Implementation data (toolkit, FPE projects) Program levelProgram level Fidelity to IMR and FPE Fidelity to IMR and FPE

3.3. CAIRS Data (ACT, case management, housing providers)CAIRS Data (ACT, case management, housing providers) Client levelClient level Services and treatment planning componentsServices and treatment planning components

4.4. Medicaid Data (all billable services)Medicaid Data (all billable services) Client levelClient level Services Services

5.5. NYISERS (vocational programs)NYISERS (vocational programs) Client LevelClient Level ServicesServices

35

Conceptualizing Performance Indicators for DIGS Question is: Question is:

How to operationalize How to operationalize EBP quality indicatorsEBP quality indicators using using available data?available data?

Focus on:Focus on: Supported HousingSupported Housing Supported EmploymentSupported Employment Famly PsychoeducationFamly Psychoeducation ACTACT Integrated Treatment for Dual DisordersIntegrated Treatment for Dual Disorders Illness Management and RecoveryIllness Management and Recovery

36

Conceptualizing Performance Indicators for DIGS (cont’d) Developed algorithms/methods for each of the Developed algorithms/methods for each of the

EBPs using different data sourcesEBPs using different data sources Census of programs (CAIRS)Census of programs (CAIRS) Services provided (CAIRS; Medicaid)Services provided (CAIRS; Medicaid) Process-based data on Process-based data on fidelityfidelity or or qualityquality of of

service (ACT licensing data; EBP projects service (ACT licensing data; EBP projects data) data)

37

Proposed DIGS Performance Indicators(Table 16)

1.1. Supported Housing (SHSupported Housing (SH))

Method Using CAIRS (Penetration only)Method Using CAIRS (Penetration only)

All persons reported in All persons reported in CAIRSCAIRS to be receiving to be receiving SHSH

Relatively straightforward because no fidelity Relatively straightforward because no fidelity measure neededmeasure needed

38

Proposed DIGS Performance Indicators(Table 16 cont’d)

2. 2. Supported Employment (SE)Supported Employment (SE) Method for ACT Teams Using ACT Licensing and Method for ACT Teams Using ACT Licensing and

CAIRS DataCAIRS DataIfIf teams meet these teams meet these fidelityfidelity conditions: conditions:

• Team has employment specialist (score = 5) Team has employment specialist (score = 5) &&• Employment specialist provides majority of service Employment specialist provides majority of service

(score = 4 or 5; e.g., job development, coaching, (score = 4 or 5; e.g., job development, coaching, follow-along support) follow-along support) &&

• There is some diversity of job options (score = 3-5; There is some diversity of job options (score = 3-5; e.g., not all jobs are working as messenger for agency)e.g., not all jobs are working as messenger for agency)

ThenThen # of clients with SE in treatment plan on # of clients with SE in treatment plan on that team (CAIRS) represents that team (CAIRS) represents penetrationpenetration and and reporting for that indicatorreporting for that indicator

39

Proposed DIGS Performance Indicators(Table 16 cont’d)

2. 2. Supported Employment (cont’d)Supported Employment (cont’d) Method Using Medicaid Data Method Using Medicaid Data

# of persons with Vocational Services rate code in Medicaid # of persons with Vocational Services rate code in Medicaid (Penetration)(Penetration)

Rate code means that person has 15 hours/week in setting that Rate code means that person has 15 hours/week in setting that integrates mental health and vocational services integrates mental health and vocational services (Proxy for fidelity)(Proxy for fidelity)

Method Using NYISERS Data for Vocational Method Using NYISERS Data for Vocational ProgramsPrograms

# of persons receiving SE # of persons receiving SE No fidelity data as yet No fidelity data as yet

40

Proposed DIGS Performance Indicators(Table 17 cont’d)

3. 3. Assertive Community TreatmentAssertive Community Treatment Method Using CAIRS DataMethod Using CAIRS Data

# of persons receiving ACT services # of persons receiving ACT services (penetration)(penetration) Because structural features of fidelity, e.g., staffing, meeting Because structural features of fidelity, e.g., staffing, meeting

structure, in place for all teams in NY, no decision rule structure, in place for all teams in NY, no decision rule

4. 4. Family Psychoeducation (FPE)Family Psychoeducation (FPE) Method Using Data from FPE implementation initiativeMethod Using Data from FPE implementation initiative

Collaboration with University of Rochester Collaboration with University of Rochester 21 programs running family groups21 programs running family groups About 100 clients; more as project continuesAbout 100 clients; more as project continues Fidelity monitoring by phoneFidelity monitoring by phone Data are still being collectedData are still being collected No standards or algorithm as yetNo standards or algorithm as yet

41

Proposed DIGS Performance Indicators(Table 17 cont’d)

5. 5. Integrated Dual Disorders Treatment (IDDT)Integrated Dual Disorders Treatment (IDDT) Method for ACT Teams Using Licensing and CAIRS dataMethod for ACT Teams Using Licensing and CAIRS data

IfIf teams meet these teams meet these fidelityfidelity conditions: conditions: • Has a substance abuse specialist (score = 5) Has a substance abuse specialist (score = 5) &&• Provides some individual substance abuse treatment Provides some individual substance abuse treatment

(score = 3-5) (score = 3-5) &&• Provides some group substance abuse treatment (score Provides some group substance abuse treatment (score

= 4 or 5)= 4 or 5) ThenThen the clients on those teams with substance abuse as the clients on those teams with substance abuse as

part of their treatment plan (CAIRS) would be counted part of their treatment plan (CAIRS) would be counted toward toward penetrationpenetration and reporting for that indicator on and reporting for that indicator on ACT ACT

42

Proposed DIGS Performance Indicators(Table 17 cont’d)

6. 6. Illness Management and Recovery (IMR)Illness Management and Recovery (IMR) Method Using Data from Pilot ProjectMethod Using Data from Pilot Project: :

4 programs4 programs Approximately 80 clients in total Approximately 80 clients in total Fidelity data collected every 6 months for 2 yearsFidelity data collected every 6 months for 2 years No algorithm or standards as yetNo algorithm or standards as yet

7. 7. Medication Management (MM)Medication Management (MM) No fidelity data available as yetNo fidelity data available as yet Refinement of fidelity scale ongoing in national project Refinement of fidelity scale ongoing in national project

OMH is participating in OMH is participating in

43

Next Steps Further refinement of licensing indicators Further refinement of licensing indicators

for ACTfor ACT Revision of licensing instrumentRevision of licensing instrument Cutpoints to reflect reasonable expectationsCutpoints to reflect reasonable expectations Ongoing evaluations of peer perspectives, Ongoing evaluations of peer perspectives,

psychosocial outcomes, and recovery-oriented psychosocial outcomes, and recovery-oriented practicespractices

CAIRS outcomes applicationCAIRS outcomes application Extension to Personalized Recovery-Oriented Extension to Personalized Recovery-Oriented

Services (PROS)Services (PROS) Client-completed form rather than clinician-Client-completed form rather than clinician-

completed completed

44

Next Steps (cont’d)

Licensing for PROSLicensing for PROS Further integration of fidelity and licensing Further integration of fidelity and licensing

standards standards Development of indicators for recovery-oriented Development of indicators for recovery-oriented

practicespractices DIGSDIGS

Ongoing refinement of data collection and analytic Ongoing refinement of data collection and analytic approach for EBPs and quality monitoring generallyapproach for EBPs and quality monitoring generally

Continuous quality improvement approach reflected Continuous quality improvement approach reflected in all initiativesin all initiatives