LOCUS MIFAST Fidelity Scale (Version 2/13/18)avcmh.s3-us-west-1.amazonaws.com/LOCUS MIFAST Report...

Developed by the MDHHS LOCUS MIFAST Project Team (2017). Based on LOCUS, GOI, and other fidelity instruments.

Use with written permission only.

LOCUS MIFAST Fidelity Scale (Version 2/13/18)

Today’s Date: 5/23/2018

Assessor’s Names:

1. Dana Fuller

2. Julie Hudson

3. Denequa Mixon-Young

Organization Name: Ausable Valley Community Mental Health

Organization Address: Street: 1199 W. Harris Avenue Suite:

City: Tawas City State/Zip: MI, 48764

Organization Contact: Name: Trish Otremba Title: Chief Quality Officer

Phone Number: 989-362-8636 Email: [email protected]

PIHP/CMHSP Name: Northern Michigan Regional Entity

PIHP Region #: 2

Sources Used for Fidelity and GOI Assessment Number of trained LOCUS Trainers: 1

Check all that apply. Indicated number as requested.

☒ Chart/Record Review

☒ Team/Practitioner Observation/Interview

☒ Supervisor Observation/Interview

☒ LOCUS Trainer Observation/Interview

☐ LOCUS Training Observation

☒ LOCUS Case Scoring #10 completed:

☒ Organizational Document Review

☐ Other staff Interview

☒ Consumer Interview #1 Interviewed:

☐ Family Member Interview #0 Interviewed:

Number of adult consumers served by organization: 600

Date LOCUS implemented within organization: 11/2016

LOCUS Implementation Work Plan from Site Review Outcome Scoring

Purpose: Michigan Fidelity Assistance Support Team (MIFAST) has

prepared this work-plan to assist with implementation and process

improvement based on the scoring of the Level of Care Utilization System

(LOCUS) Site Review Report. The astute implementation work group will

be able to use this work plan as agenda, minutes and tracking of the agencies

efforts to implement the LOCUS, as well as for a guide in choosing the focus

for the next year’s efforts to improve supports and services. We recommend

that you choose the areas from the scoring chart and use this work plan to

chart the activities around improvement of those items.

L1. Program Support



Definition: The organization has clearly articulated purpose and use processes for LOCUS

Rationale: Within behavioral health services staff members at all levels embrace and support the use of the LOCUS as a consistent measure of the

level of care.

1 2 3 4 5 L1. Program Support.

The organization has clearly articulated

purpose and use processes for LOCUS,

based on the following sources:

1. PIHP/CMHSP Organizational

Leadership

2. Organizational Leadership (as

applicable): Executive Director,

Clinical Director, Program Managers 3. LOCUS Trainers and/or Supervisors

4. Practitioners using LOCUS

5. Written materials (e.g., brochures,

policies, procedures, etc.).

Sources: Interviews, discussion, written

materials.

No more than 1 of the 5

sources shows clear

understanding of the

program philosophy

OR

All sources have

numerous major areas

of discrepancy

☐

2 of the 5 sources show

clear understanding of

the program philosophy

OR

All sources have

several major areas of

discrepancy

☐




OR

Sources mostly aligned

to program philosophy,

but have one major area

of discrepancy

☐




OR

Sources mostly aligned

to program philosophy,

but have one or two

minor areas of

discrepancy

☒

All 5 sources display a

clear understanding and

commitment to the

program philosophy.

☐

Strengths:

● There is evidence of enthusiasm and buy in that was consistently expressed in all interviews with Chief Clinical Officer, Chief Operating

Officer, Practitioner, Trainer, and Program Manager. Upper leadership is reported to have participated in formal training for LOCUS as well.

● Implementation of LOCUS was described by both leadership and clinical staff as being well received, positive and reported to feel like a

natural shift to support level of care and services provided.

● Leadership has demonstrated support for LOCUS by requiring all new staff to be trained.

Recommendation of the site review team (None if the score is a 5):

1. It is suggested that the organization develop written policies and procedures, including procedural steps, on how LOCUS supports overall

operations, including quality. Recommend that written policies are developed which address:

a. Purpose of LOCUS

b. Rationale and reason for determining frequency of use.

c. Use of LOCUS within the person centered planning process.

Work Plan Activity Log – Based upon Score and Recommendations (to be completed by the organization):

L2. Eligibility/Client Identification



Definition: All individuals in adult behavioral health/substance use programs receive a LOCUS for level of care determination.

Rationale: To ensure that individuals seeking and receiving services receive consistently appropriate services for the care needs.

1 2 3 4 5 L2. Eligibility / Client Identification.

A LOCUS Level of Care is identified for

all individuals with severe mental illness within the organization. Elements:

1. LOCUS is used as part of eligibility

determination.

2. LOCUS is used for initial level of care

determination.

3. LOCUS level of care is used for

transitions

4. LOCUS level of care is used for

discharge.

Sources: Written documents such as

policies/procedures, record reviews, etc. (based on expected number of LOCUS

scores for consumer).

No elements are

observed or it is highly

inconsistent.

☐

The majority of records

(greater than 51%) have

at least one element.

☐



at least two elements.

☒



at least three elements.

☐



all four elements

☐

Strengths:

● Once the agency started using LOCUS, a score has been completed at time of eligibility, initial level of care assessments and at annual

review.

● Clinician and Program Manager alerts have been installed within PCE EMR to support staff in completing LOCUS for each individual.

● Clinical rationale is required for all clinical overrides and must be reviewed by Program Manager.


1. While it is clear that LOCUS is being used for initial LOC determination, it is not evident that it is being used for transit ions and discharge.

It is recommended that agency consider developing a written process that can serve as a guideline for clinician’s use of LOCUS during

transitions and discharge planning. This would assist with monitoring progress and provide outcome measures.

2. It is suggested that clinician trends, such as interpretation of independent criteria and clinical overrides, in scoring LOCUS be identified by

pulling data and sharing data with staff.

3. Documentation for clinical justification for the LOCUS assessments regarding overrides were either not documented or did not support the

recommended level of care. It is suggested that clinician trends be identified and appropriate action planning occur to address the need for

including clinical rationale for overrides in the EHR record. (This may include supervision, training, measurement tracking, policy

development, etc). This would also assist with monitoring progress, identifying needs, and measuring outcomes.



Work Plan Activity Log – Based upon Score and Recommendations:

L3. LOCUS Completion

Definition: A LOCUS is completed for all adults eligible for behavioral health/substance use services.

Rationale: Completing a LOCUS for eligible adults creates a system or consistency and equity in service delivery.

1 2 3 4 5 L3. Use Measurement.

The LOCUS is used with eligible

individuals within the organization.

1. A LOCUS level of care is determined

for all individuals with severe mental

illness within the organization

regardless of service (i.e. ACT, case management, outpatient therapy, etc.)

Sources: Interview, Written documents

such a policy/procedures, record reviews,

etc.

‹20% of individuals

have LOCUS

completed.

☐

21%-40% of

individuals have

LOCUS completed.

☐

41%-60% of

individuals have

LOCUS completed.

☐

61%-80% of

individuals have

LOCUS completed.

☐

81%+ of individuals

have LOCUS

completed.

☒

Strengths:

● The organization has displayed positive commitment and support to LOCUS utilization. Based on staff and supervision interview, chart

review, management alerts, it is clear that the LOCUS is being utilized consistently at time of intake and annual review.


N/A


L4. Timeliness

Definition: LOCUS is completed within a timely manner based on organizational policy.

Rationale: Timely completion of the LOCUS results in improved quality of service.

1 2 3 4 5



L4. Timeliness

LOCUS is used within a timely manner

(minimum annually or per organizational

policy).

(Look back three years or when LOCUS was implemented, whichever is earlier).

Sources: Interview, Record Review

‹20% of individuals

have LOCUS

completed in a timely

manner.

☐

21%-40% of

individuals have

LOCUS completed in a

timely manner.

☐

41%-60% of

individuals have


timely manner.

☐

61%-80% of

individuals have


timely manner.

☐

81%+ of individuals

have LOCUS

completed in a timely

manner.

☒

Strengths:

● Based on chart review and staff/leadership interviews, there is evidence that the LOCUS is completed consistently at time of intake and

annual review.

● The electronic notification system within PCE EMR is a positive support to completing the LOCUS in timely manner.


N/A




L5. Assessment

Definition: The LOCUS is used in conjunction with other relevant tools to provide a full picture of individuals/ needs

Rationale: Comprehensive assessments will identify individuals’ level of care needs and progress toward recovery.

1 2 3 4 5 L5. Assessment.

The LOCUS is used in conjunction with

other relevant tools to provide a full picture

of consumer needs. Scoring of the LOCUS

is based upon feedback obtained from

review of clinical history, current level of

functioning, related documentation from

ancillary sources, direct input from

individual receiving services, etc. Based on interviews with Intake/Assessment

Clinical, Primary Clinician, and

Supervisory staff. Comprehensive

assessments completed prior to LOCUS

scoring include:

1. LOCUS

2. Biopsychosocial

3. Psychiatric Evaluation/Medication

Review

4. Medical Information 5. Substance Use Assessment (as

applicable)

Sources: Interviews, written documents

such as record reviews.

LOCUS alone

Or

None

☐

One assessment prior

plus the LOCUS

☐

Two assessments prior

plus the LOCUS

☐

Three assessments prior

plus the LOCUS

☒

Four assessments prior

plus the LOCUS

☐

Strengths:

● Once the LOCUS was implemented, it has been done consistently and in conjunction with Patient Health Questionnaire (PHQ-9) to assist

with initial level of care decisions.

● For 9/10 charts reviewed, it was documented that additional comprehensive assessments, including Biopsychosocial, Psychiatric

Evaluation/Medication Review, Medical Information, and Substance Use Assessments (when applicable) were completed within 4-6 weeks

of the LOCUS score being done.




1. While the intent of the relationship of the LOCUS being based on the assessments and the treatment plan was verbalized by clinical trainer

and clinician; it is not clearly evident within the clinical charts viewed and how it is part of the person centered planning process. It is

suggested that the golden thread be observed throughout treatment planning. Assessments should naturally support the scoring of the

LOCUS which then assists with determining the appropriate level of care and further guides with goal development and treatment planning.

The organization will want to consider developing policy and processes which promote this concept.




L6. Training Approach

Definition: The organization has a LOCUS training approach that promotes reliability, fidelity, and sustainability.

Rationale: The design of a consistent and thorough training approach reduces drift and promotes consistency of level of care decisions.

1 2 3 4 5 L6. Training Approach

The organization has a LOCUS training

approach that promotes reliability, fidelity,

and sustainability.

1. The LOCUS training approach

promotes reliability within training.


promotes fidelity to the use of the LOCUS tool.


contains elements promoting

sustainability of training across time.

4. The LOCUS training approach is

supported by written documentation

(i.e. policies, procedures, processes,

etc.)

5. The LOCUS training approach is

implemented successfully across the

organization.

Sources: Interviews, organizational

process documents.


measures are evident

OR

All measures have


of incompleteness.

☐

2 of the 5 measures are

evident

☒


evident

☐


evident

☐

All 5 measures are

evident

☐

Strengths:

● The organization is committed to training all clinical staff.

● Initial training of all existing staff has occurred.

● The organization is using materials from the original source.

● The LOCUS trainer has demonstrated enthusiasm, commitment to the tool and has expressed interest in refresher training.


1. It is recommended that a plan be developed for trainer sustainability. Adding additional trainers/coaches to assist with collaboration and

availability should be considered.

2. It is suggested that the agency develop written policies that address approach for initial and ongoing trainings/coaching.

3. Develop plan for ongoing training/coaching and competency assessment that assists with determination of individual level competency in

LOCUS use.



4. The agency may consider creating and utilizing organizationally specific vignettes in their LOCUS training, which reflect the organization’s

levels of care.


L7. Training Implementation

Definition: LOCUS training is implemented to promote reliability, fidelity, and sustainability.

Rationale: Practitioner training and retraining are warranted to ensure that services are provided in a standardized manner across practitioners and

over time.

1 2 3 4 5 L7. Training Implementation

Based on the LOCUS training approach,

LOCUS training is implemented that

promotes reliability, fidelity, and

sustainability.

1. LOCUS training is taught by certified

LOCUS Trainers who have completed

a LOCUS train-the-trainer course.

2. The LOCUS curriculum used for

training promotes reliability and fidelity. (Yes/No)

3. Existing practitioners successfully

complete LOCUS training. (Yes/No)

4. Practitioners successfully complete

LOCUS training within the agency

defined orientation period for new

hires.

5. The agency has a written plan to

ensure that practitioners have the

opportunity to refine their LOCUS use

skills at regular intervals as defined by

the organization’s written guidelines (policy, procedure, process, etc.).

Sources: HR Documents, Interviews,

record of staff trainings, procedures and

written documents.


measures are evident

OR

All measures have


of incompleteness

☐


evident

☐


evident

☒


evident

☐

All 5 measures are

evident

☐

Strengths:

● The agency has a training plan.

● There is an identified system in place for training new clinical staff.



● The agency’s identified LOCUS trainer has displayed positive regard and commitment to the LOCUS tool. She was trained through

MDHHS Train the Trainer and utilizes the original material.


1. Develop a plan for trainer sustainability.

2. It is recommended that the organization implements a process for enhancing LOCUS skills. This may include: ongoing/annual training,

clinician competency assessment, clinician specific skill refinement.

3. The organization should consider developing a coaching process to assist with clinical collaboration and availability.

4. It is suggested that the process is supported by written policies/procedures.




L8. Supervision

Definition: Practitioners have the opportunity to receive documented structured/scheduled, person-centered, and supportive feedback on their use of

the LOCUS. This can include one-on-one as well as group supervision. However team or similar meetings are not considered supervision.

Rationale: Regular supervision is critical for individualizing treatment but also for ensuring the standardized use of the LOCUS to adults served.

1 2 3 4 5 l.8. Supervision

Practitioners have the opportunity to

receive documented structured/scheduled, person-centered, and supportive feedback

on their use of the LOCUS. Each

supervision session includes one or more

of the following:

● Review of LOCUS reports by

clinician.

● Level of care maintenance

● Clinician trends in LOCUS scoring

(example: loading stress into all

dimensions, scoring individuals to a

desired setting, difficulties with independent criteria).

● Clinician challenges in using LOCUS

● Clinical development needs and

opportunities for improvement are

documented

Sources: Interviews, written documents,

meeting schedules, group/meeting agendas

and/or minutes.

Practitioners receive no

documented

supervision (group or

individual) and/or they express not feeling

supported in use and

understanding of

LOCUS

☐

Supervision is available

upon request or is

unstructured and/or

inconsistent in practice.

☒

Scheduled group

supervision and/or team

meetings are evident at

least once per quarter and include

documented LOCUS

discussion.

☐

1:1 or group

supervision is provided

monthly and includes

documented LOCUS discussion.

☐

1:1 structured

supervision that

includes documented

LOCUS discussion is provided at least

monthly.

☐

Strengths:

● Documented supervision notes address LOCUS training completion.

● Supervisors and Program Managers are reported to be approachable and there is an “open door” policy. They are alerted to clinical overrides

and review clinical rationale provided.

● Clinical supervision is provided by the agency at least twice monthly.


1. It is suggested that supervision include documented discussions of LOCUS usage including clinician trends, level of care maintenance, and



specific clinician challenges.

2. Recommend that clinical overrides are reviewed with staff during supervision, and it is ensured that clinically sound documentation of the

override is included in the EMR.

3. Recommend collaboration with clinical staff in determining what type of data reports will be collected and analyzed. Clarify plan as to how

these reports will be utilized in improving individual outcomes and identifying needs for additional supports/services.

Work Plan Activity Log – Based upon Score and Recommendation:



L9. Process Monitoring

Definition: The organization uses a standardized approach to LOCUS related data collection, analysis, and use.

Rationale: Systematic and regular collection of process data is imperative in evaluating fidelity to the LOCUS.

1 2 3 4 5 L9. Process Monitoring

Supervisors and program leaders monitor

the process of implementing the LOCUS

consistent with organizational policy or

procedure. Monitoring involves a

standardized approach.

1. There is a standardized approach.

2. There are predetermined timeframes

for monitoring of no less than every six (6) months.

3. Implementation of LOCUS is tied to

quality management.

4. The results of monitoring are used to

guide LOCUS use improvements.

Sources: Policies, procedures, protocols

No attempt at monitoring

process is made

☒

Informal process

monitoring is used at

least annually

☐

Process monitoring is

deficient on 2 of these 3

criteria: (1)

Comprehensive &

standardized; (2)

Completed every 6

months; (3) Used to

guide program

improvements

OR Standardized

monitoring done

annually only

☐

Process monitoring is

deficient on 1 of these 3

criteria: (1)

Comprehensive &

standardized; (2)

Completed every 6

months; (3) Used to

guide program

improvements

☐

Standardized

comprehensive

process monitoring

occurs at least every

6 months and is used

to guide program

improvements

☐

Strengths:

● The organization is in discussion and has expressed an interest in developing a process for monitoring data obtained through LOCUS

implementation.


1. It is recommended that identification of LOCUS related data collection, analysis, and utilization be determined and implemented.

Some ideas include: percent of charts with “override” disposition, focusing on levels of care and the “reason” for doing LOCUS (initial,

transitions, discharges, other times as identified), percent of up vs down override dispositions, and timeframes between assessment, LOCUS

scores, and treatment planning.

2. It is suggested that plan for data collection and utilization is supported by policies/procedures/protocols.


L10. Outcome Monitoring



Definition: Program leaders share LOCUS data with practitioners in an effort to improve services.

Rationale: Systematic and regular collection of LOCUS outcome data is imperative in evaluating level of care effectiveness. Outcome monitoring

involves a standardized approach that uses the results to improve the quality of services provided.

1 2 3 4 5 L10. Outcome Monitoring

Supervisors / organizational leaders

monitor the outcomes of LOCUS use.

Consider:

1. Are individuals served receiving

services reflective of the LOCUS level

of care (LOC) recommendation?

2. Do prior LOCUS scores reflect the

appropriate LOC? “Example: Does the LOCUS score support a transition

to an ACT team?”

3. The frequency of override use is less

than 10 percent within a program and

across the agency.

4. Clinical rationale for overrides is

documented and appropriate.

Sources: Quality documents/reports,

clinical records, etc.

No outcome monitoring

occurs.

OR

Organization has not

begun LOCUS outcome

planning.

☐

Outcome monitoring

occurs at least once a

year, but results are not

shared with

practitioners

OR

Organization is in the

beginning phases of

LOCUS outcome monitoring planning.

☒

Standardized outcome

monitoring occurs at

least once a year, and

results are shared with

practitioners

OR

Organization has a

beginning LOCUS

outcome monitoring plan in place and it

includes sharing results

with practitioners.

☐

Standardized outcome

monitoring occurs at

least twice a year, and

results are shared with

practitioners

☐

Standardized

outcome monitoring

occurs quarterly and

results are shared

with practitioners

☐

Strengths:

● Leadership has asked staff about what data they would need and this is being discussed in clinical meetings.

● Program Manager reviews clinical overrides when notified by clinician or EMR alert.


1. While the recommended frequency of override use is less than 10%, the organization should make a determination regarding their acceptable

override use. It is recommended that the agency’s current use of overrides be clarified.

2. It is suggested that the organization collect and review override data to understand clinical trends.

3. Develop a monitoring process to support the reduction and appropriate use of overrides.

4. It is recommended that completing a LOCUS score at time of transitions and discharges is implemented. This will assist with providing

clinical rationale and support for level of care changes, assist with identifying current needs, and help to measure progress.




L11. Quality Assurance (QA)

Definition: A good quality assurance approach or plan helps to guide and sustain the implementation, makes recommendations for improvement,

and monitors outcomes.

Rationale: Systematic and regular collection of outcome data is imperative in evaluating program effectiveness.

1 2 3 4 5 L11. Quality Assurance (QA)

The agency has a QA Committee,

clinical/implementation steering committee, or similar mechanism with an

explicit plan to review LOCUS

trends/gaps/outcomes/ organizational needs

a minimum of every 6 months, reports to

leadership, and implements corrective

changes/approaches.

Sources: Agendas, minutes documents,

interviews, etc.

No review or no

committee

☐

QA committee has been

formed, but no reviews

have been completed

☒

Explicit QA review

occurs less than

annually

OR QA review is

superficial (does not

fully address the issues

at hand)

☐

Explicit QA review

occurs annually

☐

Explicit review

every 6 months by a

QA group

☐

Strengths:

● Staff from all programs are represented at the Clinical Quality Improvement Committee (CQIC).

● It is reported that LOCUS training and utilization is discussed within the quality assurance process.

● It is documented in agency notes that LOCUS training is being supported and addressed.


1. It is recommended that agency develop an explicit plan to review LOCUS trends/gaps/outcomes/organizational needs including identifying

frequency of review periods and clarifying what type of data will be collected and analyzed.

2. It is recommended that these findings are shared with leadership and all levels of the organization.

3. It is suggested that a process be developed to address corrective changes/approaches as needed.


L12. Individual Choice

Definition: Use of the LOCUS is integrated and reflective of clinical best-practice and person-centered philosophy within the organization.



Rational: Individuals served play a vital role in managing their illness, making progress toward goals, and becoming effective participants in the

treatment process.

1 2 3 4 5 L12. Individual Choice

All individuals receiving services are offered choices consistent with their

recovery choices and clinical needs even if

outside of the LOCUS level of care

recommendation. Individuals are able to

access chosen services if clinically

necessary. Evidence is reflected in the

following sources:

1. Individuals receiving services

2. Clinician interviews

3. Documentation in the record

4. Supervisor and/or administrative

interviews 5. Organizational policy and

procedure (LOCUS/Person-

centered planning specific).

Sources: Individuals receiving services,

clinician interviews, documentation in the

record, supervisor/administrative

interviews, organizational policy and

procedure.

Evidence is reflected in

only one source.

☐


two sources.

☒


three sources.

☐


four sources.

☐

Evidence is reflected

in all five sources.

☐

Strengths:

● Individual interviewed was pleased with services that they were receiving. Stated that therapist had offered more services at multiple times,

and she knows that services are available if needed.

● Per chart review, it appeared as though individuals were consistently satisfied with their services.

● Based on staff and individual interview, a person centered culture was fostered and was evident.


It was noted that the LOCUS score and recommended level of care did not consistently match the level of care services utilized by the individual. It

was unclear if reason for this inconsistency was due to clinician error in scoring tool, a resource issue, or client preference.

1. It is recommended that the organization review these inconsistencies and develop an approach to encourage improvement in the areas

identified.




Agency Name: Ausable Valley Community Mental Health Date of Visit: 5/23/2018

Reviewers Names: Dana Fuller, Julie Hudson

Reviewer 1 Reviewer 2 Consensus

L1 Program Support

4

L2 Eligibility/Client Identification

3

L3 LOCUS Completion

5

L4 Timeliness

5

L5 Assessment

4

L6 Training Approach

2

L7 Training Implementation

3

L8 Supervision

2

L9 Process Monitoring

1

L10 Outcome Monitoring

2

L11 Quality Assurance

2

L12 Individual Choice

2

Total Sum of Consensus Row 35

Consensus Row Sum Divided by 12 2.92

Total Mean Score 35

LOCUS MIFAST Fidelity Scale (Version 2/13/18)avcmh.s3-us-west-1.amazonaws.com/LOCUS MIFAST Report...

Documents

Transcript of LOCUS MIFAST Fidelity Scale (Version 2/13/18)avcmh.s3-us-west-1.amazonaws.com/LOCUS MIFAST Report...