Motivational Interviewing Treatent Integrity (MITI)

6 Studies18-23

Validity Content: Derived from MISC factor structure18

Construct: Convergent validity with the MISC,18

construct validity via change in scores over time18

Reliability Internal Consistency: Poor19

Inter-rater: Adequate18,20

Clinical Utility Availability: Available in online Instructions: Excellent Format: Audio recordings, transcripts Training Time: 4 hours Administration Time: Unknown Interpretation: Easy

Diversified Placement Approach (DPA) Fidelity Scale

3 Studies9-11

Validity Content: Literature review and pilot testing9

Construct: Discriminates between DPA and non-DPA programs9,10

Reliability Internal Consistency: Not assessed

Inter-rater: Excellent9

Clinical Utility Availability: By email request9 Instructions: Unknown Format: Interview & Observation Training Time: Unknown Administration Time: Extensive Interpretation: Complex Role of the Clinician: Staff interviews

Paediatric Rehabilitation Observational measure of Fidelity (PROF)

2 Studies16,17

Validity Content: Delphi process16 and semi-structured interviews17

Construct: Unclear discriminant validity between frequency and quality scales16

Reliability Internal Consistency: Frequency Domain- Adequate16 Quality Domain- Excellent16

Inter-rater: Frequency Domain- Excellent16 Quality Domain- Poor16

Clinical Utility Availability: Available in online17 Instructions: Unknown Format: Video observation Training Time: 16 hours + experience as OT or PT Administration Time: 17 minutes per video Interpretation: Complex

Ayres Sensory Integration Fidelity Measure (ASIFM)

3 Studies1,7,8

Validity Content: Literature review,1 expert survey7

Construct: Distinguishes ASI from other interventions7 and ASI facilities from non-ASI facilities8

Reliability Internal Consistency: Excellent7

Inter-rater: Excellent7,8

Clinical Utility Availability: Available in two articles1,8 Instructions: Adequate Format: Live or video observation Training Time: 6 hours Administration time: Unknown Interpretation: Easy

Taxonomy of Behavior Change Techniques for Smoking Cessation

6 Studies24-28

Validity Content: Expert panel24

Construct: 14 competencies cited in source documents and RCTs24

Reliability Internal Consistency: Not assessed

Inter-rater: Excellent25

Clinical Utility Availability: Available in article24 Instructions: Excellent Format: Video observation Training Time: Unknown Administration Time: Unknown Interpretation: Easy

Tool for Measurement of Assertive Community Treatment (TMACT)

4 Studies12-15

Validity Content: Derived from DACTS, group of experts, pilot testing12

Construct: Sensitive to change over time,13 higher scores associated with less use of healthcare12

Reliability Internal Consistency: Not assessed

Inter-rater: Not assessed

Clinical Utility Availability: Available online Instructions: Adequate Format: Interview & observation Training Time: Unknown Administration Time: 1.5-2 days Interpretation: Easy Role of Clinician: Team surveys, interviews Role of Client: 3-5 clients interviewed

Intervention fidelity measures in rehabilitation: A systematic review Brittany Hand, MOT, OTR/L, Amy Darragh, PhD, OTR/L, FAOTA, & Andrew Persch, PhD, OTR/L

Discussion

Division of Occupational Therapy, School of Health and Rehabilitation Science, The Ohio State University

Specific Aim: §  To evaluate the quality (reliability, validity, clinical utility) of existing

rehabilitation intervention fidelity measures Search Strategy: §  Databases searched: Academic Search Complete, CINAHL, ERIC, Medline,

PsychINFO and Web of Science §  Search terms: (“fidelity” OR “intervention fidelity” OR “quality assurance,

health care”) AND (“rehabilitation” OR “occupational therapy” OR “physical therapy”)

§  Multistep review process Inclusion Criteria: §  Published in a peer-reviewed journal in English §  Development of an assessment that measures intervention fidelity in

rehabilitation OR evaluation of psychometrics of an assessment that measures intervention fidelity in rehabilitation

§  Fidelity assessment is observational in nature Quality Assessment: §  Data were extracted using the CanChild Outcome Measures Rating Form5 §  Data are summarized for all variables descriptively §  Quality ratings for the following variables were based on the indicated

criteria:

References & Acknowledgements

§  Fidelity in rehabilitation refers to faithfulness of an intervention to underlying therapeutic principles and clinical guidelines.1

§  Fidelity measures: §  Are necessary for conclusions to be confidently drawn about the

uniformity and efficacy of the intervention in research trials3,4 §  Ensure stakeholders of consistency between the treatment provided in

research trials and clinician-led interventions in the community §  It is essential that valid and reliable rehabilitation fidelity measures be

developed and utilized to ensure the consistent delivery of high-quality, evidence-based interventions in research trials and clinical practice.

Conclusions: §  Construct validity was evaluated using the known groups method,

hypothesis testing, and factor analytical approaches §  Most assessments had adequate-excellent reliability §  Clinical utility varied widely

Clinical Implication: §  Greater emphasis must be placed on the development and

implementation of fidelity measures to ensure uniformity in intervention delivery and high-quality, evidence-based care.

Limitations: §  Exclusively included observational fidelity measures in the field of

rehabilitation that were published in English Future Plans:

§  Additional validation of existing fidelity measures §  Development of novel fidelity measures:

§  Validity: Literature review, expert panel, factor analysis §  Reliability: Scoring of videos or audio recordings §  Clinical Utility: Clear instructions, relatively low training and

administration time, relatively simple interpretation, low clinician involvement

Background

Methods

Results

Reliability Coefficients6 Clinical Utility: Instructions5 Excellent ≥ 0.80 Clear, comprehensive, concise and available Adequate 0.60-0.79 Clear, concise but lacks some information

Poor <0.60 Not clear and concise or not available

Clinical Utility: Interpretation5 Easy Each part of the task can be completed within one hour with a minimal amount of

training and is easy for the average service provider to complete Complex Part of task cannot be completed within one hour and/or more than a minimal

amount of training is required

References: Available by email request (hand.58@osu.edu) Funding:

§  1R01HD074574-01 Multi-Site RCT of 3 Neurorehabilitation Therapies for Infants with Asymmetrical CP

§  5R01HD068345-03 Multi-Site RCT of Pediatric Constraint-Induced Movement Therapy

696 Articles Screened

36 Articles Reviewed

24 Articles Selected for Inclusion