The ATOMS Project: Measuring Assistive Technology Outcomes Dave L. Edyburn, Roger O. Smith, Todd D....

The ATOMS Project:Measuring Assistive Technology Outcomes

Dave L. Edyburn, Roger O. Smith, Todd D. Schwanke & Kelly S. Fonner

Center for Rehabilitation Sciences & Technology

Closing the Gap Conference, Minneapolis, MN October 17, 2002

View: Universal Access Features

October 2002Update: The ATOMS Project 2

© 2002 Center for Rehabilitation Sciences & Technology

Copyright

• This Microsoft PowerPoint file has been made available as an accessible, electronic handout for the participants of the presentation.

• You must obtain permission from the ATOMS Project before copying or further distributing this presentation.



ATOMS Projects Vitals

• Based at the University of Wisconsin – Milwaukee

• National consortium

• 5 year AT outcomes project

• NIDRR funded (US Dept of Education) DRRP - Disability and Rehabilitation Research Projects



ATOMS Project Consortium

Steve Mendelsohn

Helen HayesHospital



AT Outcomes Vision and Needs?

• What do you think an outcomes system should look like in 10 years?

• How would you want to use it?

Audience Vision

Audience Need



The Atoms Project Response

• Difficult questions for response, aren’t they?

• The ATOMS Project hopes to help clarify the field’s 10 year vision.



Overall ATOMS Project goal

• Explore, Pilot, and Test AT Outcome Measurement Ideas to Recommend an AT outcomes system

(Go Where No One Has Gone Before)



Our Overall Approach Reflected as the ATOMS Project Logo



Challenges to an AT outcomes system

• Diversity of perspectives of need for outcomes data

• Diversity of populations

• Diversity of domains for which we'd like outcomes data

• Requirements for a data system to use reliable and valid measures

• AT is only one of many interventions and it is an rarely used in isolation

• Outcomes methodology has advanced, but so have new outcomes measurement ideas



AT Outcome Model (1)- Context and Baseline

Environment

Task

Person

Function

BaselineContext



AT Outcome Model (2)- Interventions

Environment

Task

Person

Function

BaselineContext

Interventions:

Includingthe Use ofAssistive

Technology

Intervention Approaches



AT Outcome Model (3) - Outcome

Environment

Task

Person

Function

BaselineContext

Interventions:

Includingthe Use ofAssistive

Technology

Intervention Approaches

Enhanced

Function

Outcome



AT Outcome Model (4) - Interventions Detailed

Environment

Task

Person Functional

Performance

BaselineContext Intervention Approaches

Enhanced

Functional

Performance

Outcome

Reduce the Impairment

Use Assistive Technology Devices

and Services


and Services

Redesign the Activity

Compensate for the Impairment

Redesign the Environment

Use Personal Assistance



AT Outcome Model (5) - with Pre-interventions

Environment

Task

Person Functional

Performance


Enhanced

Functional

Performance

Outcome



and Services


and Services





Universal Design

Health Promotion

Pre-intervention



AT Outcome Model (6) - with Costs

Environment

Task

Person Functional

Performance


Enhanced

Functional

Performance

Outcome



and Services


and Services





Universal Design

Health Promotion

Pre-intervention

$$

$$

$$

$$



Face the Challenge or Run?

• Does that mean that a coordinated outcome system is too difficult and that we are stuck with a fragmented, haphazard, homemade, anarchistic system?

• Is a more cohesive and comprehensive AT outcomes approach possible?



Strategies

• Computerized data collection

• Centralized database

• Customized or multiple interfaces for various user perspectives

• Decision analysis data elicitation strategies



General ATOMS Project Timeline

Year 1-2: Needs assessment & field feedback

Year 2-3: Instrument/methodology exploration & development

Year 3-5: Pilot ideas

Year 5: Propose AT outcomes methods & system



ATOMS Specific Activities

1. Needs Assessment Field Scans Stakeholder Focus Groups Existing Database Analyses

2. Exploratory R&D Projects

(Instrument Development)

3. Abandonment Analysis



Field Scans Types

• Current Instruments (1,2)

• Newer methodologies & instrumentation (4,5,6)

• Feedback from field (focus groups)

• Literature reviews (3,7,8,10,12)

• Legal/Policy (9)

• Conference scientific reviews (11)



Field Scan 1 – Instrument Update and Review

Formal and informal tools

Commercial and program specific tools

Specific and general tool

• Instrument nomination form• Identification of gaps and overlaps• Searchable directory• Identification of type of measurement domains

addressed



Field Scan 2 – Coverage of AT in Current/Emerging Health & Rehabilitation Outcome Measures

~100 instruments selected for review (2 scored two-ways)Preliminary review reveals: N = 41 fail to acknowledge the use of assistive

technology in their outcome scoring N = 47 lower the outcome score if AT is used (nude

independence) (most examine limited types of AT) N = 24 allow for AT to elevate the outcome score,

but many do not differentiate among type of aids and assistance

Of the 102, 4 acknowledge that AT contributes to outcomes and isolates the outcome.



Field Scan 3 – Inventory of Measures Used in AT Research and Design Activity

Do AT developers use appropriate outcomes instruments and methodologies for their projects?

How severe is the problem? Or how is this perceived by product developers?

Request methodology of grant proposals from PI’s – 2001 funding (NIH n=34, NIDRR n=27)

Survey of product developers• Technology Exhibitors, RESNA&AOTA National

Conference 2002 n=78• Random sample (n=500/1100) from ABLEDATA

Directory of Manufacturers



Field Scan 4 – Ascertain Next Generation Data Collection Technology

• Factors considered for review of Technology & Methodology Content/outcomes information Scaling potentials Data collection processes/protocol Equipment (hardware/software) characteristics e.g.

interface, portability, cost, durability Data handling protocols Data Reporting

~ 50 hardware & software technologies being reviewed



Field Scan 5 – Evaluate Cost Comparison Methods

• Helen Hayes Hospital: Frances Harris, Ph.D.

• Initiating methodology review of cost comparison methods and how various approaches match the needs for AT Outcomes measurement procedures that include cost variables. Methods of measuring costs Methods of comparing costs

• Cost-comparison literature review relevant to assistive technology



Field Scan 6 – Assess Application of Decision Analytical Approaches

• Multi-attribute Utility (MAU) Theory Application analysis (selected articles) 47 Engineering articles using MAU methods 60 Health-related articles using MAU methods 32 General articles on MAU methodology

• Bayesian Approach analysis (initial search) 3797 articles, MEDLINE (1966-present) 140 ERIC (1967-present) 1573 Engineering Village (1970-present)



Field Scan 7 – Review Taxonomies of Outcome

What intersection of domains across taxonomies provide common language for a more universal AT outcomes discussions?

• e.g. Nagi Model (1965) ICIDH (1980, 1993) Rehabilitation Indicators (1983) NCMRR Research plan (1993) Quality of Life Taxonomy (Spilker & Revicki, 1996) Characterization of Rehabilitation Services (Duncan, Hoenig ,

Samsa , & Hamilton 1997) Institutes of Medicine Model (1997) ICIDH-2 draft (1997) ICF (2002)



Field Scan 8 – Identify Strategies for Isolating ATOutcomes

Compare methodologies that can isolate & quantify the outcomes of assistive technology. ~410 articles

e.g. Multi-variate regression analyses Structural equation modeling Direct consumer qualitative input (perceived benefit /

satisfaction of a device) Randomized controlled trials Sequential Clinical Trials Time-series concurrent differential (TSCD) Qualitative



Field Scan 9 – Describe Legal Implications of AT Outcomes Instrumentation

• What are legal & ethical issues related to AT outcomes systems? -- Steven Mendelsohn

Legal, responsible, & ethical data collection procedures

Potential legal ramifications of AT outcomes data (positive and negative)

Implications of AT outcomes for policy-making



Field Scan 10– Review of Special Education Technology Literature

• Review of 31 special education technology journals

• Reviews published in Journal of Special Education Technology (1999, 2000, 2001)

• >2700 articles• Iteration with outcome filter to identify

relevant articles.• Summarize and report findings.



Field Scan 11– Technology Conference Program Review

• Last two years of CTG, CSUN, RESNA, and TAM conferences reviewed as fugitive literature.

• Identify relevant papers on AT Outcomes.

• Summarize and report findings.



Field Scan 12– Chronology of Assistive Technology Outcomes Measurement

• Literature review reveals that At outcomes have been measured differently over the decades

• This is consistent with what types of outcomes studies have been performed and the mandate for accountability

• This field scan will develop a chronology map and accompanying discussion



Quiz on the field scans:

• Just Kidding….

• Lots of data from field scans. What do we hope to learn?

• Technical reports

• Compendium document

• Consensus meeting



Stakeholder Focus Groups

• Direct Feedback from field

• Consumer/User groups (4)

• Service Directors (for records & documentation assessment),

• Payors of AT devices & services,

• Researchers, Developers, & Manufacturers, and

• Parents & Caregivers.



Service Director Focus Group – April 2002

• Modified NGT developed current list of identified AT Outcomes data domains (“If we had a magic wand”)

1. Change in performance/function (body, structure, activity)2. Change in participation3. Usage and why or why not4. Consumer satisfaction (process, devices)5. Goal Achievement6. QOL7. Cost8. Demographics9. AT interventions (services + devices)10. Environment context

• List is similar to DeRuyter (1998)



Databases – National

• National Health Information Survey-Disability (NHIS-D)

• Assistive and information technology survey (NIDRR/RESNA/University of Michigan)



Databases – Clinical

• UW-Stout

• University at Buffalo (SUNY)

• HHH

• PROVAIL

• What data are being collected?

• Are available data sufficient for analysis?

• Is there an intersection of outcome data fields among existing service file systems?

• Can clinical programs adapt to collecting more data?



Clinical Database Update

• Data fields Commonalities (few) and differences (numerous) Setting & funding specific (education, vocation,

hospital, university) Stakeholders (variable) Electronic records (variable & minimal)

• Potentials VR data Willingness of clinical programs to modify data

collection



Exploratory R&D Projects (Instrument Development)

a) AT Device Inventory

b) AT Services Inventory

c) Cost Identification Feasibility Study

d) Assistive Technology Approach Isolation Measure (Subjective)

e) Web-based visualization

f) Environmental access assessment (WebAUDIT, MED-AUDIT)

g) Computer branching questioning (TTSS)

h) AT supplements to existing instruments (SFA-AT)



Public School AT Outcome Pilots & Collaborative Activity

a) Ohio

b) Colorado

Colorado

Ohio

Colorado



NIDRR DRRP Coordination

• CATOR (Consortium of Assistive Technology Outcomes Research) http://www.atoutcomes.org

• ATOMS Project

http://www.atoms.uwm.edu



Dissemination: What to Expect

• ATOMS Project Website Products

• Special Issues of Journals

• Conference Presentations

• 2003-2004 Symposium



Dissemination: ATOMS Project Products in the Works

AT Outcomes Primer Study Group Outline Test Your Knowledge of AT Outcomes FAQ’s Course Guide (syllabi and more) Drafts of Instruments Technical Reports Compilation of Needs & Current Outcome

Directions Implications for Next Generation AT Outcomes

System



Needs Assessment Participation Opportunities

• Conversation

• Product developer survey

• Instrument collection



Contacting the ATOMS Project

Completing interest survey

Web: www.atoms.uwm.edu

Email: [email protected]

Voice: (414) 229-6568

TTY: (414) 229-5628



Questions, Answers and Discussion

?

? ?

?

?

?

??



Thank-you…. . . . for your attention!

Slides about the universal access features of this PowerPoint presentation follow.



Universal Access Features

• The “speaker notes” associated with each slide in PowerPoint are used as a Universal Access feature.

• They contain long text descriptions of the graphics because it was not feasible to do this with PowerPoint’s “alternative text” function. The descriptions can be used by new presenters and presentation attendees, in addition to being used for accessibility by people with vision or cognitive impairments.



Viewing the Speaker Notes(does not work within PowerPoint Viewer 97 or 98)

• In the “Slide Show” view within PowerPoint: Windows: right click on the slide or use the context key

to bring up the context menu and then select “speaker notes”

Mac: [Ctrl] + click on the slide to bring up the context menu and then select “speakers notes”

• The notes can also be seen as a frame or pane in the “Normal” view or directly by using the “Notes Page” view.

• When in “Normal” view, F6 is used to switch between the slide, notes and outline panes respectively.

Go back to the opening presentation slide

The ATOMS Project: Measuring Assistive Technology Outcomes Dave L. Edyburn, Roger O. Smith, Todd D....

Documents

Transcript of The ATOMS Project: Measuring Assistive Technology Outcomes Dave L. Edyburn, Roger O. Smith, Todd D....