Evaluating Return on Investment of Statewide ... · The return on investment for demographic data...
Transcript of Evaluating Return on Investment of Statewide ... · The return on investment for demographic data...
![Page 1: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/1.jpg)
Evaluating Return on Investment of Statewide Implementation of Electronic Demographic Data Reporting
Bob Johnson, Tony Steyermark, and Amy Gaviglio
Minnesota Department of Health
http://www.health.state.mn.us/e-health/1
![Page 2: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/2.jpg)
Association of State and Territorial Health Officials
Karl Ensign, Maggie Carlin
Minnesota Department of Health
Jennifer Fritz
Stephan Gildemeister
Jill Simonetti
Karen Soderburg
Acknowledgements
2
![Page 3: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/3.jpg)
1) Public Health / e-Public Health
2) The Newborn Screening Program’s Process Improvement Project
3) Return on Investment
4) Lessons Learned
5) Next Steps
Outline
3
![Page 4: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/4.jpg)
APHL Mission: Improve the health of the public and achieve equity in health status
MDH Mission: Protecting, maintaining and improving the health of all Minnesotans
Newborn Screening:
NJ: The mission of the Newborn Bloodspot Screening Follow-up Program is to promote and protect the health of all newborns identified as having out-of-range results by the Newborn Screening Laboratory.
FL: The mission and primary goal of the Florida Newborn Screening Program is to ensure that all newborns screened receive appropriate, high-quality laboratory and follow-up services.
TX: The Newborn Screening (NBS) Clinical Care Coordination Program seeks to decrease the morbidity and mortality of infants born in Texas through customer-oriented, high quality newborn screening follow-up, case management and outreach education.
Public Health
4
![Page 5: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/5.jpg)
e-Public Health
• Not a new definition of the core mission and functions of governmental public health – Mission stays the same
• Captures the impact of the digital era on the information roles and practices of agencies - How we do our business changes so as to better carry out our mission
• e-health is the adoption and effective use of health information technology (HIT) to improve health care quality, increase patient safety, reduce health care costs, and enable individuals and communities to make the best possible health decisions. (MDH definition)
5
![Page 6: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/6.jpg)
The Process Improvement: The MNScreen Project
Step 1 Step 2 Step 3
Demographics directly from EHR
Screening results directly from
screening devices
Integrated Newborn Record
6
![Page 7: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/7.jpg)
The Process Improvement: Why go Electronic?
7
• Previous Experience with Newborn Blood Spot Screening
• Missing demographic information sometimes delays result reporting
• Illegible handwriting led to errors
• Legal name often not available
![Page 8: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/8.jpg)
(Presumed) Value-add of Electronic Data Exchange (less, or no, human dependency):
• Simpler workflows = fewer error points
• No (less) manual data entry = higher data quality
• No (less) manual data entry = time savings for BC and NBS staff
• Fewer human touchpoints = more time for other work
• Improved data quality = less NBS staff time correcting data
• Improved data validity = less Birth Center time providing correct data
• Faster receipt of data for the NBS Program
The Process Improvement: Manual / Paper to Electronic
8
Main Return on Investment Outputs
![Page 9: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/9.jpg)
• ROI is a form of cost analysis that compares the net costs of an intervention with its net benefits in financial or monetary terms.
• Because ROI follows a business model, the goal is to realize a positive case return. A positive ROI means the investment gains compare favorably to the investment costs.
Return on Investment
9
![Page 10: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/10.jpg)
What's Your ROI? A Web-based Tool to Estimate Economic Returns on Investments for Public Health Agency Projects
http://www.astho.org/programs/evaluation/
For more information, contact Karl Ensign, chief, performance improvement, research and evaluation at ASTHO.
Phone: (571) 527-3143Email: [email protected]
ROI – the ASTHO Tool
10
![Page 11: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/11.jpg)
Our Process
11
![Page 12: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/12.jpg)
0
10
20
30
40
50
60
70
80
90
100
5/5/2014 11/5/2014 5/5/2015 11/5/2015 5/5/2016 11/5/2016 5/5/2017
Contract signed with vendor All birth hospitals live!
6/19/2017
ROI – the ASTHO Tool: Project PhasesN
umbe
r of B
irth
Hos
pita
ls L
ive
Pilot Live1/22/15
![Page 13: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/13.jpg)
0
20
40
60
80
100
5/5/2014 11/5/2014 5/5/2015 11/5/2015 5/5/2016 11/5/2016 5/5/2017
ROI – the ASTHO Tool: Project Phases
Num
ber o
f Birt
h H
ospi
tals
Live
Phase Description Dates Activities
Plan Planning the project May 2013 – May 2014 Requirements gathering, RFP
Do Initial implementation of the project May 2014 – May 2015 Contract Yr 1, initiate implementation
Check Implementation is reviewed May 2015 – May 2016 Contract Yr 2, adjust/continue implementation
Act Adjustments learned from early implementation are made
May 2016 – May 2019 Contract Yrs 3 – 5, complete implementation, modifications
Act 2 Adjustments continue to be made May 2019 – May 2024 Contract Yrs 6 – 10, modifications, enhancements
![Page 14: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/14.jpg)
• The ROI tool makes comparisons over time, using:
• Investment costs - Planning and implementing the initiative
• Routine operating costs - Operating costs of the program, service line, or operational unit that is to be altered, improved or changed by the initiative
• Outputs or outcomes - Additional benefits realized through the initiative
ROI – the ASTHO Tool: Inputs & Outputs
14
![Page 15: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/15.jpg)
ROI – the ASTHO Tool: Inputs & Outputs
15
Investment Costs ($)
MDH Staff Requirements gathering, RFP
Training
Implementation
Birth Center Staff IT – ADT feed build
OZ Systems Contract NANI; Yr 1, Yr 2, ongoing
![Page 16: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/16.jpg)
ROI – the ASTHO Tool: Inputs & Outputs
16
Operating Costs ($)
MDH Staff Manual data entry
STFU, data quality
Ongoing informatics support
Birth Center Staff • Manual data entry onto card• Tracking down
missing/incomplete data
![Page 17: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/17.jpg)
ROI – the ASTHO Tool: Inputs & Outputs
17
Outputs / Outcomes ($)
Change in production time (minutes)
Manual data entry
![Page 18: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/18.jpg)
ROI – The ASTHO Tool
18
![Page 19: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/19.jpg)
ROI – The ASTHO Tool
19
![Page 20: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/20.jpg)
ROI – The Results
20
![Page 21: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/21.jpg)
The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific inputs and assumptions, is $0.77.
For every $1.00 spent on the project, the fiscal return is $0.77
ROI – The Results
21
![Page 22: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/22.jpg)
ROI – the Results
22
Phase Phase 1: Plan Phase 2: Do Phase 3: Check Phase 4: Act 1
Phase 5: Act 2 Cumulative
Date May 2013 –May 2014
May 2014 –May 2015
May 2015 –May 2016
May 2016 –May 2019
May 2019 –May 2024
ROI $0.20 $0.31 $0.46 $0.77 $0.77
![Page 23: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/23.jpg)
Retrospective vs Prospective
Lessons Learned – Ways to use the ASTHO ROI tool
23
![Page 24: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/24.jpg)
Retrospective vs Prospective
Retrospective• Was it worth it?
• How do you assign “worth”? Fiscal? Mission?
• Could the implementation have been more efficient?
• Did changes in workflows lead to expected changes in outcomes?
• What could we have done differently in the project planning & implementation?
• What can we do differently in the coming years of the project?
Lessons Learned – Ways to use the ASTHO ROI tool
24
![Page 25: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/25.jpg)
Retrospective vs ProspectiveProspective
• Will it be worth it?• How do you assign “worth”? Fiscal? Mission?
• If you want to achieve 0 net cost, how can I structure the project to achieve that? (e.g. contract cost, planning costs, training costs)
• If you to achieve 0 net cost, and can estimate fixed costs, what does your Δ output need to be?
• Δ time savings?
• Δ data quality? (e.g. amendments?)
Lessons Learned – Ways to use the ASTHO ROI tool
25
![Page 26: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/26.jpg)
Lessons Learned
26
![Page 27: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/27.jpg)
What data will you need?
• Time studies for the planning
• Time studies for IT implementation, training, contract work
• Time studies on public health staff & birth center data entry
• How to quantify Δ in data quality?
• What are staff doing with the time saved? Does that translate to better health outcomes?
• How do you quantify changes in health outcomes?
Lessons Learned: What we wished we knew
27
![Page 28: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/28.jpg)
1) Continue to validate our assumptions
2) Continue to quantify Δ in data quality, and integrate into ROI model (e.g. time spent on amended reports; time spent tracking down missing data)
3) Quantify how Δ time production results in changes in health outcomes (e.g. what else are public health staff doing now with time saved)
4) Quantify how Δ in data quality results in changes in health outcomes (e.g. how does fewer manual entry errors translate to better health outcomes)
Next Steps
28
![Page 29: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/29.jpg)
Lessons Learned
29
![Page 30: Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific](https://reader033.fdocuments.us/reader033/viewer/2022050520/5fa4188cd91e9757be2a9195/html5/thumbnails/30.jpg)