Using ROI to promote simulation - laerdaltraining.com · Using ROI to Promote Simulation! Amar P....

Raleigh, North Carolina

WakeMed Health & Hospitals

Using ROI to Promote Simulation

Amar P. Patel, MS, NREMT-P, CFC Director Center for Innovative Learning

Gina Della Porta, MS Grants Specialist WakeMed Foundation

Overview

•  ROI •  ROE •  Educa/onal Principles & Kirkpatrick •  Data in simula/on courses •  Link simula/on to ROI •  Calcula/ons and examples •  Where can ROI be calculate from? •  ROE or ROI?

Objec/ves

•  Define Return of Investment (ROI). •  Describe how to calculate Return of Investment.

•  Discuss how to link simula/on and ROI. •  Demonstrate an ROI calcula/on u/lizing informa/on provided through audience feedback.

•  Determine 5 key areas ROI can be calculated from.

Questions?

•  How do we utilize existing technology to integrate a multi-disciplinary group?

•  How do we utilize “simulation” to teach multiple disciplines?

•  What are these opportunities?

A simple understanding of history

Circa 1930-1940

Average Age

37 Entertainment Software Association. (2010). Industry facts. Retrieved from

http://www.theesa.com/facts/index.asp

Growing Issues

•  “Medical procedures are becoming more numerous and more complex –medical knowledge has ‘hypertrophied.’” (Cooke, et al, 2006)

•  Clinical rota/ons are becoming more difficult to schedule.

•  Clinical seYng is not the place to “prac/ce” skills.

Cooke, M., Irby, D., Sullivan, W., & Ludmerer, K. (2006). American medical educa/on 100 years a]er the flexner report. The New England Journal of Medicine, 355(13), 1339-‐1344.

What does simula/on offer?

•  Repe//on which leads to proficiency •  Understanding of the cogni/ve and the prac/cal

•  Training opportuni/es •  Established objec/ves •  Clear expecta/ons •  Quality & Quan/ty •  Immediate applica/on

Funding

•  How do you secure funding? •  What does that mean? •  For what period of /me? •  Can a simula/on program be self-‐sufficient? •  What do investors want? •  Repor/ng on funding?

Consider these costs…

Developing a “sim” center

•  Are you looking to develop a new building or use an exis/ng building?

•  A cost is based on… – program objec/ves? – short & long term goals? – your target audience?

•  Why does it majer?

Investments

•  Program investment = $1 million •  Student count = 1,000 people/year

•  Cost per student = $1,000.00

•  What about intangibles? Long term expenses? Supplies? Gases? -‐-‐-‐ What does that cost?

What are the true financial needs?

Intangibles

•  Community Outreach •  Dona/ons •  High visibility –show you mean business!

•  Does there need to be a business model?

•  What is “free” and what “costs money?”

Is simula)on profitable?

Is it?

Powerful Mo>vators

Smith, R. (2009). Embracing game technology for medical educa/on. Retrieved from hjp://www.peostri.army.mil/CTO/FILES/2009_RSmith_MT3.pdf

} Mo/ve 1: Lower Cost Mo/ve 2: Bejer Access to Symptoms/Cases Mo/ve 3: Reduced Training Time Mo/ve 4: Reduced Errors

Business Plan

Defined as:

“A wrijen document which describes the business, its objec/ves, its strategies, the market in which it operates and its financial forecasts.” (Markow, 2011)

Markow, M. (2011). Business terms glossary. BusinessWings. Retrieved from hjp://www.businesswings.co.uk/ar/cles/Business-‐terms-‐glossary

Consists of:

SBA.gov. (2011). Essen/al elements of a good business plan. Retrieved from hjp://www.sba.gov/category/naviga/on-‐structure /star/ng-‐managing-‐business/star/ng-‐business/wri/ng-‐business-‐plan/essen/al-‐elements-‐good-‐busines

•  Business Plan Execu/ve Summary* • Market Analysis •  Company Descrip/on •  Organiza/on & Management • Marke/ng & Sales Management •  Service or Product Line •  Funding Request •  Financials •  Appendix

Remember this:

“…hospitals typically measure ROI from a business perspec/ve—cost, revenues or opera/ng efficiencies —but many benefits of clinical applica/ons fall into quality and safety realms that do not easily translate into dollars.” (Page, 2010)

Page, D. (2010). IT’s return on investments is tricky to pin down. Hospital & Health Networks. Retrieved from hjp://www.hhnmag.com/hhnmag_app/jsp /ar/cledisplay.jsp?dcrpath=HHNMAG/Ar/cle/data/06JUN2010/1006HHN_Fea_MostWired&domain=HHNMAG

A business plan will only make you stronger…

The reality is:

"We should measure clinical ROI in terms of measurable impact on pa/ent care.” (Page, 2010)

Page, D. (2010). IT’s return on investments is tricky to pin down. Hospital & Health Networks. Retrieved from hjp://www.hhnmag.com/hhnmag_app/jsp /ar/cledisplay.jsp?dcrpath=HHNMAG/Ar/cle/data/06JUN2010/1006HHN_Fea_MostWired&domain=HHNMAG

A Return on Investment

Tradi/onal ROI is:

Investopedia. (n.d.). Return on investment -‐ ROI. Retrieved from hjp://www.investopedia.com/terms/r/returnoninvestment.asp

“…if an investment does not have a posi/ve ROI, or if there are other opportuni/es with a higher ROI, then the investment should be not be undertaken.” (“Investopedia,” n.d.)

“…performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments.” (“Investopedia,” n.d.)

Solu/on Matrix. (2011). Return on investment. Retrieved from hjp://www.solu/onmatrix.com/return-‐on-‐investment.html

Best Prac/ce Commijee. (2002). The Value of IT investments: It’s not just return on investments. Retrieved from hjp://www.cio.gov/documents /thevalueof_it_investments.pdf

Tradi/onal ROI

•  “ROI evaluates an investment’s poten/al by comparing the magnitude and /ming of expected gains to the investments costs.” (“Best,” 2002)

gains – investments costs

$700,000 (saved) – $500,000 sim center $500,000

Investments costs ROI =

= 40%

Healthcare ROI is:

“the amount of improvement in care brought about by a certain investment. ROI can also refer to the theory that if you invest in health care quality now, then the quality of care for pa/ents will improve in the future.” (“Robert Wood,” 2011)

Robert Wood Johnson Founda/on. (2011). Glossary of healthcare terms. Quality/Equality. Retrieved from hjp://www.rwjf.org/qualityequality/glossary.jsp

Healthcare ROI

1.  Examines impact to system processes. 2.  Determines a cost to each process related

change and if the cost was favorable. 3.  That cost is consider the “return on

investment.”

((# Process errors found x Es/mated cost per error)-‐SIM COSTS) x (% $ Paid Out for errors) SIM COSTS

Healthcare ROI

ROI Total: 5.93% Number of Process Errors Found: 14 Es/mated Cost per Error: $35,000.00 % Hospital $ Paid Out: 0.25% SIM Costs: $19,821.00

} $490,000

Healthcare ROI Issues

•  Who is actually providing the training that is resul/ng in the ROI being created?

•  How are you determining process issues? •  Are there clear “program” objec/ves? •  How are you being funded? •  Who benefits from the ROI? •  Who are your stakeholders?

Return on Investment “Defensive isola/on approach that separates func/ons.” “Defines training as an end in itself.” “Value defined by predetermined formula.” “Focus on single metric of numeric proof.” “Complex, rigid, and expensive.”

Kirpatrick, JD, Kirkpatrick, WK. (2010). ROE’s rising star: Why return on expecta/ons is geYng so much ajen/on. Training + Development. Retrieved from hjp://www.astd.org/TD/Archives/2010/Aug/Free/1008_ROEs_Rising_Star.htm

Let’s try a few calcula)ons…

To create ROI, we must develop a “value measuring methodology.” (“Best,” 2002)


“Value Measuring Methodology”

1.  Develop a decision framework •  Customers •  Social •  Government Opera/onal/Founda/onal •  Strategic/Poli/cal •  Government Financial

2.  Perform an alterna/ve analysis 3.  Gather all the informa/on 4.  Communicate and document


What does all this mean? •  There has been this ongoing debate in medical simula/on about

the ability to prove ones worth -‐then been able to u/lize that proof to secure addi/onal funding from donors.

•  Does simula/on make an impact in educa/on?

•  Does it make an impact in healthcare, in pa/ent safety, in changes in human behavior or system processes?

•  There are numerous issues that result from trying to jus/fy that simula/on was the sole reason a system benefited. –  Was it the single cause or was there other educa/onal programs that

helped shaped changes? –  What happens if the data shows decline? Was simula/on to blame?

The mo/va/on…

•  The want and desire to include simula/on. It’s the “in” thing to do…

•  The data shows improvement in cogni/ve learning.

•  The data shows improvement in prac/cal applica/on and reten/on.

But…

You have to think through it!

Business side

•  Who are your “key stakeholders”? –  Internal – External

•  Have you considered developing a business plan?

•  Do you have opera/onal & strategic goals? •  Who is managing your program data & finances?

When can I see a return?

Making the case

•  Understand the program requirements •  Establish a “board of directors” or “steering commijee”

•  Involve key stakeholders in important decisions •  Provide updates •  Show evidence that simula/on works •  Show other people’s ROI, if allowed •  BE PATIENT!

People

•  Who are the experts? •  Where do you find experts?

•  How many people do you need to succeed?

Kirkpatrick Model

‘Four’ Level Model

•  Level 1: Par/cipant Reac/on

•  Level 2: Learning

•  Level 3: Job Impact

•  Level 4: Business Impact

Trainingcheck.com. (n.d.). Kirkpatrick ROE model of training evalua/on. Retrieved from hjp://trainingcheck.com/training-‐evalua/on /kirkpatrick-‐ROE-‐model-‐of-‐training-‐evalua/on/

Level 1

REACTION: “how the learners react to the learning process”

Big Dog & Lijle Dog’s Performance Juxtaposi/on. (n.d.). Kirkpatrick’s four-‐level training evalua/on model. Retrieved from hjp://www.nwlink.com /~donclark/hrd/isd/kirkpatrick.html

Level 2

LEARNING: “the extent to which the learners gain knowledge and skills”


Level 3

BEHAVIOR: “capability to perform the learned skills while on the job”


Level 4

RESULTS: “includes such items as monetary, efficiency, moral, safety, etc.”


The heart of any model is understanding the basic principles…

Return on Expecta>on

Main principles

•  “Business/[organiza/onal] objec/ves are seen as a star/ng point.” (“Trainingcheck.com,” n.d.)

•  “’Return on Expecta/on’ (ROE) is key” (“Trainingcheck.com,” n.d.)

•  “Collec/ve efforts are needed throughout an [organiza/on] to achieve success.” (“Trainingcheck.com,” n.d.)

Trainingcheck.com. (n.d.). Kirkpatrick ROE model of training evalua/on. Retrieved from hjp://trainingcheck.com/training-‐evalua/on /kirkpatrick-‐ROE-‐model-‐of-‐training-‐evalua/on/

Return on Expecta/on “Proac/ve, business partnership approach that unified teams.” “Defines training as a contributor to key business results.” “Value defined by business stakeholders in coopera/on with training.” “Focus on comprehensive evidence and a compelling story of value.” “Easy to understand, flexible, and cost-‐effec/ve.”


The key is…

Numeric data


Tes)monials


Stories


“The combina)on of evidence shapes a story that appeals to and is understood by everyone.”


You must s/ll:

1.  Iden/fy key stakeholders. 2.  Create a cross-‐func/onal advisory board. 3.  Develop a strategic ini/a/ve, and use an

ROE process.


Collect Data!

Target U/liza/on

•  How much simula/on is enough?

Can I make money?

Genera/ng Revenue

•  Is it possible? •  Can I sell programs? •  How do I jus/fy selling simula/on, crea/ng ROI or ROE, and keeping up with my internal customers?

•  What programs do I sell? •  What is required? Contracts? Legal concerns?

GeYng the word out…

•  Take home items (trinkets?) – Pins – Pens – Folders

•  Journal ar/cles, news stories •  Demonstra/ons •  Discussions •  Presenta/ons

} “a Return”

A Return = Outcomes + Simula>on

The Link

•  Clinical outcomes •  Variability of outcomes •  Transi/on to prac/ce •  Pa/ent safety •  Change in behavior as a result of educa/on •  Iden/fying cogni/ve and prac/cal gaps

Where can I calculate “a return” from?

1.  System-‐wide process changes 2.  Changes in programs as a result of simula/on 3.  Increase in program demand 4.  Iden/fica/on of gaps 5.  The stories….

What is the difference between ROI & ROE?

ROE verse ROI


Overview

•  ROI •  ROE •  Educa/on Principles & Kirkpatrick •  Data in simula/on courses •  Link simula/on to ROI •  Calcula/ons and examples •  Where can ROI be calculate from? •  ROE or ROI?

References Best Prac/ce Commijee. (2002). The Value of IT investments: It’s not just return on investments. Retrieved from hjp://www.cio.gov/documents

/thevalueof_it_investments.pdf Big Dog & Lijle Dog’s Performance Juxtaposi/on. (n.d.). Kirkpatrick’s four-‐level training evalua/on model. Retrieved from hjp://www.nwlink.com/~donclark/hrd

/isd/kirkpatrick.html Cooke, M., Irby, D., Sullivan, W., & Ludmerer, K. (2006). American medical educa/on 100 years a]er the Flexner report. The New England Journal of Medicine, 355(13),

1339-‐1344. Entertainment So]ware Associa/on. (2010). Industry facts. Retrieved from hjp://www.theesa.com/facts/index.asp Investopedia. (n.d.). Return on investment -‐ ROI. Retrieved from hjp://www.investopedia.com/terms/r/returnoninvestment.asp Kirpatrick, JD, Kirkpatrick, WK. (2010). ROE’s rising star: Why return on expecta/ons is geYng so much ajen/on. Training + Development. Retrieved from

hjp://www.astd.org/TD/Archives/2010/Aug/Free/1008_ROEs_Rising_Star.htm Markow, M. (2011). Business terms glossary. BusinessWings. Retrieved from hjp://www.businesswings.co.uk/ar/cles/Business-‐terms-‐glossary Page, D. (2010). IT’s return on investments is tricky to pin down. Hospital & Health Networks. Retrieved from hjp://www.hhnmag.com/hhnmag_app/jsp

/ar/cledisplay.jsp?dcrpath=HHNMAG/Ar/cle/data/06JUN2010/1006HHN_Fea_MostWired&domain=HHNMAG Robert Wood Johnson Founda/on. (2011). Glossary of healthcare terms. Quality/Equality. Retrieved from hjp://www.rwjf.org/qualityequality/glossary.jsp SBA.gov. (2011). Essen/al elements of a good business plan. Retrieved from hjp://www.sba.gov/category/naviga/on-‐structure/star/ng-‐managing-‐business

/star/ng-‐business/wri/ng-‐business-‐plan/essen/al-‐elements-‐good-‐busines Smith, R. (2009). Embracing game technology for medical educa/on. Retrieved from hjp://www.peostri.army.mil/CTO/FILES 2009_RSmith_MT3.pdf Solu/on Matrix. (2011). Return on investment. Retrieved from hjp://www.solu/onmatrix.com/return-‐on-‐investment.html Trainingcheck.com. (n.d.). Kirkpatrick ROE model of training evalua/on. Retrieved from hjp://trainingcheck.com/training-‐evalua/on

/kirkpatrick-‐ROE-‐model-‐of-‐training-‐evalua/on/

Sample RFP

ANY QUESTIONS??

Raleigh, North Carolina

WakeMed Health & Hospitals

Using ROI to Promote Simulation

Amar P. Patel, MS, NREMT-P, CFC Director Center for Innovative Learning

Gina Della Porta, MS Grants Specialist WakeMed Foundation

Using ROI to promote simulation - laerdaltraining.com · Using ROI to Promote Simulation! Amar P....

Documents

Transcript of Using ROI to promote simulation - laerdaltraining.com · Using ROI to Promote Simulation! Amar P....