11/25/2015

1

Analytics, Big Data, and Partnerships

Connie White Delaney, PhD, RN, FAAN, FACMI

Dean & Professor, School of Nursing*

Tom Clancy, PhD, MBA, RN, FAAN

Clinical Professor & Associate Dean for Faculty Practice, Partnerships, & Professional Development*

Karen Monsen, PhD, RN, FAAN

Associate Professor & NI Specialty Coordinator*

Judy Pechacek, DNP, RN

Clinical Assistant Professor & Director of DNP Programs*

* University of Minnesota

The presenters have nothing to disclose

November 7, 2015

8:30-4:00

Orlando, FL USA

#27FORUM

M19

Knowledge Discovery Data Analytics Methods:

Evaluating and Optimizing Healthcare Value Using Big Data Methods

Karen A. Monsen, PhD, RN, FAAN

December 7, 2015

mons0122@umn.edu

11/25/2015

2

Three Major Shifts of Big Data Research

• The first is the ability to analyze vast amounts of data about a topic rather than be forced to settle for smaller sets.

• The second is a willingness to embrace data's real-world messiness rather than privilege exactitude.

• The third is a growing respect for correlations rather than a continuing quest for elusive causality.

• Viktor Mayer-Schönberger and Kenneth Cukier: Big Data - A revolution that will transform how we live, work and think; John Murray Publishers, London, 2013

“More, Messy, Good Enough” (p. 12, Mayer-Schonberger & Cukier, 2013)

• ‘Big Data’ changes the scientific paradigm

• More data means less sampling error

• Messy means we can try to understand and account for the biases inherent within observational data

• Good enough means we can let go of our fixation on causation

• Description

• Pattern Discovery

• Hypothesis generation

• Letting the data give voice to nurses and patients

Hey, T., Tansley, S., & Tolle, K. (2009). The fourth paradigm: Data intensive

scientific discovery. Redmond, WA: Microsoft Research.

Mayer-Schonberger, V. & Cukier, K. (2013) Big Data: A revolution that will

transform how we live, work, and think. Houghton Mifflin Harcourt, Boston.

11/25/2015

3

Nursing Data Nursing Context Data

Nursing Minimum Data Sethttp://www.nursing.umn.edu/prod/groups/nurs/@pub/@nurs/documents/asset/nurs_71413.pdf

• a minimum set of elements of information with uniform definitions and categories concerning the specific dimensions of nursing, which meets the information needs of multiple data users in the health care system.

• Client characteristics & outcomes

• Nursing assessments & interventions

Nursing Management Minimum Data Sethttp://www.nursing.umn.edu/icnp/usa-nmmds/

• core essential data needed to support the administrative and management information needs for the provision of nursing care. The standardized format allows for comparable nursing data collection within and across organizations.

• Nurse and health system characteristics

• Nurse and health system credentials

Werley HH. Nursing minimum data: abstract tool for

standardized comparable, essential data. Am J Public Health.

1991;81(4):421–6. doi: 10.2105/AJPH.81.4.421.

Huber D, Delaney C. The American Organization of Nurse

Executives (AONE) research column. the Nursing Management

Minimum Data Set. Appl Nurs Res. 1997;10:164-165.

Recognized Nursing TerminologiesAmerican Nurses Association

Terminology Nursing Specific Items from the NMDS

Nursing Problem Nursing Intervention Nursing Outcome Nursing Intensity

Nursing Specific Terminologies

NANDA (1992) x

NIC (1992) x

NOC (1997) x

The above three terminologies must be used together to obtain information about the nursing problem (diagnosis), intervention and

outcome. The below terminologies all have terms for the nursing problem, intervention, and outcome.

CCC (HHCC) (1992) x x x

PNDS (1997) x x x

ICNP (2000) x x x

Interdisciplinary Terminologies

SNOMED-CT (1999) x x x

SNOMED-CT Nursing Subset x

LOINC (2002) x

Omaha System (1992) x x x

Sewell, J. P. & Thede, L. Q. (2012). Nursing and Informatics: Opportunities and

Challenges. Nursing Documentation in the Age of the EHR. Available at:

http://dlthede.net/informatics/Chap16Documentation/anarecterm.html

http://dlthede.net/informatics/Chap16Documentation/chap16.html

11/25/2015

4

Martin KS. (2005). The Omaha System: A key to practice, documentation,

and information management(Reprinted 2nd ed.). Omaha, NE: Health

Connections Press.

Big Data Laboratory

• 2010: Dean Delaney invited the Omaha System Partnership for Knowledge Discovery and Healthcare Quality within the University of Minnesota Center for Nursing Informatics

• Scientific teams

• Affiliate members

• Data warehouse

• 2010: Dean Delaney invited the Omaha

System Partnership for Knowledge

Discovery and Healthcare Quality within the

University of Minnesota Center for Nursing

Informatics

– Scientific teams

– Affiliate members

– Data warehouse

11/25/2015

5

Using a Logistical Mixed-effects Model with Nursing Data

• How do nurses and interventions contribute to variability in patient and population health?

This research is partially supported by the National Science Foundation

under grant # SES-0851705, and by the Omaha System Partnership.

Monsen, K. A., Chatterjee, S. B., Timm, J. E., Poulsen, J. K., & McNaughton, D.

B. (2014). Factors explaining variability in health literacy outcomes of public

health nursing clients. PHN. Epub ahead of print: doi: 10.1111/phn.12139.

• Nurse (17%)

• Client (50%)

• Problem (17%)

• Intervention (17%)

• Client age was significantly positively associated with knowledge benchmark attainment in all models

Using Data Visualization to Detect

Client Risk PatternsMonsen, K. A. et al., 2014

Each image (sunburst) was created in

d3 from public health nursing

assessment data for a single patient.

Data were generated by use of the

Omaha System signs and symptoms

and Problem Rating Scale for

Outcomes

Key:

• Colors = problems

• Shading = risk

• Rings = Knowledge, Behavior, and

Status

• Tabs = signs/symptoms

Documentation patterns suggest a

comprehensive, holistic nursing

assessment.

Kim et al. found that the presence of

mental health signs and symptom

tends to be associated with more

diagnostic problems and worse patient

condition

Kim, E., Monsen, K. A., Pieczkiewicz, D. S. (2013). Visualization of Omaha System data enables

data-driven analysis of outcomes. American Medical Informatics Association Annual Meeting,

Washington D. C. Funded by a gift from Jeanne A. and Henry E. Brandt.

11/25/2015

6

Population Perspectives on Assessment

• Our assessments may be guided by evidence and policy

• Required documentation protocols

• Population characteristics

• Maternal-child health

• Diabetes

• Tuberculosis

• Community-dwelling elders

Diabetes

IncomeNutrition

Physical activity

Medication regimen

Skin

Sleep and rest patterns

Pain

Circulation

Digestion-hydration

11/25/2015

7

Tuberculosis

IncomeCommunicable/infectious condition

Medication regimen

Health care supervision

Interpersonal relationship

Neighborhood/workplace safety

Social contact

Community Dwelling Elders

Income

ResidenceMental health

Personal care

Abuse

Substance use

Neglect

Cognition

Neuro-musculo-skeletal function

11/25/2015

8

Using Generalized Estimating Equations for Cohort Comparison

• Mothers with intellectual disabilities have twice as many problems as mothers without intellectual disabilities

• Receive more public health nursing service

• Twice as many encounters and interventions

• Show improvement in all areas

• Do not reach the desired health literacy benchmark in Caretaking/parenting

Monsen, K. A., Sanders, A. N., Yu, F., Radosevich, D. M, & Geppert, J. S. (2011). Family

home visiting outcomes for mothers with and without intellectual disabilities. Journal of

Intellectual Disabilities Research, 55(5), 484-499. doi:10.1111/j.1365-2788.2011.01402.x

Using Graphing Methods with Multilevel KwayPartitioning to Form Non-Overlapping Intervention Clusters

This research was supported by a Midwest Nursing Research Society New Investigator

Seed Grant. Monsen, K. A., Banerjee, A., & Das, P. (2010). Discovering client and

intervention patterns in home visiting data. Western Journal of Nursing Research, 32(8),

1031-1054. doi:10.1177/0193945910370970

11/25/2015

9

Using Kaplan-Meier Curves to Depict Problem Stabilization

This research was supported by the National Institute of Nursing Research (Grant #P20 NR008992;

Center for Health Trajectory Research). The content is solely the responsibility of the authors and does

not necessarily represent the official views of the National Institute of Nursing Research or the National

Institutes of Health. Monsen, K. A., McNaughton, D. B., Savik, K., & Farri, O. (2011). Problem

stabilization: A metric for problem improvement in home visiting clients. Applied Clinical Informatics, 2,

437-446 http://dx.doi.org/10.4338/ACI-2011-06-RA-0038

COMPREHENSIVE WOUND CARE

BASIC

WOUND

CARE

Treatments &

procedures

Case

management

Surveillance

Monitoring

Teaching, guidance, & counseling

Informing

Providing Therapy

Using Inductive and Deductive Approaches to Create Overlapping Intervention Groups

Relationships between

four intervention

grouping/clustering

methods for

wound care.

Monsen, K. A., Westra, B. L., Yu, F., Ramadoss, V. K., & Kerr, M. J. (2009). Data

management for intervention effectiveness research: Comparing deductive and inductive

approaches. Research in Nursing and Health, 32(6), 647-656. doi:10.1002/nur.20354

11/25/2015

10

Using Receiver Operating Curves to Understand Model Fit

• Comparison of Intervention Modeling Approaches and Hospitalization Outcomes for Frail and Non-frail Elderly Home Care Patients

Monsen, K. A., Westra, B. L., Oancea, S. C., Yu, F., & Kerr, M. J. (2011). Linking home care

interventions and hospitalization outcomes for frail and non-frail elderly patients.

Research in Nursing and Health, 34(2), 160-168. doi:10.1002/nur.20426. NIHMS274649

Using Logistic Regression to Associate Home Care Interventions and Hospitalization Outcomes

• Too little care may result in hospitalization when patients have more intensive needs

• Frail elders are more likely to be hospitalized if they have low frequencies of four skilled nursing intervention clusters

Monsen, K. A., Westra, B. L., Oancea, S. C., Yu, F., & Kerr, M. J. (2011). Linking home care

interventions and hospitalization outcomes for frail and non-frail elderly patients.

Research in Nursing and Health, 34(2), 160-168. doi:10.1002/nur.20426. NIHMS274649

11/25/2015

11

Using Pattern Comparison Pre- and Post-Intervention to Demonstrate Intervention Effectiveness

Knowledge scores across problems over time• Pre-intervention, patterns by race/ethnicity

• Post-intervention, patterns by problem

Benchmark = 3

Monsen, K. A., Areba, E. M., Radosevich, D. M., Brandt, J. K., Lytton, A. B., Kerr, M. J., Johnson, K. E., Farri, O,

& Martin, K. S. (2012). Evaluating effects of public health nurse home visiting on health literacy for

immigrants and refugees using standardized nursing terminology data. Proceedings of NI2012: 11th

International Congress on Nursing Informatics, 614..

Toward Personalized Algorithms to Improve Nursing Care Quality and Efficiency

• Using data from PHN documentation for 1,618 low income/high risk clients including 113,989 interventions, determine the feasibility of data-driven personalized care approaches using a machine learning approach to answer resource-related research questions:

(1) which clients will need more services,

(2) which clients will have better outcomes if they receive more services

(3) which intervention patterns are used to address the Oral health problem

(4) how to personalize interventions to maximize client outcomes based on client characteristics.

Chih-Lin Chi, PhD, MBA; Brady Alsaker, MN, RN; Karen A. Monsen, PhD, RN, FAAN

Manuscript under review

11/25/2015

12

Methods

Question 1 (intervention need) method employed support vector machines in Matlab softwareInput: Client characteristics (demographics and signs/symptoms from first encounter)Output: Intervention resource across all clients (compared to 50th and 75th percentiles)

Question 2 (responsiveness) method employed support vector machines in Matlabsoftware with sensitivity analysis

Input: Client characteristics (demographics and signs/symptoms from first encounter) Output: Responsive score (sensitivity) based on personal characteristics

Question 3 (care planning) method employed simple cluster analysis in Excel using round-up or round-down technique using proportion of interventions by category observed in the data

Question 4 (outcome optimization) method employed support vector machines in Matlab software with optimization

Input: Client characteristics (demographics and signs/symptoms from first encounter) and intervention patterns for each client Output: Any KBS improvement from admission to discharge

Algorithm 1: Predicting Intervention Need

• Approximately 20% of clients received over 70% of interventions. The intervention needs of clients were identified in two scenarios (50% of clients and 25% of clients)

Model 1: 50% of clients received 92% of interventions.

Accuracy: 60%

Area Under Curve: 75%

Model 2: 25% clients received 76% of interventions.

Accuracy: 74%

Area Under Curve: 77%

11/25/2015

13

Algorithm 2: Predicting Responsiveness

• Improvement was twice as great for clients who were predicted to exhibit high responsiveness than for those who are predicted low responsiveness

Algorithm 3: Predicting Personalized Care

• Round up-round down techniques identified eight combinations of interventions that were used to personalize care.

11/25/2015

14

Algorithm 4: Optimizing Outcomes

• For clients with improvement space, the probability of improvement using predicted personalized care was 49% (p < .001) based on a client's characteristics, needs, and responsiveness.

• Improvement space (average of the difference between the highest and lowest predicted outcome improvement probabilities across all clients): 1.18%

• The relative outcome improvement was 49% (p<0.001), which was calculated by comparing probability increase (0.58%) to the largest possible improvement space of 1.18% .

Relative outcome improvement 49%

Next Steps

• Standardize the process and develop data-mining pipeline for other Problems

• Validate with world-wide structured nursing data

11/25/2015

15

Further Research

• Examine the impact of evidence-based care

• What is the impact of evidence-based care effect relative to patient assessment, intervention tailoring, and patient outcomes?

• What is the value of evidence-based care?

• Examine associations between intervention patterns and client outcomes

• Is there differential effectiveness of intervention patterns for similar client profiles?

• What is optimal intervention tailoring?

• What is the nurse effect?

• Evaluate patterns across agencies and programs

• Do patterns persist across agencies?

• What aspects of nursing interventions are similar across programs and populations?

• How does individualized care relate to evidence-based practice?

Monsen, K. A. et al., 2014

Monsen, K. A., Chatterjee, S. B., Timm, J. E., Poulsen, J. K., & McNaughton, D. B. (2014). Factors

explaining variability in health literacy outcomes of public health nursing clients. PHN. Epub

ahead of print: doi: 10.1111/phn.12139

Recommendations

• Expand the development of data analytics methods to incorporate nursing and interprofessional datasets and encompass all standardized terminologies and structured data.

• Test new methods on existing datasets.

Recommendations

• Expand the development of data analytics

methods to incorporate nursing and

interprofessional datasets and encompass all

standardized terminologies and structured

data.

• Continue to develop and test new methods

on existing datasets.