Guerrilla Reporting

Leveraging Free and Existing Resources to Rapidly Disseminate TCOM Information

Nathaniel Israel, Mark Lardner, Nick Mader

Overview

• Origins and Intent

• Framework and Challenges

• Front-End Development: Interface

• Back-end Tools and Case Study: R Reports

• Futureshock

2

Origin Story3

Origin Story4

• Context 1: Failure to launch

• Context 2: Losing what we had

• Context 3: Too small to scale

Intent

• Insure that anyone, anywhere gets value-added information when entering the CANS ratings with a child / family

• Provide cross-time group-based reports with minimal infrastructure

5

Framework / Principles

• Use software which (nearly) everyone already has

• Provide:

– individual-level reports without an internet connection

– HIPAA-secure upload of individual CANS

6

Framework / Principles

• Generate all group-based Standard Reports for stakeholders

• Leverage report code from across sites to spread innovation

7

Challenges

• All settings: different versions of the CANS

• Sector-based differences in Goal Setting / Treatment Planning

• Managing permissions

• Iron Triangle

8

Excel ‘Guerilla’ Reports

• The importance of data feedback loops in TCOM implementations cannot be overstated.

• In some systems, there are insurmountable cost implications of building or adapting data systems.

• One potential solution is using existing infrastructure and expertise to to support data feedback loops.

Guerrilla FAST: Child Welfare Application

10

Guerrilla FAST: Child Welfare Application

11

Guerrilla FAST: Child Welfare Application

12

Guerrilla FAST: Child Welfare Application

13

Guerrilla Standard CANS

14

Guerrilla Standard CANS

15

Guerrilla Standard CANS

16

Guerrilla Standard CANS

17

Guerrilla Standard CANS

18

Tools for Analysis and Reporting --19

• CHIC has significantly picked up on R in the last year for:

– Data pulls

– Cleaning, transformation, automated documentation

– Analysis

– Data visualization

– Report generation

A universal remote formula bar

as a tool

• Free– Easy to get– Easy to agree on

• Open source– Rapidly advancing – more functional, easier to use– All advances are free– Large community of users creating help files

• Increasingly popular– More collaborators and potential hires who can use it– More training (often free)

20

as a tool

Data interactivity• Reads/writes to almost any file format

• Connects to databases, APIs, can automate SFTP push/pull, scrape websites

• Ability to handle/implement Big Data methods

Analysis• All social science methods: ANOVA, regression, time

series, factor analysis, …

• Predictive analytics (“machine learning”): clustering, classification, risk assessment

21

as a tool

Data visualization• Full modern suite of data visualization

• Interactive figures and maps

Report generation• In addition to outputting csv and Excel tables, can

generate --

• Narrative reports (Word, pdf, html) and interactive dashboards

• Each can be data driven, fully customized, and scaled to audience of any size

22

case study (part 1)

• Received large amount of CANS data across many episodes from Washington State Behavioral System

• Wrote R code to clean and prepare data– Recode problematic values– Identify first full assessment after screeners per episode– Identify when reassessments occur and deduplicate– Merge data CANS items with meta data on item domains– Generate algorithms for care

Data steps—including results from each step—are self-documenting

23

case study (part 2)

• Construct a report template with narrative, and interactive tabs

• Automatically generate dozens of reports with custom information, at different levels of the system

Upshot• Once built, reports can be regenerated (and documentation

updated) with minimal effort and turnaround time• Over time, staff time can focus on building richer content• Because code allows for replication of the full process,

Chapin Hall will build tools and tutorials useful across all systems

24

Future Plans – Report Formats

1. Dashboard development• R has means to create Tableau/Power BI-like dashboards• Same as Tableau/Power BI:

– Interactive graphics and maps– Slicers– Flexible layout combining text, figures, call-out values– Multiple tables and storyboard

• Beyond Tableau/Power BI:– No cost or complications to get many stakeholders on a common

platform– Ability to generate completely custom dashboards for audience

of any size

25

Future Plans – Support Structures

2. Open Source Materials and Tutorials• CHIC staff will create public versions of tools that we

develop, for adaptation to different systems

• This will include read-me, tutorials, and FAQs for how to adapt each materials, and for overall R FAQs

• This will use GitHub, which has become the “industry” standard for open sourcing materials

• Other community members will be able to contribute both new functions, and suggestions for what to address

26

Future Plans - Analytics

3a. Classifying clients on intake

– Identify clusters of types of needs, distill CANS item information into common factors, and assess future risk

– Uses: referral and support systems, easier “noticing” of trends across time and demographics

27

Future Plans - Analytics

3b. Identify factors associated with successful courses

– Identify how client needs and strengths tend to evolve over the treatment course

– Help notice cases where clients are doing well, or need more support, to sponsor low stakes conversations around practices and resources

28

Future Plans - Analytics

3c. Identify differential effectiveness of different care circumstances with different client needs/strengths

Overall – use academically rigorous methods for CQI purposes, fit to guidance from systems and clinicians themselves.

29

From Futureshock to Anti-Fragility30

https://cdn.static-economist.com/sites/default/files/imagecache/640-

width/images/print-edition/20121117_BKD001_0.jpghttps://upload.wikimedia.org/wikipedia/commons/thumb/0/02/100_Yen_lighter.JPG/

220px-100_Yen_lighter.JPG

Questions/Comments

• Reactions……

• How do you provide feedback?

• What are we missing?

• Where do you see this going?

• How do we make sure that this work really helps children, youth and families?

31