6/20/17

1

Alameda County Health Care Services Agency

Center for Healthy Schools and Communities

Measuring Success: Improving School Health Center Evaluation Using a Results-Based

Accountability Framework

School-Based Health Alliance 2017 Convention

Kimi Sakashita, MPH Sara Geierstanger, MPH

Samira Soleimanpour, MPH, PhD

Presenter Disclosures

The following personal financial relationships with commercial interests relevant to this presentation existed

during the past 12 months:

No relationships to disclose

Kimi Sakashita, MPH Sara Geierstanger, MPH

Samira Soleimanpour, MPH, PhD

Learning Objectives

•  Explain how a Results-Based Accountability (RBA) Framework can be used to evaluate school health centers (SHCs) and “tell the story.”

•  Describe SHC evaluation results and measures of health and

education equity using a RBA Framework. •  Discuss how to incorporate the national SHC performance

measures into an evaluation that uses the RBA framework. •  Discuss how to leverage existing data systems or implement new

strategies to collect data.

AlamedaCounty,California

Population: 1,578,891

School Districts/Schools : 18/389

Number of Students: 222,681

Free/Reduced Lunch Percentage: 45%

College graduates, persons 25 and over: 42%

Median household income: $72,399

Source:AmericanCommunitySurvey,2013.

6/20/17

2

29 Current School Health Centers A l a m e d a C o unt y H e a l t h C a r e Se r v i c e s A g e nc y

Berkeley High School Health Center Emeryville Unified

BTA Health Center

Chappell Hayes Health Center

West Oakland Middle School

uu

u u u u

u

TechniClinic

Piedmont Wellness Center

Shop 55 Wellness Center

Hawthorne Elementary

United for Success Skyline High School

Frick Middle School

Youth Uprising Health Center Youth Heart Health Center

Roosevelt Health Center

Rising Harte Wellness Center ACLC/NEA Campus Encinal High School Alameda High School

Fremont Tiger Clinic

Havenscourt Campus

uu u u

u uu u u u u u u u

u

u

u

uu

Elmhurst/Alliance Campus

Fuente - REACH AYC

Madison Park Academy

San Leandro High School Health Center

u

u

San Lorenzo High School Health Center

Hayward High Mobile Clinic u

Tennyson Health Center

Logan Health Center

LI N K IN G H E A LT H A N D E D U C A T I ON T O C H A N G E L I VE S AN D AC H I E V E E Q U I T Y 10 00 SA N L E A N D RO B O U L E VA R D S U I TE 3 00 | S A N L EA N D R O , C A 94 5 7 7 | WWW . A C H E A LT H Y S C H O OL S . OR G

Updated 3/4/16

Alameda County Health Care Services Agency

Center for Healthy Schools and Communities

Why RBA?

RBA Framework

h<ps://clearimpact.com/results-based-accountability/

RBA in Alameda County

STEPS 1.  Choose RESULTS or conditions of well-being that you want to achieve for youth,

families and communities.

2.  Identify the overall population you hope to influence and the POPULATION INDICATORS that measure these results at the population level.

3.  Identify the people you are directly serving and choose PERFORMANCE MEASURES

that measure the impact of your programs/services on them. Find measures that answer: •  How much did you do? •  How well did you do it? •  Is anyone better off?

6/20/17

3

Alameda County SHC Evaluation Logic Model QUESTIONS

1.  Name one overall RESULT (condition) you are working towards.

a.  Name a POPULATION INDICATOR that matches this result and the population you hope to influence. Remember, the population you choose is LARGER than the clients you serve.

Alameda County Health Care Services Agency

Center for Healthy Schools and Communities

Evaluation Methods & Findings

2015-16 Data Collection Tools •  Client Services Form (Efforts to Outcomes/ETO)

Ø  14,790 clients

•  Community Functioning Evaluation Form (ETO) Ø  227 behavioral health clients

•  Student and School Community Activity Log (Activity Log) Ø  113,234 youth

•  Quarterly Reports Ø  27 SHC directors

6/20/17

4

2015-16 Data Collection Tools (cont.)

•  California Healthy Kids Survey, SHC Module (CHKS) Ø  11,786 students Ø  58% students from 24 schools

•  Client and Youth Program Participant Survey Ø  1,632 clients Ø  50% of all clients over a 2-month period

Sharing Data: QI, Accountability, Making the Case

•  School Health Centers Ø Agency/District Level Reports Ø Site Level Workbooks

•  County Level Ø Measure A Report Ø Insurance Trends Ø Program Level Ø Presentations to stakeholders

•  Making the Case Ø Issue Brief Ø Presentations to stakeholders

Sharing 2015-16 Findings with you today!

•  Access to Care •  Reproductive Health Services • Oral Health Services •  Behavioral Health Services •  Youth Engagement ** NQI Measures

Access to Care

6/20/17

5

6,624 6,642 7,276 8,968 9,514 11,399 11,813 13,017 14,446 14,790

27,078 27,394

33,614

41,519 41,802

51,715 52,226 56,967

60,780 63,345

2006-07 (11 SHCs)

2007-08 (12 SHCs)

2008-09 (12 SHCs)

2009-10 (14 SHCs)

2010-11 (18 SHCs)

2011-12 (23 SHCs)

2012-13 (23 SHCs)

2013-14 (25 SHCs)

2014-15 (27 SHCs)

2015-16 (28 SHCs)

Clients Served Clinic Visits

PERFORMANCE MEASURE How much? Increased clients & visits

2015-16 School Health Centers serve: •  2 elementary schools •  12 middle schools •  24 high schools

Data Source: ETO Client Service Form

Howmuch?Access to Care Client Demographics

Students who had not used SHC (as a registered

client) 69%

SHC student clients 31% Female

58%

Of the nearly 37,000 Alameda County students with access to a School Health Center, 31% were registered clients

Data Source: ETO Client Service Form and California Dept. of Education

Male 42%

Data Sources: ETO Client Service Form and California Dept. of Education

40%

22%12%

7% 10%

43%

18% 22%12%

3%

LaHno/a AfricanAmerican Asian/PacificIslander/Filipino

White Bi/MulH-RacialorOther

SchoolHealthCenterClients(Studentsonly) StudentPopulaHon

PERFORMANCE MEASURE How much? Client demographics

Experiences with the School Health Center (n=1,484-1,567)

% Agree or Strongly Agree

The people who work there helped me to work through my problem

98%

It felt like a safe place 99%

The people who work there were easy to talk to 98%

§ Demonstrating satisfaction… §  62% clients returned for ≥1 visit §  30% returned for a different type of service

PERFORMANCE MEASURE How well? Client satisfaction

Data Sources: Client Survey and ETO

6/20/17

6

§  School Health Center clients were more likely than non-clients to “always” receive counseling when needed (32% vs. 27%, p<0.001)

Experiences with the School Health Center (n=1,484-1,567)

% Agree or Strongly Agree

They helped to get help sooner than I normally would 97%

They helped me to get services I wouldn’t otherwise get

96%

Data Sources: CHKS and Client Survey

PERFORMANCE MEASURE Better off? Increased access

PERFORMANCE MEASURE How much? Increasing insurance enrollment

Data Sources: ETO Client Service Form and California Dept. of Education (2015-16)

•  24 of the 27 SHCs conducted some amount of application assistance to educate and/or enroll 7,220 families in health coverage and other benefits programs, such as CalFresh.

28%

24%

25%

19% 17%

2011-12 2012-13 2013-14 2014-15 2015-16

% SHC Clients with No Reported Insurance

Data Source: ETO Client Service Form representing 56% (n=8,235) of clients with insurance documentation.

PERFORMANCE MEASURE Better off? Increased insurance enrollment

POPULATION INDICATOR Better off? Increased insurance enrollment

14%

10% 10% 9% 8%

8% 7%

7% 8% 7%

4%

7%

10%

4%

2001 2003 2005 2007 2009 2011 2013

% Uninsured at Any Point in Last Year

California Alameda County

Definition: % children ages 0-17 uninsured at any point during the 12 months prior to the survey.

Data Source: California Health Interview Survey, UCLA

6/20/17

7

Reproductive Health Services

§  Provided during 28% of all School Health Center visits (Client Service Form, ETO) §  Contraceptive counseling (87% of reproductive/sexual health

services) §  STI screening/counseling (37%) §  HIV screening/counseling (21%) §  Chlamydia screening (19%)

§  School Health Centers also provided 1,033 HPV immunizations

Data Source: Client Service Form, ETO

How much? Reproductive Health Services

§  Stay tuned….

How well? Reproductive Health Services

POPULATION INDICATOR Better off? Teen birth rates

40.2 40.1 38.3 35.4

31.6 28.3 26.2

23.2 27.6 26.0 25.6 24.7

21.8 19.3 17.7

14.7

2006 2007 2008 2009 2010 2011 2012 2013

California Alameda County

Definition: Number of births per 1,000 young women ages 15-19.

Data Source: CA Dept. of Finance, CA Dept. of Public Health, & CDC

6/20/17

8

POPULATION INDICATOR Better off? Chlamydia rates

742.2 779.7 815.0 803.0 803.7 816.6 779.3 725.2 684.4

1125.0 1306.5 1316.1 1277.6 1302.2

1164.2

973.7 885.1

788.4

2006 2007 2008 2009 2010 2011 2012 2013 2014

California Alameda County

Definition: Rate of chlamydia and gonorrhea infections per 100,000 youth ages 10-19.

Data Source: CA Dept. of Public Health, CA Dept. of Finance, CDC, & US Census

Oral Health Services

1,533

2,444

3,072

3,871

4,484

506 763 766

1,041 1,339

2011-12 (4 SHCs)

2012-13 (5 SHCs)

2013-14 (6 SHCs)

2014-15 (8 SHCs)

2015-16 (10 SHCs)

SHC Dental Visits

SHC Dental Clients

Data Source: ETO Client Service Form

PERFORMANCE MEASURE How much? Oral health visits and clients

•  18%ofallvisits(1,341clients)hadadentalserviceprovidedforscreeningexamsandcleanings,andrestoraHvetreatment.

•  2,489 students participated in one of 27 school-wide

dental screenings

Data Source: Activity Log and Client Service Form, ETO

PERFORMANCE MEASURE How much? Oral health screenings

6/20/17

9

•  91% School Health Center dental clients had a clinical baseline dental assessment §  85% were found to have some dental decay

Data Source: Activity Log and Client Service Form, ETO

PERFORMANCE MEASURE How well? Oral health screenings

At the 10 sites that provided oral health services: §  85% of clients had suspicious areas of decay identified

at first oral health assessment Ø  20% had urgent dental needs identified Ø  Decay improved or did not worsen over time in 82% of

clients screened at baseline and follow-up

Data Source: ETO Client Service Form

PERFORMANCE MEASURE Better off? Improved oral health

Behavioral Health Services

School-Wide Prevention (Tier 1 and Tier 2) # Contacts Restorative justice/circle activities 2,901

School/teacher consultation (i.e., Coordination of Services Team) 2,400

Screening, trauma, 9th and other grades 1,753

School safety/climate presentation/activity 1,710

Social skills/communication/anger management/conflict resolution 1,175

Data Source: Activity Log and ETO Client Service Form

PERFORMANCE MEASURE How much? Behavioral health services

Client Early/Intensive Intervention (Tiers 2 & 3) Services, by type of Behavioral Health Service (>1 service can be selected per visit; N=21,587 visits) Individual therapy 44%

Assessment/intake 17%

Psychosocial screening 15%

Individual contact 8%

Plan development 7%

6/20/17

10

§  Among 6 OUSD School Health Centers (in high schools) providing trauma-informed interventions: §  21% of unduplicated clients received trauma screenings during

a visit to the School Health Center §  35% of those screened positive for trauma and needed

follow-up care §  86% were connected to services/supports to address

their needs

Data Source: Client Survey and ETO Client Service Form

PERFORMANCE MEASURE How well? Behavioral health services Experiences with the School Health Center (n=918-1,464)

% Agree or Strongly Agree

Feel like I had an adult I could turn to if I needed help or support 97%

Feel safe talking about my problems 97%

Data Source: Client Survey

Experiences with the School Health Center (n=918-1,464)

% Agree or Strongly Agree

Deal with stress/anxiety better 94%

Concentrate better 93%

Stop using or use less tobacco, alcohol or drugs 91%

Feel less irritable or have less angry outbursts 91%

PERFORMANCE MEASURE Better off? Behavioral health services

% Change in Presenting Concerns, Baseline and Follow-Up (N=227) *significant at p≤0.005, ** significant at p<0.0001

Data Source: Community Functioning Evaluation, ETO

17%*

18%**

24%**

32%**

35%**

36%**

Academic Functioning

Living Arrangements & Family Functioning

Exposure

Emotional & Behavioral Functioning

Health/Basic Needs

Social Relationships

PERFORMANCE MEASURE Better off? Behavioral health services

POPULATION INDICATOR Better off? Depression rates

Definition: Percentage of public school students in grades 7, 9, 11, and non-traditional students reporting whether in the past 12 months they had felt so sad or hopeless almost every day for two weeks or more that they stopped doing some usual activities.Data Source: CHKS & California Student Survey (2011-2015); Note: Alameda County ”2013-15” data are from the 2015/16 school year only and should be considered estimates.

30.0% 30.4%

29.1%

26.9%

2011-2013 2013-15

California Alameda County

6/20/17

11

Youth Engagement School Community Supports

Additional Contacts with Youth # Sessions # Contacts

Health fairs/outreach 214 26,196

School-wide assembly or special event 34 5,222 Job training/career exploration (e.g., applying, internships, shadowing)

305 2,801

Youth advisory board/leadership/research/advocacy 299 2,663

Peer health education group/peer counseling/mentoring 238 2,442

Physical activity/recreation/dance/yoga 148 1,877

Acculturation support (for newcomers, unaccompanied youth, etc.)

116 1,376

Data Source: Activity Log

PERFORMANCE MEASURE How much? Youth engagement

Data Source: Client Survey

Impact of SHC Services (n=1,042-1,272) % Agree or Strongly Agree

Have goals and plans for the future 95%

Stay in school 93%

Do better in school 89%

Have better attendance (cut classes less) 87%

Impact of SHC Group Programs (n=370-437) % Agree or Strongly Agree

Learn skills that will help in future 98%

Feel more confident 96%

Make school a better place 96%

Feel more connected to people at school 93%

PERFORMANCE MEASURE Better off? Youth enrichment

§ More SHC clients reported higher levels of school assets than non-clients, including: § Having caring relationships with teachers or other adults at

school (36% vs. 29%, p<0.001)

§ Having a teacher or other adult at school who had high expectations for them (49% vs. 41%, p<0.001)

Data Source: CHKS

PERFORMANCE MEASURE Better off? School assets

6/20/17

12

§  After 96% of medical visits, clients were sent back to class potentially saving an absence from school for the rest of the school day

§  With an estimated Average Daily Attendance (ADA) payment of $56/day, this saved school districts nearly $900,000 (16,017 medical visits x $56) in ADA in 2015-16

Data Source: Client Service Form, ETO; ADA estimate from OUSD (2014-15)

PERFORMANCE MEASURE Better off? Seat time

Suspension Rates Graduation Rates

2010-11 2014-15 % Change

2010-11 2014-15 % Change

California 11.3 3.8 -66% 77.1 82.3 +7%

Alameda County 9.6 3.3 -66% 78.0 85.0 +9%

Alameda County School Health Center schools

18.4 6.3 -66% 72.3  

79.5  

+10%

Data Source: California Dept. of Education, Data Reporting Office: California Longitudinal Pupil Acheivement Data Systems (CALPADS)

PERFORMANCE MEASURE POPULATION INDICATOR Better off? Suspension and Graduation rates

QUESTIONS

2. What is one thing you wish you could measure?

a.  Is there a proxy measure that would not be resource intensive?

National Quality Initiative

6/20/17

13

SBHA National Quality Initiative

§  School-Based Health Alliance, with support from the Maternal and Child Health Bureau (MCHB), launched the School Health Services National Quality Initiative (NQI)

§  Central to NQI is the adoption SHC standardized performance measures to help describe the quality of care delivered in SHCs and demonstrate our collective impact

§  Performance measures include: 1.  Annual Well Child Visits 2.  Annual Risk Assessments 3.  BMI Screening and Nutrition/Physical Activity Counseling 4.  Depression Screening with Follow-up Plan 5.  Chlamydia Screening

§  For more information, visit: http://www.sbh4all.org/resources/quality-counts/

Evaluation Client Service Form –NQI-Related Fields

SCREENINGS/NEEDS AND DIAGNOSES/PROCEDURES Providers: complete sections that are applicable to each visit and check all boxes that apply.

PHYSICAL AND BEHAVIORAL HEALTH SCREENINGS/NEEDS (during this visit)

PHYSICAL AND BEHAVIORAL HEALTH DIAGNOSES/PROCEDURES

Annual Risk Assessment (GAPS, RAAPS, HEEADSSS, etc.; 99420): ¡ Yes   BMI Screening Percentile: ¡ < 5% (underweight) ¡ 5-84% (normal) ¡ 85%-94% (overweight) ¡ 95%+ (obese)   Depression Screening (PHQ-2, PHQ-9, etc.; 96127): ¡ Yes

q  Nutrition counseling (E66, Z68.51-4, Z71.3)

q  Physical activity counseling (E66, Z68.51-4, Z71.3)

q  Well care visit (99381-5, 99391-5, Z00.00-.01, Z00.121, Z00.129)

SEXUAL/REPRODUCTIVE HEALTH SCREENING/NEEDS

SEXUAL/REPRODUCTIVE HEALTH DIAGNOSES/PROCEDURES

Is client sexually active (past 3 months)? ¡ Yes ¡ No

q  Chlamydia screening (86631-2, 87110, 87270, 87320, 87490-2, 87810, Z11.8)

% Unduplicated clients with documentation of ≥1 of the following during the school year:

Alameda County SHCs

National Data

Well Child Visit: ≥1 comprehensive well-care visit with a PC or an OB/GYN practitioner regardless of where the exam was provided, including documentation of health and developmental history, physical exam, and health education/anticipatory guidance

9% 44%*

Annual Risk Assessment: ≥1 age-appropriate annual risk assessment Not documented N/A

BMI Screening & Nutrition/Physical Activity Counseling (ages 3-20 years):

•  BMI percentile AND counseling for nutrition AND physical activity

•  BMI ≥85% AND counseling for nutrition AND physical activity

8% all clients

37% clients w/ BMI ≥85%

58%**

TBD

Depression Screening & Follow-up Plan (ages ≥12 years): ≥1 screening for clinical depression using age appropriate standardized tool AND a follow-up plan documented if positive screen

Not documented 51% (includes adults)**

Chlamydia Screening: ≥1 test for Chlamydia if identified as sexually active 52% female 47% male

48%* female N/A male

Data Source: ETO Client Service Form (2015-16); *DHHS: https://www.medicaid.gov/medicaid/quality-of-care/downloads/2015-child-sec-rept.pdf; **https://bphc.hrsa.gov/uds/datacenter.aspx

NQI PERFORMANCE MEASURES How well?

Questions?

For more information contact: Kimi Sakashita, MPH

Center for Healthy Schools and Communities Alameda County Health Care Services Agency Kimi.Sakashita@acgov.org

www.achealthyschools.org