What Outcomes Are Important What Outcomes Are Important for CACs?: for CACs?:

Survey Results and Survey Results and ImplicationsImplications

Ted Cross, Ph.D.

Lisa Jones, Ph.D.

Crimes Against Children Research Center

University of New HampshireSeventeenth National Symposium on Child Sexual Abuse

Huntsville, Alabama March 13-16, 2001

Children’s Advocacy CentersChildren’s Advocacy CentersPlay an Important RolePlay an Important Role

In 2000…

– 105,039 children received services

from CACs

– in 362 affiliated centers representing…

– 48 states, DC, and U. S. Virgin Islands

Growth of CACsGrowth of CACs

0

50

100

150

200

250

300

350

400

1994 2000

Number ofNCACenters

Many Other Organizations Many Other Organizations Use Components of CACsUse Components of CACs

Multi-disciplinary investigation teams

Child-friendly facilities

Case review

Important Questions About Important Questions About CACsCACs

What is the impact of CACs on children, families, and partner agencies?

What is the impact of CACs on the prosecution of child abuse and the court system?

Are there other important outcomes of CACs for their community?

National Trend TowardNational Trend Toward Outcomes-Based Outcomes-Based

AccountabilityAccountability Many systems are measuring outcomes

and being evaluated on them– Education– Health Care– Mental Health– Criminal Justice– Child Welfare

CACs are likely to need to examine outcomes

Evaluation of CAC OutcomesEvaluation of CAC OutcomesMay Help...May Help...

Document the impact of CACs

Refine strategies for helping children

National Evaluation of National Evaluation of Children’s Advocacy Centers Children’s Advocacy Centers

(CACs)(CACs)

Multi-year, multi-site study of the efficacy of CACs

CACs across the country participating in evaluation

Cases enrolled in the study starting April 2001

One Step In The National One Step In The National Evaluation:Evaluation:

Survey on CAC OutcomesSurvey on CAC Outcomes

Identifies what outcomes are important to professionals working in CACs

First data from the National Evaluation Project

Development of the Outcome Development of the Outcome Survey InstrumentSurvey Instrument

Consulted standards of the National Children’s Alliance

Examined other publications about CACs

Consulted national CAC experts

Survey with 84 outcomes in 6 categories

2 Phases of CAC Outcomes2 Phases of CAC Outcomes

Investigation outcomes

Post-investigation

outcomes

What happens duringinvestigations?

What happens later?

3 “Groups” That Have CAC 3 “Groups” That Have CAC OutcomesOutcomes

Children and FamiliesAgenciesCommunities

6 Categories of CAC Outcomes6 Categories of CAC Outcomes

InvestigationPost-investigation

Children &Families

X X

Agencies X X

Communities X X

Survey Instrument FormatSurvey Instrument Format

0-100 scaleMost important outcome in each

category was scored 100Least important outcome was rated

relative to the most important outcome

Other outcomes were scored in-between

Outcomes In The Instrument Outcomes In The Instrument By CategoryBy Category

Investigation Post-investigation

Child & Family 20 outcomes 17 outcomes

Agency 19 outcomes 15 outcomes

Community 13 outcomes 7 outcomes

RATE THE OUTCOMES SEPARATELY FOR EACH PAGE

Child and Family Investigation Outcomes

On a scale from 0-100, please score the CAC outcomes on this page based on how important you think they are for the CAC in your community. Give a score of “100” to the outcome that is most important to you. Score the outcome that is least important to you between "0" and "100" based on how important it is relative to your most important category. Now rate each of the other outcomes on the page relative to the outcomes you chose as most and least important.

Investigation Outcomes:

Importance:

Fewer interviewers ___________

Fewer repetitive interviews ___________

Fewer interview settings ___________

Shorter interviews ___________

More effective interviews ___________

More thorough investigations ___________

Fewer medical exams ___________

Decreased discomfort/stress for family and child during investigation ___________

Child feels supported by someone on team ___________

Excerpt from theOutcomes Survey

The SampleThe Sample

Professionals working in or with CACs in the 5 national evaluation communities

Surveys were mailed to professionals identified by each CAC as most knowledgeable about it

69 respondents returned the survey

Child & Family OutcomesChild & Family Outcomes

Child and Family Child and Family Investigation OutcomesInvestigation Outcomes

30 40 50 60 70 80 90 100

Shorter interviews

Fewer interviewers

Fewer repetitive interviews

More thorough investigations

More effective investigations

Avg.Avg.

Child and Family Child and Family Investigation OutcomesInvestigation Outcomes

30 40 50 60 70 80 90

Parent feels morerespected

Parent feelssupported

Child/family feelless distress

Child feelssupported

Avg.Avg.

Child and Family Child and Family Investigation OutcomesInvestigation Outcomes

30 40 50 60 70 80 90 100

Increased use ofservices

Increasedavailability of

services

Prompt deliveryof services

Avg.Avg.

Child and Family Child and Family Investigation OutcomesInvestigation Outcomes

Outcomes rated as most important by survey respondents:– Increased quality of the child interview– Increased support for the child– Increased identification and delivery of

needed services during investigationMore important than:

– Specific limitations on interviews (fewer, shorter, etc.)

– Improving experience for non-offending caregiver

Child and Family Child and Family Post-Investigation OutcomesPost-Investigation Outcomes

30 40 50 60 70 80 90 100

Greater satisfaction by child inprocess

Greater sense of justice by child

Decreased stress by child

Child less likely to experiencerepeat abuse

Avg.Avg.

Child and Family Child and Family Post-Investigation OutcomesPost-Investigation Outcomes

30 40 50 60 70 80 90 100

Greater justiceexperienced by

family members

Greatersatisfaction by

family members

Decreased stressby familymembers

Avg.Avg.

Child and Family Child and Family Post-Investigation OutcomesPost-Investigation Outcomes Outcomes rated as most important by

survey respondents:– Decreased chance that child will experience

further abuse.– Improved emotional well-being for child.– Decreased stress for child.

More important than:– Satisfaction and sense of fairness for non-

offending caregivers.

Agency OutcomesAgency Outcomes

Important Agency Important Agency Investigation OutcomesInvestigation Outcomes

30 40 50 60 70 80 90 100

Increased inter-agency

communication

Faster firstresponse time

Increased caseinformation

shared

Avg.Avg.

Important Agency Important Agency Investigation OutcomesInvestigation Outcomes

30 40 50 60 70 80 90 100

Better evidence

More expertise available

Better case dispositions

Accurate decisions aboutallegations

Avg.Avg.

Less Important Agency Less Important Agency Investigation OutcomesInvestigation Outcomes

30 40 50 60 70 80 90 100

Better location for observation ofinterviews

Improved coordination with DVinvestigations

Decreased investigator burnout

Better understanding of otheragencies

Avg.Avg.

Agency Investigation OutcomesAgency Investigation Outcomes

Outcomes rated as most important by survey respondents:– Faster response time– Increased case information shared, inter-

agency communication,– Better expertise, better evidence and

more accurate decisions– Better case dispositions

More important than outcomes one step removed from child and perpetrator

Important AgencyImportant Agency Post-Investigation Outcomes Post-Investigation Outcomes

30 40 50 60 70 80 90 100

Increased % of confessions

Increased % of convictions at trial

Increased % of substantiated casesfiled for prosecution

Increased % of at-risk childrenprotected

Avg.Avg.

Less Important Agency Less Important Agency Post-Investigation OutcomesPost-Investigation Outcomes

30 40 50 60 70 80 90 100

Increased % non-child abuse

charges

Increased %dependency

filings

Increasedarraignment

Avg.Avg.

Agency Agency Post-Investigation OutcomesPost-Investigation Outcomes

Outcomes rated as most important by respondents:– Increased % of at-risk children protected– Increased prosecution, conviction at trial,

confessionsMore important than more specialized

law enforcement or child protection outcomes

Community OutcomesCommunity Outcomes

Important Community Important Community Investigation OutcomesInvestigation Outcomes

30 40 50 60 70 80 90 100

Greateradherence to best

practicestandards

More resourcesfor investigation

Avg.Avg.

Important Community Important Community Investigation OutcomesInvestigation Outcomes

30 40 50 60 70 80 90 100

Better relationships ofprofessionals with investigatory

agencies

More joint training

Better coordination ofinvestigations

Avg.Avg.

Less Important Community Less Important Community Investigation OutcomesInvestigation Outcomes

30 40 50 60 70 80 90 100

Increaseduniformity in

media relations

Reduced publiccriticism of

investigation

Cross-agencypeer review

Avg.Avg.

Community Community Investigation OutcomesInvestigation Outcomes

Outcomes rated as most important by survey respondents:– More resources and training for investigation– Adherence to best practice– Coordination & relationships with

professionalsMore important than:

– Cross-agency peer review and uniform media relations

– Reduced public criticism

Community Community Post-Investigation OutcomesPost-Investigation Outcomes

30 40 50 60 70 80 90 100

More private financial support foragencies

Greater community support forCACs

Growth in public awareness ofchild abuse

More community resources forchild abuse

Avg.Avg.

Community Community Post-Investigation OutcomesPost-Investigation Outcomes

Outcomes rated as most important by survey respondents:– Increased awareness of child

maltreatment and resources for victims in community

More important than:– Support for CACs– Increased private financial support of

agencies

How Much Do Professionals How Much Do Professionals Agree on Outcomes?Agree on Outcomes?

Very high agreement on very important outcomes

But people disagreed somewhat about the importance of most outcomes

“Low” importance scores were very important to some people

More effective interviews

Very high agreement on very high scoresF

req

ue

ncy

10

8

6

4

2

0

Std. Dev = 4.68

Mean = 96

N = 14.00

Everyone ratedMore Effective Interviewsgreater than 80 in this site

Increased availability of needed services during investigation

98

93

88

83

78

73

68

63

58

53

48

43

38

33

28

23

18

13

8

3

Some disagreement

even on important outcomesF

req

ue

ncy

7

6

5

4

3

2

1

0

Std. Dev = 19.93

Mean = 81

N = 20.00

17 of 20people gavethis > 70

3 people in this sitescored this 50 or less

Improved coordination with domestic violence investigations

93

83

73

63

53

43

33

23

13

3

"Low" importance scores

were very important to some peopleF

req

ue

ncy

5

4

3

2

1

0

Std. Dev = 21.30

Mean = 73

N = 19.00

5 of 19 peoplescoredthis > 90

Most peoplein this site gavethis low scores

ConclusionsConclusions

CACs Have Many Important OutcomesCACs Have Many Important Outcomes

Responding rapidly Coordinating effectively Investigating effectively Making accurate decisions Making appropriate dispositions Reducing child & family stress

(cont.)

CACs Have Many Important OutcomesCACs Have Many Important Outcomes

Facilitating use of servicesProsecuting offendersReducing risk of re-offendingProviding trainingEncouraging best practice

Possible Implications of Possible Implications of Numerous Outcomes Numerous Outcomes

CACs have many demands on time and resources

Programs may need to develop gradually over time

Different CACs may specialize in different outcomes

CACs role in community larger than coordinating investigations

Lack of Consensus on Lack of Consensus on OutcomesOutcomes

Lack of consensus on the importance of many outcomes underlines the need for coordination, communication and relationship-building

CACs may need to accommodate small groups of their participants invested in specific outcomes

No Simple BenchmarkNo Simple Benchmark

Important outcomes tend to be holistic: quality of response and child well-being

There is no simple benchmark

Number of interviews is potentially a misleading indicator

Implications for Implications for Program EvaluationProgram Evaluation

No simple measure of CAC success may exist

Given the number of outcomes, CACs may need to do a number of small, focused evaluations over time