Persuading Policy Makers: Effective CIT Program Evaluation and Public

Relations

Maria Felix-Ortiz, Ph.D.&

Megan BrodieUniversity of the Incarnate Word

Presentation for the CIT International Conference in San Antonio, TX

June 2010

Today… The connection between good program

development and evaluation, and policymaking

Advocacy and community involvement in program development/evaluation, and in selling CIT to policy makers

Program evaluation “in a nutshell” Resources

How can you convince policy makers and stakeholders to support your CIT program?

• Become familiar with the policy making process

• “Policy making is a cyclical process. It begins with recognition and definition of a significant public problem ... In response, government may formulate, adopt, and implement a strategy for addressing the problem. Analysis of policy effectiveness in turn often reveals shortcomings...”http://www.laits.utexas.edu/txp_media/html/bur/features/0303_01/policy.html

How can you convince policy makers and stakeholders to support your CIT program?

Build relationships with friends in your state capitol. Advocacy: the process of

education through relationship building and sharing personal experiences that “move” people to action.

Lobbying: telling someone how to vote.

Defining goals and showing “results.”

Aim for handshaking, not head butting

“Tongue handshake” by Skorpion of the Independents

The Old and New in Persuading Policy Makers

USE THE NEWThe use of scientific data obtained through regular program evaluation numbers as well as interviews or observations. National Alliance for the Mentally Ill and other advocacy groupsFacebook, blogs, TwitterWork with the PTA/O, faith-based communities, and business

BUT KEEP THE OLDTake consumers with you on visits to the policymaker and or their aides. Policymakers are influenced by

consumers, stakeholders, other policymakers, and “campaign contributions”

Visit gatekeepers of powerful political institutions, social services, and neighborhood institutionsletters to the editor of your newspaper, television, radioInvite the community to an open-house or panel discussion

The Old and New in Persuading Policy MakersGet to know your opponents and their positions so you can respond to/address their concernsIn one page of bullets, show that your program

Stretches the dollar Impacts real people for whom your community has

sympathy Women Children Veterans

Thank you notes build relationships

Why involve the community in your CIT program

development and evaluation?

can offer insightful perspectives on what works and what needs changing, but getting community input can be time consuming, and sometimes it’s difficult to identify consensus or the main themes.

Community “buy in” This means people will use

your program, refer others, and be your best publicity.

Constituents who vote can be very persuasive to policymakers.

Stakeholders can provide a variety of resources to the program.

Program Evaluation (in a nutshell)

Needs AssessmentResearch design and time frameCollecting dataAnalysis and feedbackProgram modification and second evaluationAnalysis and feedbackProgram modification…

Needs AssessmentWhat is it?

A needs assessment is a lot like what you do when your TV cuts off.

A needs assessment is about getting lots of information about what’s happening, and using it to develop a response, a program.

Benefits of? Important so you can show results (get the TV going again…).Challenge of? In a needs assessment, you often discover that you were wrong about what “the community” needs. You might have to add to the mental health programming as well as develop a diversion program…

Needs AssessmentHow’s it done?

Begin with formal or informal talks with EVERYONE

Check the information gathered Identify resources Organize a “town hall”-style meeting or

legislator’s lunch

Research Design and Time Frame

Research Design? “Experimental” vs. “Control” Group

Compare people in your program to those who get the usual

“pre” and “post” intervention comparison Compare people in your program to themselves Compare the number of their arrests before and after

being in your programWhat is your time frame?

Looking for short-term effects Looking for long-term effects: 3, 6, or 12 months after

the program

Collecting DataFocus of study

Consumers/Families of consumers Rookie/Cadet vs. Veteran Officer’s Perspective First Responders Linear Study

Gathering Information Observation Visiting other programs Case Study: Following various “typical” individuals

before the program, through the program, and after Interviews Surveys

What outcomes can you measure? Outcomes in your consumers

How many referrals were diverted from jail? When were they identified (pre or post booking)? How were they identified (What kind of screening, who did it)? Where did they go?

What are the demographics of those diverted? What kind of illness is going untreated in your community? Are there racial/ethnic or gender disparities?

Gains in independent living skills, reduced drug use, better “quality of life,” and reduced psychological distress one year later (Cosden et al., 2003)

Lower re-arrest, violence, homelessness, hospitalizations on year later (Lamb, Weinberger, and Reston-Parham, 1996)

Less jail time two months later (Steadman, Cocozza, and Veyzey, 1999) Changes in the type of arrest Health problems, social support, financial circumstances, living

arrangements, ongoing use of outpatient mental health services.Outcomes in your officers

Attitudinal change Increase in skills and knowledge Behavior in the field

Outcomes in the system Document changes Keep track of costs

EXCELLENT RESULTS It met your goals. It was accepted by the community. Things you measured changed in a positive direction. There are better outcomes among your program

participants compared to those who didn’t participate. LITTLE OR NO CHANGE?

There was no improvement, BUT there was also no deterioration or worsening of the problem AS MIGHT BE EXPECTED

Interpret changes in a larger context. How might the small differences be a big deal?

Allow stakeholders and consumers themselves to see the data and help you interpret your findings.

Analysis: How do you know that your program worked?

Choosing an Evaluatorhelps you plan your program so that it can be evaluated for effectivenessAdvantages of a University-based evaluator

less costly than professional evaluators (or work for free in exchange for access to the data for publications),

can help develop your local coalition and collaboration, provide student interns, and be more willing to develop measures/tools specific to your needs.

Advantages of a professional program evaluation company more resources, can draw from their experience with programs from other regions, might be able to devote more time to your project since evaluation is their

primary work. An evaluator should have a background in statistics and measurement, publications that showcase this expertise, and experience with making programs work in real-world settings. American Evaluation Association: www.eval.org

ResourcesUniversity relationshipsPartner with a county that already has a programNAMI and In Our Own Voice, their speaker programAmerican Evaluation AssociationSociety for Prevention ResearchThe Center for Mental Health Services GAINS Center

GAINS ResourcesWith the help of its supporters, GAINS has produced several documents that focus on increasing meaningful peer involvement in jail diversion and reentry initiatives. Click on the links below to read more on best practices used to increase the inclusion of peers in systems change. Leveling the Playing Field: Practical Strategies for Increasing Veterans' Involvement in Diversion and Reentry Programshttp://www.gainscenter.samhsa.gov/pdfs/veterans/levelingthefield_veterans.pdf Overcoming Legal Impediments to Hiring Forensic Peer Specialistshttp://www.gainscenter.samhsa.gov/pdfs/integrating/Miller_Massaro_Overcoming.pdf Peer Support within Criminal Justice Settings: The Role of Forensic Peer Specialistshttp://www.gainscenter.samhsa.gov/pdfs/integrating/Davidson_Rowe_Peersupport.pdfAdditional ResourcesMedicaid Coverage of Peer Support for People with Mental Illness http://cms.hhs.gov/PromisingPractices/downloads/PeerSupport.pdfMental Health Consumer Providers by the Rand Corporationhttp://consensusproject.org/bja-ta-training-event-july-2009/materials-bja-ta-09/Rand_Article.pdfMentoring Formerly Incarcerated Adultshttp://www.workingventures.org/ppv/publications/assets/265_publication.pdfNavigating the Child Support Systems http://ppv.org/ppv/publications/assets/274_publication.pdfPeer Specialist Compensation/Satisfaction 2007 Survey Report by NAPShttp://www.ncmhcso.org/downloads/NAPS_survey_report.docAdditional Resource Sites:Community Access: Howie the Harp http://www.communityaccess.org