Using Research to Inform Your Mentoring Program · Instructions for how to access PDF of...

Collaborative Mentoring Webinar Series

Using Research to Inform Your Mentoring Program

February 19, 2015


2015 Collaborative Mentoring Webinar Series Planning Team


Good to Know…

After the webinar, all attendees receive:

Instructions for how to access PDF of presentation slides and webinar recording

Link to the Collaborative Mentoring Webinar Series webpage, where all slides, recordings, and resources are posted.

Please help us out by answering survey questions at the end of the webinar.


• All attendees muted for best sound

• Type questions and comments in the question box

• Respond to polls

• Who is with us today?

Participate in Today’s Webinar


Today’s Webinar

• David DuBois - Research Board Chair, National Mentoring Resource Center

• Jennifer Tyson – Social Science Analyst, Office of Juvenile Justice and Delinquency Prevention

• Why EBP matters to the mentoring field • The NMRC as a tool supporting your work • Tips for using evidence for program improvement

Use the Question panel to ask questions of

our presenters!


Is your mentoring program evidence-based? Seems simple…

• What constitutes that evidence?

• Where did that evidence come from?

• Who determined that the “evidence” was actually meaningful?

• How closely (or globally) do you adhere to the evidence?

• Can you prove that you are evidence-based?


Evidence-Based Practice and Mentoring

• Growing trend in policy circles for many years

• Meaningful example in the education field – IES established in 2002; What Works Clearinghouse shortly

thereafter – Goal is to provide educators with the information they need to

make evidence-based decisions – Over 500 reports and reviews of evidence on the WWC site

• Getting research to practitioners remains a challenge – Only 42% of school districts had even heard of the WWC in 2010 – Only 34% had accessed it more than once – WWC has developed different types of products – RELs are now built around Research Alliances with practitioners


The Push for Evidence-Based Mentoring

Other fields: Theory Research Practice

Mentoring: Theory Practice Research

• Exciting things are happening in the field

• Easier than ever for programs to build on research and a growing body of evidence


David DuBois

• Professor of Community Health Sciences at University of Illinois at Chicago

• Co-editor of Handbook of Youth Mentoring

• Lead author of meta-analyses of mentoring program effectiveness

• Currently leads the Research Board of the National Mentoring Resource Center


Jennifer Tyson

• Social Science Analyst in the Innovation and Research Unit at the Office of Juvenile Justice and Delinquency Prevention

• Oversees approximately $45 Million in Federal funding for OJJDP’s research and research translation projects, including the majority of the Office’s mentoring research activities.

• Worked on the development of the Office of Justice Program’s Evidence Integration Initiative and launch of CrimeSolutions.gov


What is EBP?

• Term originates in the medical fields around how to best decide how to care for patients

• EBP is “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research.” (Sackett, D., 1996)


Components of “Evidence”


Researchers

Practitioners

Communities

Service Users

Provider

Organizations &

Intermediaries

Policy-

Makers

Optimal EBP

Adapted from Pawson, Boaz, Grayson, Long, & Barnes (2003)

Who Contributes to Evidence-Based Practice (EBP)?


Practice vs Decision-making vs Policy

• EB Decision-making – Using research and evidence to provide the best services to a specific client

• EB Practice – Using research and evidence to design and deliver better programs

• EB Policy – Using research and evidence to determine which programs to support, scale, or re-calibrate for maximum impact


Mentoring and “evidence”

Available Mentoring Research 1990-2010 (Blakeslee & Keller, 2012)



Wheeler & Keller, 2011



DuBois et al., 2011

• Use of evidence-based practices in mentoring programs

– Appears to be increasing use over last decade plus, at least in published evaluations of mentoring programs

– But program effectiveness estimates have stayed more or less the same (see next slide)

– Why might this be?

Sources: DuBois et al. (2002, 2011)

0

0.2

0.4Ef

fect

Siz

e Youth Mentoring(Pre-1999)

Youth Mentoring(1999-2010)


Why Use of Evidence Matters

• Policy perspective – Easier to scale impact and “move the needle” on

issues that matter

– Wise allocation of resources

– Less trial and error

– Hopefully, fewer decisions based on politics and more on “what works”


Why Use of Evidence Matters

• Program and professional perspective

– Improved outcomes for youth

– Better positioned for funding

– Empowered staff

– Defines what formal mentoring is and differentiates it from other interventions

– Supports effective integration of mentoring into other services and initiatives

– Draws attention to the programs that are offering quality services


Challenges of EBP for Mentoring

• Defining evidence

• Lack of evidence

• Applying evidence to program settings

• Knowing how and when to innovate in this landscape


The National Mentoring Resource Center

Established to help improve the quality and effectiveness of youth mentoring programs across the nation by more fully incorporating evidence-based practices with training and technical assistance. Three central features include:

– a Research Board; – an interactive website with resources

for the youth mentoring field; and – no-cost specialized technical assistance

for mentoring programs.

http://www.nationalmentoringresourcecenter.org/

http://www.nationalmentoringresourcecenter.org/


The National Mentoring Resource Center

• Role of the Research Board

– Review and summarize research evidence on effectiveness of:

• mentoring programs

• program practices

• resources

– Assist with ensuring that trainings and TA are optimally informed by available research


Program Reviews

• Program reviews are carried out using procedures and criteria of Crimesolutions.gov – Rigorous evaluation design required to be eligible for review

(non-mentored comparison group) – Result is Program Profile with Evidence Classification: Effective,

Promising, or No Effects

• How users should interpret and use the results – Classifications reflect only the current state of evidence for a

program – Intended to be used as just one source of information in making

decisions about programming – Should read the entire profiles – lots of important context and

detail (e.g., significant outcomes even when No Effects overall) – Read accompanying Insights for Pracitioners prepared

specifically with mentoring professionals in mind

http://www.crimesolutions.gov/


Program Review Example

• Example Program: – Significant effects:

• High school diploma/GED attainment (72 vs. 56%)

• Increased likelihood of being employed (58 vs. 51%)

– Non-Significant effects: • Number of Arrests, Delinquent Behavior, Marijuana

Use, Other Illicit Drug Use, Psychological Distress

• What Evidence Classification would you expect?


Other Reviews and Resources

• Reviews of practices, program models, special populations, and resources

• Nominate your best resources and program practices! – Resources do NOT need to have undergone rigorous

evaluation, but will be screened for alignment with existing, related research

• Technical assistance is available to help you improve your program!


Q&A for all Panelists

Type your questions in the question box:


Effective Use of Evidence

• Begin with your logic model and theory of change

• Identify the components of your program that can be modified or improved

• Encourage fresh thinking about why you do what you do

• Replication and planned adaptation are critical

R E S E A R C H


Sources of Research-to-Practice Info

• National Mentoring Resource Center

• Chronicle of Evidence-Based Mentoring

• Mentoring Partnership Network

• New Elements of Effective Practice for Mentoring™ coming soon!


Borrowing from Research in Other Fields

• Education

• Psychology and mental health

• Youth development

• Management

• Juvenile justice and criminology


Making Time for Research

• Framework for interpreting and understanding mentoring research

• Inquiry and understanding

• Compatibility and utility The most valuable information will always be the

evidence you gather on your own program.


Tips f0r Using Evidence Wisely

• Final thoughts from Jen and David

• What do our participants want to know more about?


Q&A for all Panelists

Type your questions in the question box:


Remember…

After the webinar:

Please help us out by answering survey questions at the end of the webinar.

Everyone will get an email with information on how to

download the slides, recording, and resources on the CMWS webpage on the MENTOR website:

http://www.mentoring.org/program_resources/training_opportunities/collaborative_mentoring_webinar_series/




Access CMWS Information All Collaborative Mentoring Webinar Series information for upcoming and past webinars

is on the CMWS webpage on the MENTOR website:


A New Lens for Mentoring: Trauma Informed Care Thursday, March 19th, 2015

1 - 2:15 pm Eastern

Join Us Next Month!

Using Research to Inform Your Mentoring Program · Instructions for how to access PDF of...

Documents

Transcript of Using Research to Inform Your Mentoring Program · Instructions for how to access PDF of...