CVPP Evaluation:Youth Employment Program &

Parent ProgramPresentation by:

Jessica Reichert, Senior Research AnalystIllinois Criminal Justice Information Authority

CVPP Steering Committee MeetingFebruary 13, 2014

EVALUATION OF YOUTH EMPLOYMENT PROGRAM

YEP evaluation methods• Administrative data• Job readiness training survey• Mentor training survey• Employer exit survey• Coordinator manager survey• Mentor exit survey• Youth exit survey• Youth pre- and post-assessment

Lead Agencies: Program data

• 7,254 youth applied to the program • 1,894 were accepted• 1,604 completed job readiness training• 1,838 were assigned a mentor• 1,466 completed employment

Youth: On job readiness training Agreed or strongly agreed about the training:

Aspect of training PercentTrainer knowledgeable, helpful 84%Offered better sense how to obtain, maintain a job

82%

Answered questions about employment 80%Materials, handouts useful 79%Well-designed 72%

Youth: On job readiness training

Spend more time on•Resumes, applications (n=48)•Interview techniques (n=44)•Money management (n=29)

Spend less time on:•Dressing appropriately for a job (n=45)•Hygiene (n=31)

Youth: On job readiness trainingLiked best about the training:•Learning job readiness/life skills (n=152)•Activities/role plays/games (n=95)•Teamwork/ group discussion (n=81)

Change about the training•Nothing (n=228)•More interactive activities (n=131)•More organized/prepared (n=69)

Mentors: On training Agreed or strongly agreed about the training:

Aspect of training PercentWell designed 88%Information about mentoring 89%Materials, handouts useful 89%

Trainer knowledgeable, helpful 91%Learned what it takes to be a mentor 84%

Mentors: On training

Spend more time on:•Nothing (n=31)•More time with youth (n=13)•Dealing with problems, crises, emergencies (n=10)

Employer: Program ratings

Ratings:•YEP was successful or very succesful-85%•Satisfied or very satisfied with their experience as a YEP employer- 84%•Satisfied or very satisfied with matching of youth with their agency- 81%•Good or very good communication with YEP staff - 80%•YEP youth were prepared or very prepared 71%

Employer: Interested in serving as YEP employer again

Employer: Hiring YEP youth

Employer: Change about program

• Nothing (n=15)• More preparation for youth (n=15)• Longer program/more hours for youth (n=13)• Better communication with program (n=12)

Coordinators/Managers: Ratings

Aspect of program PercentEmployment component 84%ICJIA support 75%Lead agency support 75%Mentor component 68%Training 50%Service and support from CAPs 24%

Rated aspects of YEP good or very good:

Coordinator/Manager: Comment

“I really appreciated the fact that it kept a lot of kids off the street this summer and gave them a chance, not only to do something positive, but earn money.”

Mentors: Ratings

Aspect of program PercentMentor component 86%Employment component 84%Manager/Coordinator support 74%Mentor training 68%

Rated aspects of YEP good or very good:

Mentor: On relationship w/ mentee

Aspect of relationship PercentMade a positive connection with their mentees

83%

Interested in serving as a mentor again 82%Found it easy to be a mentor 63%Learned new things about themselves 58%

Rated aspects of YEP good or very good:

Youth: How learned about YEPHow learned about YEP

Youth: Ratings of program

Aspect of program PercentMentor 91%Job tasks 89%Job readiness training 88%YEP overall 87%Supervision on the job 86%

Rated aspects of YEP good or excellent:

Youth: Job training prepare you for job?

Youth: Training skills used

• Time management- 77%• Dressing appropriately for the job- 75%• Professional vocab, communication- 70%• Money management- 59%• Conflict resolution- 54%

Youth: Job a good match?

Youth: Work benefitted employer?

Youth: Offered a job?

Youth: Gain from mentoring?

Most common responses:•Guidance/general advice (n=85)•A Relationship/Someone I can talk to and trust (n=82)•Confidence/self-esteem (n=62)•Communication skills (n=52)

Pre- and post-assessmentMeasured:•attitudes toward employment•attitudes toward violence•conflict resolution•self-esteem

Received:•2,068 forms •1,446 pre- and 622 post-assessments•368 matched pre- and post-assessments

Pre- and post- results n Mean 1

(pre)Mean 2 (post)

Change in means

Attitudes toward employment

359 4.13 4.11 -.02

Attitudes toward violence*

358 3.99 3.92 -.07

Conflict resolution*

358 4.30 4.18 -.12

Self-esteem 356 4.26 4.26 0

Combined measures

355 4.17 4.12 -.05

Conclusions from YEP evaluation

• Youth very satisfied with program, job training, job, and mentor

• Small decreases in some mean pre- and post- scores (started and ended high)

• Mentors rated training and program high• Recommendations to improve payroll• Employers satisfied with program, some suggestions

to better prepare youth

Suggestions to enhance YEPFocus on youth at-risk, in need•Selection process

– Limit college students– Limit prior participants– Priority to those with jj or cj involvement– Priority to those with no prior employment– Priority to those with limited, no opportunities for

another job

Suggestions to enhance YEPEncourage education for higher level, higher paying jobsEnhance mentoring component•Mentors should discuss, help mentees with

– conflict resolution– self-esteem– violence

Suggestions to enhance YEP

• Enhance evaluation– Improve survey administration, completion rate– Repeated instruction by Evaluator– Encourage administer the same way each time, no

distractions, enough time allotted– Add questions

EVALUATION OF PARENT PROGRAM

Evaluation of PP methods

• Administrative data• Parent Leader training evaluation• Coordinator/ Manager exit survey• Parent Leader exit survey• Parent Leader pre- and post-survey

Lead Agencies: Program data

• 983 Parent Leaders recruited• 834 Parent Leaders trained• 4,816 hours spent on service projects• 107 projects completed

Parent Leader: Training ratings

Aspect of training PercentWell designed 94%Trainers knowledgeable 96%Materials, handouts useful 95%Protective factors covered 96%

Agreed or strongly agreed:

Parent Leader: Learned from training

Learned at training PercentHow to strengthen own family 95%How to strengthen community 90%How to form parent teams 89%How to create service teams 87%

Agreed or strongly agreed:

Coordinators/Managers: On program

Aspect of program PercentSupport from ICJIA 75%Support from BSF 90%Preparation for role 88%Training for role 86%Support from Lead Agency 71%

Rated good or very good:

Coordinators/Managers: Community service

Improved community Increased protective factors

Coordinators/Managers: Continuing service project

Parent Leaders: On program

Aspect of program PercentSupport from Manager/Coordinator 96%Training for your role 92%Resources to complete service projects 85%Successfulness of program 78%

Rated good or very good:

Parent Leaders: Incorporate what learned in daily life?

Parent Leaders: Community service

Improved community Increased protective factors

Pre- and post- surveyMeasured:•family functioning and resiliency•social and concrete support•nurturing and attachment•Child development/ knowledge of parenting

Received:•889 surveys•613 pre- and 276 post-surveys•204 matched pre- and post-surveys

Pre- and post- results n Mean 1

(pre)Mean 2 (post)

Change in means

Family functioning and resiliency*

204 5.57 5.80 .23

Social and concrete support

204 4.48 4.49 .01

Nurturing and attachment

202 3.95 4.04 .09

Child development and knowledge of parenting

202 5.57 5.66 .09

Combined measures* 204 4.86 5.00 .24

Conclusions from Parent Program evaluation

• Program increased protective factors• Parent Leader trainings well received• Administrative teams’ responses were

favorable to the program• Parent Leaders thought program was well

conducted and successful

Suggestions to enhance PP

• Increase protective factor of social and concrete support

• Reduced change in means on 3 items:– I have others who will listen when I need to talk about my problems.– I wouldn’t know where to go for help if I had trouble making ends

meet. (reverse coded)– If I needed help finding a job, I wouldn’t know where to go for help.

Suggestions to enhance PP• Recruit younger parents with young children

– More at-risk for child abuse, neglect

• Continue to recruit primary caregivers– To have greatest impact on parenting; training

material most relevant to them

• Recruit more fathers– Involvement in child’s life is a protective factor, can

reduce child maltreatment

Suggestions to enhance PPEnhance evaluation•Improve survey administration, completion rate•Add questions:

– Parent Leaders: Age; gender; number & ages of children; status as primary caregiver of own children or grandchildren; loss of custody of children, prior DCFS involvement

– Service projects: Number of participants/attendance per project; duration of service project

Thank you!

Questions/comments Jessica.Reichert@Illinois.gov

Reports will be available Spring 2014 on website www.icjia.state.il.us