Evaluation Overview Plus 50 Conference July 29 - 30, 2009 Washington, DC.
-
Upload
regan-boyers -
Category
Documents
-
view
214 -
download
0
Transcript of Evaluation Overview Plus 50 Conference July 29 - 30, 2009 Washington, DC.
Evaluation Overview
Plus 50 Conference July 29 - 30, 2009Washington, DC
Faces to Names and Voices: Your Evaluation Liaisons
Kathleen Hentz• Spokane• Luzerne• Richland
Laura Bekes• Central Florida• Wake Tech• Northern Virginia
Samantha Chen• Cape Cod• Santa Fe• Saint Louis• Clark• Western Dakota
Overview
• Why Evaluate?
• How Do You Evaluate the Plus 50 Initiative?
• How Do You Evaluate the Plus 50 Grantees?
Why Evaluate?
• DOCUMENT implementation
• CAPTURE lessons learned
• TRACK outcomes over time
• SHOW the effectiveness of a model
Overview
• Why Evaluate?
• How Do You Evaluate the Plus 50 Initiative?
• How Do You Evaluate the Plus 50 Grantees?
How Do You Evaluate the Plus 50 Initiative?
Putting the evaluation puzzle pieces together:
– Research Questions– Research Tools
•Program Implementation:
•What progress have grantees made in implementing programs?
•Lessons Learned:
•What have grantees learned about what works and doesn’t work in program implementation?
•Participant Outcomes:
•Is program participation having the hoped-for results among the participants? Are they satisfied with the program?
How Do You Evaluate the Plus 50 Initiative?
Identifying Research Questions
•Data Collection Worksheets •Participant Surveys•Key Informant Interviews
How Do You Evaluate the Plus 50 Initiative?
Identifying Appropriate Research Tools
How Do You Evaluate the Plus 50 Initiative?
Implementation Assessment
• What is the progress on workforce training and career development goals for each college?
•Data Collection Worksheets
•Key Informant Interviews
How Do You Evaluate the Plus 50 Initiative?
How is implementation progressing?
2 4125
18
1 20
5
10
15
20
25
30
Unique Total Unique Total
Mentor Colleges(n=4)
Demonstration Colleges(n=10)
Ave
rag
e N
um
ber
of C
ou
rses
for S
tud
ents
Ag
e 50
+
Plus 50 Initiative Baseline DataTraining and Re-training Non-credit Courses
for Students Age 50+July 1, 2007 - June 30, 2008
Newly designed Redesigned Existing marketed
How Do You Evaluate the Plus 50 Initiative?
Lessons Learned
"The fact that we have monthly enrollment is very helpful for the students. …People are laid off year-round not necessarily aligned with an academic calendar.”
-Plus 50 College
"Short-term is very much in demand, as opposed to something long-term over many semesters. We’re looking at short-term programming that gets people out and on their way.”
-Plus 50 College
"The registration process has a lot of steps. We do it for them – if they had to jump through all those hoops they might give up. The college has also allowed us to skip a few steps.”
-Plus 50 College
How Do You Evaluate the Plus 50 Initiative?
Participant Outcomes Evaluation
How Do You Evaluate the Plus 50 Initiative?
Participant Outcomes Evaluation
• What outcomes do Plus 50 participants experience as a result of the their participation in training courses and career services?
•Participant Surveys
How Do You Evaluate the Plus 50 Initiative?
Participant Outcomes Evaluation
6%
5%
5%
6%
6%
5%
34%
35%
31%
55%
54%
59%
0% 25% 50% 75% 100%
Improvement in Technology Skills (n=119)
Improvement in Job Skills (n=131)
Acquisition of New Job Skills (n=130)
Percentage of Respondents
Skills Acquired by Plus 50 Workforce Training Participants
Completely Disagree Mostly Disagree Mostly Agree Completely Agree
How Do You Evaluate the Plus 50 Initiative?
Participant Outcomes Evaluation
20%
18%
20%
15%
25%
29%
13%
14%
33%
29%
38%
31%
22%
25%
30%
40%
0% 25% 50% 75% 100%
Got a raise(n=49)
Got a promotion(n=49)
Advanced in my job
(n=56)
Landed a job(n=107)
Percentage of Respondents
Job Outcomes of Plus 50 Workforce Training Participants
Completely Disagree Mostly Disagree Mostly Agree Completely Agree
How Do You Evaluate the Plus 50 Initiative?
Participant Outcomes Evaluation
“I will always remember you for caring and the important part you played in my success in securing a position with FedEx Express. …I received a call on Thursday to sign my offer letter on Friday at 10AM!!! ….There were times when I was almost running on empty. I wish many more success stories for the 50+ workforce people.”
-Plus 50 Participant
How Do You Evaluate the Plus 50 Initiative?
Participant Outcomes Evaluation
“The classes made me more aware of where I wanted to be going in my career at this time in my life. Also reviewing my skills and abilities, personality traits and strengths, I was able direct my energies toward solid decisions.”
-Plus 50 Participant
How Do You Evaluate the Plus 50 Initiative?
Participant Outcomes Evaluation
“Perhaps you could offer training in specifically targeted skills. For example, a one-hour [practical workshop] on Saturday morning – practice with e-mailing attachments, practice with storing folders, files, etc…..Practice time with a knowledgeable person would be good.”
-Plus 50 Participant
How Do You Evaluate the Plus 50 Initiative?
Additional Research Questions
How Do You Evaluate the Plus 50 Initiative?
Other Things we Want to Know
• How is the Plus 50 Initiative raising public awareness about the issues of Plus 50 learners, and the model?
• What have we learned about what works well in mentorship relationships?
• How has the AACC staff supported grantees?• Are the programs becoming sustainable, and
what strategies work for sustainability?
Overview
• Why Evaluate?
• How Do You Evaluate the Plus 50 Initiative?
• How Do You Evaluate the Plus 50 Grantees?
How Do You Evaluate the Plus 50 Grantees?
Looking at Individual Colleges
• Are colleges making progress against the goals set out in the proposal?
• Are colleges appropriately responding to lessons learned by finding ways to stay true to the overarching program objectives while changing implementation tactics to adjust to circumstances?
How Do You Evaluate the Plus 50 Grantees? What is Success for Individual Colleges?
•Information that helps inform AACC on individual college Plus 50 implementation and development:
•Progress Reports
•Site Visits
•Conference Calls
•Data Collection Worksheets
•Participant Surveys
How Do You Evaluate the Plus 50 Grantees? What is Success for Individual Colleges?
• What’s working?
• What would work better in the coming year?
Thoughts on the Year 1 Evaluation
• Participant Surveys
• Data Collection Forms for Year 2
• Key Informant Interviews
Evaluation Next Year
Questions
Thank You!