Post on 03-Jul-2018
Evaluating the Effectiveness of a Career Certificate Program for International Students
Jessamyn Perlus, B.A., Julia Panke Makela, Ph.D., Daniel Scholes, Ph.D., Un Yeong Park, M.Ed., Gaeun Seo, M.S., & Kevin Hoff, B.A.The Career Center, University of Illinois at Urbana-Champaign
The Career Center Research Team
We work at the intersection of research and practice
• Four team members (1 full time, 3 PhD students)
• Located within a practice environment
• Advising and leading a cross-functional Assessment Team
• Exploring research questions, with opportunities for immediate application to practice
2
International (INTL) Students in the U.S.
• US: The world’s TOP education destination for INTL students• Number of International students studying in the U.S. increased continuously
from 526,670 during the 2005/06 academic year to 854,639 in the 2014/15academic year
526,670 541,324 567,039605,015 623,119 647,246 679,338
724,725780,055
854,639
2005/06 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 2012/13 2013/14 2014/15
Number of International Students Studying in U.S. Colleges and Universities
+ 62%
Data Source: Institute of International Education, 2015
3
INTL Students’ Unique Career Challenges
Career challenges to obtaining employment in the U.S.• Cultural adjustment and language barriers (Lin & Flores, 2011; Lopez,
2002; Reynolds & Constantine, 2007)• Complicated employment regulations (e.g., CPT, OPT, H1BVisa) (Kisch,
2015; Lin & Flores, 2011)
Career development responses to challenges• Challenges have been shown to lower career aspirations or limit
consideration of options (Lopez, 2002; Reynolds & Constantine, 2007)• International students express strong needs for assistance with career
exploration and the job search process (Shen & Herr, 2004; Spencer-Rodgers, 2000)
• A growing number of higher education career services offices have added dedicated programs/staff to serve international students (NCDA, 2015)
4
Study Purpose
The literature acknowledges:
• Growing population of international students
• The unique challenges that these students face
• The development of programs and services to address their needs
Yet, what do we know about the outcomes of those programs? How well do career services programs meet the needs of international students?
5
Career Intervention OutcomesMeta-Analytic Studies
• Consistently demonstrate moderate effect sizes “career interventions are effective” in relation to a wide variety of career choice outcomes…
• E.g., career maturity, career decision-making self-efficacy, choice certainty, choice satisfaction, congruence, decidedness, indecision, vocational identity
(E.g., Baker & Popowizc, 1983; Brown & Ryan-Krane, 2000; Brown et al., 2003; Oliver & Spokane, 1988; Whiston et al., 1998)
Critical Intervention Components
• Written exercises
• Individualized interpretation and feedback
• World of work information
• Modeling opportunities
• Attention to building support for choices within one’s social network
(Brown & Ryan Krane, 2000)
6
Career Certificate – International (CCI)
• Innovative 9-week career certificate program dedicated to helping international students become a more competitive job/internship candidate in the U.S.
• Includes weekly workshops, peer mentoring, and participation in a variety of career services throughout one academic semester
• Offered by The Career Center at Illinois (TCC) since Fall 2012
7
Method
• Data collected first and last weeks of 9-week CCI program
• 3 data sources
– Indirect Measure:
• Career Search Efficacy Scale (CSES)
– Direct Measures:
• Resumes
• Written networking introduction statements
8
Career Search Self-Efficacy
Self-efficacy
Individuals’ beliefs concerning his or her capability to successfully perform a certain task or behavior (Bandura, 1977, 1986)
“Can I do this?”
Career Search Efficacy Scale (CSES, Solberg et al., 1994)
Focus on career/job search related domains
9
Method - Career Search Efficacy Scale (CSES)
• 4 subscales: Job Search Efficacy, Interviewing Efficacy, Personal Exploration Efficacy, & Networking Efficacy
• 1 new subscale: “International Student Job Search Efficacy”
• Analysis: Paired samples t-tests to examine change over time
1. Discuss CPT/OPT options with an employer
2. Communicate effectively with an employer in English
3. Identify employers more friendly to international students
4.Utilize social media like LinkedIn to identify and connect with individuals who can support international students' career development
5.Identify and utilize career resources dedicated to international students
10
Method – Document Analysis
• Documents were blind-reviewed by trained reviewers
– Resume uses established rubric (Rooney et al., 2012)
– Networking script rubric designed for this study
• Analysis: Paired samples t-tests to examine change over time
11
Participants
• 49 students completed the Spring 2015 program; 43 (88%) participated in the evaluation study
• Student Response Rates:
– 31 (63%) completed self-reported pre- and post-assessment of Career Search Efficacy Scale (CSES)
– 43 (88%) submitted pre- and post-resumes
– 40 (82%) submitted pre- and post-elevator pitches
• Countries Represented: Bangladesh, China, Ecuador, Indonesia, Malaysia, Mongolia, and South Korea
12
Demographics of Program Participants
Freshman13%
Sophomore16%
Junior26%
Graduate35%
Missing10%
CLASS STANDING
Male29%
Female61%
Missing10%
GENDER
Heterosexual68%
Bisexual3%
Prefer not to respond
16%
Missing13%
SEXUAL IDENTITY
84%
3%
3%10%
RACIAL / ETHNIC IDENTITY
Asian Hispanic or Latino Prefer not to respond Missing
13
Results – Career Search Efficacy
5.7 6.05.4 5.7 6.1
5.3
7.7 7.8 7.6 7.7 7.6 7.9
0
1
2
3
4
5
6
7
8
9
10
***OverallCareer SearchEfficacy Scale
***Job SearchEfficacy
***InterviewEfficacy
***NetworkingEfficacy
***PersonalExploration
Efficacy
***Int’l Student Job Search
Efficacy
Career Search Efficacy Scale
Pretest Posttest
14
*** p < .001
Results – Resumes
2.5
2.2
2.5
1.4
3.6
2.9
2.3
2.8
2.4
2.7
2.42.6
1.5
3.8
3.0
2.6
2.9 2.9
1
1.5
2
2.5
3
3.5
4
**Overall *30 SecondReview
PresentationFormat
Spelling/Grammar
*Heading Education *DescribedExperience
OptionalSections
*PositiveProfessional
Language
Pretest Posttest
15
* p < .05, ** p < .01, *** p < .001
1 =
Inco
mp
lete
, 2 =
Dev
elo
pin
g, 3
= P
rofi
cien
t, 4
= P
rofe
ssio
nal
Results – Networking Scripts
2.4
2.62.5
2.6
2.3
1.9
3.5
1.5
2.4
2.6
2.82.9
2.8
2.3
1.7
3.6
1.7
2.8
1
1.5
2
2.5
3
3.5
4
Overall Hook *DesribedExperiences
Goals/Aspirations
Assess Fitwith
Company
Closing Length Grammar &SentenceStructure
*Clear Story& PositiveLanguage
Pretest Posttest
16
* p < .05
1 =
Inco
mp
lete
, 2 =
Dev
elo
pin
g, 3
= P
rofi
cien
t, 4
= P
rofe
ssio
nal
Summary of Findings
• Significant increases in all domains of Career Search Efficacy Scale (CSES) indicate that the CCI program is making a difference in students’ beliefs in their ability to successfully engage in career search tasks.
• Significant increases in quality on written resumes and elevator pitches, particularly on rubric subscales that are targeted by the program (e.g., 30 Second Review, Described Experience, Positive Professional Language, Clear Story).
17
Implications for Practice
• Share Successes– Communicate successful aspects of the program to colleagues and
future participants
• Modifications to CCI Program– Strengthen “closings” on networking scripts
– Recognize program boundaries
• Provide list of common grammatical mistakes that international students make and direct participants to resources on campus for assistance
• New Resources: Networking Script Rubric– Created a version to use in staff training and student workshops
– Updated research version to clarify criteria to improve interrater reliability
18
Implications for Research
• What about this intervention makes a difference? How can it be improved?
• Can relationships be observed between programs for international students (the intervention) and:
– Developmental outcomes, e.g. reduced influence of perceived barriers on career aspirations
– Process outcomes, e.g. increased knowledge and strategies for completing career development tasks
– Summative outcomes, e.g. obtained employment
– Comparison outcomes, e.g. vs. domestic students, country of origin
19
Future Directions
Qualitative Project in Progress• Grant from Midwest ACE to conduct
interviews of CCI alums• How students perceive each factor of
the program contributed to their experience and learning: – Mentoring? – Peer relationships built with other
participants and volunteers? – Skills practiced? – Resources or information gained
during the workshops?– What could have helped students
make even greater leaps forward?
20
Resources from The Career Center
• Makela, J. P., Scholes, D. R., Perlus, J. G., Seo, G., & Park, U. (2016). Outcomes of the Career Certificate-International Program, Report. Available upon request.
• INTL Connect Website: http://www.intlconnect.illinois.edu/tccservices
21
References (Page 1 of 2)
22
• Baker, S. B., & Popowicz, C. L. (1983). Meta-analysis as a strategy for evaluating effects of career education interventions. Vocational Guidance Quarterly, 31, 178–186.
• Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.
• Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.
• Brown, S. D., & Ryan Krane, N. E. (2000). Four (or five) sessions and a cloud of dust: Old assumptions and new observations about career counseling. In S. D. Brown & R. W. Lent (Eds.), Handbook of counseling psychology (3rd ed., pp. 740–766). New York, NY: John Wiley & Sons.
• Brown, S. D., Ryan Krane, N. E., Brecheisen, J., Castelino, P., Budisin, I., Miller, M., & Edens, L. (2003). Critical ingredients of career choice interventions: More analysis and new hypotheses. Journal of Vocational Behavior, 62, 411-428. doi:10.1016/S0001-8791(02)00052-0.
• Institute of International Education. (20142015). International student enrollment trends, 1948/49-20134/154. Open Doors Report on International Educational Exchange. Retrieved from http://www.iie.org/opendoors
• Kisch, M. (2015). Helping international students navigate career options. International Educator, 24(3), 66-69. Retrieved from http://www.nafsa.org/_/File/_/ie_mayjun15_fsa.pdf
• Lin, Y. & Flores, L. Y. (2011). Job search self-efficacy of East Asian international graduate students. Journal of Career Development, 40(3), 186-202. doi:10.1177/0894845311418655.
• Lopez, A. E. (2002). Factors influencing career decision-making and post-secondary career choices of a cohort group of female Cuban immigrants. Doctoral dissertation. Retrieved from ProQuest. 304788595.
References (Page 2 of 2)
• NCDA International Student Work Group (2015). Resources for partnering with international students. National Career Development Association (NCDA). Retrieved from http://ncda.org/aws/NCDA/asset_manager/get_file/64588/resources_for_partnering_with_international_students.pdf
• Oliver, L. W., & Spokane, A. R. (1988). Career-intervention outcome: What contributes to client gain? Journal of Counseling Psychology, 35(4), 447–462.
• Reynolds, A. L., & Constantine, M. G. (2O07). Cultural adjustment difficulties and career development of international college students. Journal of Career Assessment, 15(3), 338–350. doi: 10.1177/1069072707301218.
• Rooney, G. S., Pipkins, K., DiVietro, M. & Sun, H. (2012). Fall 2012 resume review evaluation findings.[Technical Report]. Champaign, IL: The Career Center, University of Illinois at Urbana-Champaign.
• Shen, Y. J., & Herr, E. L. (2004). Career placement concerns of international graduate students: A qualitative study. Journal of Career Development, 31(1), 15-29. doi: 10.1023/B:JOCD.0000036703.83885.5d.
• Solberg, Good, G. E., Nord, D., Holm, C., Hohner, R., Zima, N., Heffernan, M., & Malen, A. (1994). Assessing career search expectations: Development and validation of the Career Search Efficacy Scale. Journal of Career Assessment, 2(2), 111-123. doi: 10.1177/106907279400200202.
• Spencer-Rodgers, J., & Cortijo, A. (1998). An assessment of the career development needs of international students. Journal of College Student Development, 39(5), 509–513. Retrieved from http://eric.ed.gov/?id=EJ581220.
• Whiston, S. C., Sexton, T. L., & Lasoff, D. L. (1998). Career-intervention outcome: A replication and extension of Oliver and Spokane. Journal of Counseling Psychology, 45, 150-165.
23
Thank You
• Un Yeong Park and Tina Yeo for their coordination of the CCI program Fall 2015
• Assessment Committee for their feedback (Brian Neighbors, Taisha Mikell, Amanda Cox)
• Midwest ACE for their generous support of our future research in this area
24
25
26
27
28