CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length !...

28
CO-OPERATIVE EDUCATION ANALYTICS: SATISFACTION & RELATIONSHIPS AMONG PROGRAMS Presenter: Yuheng Helen Jiang Authors: Yuheng Helen Jiang, Sally Lee (Satisfaction) Lukasz Golab (Supervisor) Department of Management Sciences September 22, 2015

Transcript of CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length !...

Page 1: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

CO-OPERATIVE EDUCATION ANALYTICS: SATISFACTION & RELATIONSHIPS AMONG PROGRAMS

Presenter: Yuheng Helen Jiang Authors: Yuheng Helen Jiang, Sally Lee (Satisfaction)

Lukasz Golab (Supervisor)

Department of Management Sciences

September 22, 2015

Page 2: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Who we are… 2

¨  Professor Lukasz Golab ¤  Assistant Professor in Management Sciences, cross appointed to Computer

Science ¤  BSc in Computer Science from the University of Toronto (2001) ¤  PhD in Computer Science from the University of Waterloo (2006)

¤  Research Interests: big data; applications of big data on energy and education

¨  Yuheng Helen Jiang ¤  BASc. In Management Engineering from the University of Waterloo

(2013) n  Accomplished 6 co-op terms using the Jobmine system

¤  (almost) MASc in Management Sciences from the University of Waterloo (2015)

¤  Research Interests: applications of data mining on energy and education

Page 3: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Agenda 3

¨  Satisfaction ¤  Jiang, Yuheng Helen, Lee, Sally Wai Yin, Golab, Lukasz.

"Analyzing student and employer satisfaction with cooperative education through multiple data sources." Asia-Pacific Journal of Cooperation Education, 16(4), 225-240.

¤  Objective: determine the factors affecting student and employer success and satisfaction with the co-op experience

¨  Relationships among academic programs ¤  Master thesis: On Competition for Undergraduate Co-op

Placements: A Graph Mining Approach ¤  Objective: improve the co-op process by characterizing the

relationships and extent of competition for co-op placements among students from various academic programs

¨  Future work

Page 4: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

SATISFACTION

Title: Analyzing student and employer satisfaction with cooperative education through multiple data sources

Page 5: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Satisfaction: Data 5

¨  3 years (Winter 2009 – Fall 2011) of ¤  employers’ evaluations of students (19 sub-categories 1-4/

not applicable & overall evaluation 1-5) ¤  students’ evaluations of employers (overall evaluation 1-10)

¨  Engineering students only ¨  Other factors:

¤ Work term number ¤  Length of co-op terms: 4 months or 8 months ¤  Timing of the first work term: after 1 or 2 terms ¤  Location: abroad, domestic ¤ How to find a co-op job: regular process, self-arranged,

return (work term status)

Page 6: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Satisfaction: Key Finding 1 6

¨  Overall: students are generally willing to learn new skills, but may not have much leadership experience

3.45 3.30 3.13 3.313.59 3.42 3.40

3.013.40 3.23 3.21

3.53 3.54 3.51 3.653.22 3.26

2.92

3.52

0.70 0.76 0.66 0.63 0.61 0.66 0.67 0.68 0.58 0.62 0.67 0.65 0.61 0.54 0.57 0.63 0.62 0.70 0.63

0.000.501.001.502.002.503.003.504.00

Average Standard  DeviationFigure 2: Average and standard deviations of the scores of the 19 sub-categories of employers’ evaluations of students

Page 7: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Satisfaction: Key Finding 2 7

¨  Overtime: students with more work experience receive higher scores

Figure 3: Percentage of co-op students in each evaluation category from outstanding to unsatisfactory and average of employer’s evaluation over first to sixth work terms

Page 8: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Satisfaction: Key Finding 3 8

¨  50% of evaluations are not applicable: conflict management and leadership

¨  Over 6 terms: ¤  integration of prior learning, setting goals, and

leadership decreased significantly

¨  Returning students: ¤ conflict management, leadership and integration of

prior learning decreased significantly

Page 9: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Satisfaction: Key Finding 4 9

¨  Keywords in job titles and employer names ¤ First year

Employer  Name  Keywords Job  Title  Keywords

University Engineering

Ontario Assistant

Toronto Developer

General Software

Research Junior

System Architectural

Engineering Web

Canadian Technical

City Research

Environment IT

TABLE 2: Top 10 keywords from employer names and job titles for first-year engineering students

Page 10: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Satisfaction: Key Finding 5 10

¨  Effect of work term length ¤ Nearly 70% of students stayed 8-month work terms with the same

employer ¤  4-month with two positions:

n Students were rated higher on their ability to learn, quality of work, quantity of work, creativity, problem solving and reliability

¤  8-month with one position: n Students were rated higher in goal setting, judgment, conflict

management, initiative and leadership n N/A% decreased in goal setting and integration of prior

learning n Students rated their employer 10% lower

Page 11: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Satisfaction: Key Finding 6 11

¨  Timing of first work term ¤ Students’ evaluation of employers

n After 1 term > After 2 terms

¤ Employers’ evaluation of students n No significant difference

Page 12: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Satisfaction: Key Finding 7 12

¨  International vs. Domestic ¤ 10% positions were outside of North America ¤ Keywords in job titles

n  International: trainee, intern n Domestic: co-op

¤ First term working abroad n More self-arranged positions n Students were rated worse, and less satisfied

¤ Upper years n Students were rated better, and more satisfied

Page 13: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Satisfaction: Conclusion 13

¨  Students’ perspective ¤ Expect N/A ratings in some categories ¤ Stay with the same employer? ¤ Work abroad?

¨  Employers’ perspective ¤ N/A option exists

¨  Institutions’ perspective ¤ When to start the first work term? ¤  International positions? ¤ Data collection

Page 14: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

RELATIONSHIPS AMONG ACADEMIC PROGRAMS

Title: On Competition for Undergraduate Co-op Placements: A Graph Mining Approach

Page 15: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Data Overview 15

¨  One term of interview data ¤ 16,855 student-job interview pairs ¤ 2,890 jobs ¤ 4,194 students from 93 academic programs

¨  Job ¤ Job title, advertised programs, advertised seniority

¨  Student ¤ Academic program, academic year

Page 16: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Graph Definition 16

Source: http://rubymediagroup.com/wp-content/uploads/2015/06/facebook-graph-search-job-search.jpg

Page 17: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Graph Definition (Cont’d) 17

Figure 4.1: Full program graph

Page 18: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Degrees of separation 18

Source: https://www.careeraddict.com/img/candidate-606031-2013-11-14-11-37-24.jpeg

¨ Six degrees of separation

Page 19: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Degrees of separation 19

A program

Random program

1.57

¨ Programs are REALLY connected!

Page 20: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Two graphs 20

¨  Full program graph vs. Senior program graph ¨  Finding:

¤ Senior students compete for jobs with students from fewer programs (less edges)

¤ Relationships that do exists are stronger (thicker edges)

Page 21: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Motivation 21

¨  Unclear differences among academic programs ¤ Similar programs: clusters

¨  Increasing need for multi-disciplinary and well-rounded education [9,10,18,71,119] ¤ Multi-disciplinary programs: outliers

¨  Example of jobs for promotion ¤ Competing programs: fan-out metric

Page 22: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Similar programs 22

Figure 4.17: Hierarchy of partition results of senior program graph

Page 23: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Multi-disciplinary programs 23

¨  Management Engineering (senior): 5th highest entropy

Junior students Senior students

Figure 4.22 & 4.23: Word cloud for job titles of jobs that interviewed junior/senior Management Engineering students

Page 24: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Multi-disciplinary programs 24

¨  Question: well-rounded students or sets of specialized students?

Figure 4.36: Cumulative percentage of students over number of clusters of direct competitors (7 clusters)

Page 25: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Competing programs 25

16 programs do not have any jobs that interviewed only their students

8 programs have more than 30 percent of the jobs only interviewed their students

Figure 4.24: Level of competition of programs in the senior program graph in descending order

Page 26: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Relationship: Conclusion 26

¨  Similar programs ¤ Academic programs did not always align well with the

groups of closely connected programs ¤ Clusters can be used to create job categories and

academic specializations ¨  Multi-disciplinary programs

¤  Identification and verification ¨  Competing programs

¤ Attract more employers that offer jobs to programs that face high competition

Page 27: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

Future Work 27

¨  Develop recommender systems ¤  Recommend jobs to students ¤  Recommend students to employers

¨  Temporal analysis ¤ What has changed over the years? ¤ How does co-op behavior relate to key events or social

factors? ¨  Ranking

¤ How do employers and students play the ranking “game”? ¨  Collaboration!

Page 28: CO-OPERATIVE EDUCATION ANALYTICS · Satisfaction: Key Finding 5 10 ! Effect of work term length ! Nearly 70% of students stayed 8-month work terms with the same employer ! 4-month

THANK YOU! QUESTIONS?

Contact: Helen Jiang: [email protected] [email protected] Lukasz Golab: [email protected]