Long-Term Software Long-Term Software Projects Development – Projects Development – The Affect of Students' The Affect of Students' Self-Appreciation and Self-Appreciation and

Initial ExpectationsInitial Expectations

Cecile Yehezkel Bruria Cecile Yehezkel Bruria HabermanHaberman

Davidson Institute of Science Davidson Institute of Science EducationEducation

Weizmann Institute of ScienceWeizmann Institute of Science

1

Computer Science, Computer Science, Academia & IndustryAcademia & Industry

Educational Program for Educational Program for Establishing an Entry Establishing an Entry

Point into the ComputingPoint into the Computing Community of Practice Community of Practice

2

3

High School High School computing computing programsprograms

During the last two decades a program in During the last two decades a program in computer sciencecomputer science and a program and a program inin softwaresoftware engineeringengineering designed for the high- designed for the high-

school level, have been in operation in Israelschool level, have been in operation in Israel . .

With the aim:With the aim: to expose young students to the to expose young students to the fundamentals of computing and to motivate fundamentals of computing and to motivate them to seek expertise in this fieldthem to seek expertise in this field..

(Gal-Ezer et al, 1995; Gal-Ezer & Harel, 1999; (Gal-Ezer et al, 1995; Gal-Ezer & Harel, 1999; Haberman & Cohen, 2005)Haberman & Cohen, 2005)

4

• The programs have evolved over the The programs have evolved over the years. years.

• However, However,

• There still exists a gap between school and There still exists a gap between school and the “real world” of computing: the “real world” of computing:

•ContentContent•Learning styleLearning style•professional normsprofessional norms

5

MotivationMotivation

The existing gap motivated us to initiate The existing gap motivated us to initiate the the

Computer Science, Computer Science,

Academia and Industry Academia and Industry

extra-curricular educational programextra-curricular educational program

in the Davidson Institute of Science in the Davidson Institute of Science EducationEducation

(Weizmann Institute of Science)(Weizmann Institute of Science)

6

GoalsGoals

To expose young students To expose young students directly by leading expertsdirectly by leading experts

to an up-to-date field of computingto an up-to-date field of computing

• R&D processes.R&D processes.• Common issues that professionals cope with. Common issues that professionals cope with. • Advanced technologies and methodologies.Advanced technologies and methodologies.• Professional norms.Professional norms.

7

Our VisionOur Vision

Our program represents an initiative to Our program represents an initiative to establish a different establish a different "breaking out of the "breaking out of the

box learning spacebox learning space""

in the sense that it enables students toin the sense that it enables students to : :

• Bridge the gapBridge the gap between school and between school and the the “real world” of computing “real world” of computing • Experience aExperience a different learning style different learning style• ApplyApply different learning and thinking different learning and thinking skillsskills

7

8

A new learning A new learning cultureculture

Traditional approachTraditional approach• Students should acquire explicit knowledge Students should acquire explicit knowledge

based on a thorough understanding of the based on a thorough understanding of the topic learned. topic learned.

New approachNew approach• Students should be taught to employ a Students should be taught to employ a

breadth-oriented "breadth-oriented "tastingtasting"-based learning "-based learning style.style.

• The initial exposure to an unfamiliar topic The initial exposure to an unfamiliar topic will be accomplished by getting acquainted will be accomplished by getting acquainted only with its essenceonly with its essence. .

9

The programThe program

• (Stage A)(Stage A) - Enrichment meetings - Enrichment meetings Breadth-oriented learning styleBreadth-oriented learning style

• (Stage B) - (Stage B) - Development of software Development of software projectsprojects

• In a setting that simulates a "real world" In a setting that simulates a "real world" environmentenvironment

• Depth-oriented learning styleDepth-oriented learning style

10

The student The student populationpopulation

Excellence - a relative propertyExcellence - a relative property

• Stage AStage A - 11 - 11thth grade students who major in grade students who major in CS/SE and who their teachers consider as CS/SE and who their teachers consider as "excellent students“ "excellent students“ (relatively to their classmates).(relatively to their classmates).

• As a result, there is a As a result, there is a great diversitygreat diversity in the student in the student population.population.

• Stage BStage B – 12 – 12thth grade students who exhibit grade students who exhibit the following characteristics: the following characteristics:

– High motivation, creativity, self-learning and inquiry High motivation, creativity, self-learning and inquiry ability ability

– Persistence, consistency, ability to follow up a time table Persistence, consistency, ability to follow up a time table

Participation in Participation in the programthe program

%Grad .***

#Students

Stage B**

#StudentsStage A*

#Schools

Years

52%257192004-

2006

56%50140202005-

2007

56%80180302006-

2008

61%85210302007-

2009Still

working93220302009-

201011

*Enrichment meetings **Project development activity

***Students who successfully finished their project

12

The programThe program

Stage A – Enrichment meetingsStage A – Enrichment meetings

– Students accompanied by their teachers, Students accompanied by their teachers, attend a 7-month enrichment workshop. attend a 7-month enrichment workshop.

– Each meeting consists of a lecture by a Each meeting consists of a lecture by a CS/SE scientist, a lecture by a SE CS/SE scientist, a lecture by a SE practitioner, and related class activities. practitioner, and related class activities.

– A "visiting the industry" tours are A "visiting the industry" tours are conducted. conducted.

13

Sample of topics•Advanced programming paradigms - scenario-based programming, aspect-based programming;

•Development of complex systems - model-based development, advanced software development tools, computing in space;

•Artificial intelligence - machine learning; neural networks, the control of motion in biological and robotic systems;

•The synergy between computer sciences and biology – biological computers; modeling of biologic evolution; using computational models to comprehend behavior of biological systems; transmission of odor by a computer.

•Professional norms - standards, the importance of testing and controlled reuse of code;

•Computer science educational research - misconceptions and their implication regarding the quality of software.

14

Sample of activities

• Construction and programming robots;

• Challenging algorithmic problems;• Role-playing simulation games;• Creative thinking in computer

science;• Model-based development;• Competition in testing software;

15

The programThe program

Stage B – Project developmentStage B – Project development

• Students develop projects under the Students develop projects under the apprenticeship-based supervision of apprenticeship-based supervision of professional mentors.professional mentors.

– solving “real-world” problems for a real solving “real-world” problems for a real clientclient

– utilizing advanced industrial development utilizing advanced industrial development tools tools

• The school teachers are actively involved in The school teachers are actively involved in supporting the students throughout the supporting the students throughout the entire development process.entire development process.

16

Recruiting leadingRecruiting leading experts experts

We believe that the interaction of the We believe that the interaction of the students with students with leading representativesleading representatives

of communities of practiceof communities of practice,,

who actually becomewho actually become

role modelsrole models for the students for the students , ,

may motivate them to pursue their may motivate them to pursue their studies further or pursue a career in the studies further or pursue a career in the

fieldfield..

17

• Project development experience under the Project development experience under the supervision of professional experts may supervision of professional experts may motivate students to acquire in-depth motivate students to acquire in-depth knowledge in computing, promote knowledge in computing, promote creativity, as well as enhance self-learning creativity, as well as enhance self-learning and inquiry ability.and inquiry ability.

• The interaction with role models may The interaction with role models may contribute to establish professional norms. contribute to establish professional norms.

17

18

Sample ProjectsSample ProjectsMentors from industryMentors from industry - practical characteristics - practical characteristics• computerized homes; sun-heated water tankcomputerized homes; sun-heated water tank• programming a robot programming a robot • managing a multimedia-shopmanaging a multimedia-shop• Antivirus Anti-wormAntivirus Anti-worm

Mentors from academiaMentors from academia – – theoretical, research-theoretical, research-basedbased• computerized graphicscomputerized graphics• image processing, voice recognitionimage processing, voice recognition• automatic text categorizationautomatic text categorization• modeling-based development of a control systemmodeling-based development of a control system• disassembling and reassembling DNAdisassembling and reassembling DNA• simulation of the theory of natural selection simulation of the theory of natural selection • games based on learning machine theory games based on learning machine theory 18

19

Stage BStage B

Project developmentProject development– –

The Process and The Process and ChallengesChallenges

19

20

Phases of the Phases of the project project

development development activityactivity

Phase # Description

Phase IStudying the theoretical background needed for developing the project development

Phase IIIdentifying the main algorithmic ideas

Phase IIIAcquiring the needed technical knowledge - studying a suitable programming language and a development environment

Phase IVImplementing the project (writing and testing the code)

20

The “Players” in the The “Players” in the ProjectProject

The challenge: productive cooperation between all four players

The Student

Matan Peled, a 17 years old high-school student ,

selected according to personal skills and motivation

“CS, Academia

and Industry” Program

Centered at the Davidson Institute of Science Education

High-School & Teacher

A high School in an underprivileged Israeli town that

specializes in students majoring

in the sciences

The MentorBarak Raveh, a 28 years old graduate research student

specializing in applications of

machine learning

21

22

Employment FairEmployment Fair

MentorsStudents

List of Projects

Personal Preferences

Mutual intervie

w

The The Employment Employment

Fare SimulationFare Simulation

Final assignments of students to mentors aim to match personal preferences and skills of both

student and mentor23

ChallengesChallenges

Scientific Value

Student Independenc

e and Creativity

Adhering to Professional Time-Tables

Mentor and Student Working Relations • Small gap of age can

become an advantage• But the mentor lacks

official authority and expertise

Independence = essentiality of self-discipline by students

Students are motivated if they believe their work has real value

24

Project DesignProject Design& Implementation& Implementation

A feedback loop between Mentor and Student

Mentor provides student an initial

framework for project

Student comes up with his own ideas

Mentor provides algorithmic

knowledge and guidance to ideas of

the student

25

31

AssessmentAssessmentThe framework of the The framework of the program is complexprogram is complex

32

AssessmentAssessment

• Many actors are involved:Many actors are involved:– Students, teachers, lecturers, mentors.Students, teachers, lecturers, mentors.

• Diversity with respect to:Diversity with respect to:

– Students’ attitudes, preconceptions, Students’ attitudes, preconceptions, norms, background and skills.norms, background and skills.

– Mentor’s perception of the mentoring Mentor’s perception of the mentoring process.process.

– Teacher’s perception of their roles.Teacher’s perception of their roles.

A Long-Term A Long-Term FormativeFormative

Evaluation of the Evaluation of the ProgramProgramWe believe that in order to ensure that the We believe that in order to ensure that the

students will benefit as much as possible from students will benefit as much as possible from the program, it is important to cultivate the program, it is important to cultivate

collaboration and establish good communication collaboration and establish good communication between all the "players" (students, mentors, between all the "players" (students, mentors,

teachers, and the leading team)teachers, and the leading team) . .

This can be fostered by accompanying the This can be fostered by accompanying the development of the program with a long-term development of the program with a long-term

formative evaluation while monitoring different formative evaluation while monitoring different processes that take place throughout its processes that take place throughout its

operationoperation . .

3333

A Long-Term A Long-Term FormativeFormative

Evaluation of the Evaluation of the ProgramProgramThe uniqueness of our evaluation model is its The uniqueness of our evaluation model is its

being naturally integrated as part of the being naturally integrated as part of the program's operation, whereas the research program's operation, whereas the research

tools serve as continuoustools serve as continuous

""built-in" activities of the operational modelbuilt-in" activities of the operational model . .

When evaluation is performed this wayWhen evaluation is performed this way , ,

the study subjects do not feel inconveniencedthe study subjects do not feel inconvenienced , ,

and may benefit from the related activitiesand may benefit from the related activities..

3434

35

Evaluation - GoalsEvaluation - GoalsTo identify and analyze:To identify and analyze:1. Expectations of newcomers.1. Expectations of newcomers.2. Projects development processes.2. Projects development processes.

• Use of resourcesUse of resources• ManagementManagement

3. Students’ attitudes towards the 3. Students’ attitudes towards the “different-from-school” style of “different-from-school” style of learning.learning.

(Yehezkel & Haberman, 2006; (Yehezkel & Haberman, 2006;

Haberman & Yehezkel, 2008; Haberman & Yehezkel, 2008; Haberman, Yehezkel & Salzer, Haberman, Yehezkel & Salzer, 2009)2009) 35

36

Project developmentProject development

136 students (4 cycles) 136 students (4 cycles) accomplished the developmentaccomplished the development

of comprehensive projects (~60%)of comprehensive projects (~60%)

One main drawback in the stageOne main drawback in the stage of project developmentof project development

is the high number of drop-outsis the high number of drop-outs..36

MotivationMotivation

The main goal was to find out:The main goal was to find out:

• How to better select candidates How to better select candidates capable of accomplishing their capable of accomplishing their projectsprojects

• How to improve the mentoring How to improve the mentoring modelmodel

37

38

Project Project developmentdevelopment

1. Students’ motivation and 1. Students’ motivation and initial expectations initial expectations

conducted on students of the 3conducted on students of the 3rdrd cycle (N=cycle (N=7575) )

38

EvaluationEvaluationCandidates for project development (N=75) Candidates for project development (N=75)

were given a questionnaire related to: were given a questionnaire related to: – Initial motivation to develop a project;Initial motivation to develop a project;– Personal qualities considered valuable Personal qualities considered valuable

for developing a project;for developing a project;– Expected benefits;Expected benefits;– Pre-assessment of the time needed for Pre-assessment of the time needed for

the project (hours per week). the project (hours per week). We compared the answers of two groups of We compared the answers of two groups of

students: those who accomplished their students: those who accomplished their projects ("graduates") and those who projects ("graduates") and those who dropped-out. dropped-out.

39

The questions The questions (open)(open)

1.1. Motivation:Motivation:Why do I want do participate in Why do I want do participate in stage B?stage B?

2.2. Aptitudes:Aptitudes:Which qualities that I posses will Which qualities that I posses will help me to complete the project?help me to complete the project?

3.3. Expectations:Expectations:What benefits can I expect from the What benefits can I expect from the project?project?

4.4. Pre-assessment of time needed for Pre-assessment of time needed for the project.the project. 40

41

1 .Why do I want do participate in stage B?

MotivationGraduatesDrop-outs

Matriculation units 25%23%

Creativity 2%13%

Acquiring software development experience

23%10%

Knowledge enrichment93%97%

Acquaintance with industrial proficiency27%23%

Acquaintance with professionals (experts)

20% 17%

Fondness of the domain20%27%

Self-satisfaction7%3%

Frequencies Frequencies Motivation & Motivation & ExpectationsExpectations

42

39%

50%

36%

20%

20%

2%

20%

37%

27%

33%

27%

10%

13%

27%

0% 10% 20% 30% 40% 50%

Matriculation units

Acquiring software developmentexperience

Acquaintance with industrialproficiency

Acquaintance with professionals(experts)

Self-satisfaction

Creativity (motivation)

Fondness of the domain(motivation)

Graduates 39%50%36%20%20%2%20%

Drop-outs 37%27%33%27%10%13%27%

Matriculation units

Acquiring software

development

Acquaintance with industrial

proficiency

Acquaintance with

professionals

Self-satisfaction

Creativity (motivation)

Fondness of the domain (motivation)

Frequencies for Frequencies for each item of each item of

aptitudesaptitudes

43

25%

23%

7%

77%

30%

0%

0%

64%

20%

27%

10%

90%

13%

13%

10%

60%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Creativity

Having experience

Knowledge

Diligence

Fondness of the domain

Fulfilling requirements

Fulfilling standards

Self-learning

Graduates 25%23%7%77%30%0%0%64%

Drop-outs 20%27%10%90%13%13%10%60%

Creativity Having

experience

Knowledge

DiligenceFondness

of the domain

Fulfilling requireme

nts

Fulfilling standards

Self-learning

Pre-assessment of Pre-assessment of time needed for the time needed for the

projectproject• We found that the groups differed in We found that the groups differed in

their pre-assessment of time needed for their pre-assessment of time needed for the project (the project (near to statistical significant near to statistical significant difference)difference) t-test, p=0.06; t-test, p=0.06; Graduates: Graduates: N=43 AVR=4.64 SD=2.54;N=43 AVR=4.64 SD=2.54; Drop-outs: Drop-outs: N=30 AVR=3.60 SD=1.91N=30 AVR=3.60 SD=1.91

• The findings indicated that the The findings indicated that the graduates had a more realistic graduates had a more realistic appreciation of the time they will need appreciation of the time they will need to invest in their project.to invest in their project. 44

Pre-assessment of Pre-assessment of time needed for the time needed for the

projectproject

45

47

EvaluationEvaluation

2. 2. Analysis of projectsAnalysis of projects

development processesdevelopment processes. . Use of ResourcesUse of Resources

conducted on students of the 2conducted on students of the 2ndnd cycle (N=28) cycle (N=28)

48

Students' Students' assessment of assessment of

resourcesresources• The students were asked to answer a The students were asked to answer a

reflective questionnaire just after submitting reflective questionnaire just after submitting their final projects (N=28)their final projects (N=28)

a series of questions of that type: a series of questions of that type:

• When performing Phase When performing Phase # X# X, to , to what extent did you use the what extent did you use the following resources? following resources? (high=5, (high=5, low=1) low=1)

49

ResourcesResources

• Bibliographic ResourcesBibliographic Resources• The Web, professional articles, professional books.The Web, professional articles, professional books.

• SchoolSchool• The school teacher, school learning (materials and The school teacher, school learning (materials and

methods).methods).

• Human Resources (Informal)Human Resources (Informal)• A classmate, a family member, a grown-up A classmate, a family member, a grown-up

acquaintance.acquaintance.

• Mentoring Mentoring • The mentor.The mentor.

• Self-StudyingSelf-Studying• The student.The student.

50

Students' assessment of Students' assessment of resources used for project resources used for project

developmentdevelopment

Phases Web

#1

Arti

cles

#2

B

ooks

#3

Te

ache

r

#4

Sc

hool

Le

arni

ng

#5

Acq

. G

row

nup

#6

Fam

ily

Mem

ber

#7

A

Cla

ss-

mat

e

#8

Men

tor

#9

St

uden

t

#10

#1 #2 #3 #4 #5 #6 #7 #8 #9 #10 ? I. Theoretical

background 4.4

(1.0) 2.9

(1.4) 3.3

(1.3) 2.1

(1.2) 2.3

(1.3) 1.5

(0.9) 1.7

(1.1) 2.5

(1.6) 3.8

(1.5) 4.6

(0.6)

? II. Identification of main ideas

3.9 (1.3)

2.6 (1.6)

2.8 (1.6)

1.7 (1.1)

2.0 (1.4)

1.3 (0.8)

1.6 (1.1)

1.9 (1.5)

3.8 (1.5)

4.3 (0.7)

? III. Technical Knowledge

3.8 (1.3)

2.8 (1.5)

3.0 (1.6)

2.0 (1.3)

2.1 (1.4)

1.4 (0.8)

1.6 (1.1)

2.1 (1.4)

3.5 (1.5)

4.6 (0.8)

? IV. Project Implementation

3.4 (1.5)

2.3 (1.6)

2.6 (1.7)

1.7 (1.1)

1.9 (1.3)

- 1.7 (1.2)

2.1 (1.5)

3.5 (1.8)

4.7 (0.8)

51

Students' assessment of Students' assessment of resources used for project resources used for project

developmentdevelopment

1 1.5 2 2.5 3 3.5 4 4.5

Student

Mentor

A classmate

Family Member

Acquaintance (grown-up)

School Learning

Teacher

Books

Articles

WEB

Mean

52

FindingsFindings• The students exhibited self-efficacy.The students exhibited self-efficacy.• They relied more on themselves than on other They relied more on themselves than on other

resources especially toward the complete resources especially toward the complete implementation of the project. implementation of the project.

• Low reliance on classmates their own age, and Low reliance on classmates their own age, and higher reliance on school learning. higher reliance on school learning.

• The school teachers were low-appreciated but The school teachers were low-appreciated but better appreciated than a family member and a better appreciated than a family member and a grown-up acquaintance. grown-up acquaintance.

• Self-studying and the web were perceived as the Self-studying and the web were perceived as the most significant resources. most significant resources.

It may imply that the mentors' guidance It may imply that the mentors' guidance inspired the students' self-inquiry and self-study.inspired the students' self-inquiry and self-study.

55

EvaluationEvaluation

2. 2. Analysis of projectsAnalysis of projects

development processesdevelopment processes. .

ManagementManagement

conducted on students of the 2conducted on students of the 2ndnd cycle (N=28) cycle (N=28)

63

EvaluationEvaluation

3. 3. Students’ attitudes towards Students’ attitudes towards the “different-from-school” the “different-from-school”

style of learning.style of learning. conducted on students of the 3conducted on students of the 3rdrd

cycle (N=85) cycle (N=85)

63

64

Attitudes towards Attitudes towards Stage AStage A

The students viewed the program as fulfilling The students viewed the program as fulfilling their expectations.their expectations.

• They indicated that the program provides They indicated that the program provides an acquaintance with up-to-date subjects. an acquaintance with up-to-date subjects.

• They appreciated more the wide exposure They appreciated more the wide exposure to a variety of new subjects than focusing in-to a variety of new subjects than focusing in-depth on one subject. depth on one subject.

• They rarely expected the program to focus They rarely expected the program to focus more on subjects learned at school.more on subjects learned at school.

65

Attitudes towards Attitudes towards Stage AStage A

• They highly appreciated the experience of They highly appreciated the experience of meeting researchers and professionals in the meeting researchers and professionals in the field.field.

• The program increased their motivation to The program increased their motivation to pursue further studies of CS/SE and pursue further studies of CS/SE and professional carriers in the field.professional carriers in the field.

• They were highly interested in participating They were highly interested in participating again in enrichment workshop and were again in enrichment workshop and were willing to recommend to their friends to willing to recommend to their friends to participate in the program. participate in the program.

67

Triggering a new culture of learning

• The students underwent a change on The students underwent a change on both a personal and a class level. both a personal and a class level.

• The teachers reported to us that The teachers reported to us that new new types of class activities were initiatedtypes of class activities were initiated by the attendees of the extra-curricular by the attendees of the extra-curricular program. program.

70

SummarySummary

We hope that his programWe hope that his program

will will trigger more representativestrigger more representatives

of academia and the high-tech industry of academia and the high-tech industry

to take an active part in to take an active part in

educating potential newcomerseducating potential newcomers

and that way contribute to and that way contribute to

make the computing professional domainmake the computing professional domain

more attractivemore attractive..

70

71

SummarySummary

We hopeWe hope

that our program will promotethat our program will promote

a culture of learning and work a culture of learning and work

befitting the dynamic world of befitting the dynamic world of industrial computing industrial computing

and provide the students withand provide the students with

an entry point into the computing an entry point into the computing community of practicecommunity of practice. .

71

72

Thank YouThank You

72

The The Employment Employment

FareFare

73