Programming Aptitude Testing Materials, Data analysis Saeed Dehnadi Middlesex University.

Programming Aptitude Testing Materials, Data analysis

Saeed Dehnadi

Middlesex University

I talk about:

• Test materials– Questions– Mental Models– Answer Sheet– Mark Sheet– Interpretation Algorithms

• Results– Middlesex University (initial test) - 2005– University of Newcastle (Australia) - 2006– York - 2006– Strathclyde - 2006– Middlesex University (second test) - 2006

• Collaborators• Summary

Questions 1 (single assignment)

(M1) Mental model “move value”

a = b;

• The value of b is given to a and b changes its value to zero.

a b // b 0

Ans ( a = 20 , b = 0 ) 8th Answer

(M2) Mental model“copy value”

a = b;

• The value of b is given to a and b keeps its original value.

a b // b unchanged

Correct conception of LHS RHS

Ans ( a = 20 , b = 20 ) 4th Answer

(M3) Mental model “move value”

a = b;

• The value of a is given to b and a changes its value to zero.

b a // a 0

Ans ( a = 0 , b = 10 ) 3rd Answer

(M4) Mental model“copy value”

a = b;

• The value of a is given to b and a keeps its original value.

b a // a unchanged

Ans ( a = 10 , b = 10 ) 1st Answer

Mental models (M5) (copy + add) value

a = b;

• The sum of a and b is given to a, and b keeps its original value.

a (a + b) // b unchanged

Ans ( a = 30 , b = 20 ) 2nd Answer

Mental models (M6) (move + add) value

a = b;

• The sum of a and b is given to a, and b changes its value to zero.

a (a + b) // b 0

Ans ( a = 30 , b = 0 ) 10th Answer

Mental models (M7) (copy + add) value

a = b;

• The sum of a and b is given to b, and a keeps its original value.

b (a + b) // a unchanged

Ans ( a = 10 , b = 30 ) 9th Answer

Mental models (M8) (move + add) value

a = b;

• The sum of a and b is given to b, and a changes its value to zero.

b (a + b) // a 0

Ans ( a = 0 , b = 30 ) 5th Answer

Mental models (M9) “no changes”

a = b;

• a and b keep their original values.

a unchanged // b unchanged

Ans ( a = 10 , b = 20 ) 6th Answer

Mental models (M10) Equality

a = b;

• Assignment is a simple equation, and then all equal values of a and b are acceptable.

Ans ( a = 10 , b = 10 ) and

Ans ( a = 20 , b = 20 )

both 1st and 4th answers

Mental models (M11) “Swap values”

a = b;

• a and b swap their values simultaneously.

a b a gets b’s value

b a b gets a’s value

Ans ( a = 20 , b = 10 ) 7th Answer

Models for a single assignment int a = 10; int b = 20;

a = b;

Answer Sheet (Question1)

Question 4 (two assignments)

Mental model“M1 + Sequence”

a = b;b = a;

Model is M1 that applies sequentially through both statements:

• L1) a b and b 0 then a = 20 and b = 0 • L2) b a and a 0 then b = 20 and a = 0

Single answer: a = 0 , b = 20

Mental model“M1 + Independent (M1 I)”

a = b;b = a;

• Model is M1 that applies independently for each individual line.

• L1) a b and b 0 • L2) b a and a 0

Multiple answers:a = 20 and b = 0a = 0 and b = 10

Mental model“M1 + Simultaneous-single (M1 Ss)”

a = b;b = a;

• M1 Ss Same as (M1 I), subjects only interested on Left-hand-side values of statements and ignores the right-hand-side values.

• L1) a b and b ignores • L2) b a and a ignores

Single answer: a = 20 , b = 10

Answer Sheet (question4)

Question 7 (three assignments)

Mark Sheet

Mark Sheet Interpretation(Tree Structure)

Mark Sheet Interpretation (Mode Structure)

Data sample


Barnet CollegeFirst Experiment

2005

Data provided by: Saeed DehnadiMiddlesex University

Experiment

• Subjects:– 30 students from Middlesex University

Introductory to programming (Java)– 31 students from Barnet College Further

Education Programming Course

• Time:– Week 0 of the course

• Questions– 12 questions

CategoriesWeek 0

• Consistent (45%)

• Inconsistent (39%)

• Blank (16%)

CategoriesWeek 3

• Consistent (67%)

• Inconsistent (33%)

• Blank (0%)

Stable group membership(Week3)

Data provided by: Saeed Dehnadi


Pass/Fail statistics(Initial Result)

P < = 0.01



Two populations(Initial Result)

Australian Data

University of Newcastle2006

Data provided by: SimonSchool of DCIT (Design, Communication, and

Information Technology)

Pass/Fail statistics(First glance)

P < = 0.01

Data provided by: Simon

The University of Newcastle

Pass/Fail statistics Exclude m2

P < = 0.025



Pass/Fail statistics Exclude m2

P < = 0.001



Strathclyde Data

The University of Strathclyde

2006

Data provided by: Linxiao Mu

Mental models in week 20



Value assignment

Reference assignment

Value assignment mental models in final exam

P < = 0.05



Value assignment mental models in 4 in-course tests

Data provided by: Linxiao Mu The University of Strathclyde

P < = 0.01

Reference assignment mental models in final exam

P < = 0.025



Reference assignment mental models in 4 in-course tests

Data provided by: Linxiao Mu The University of Strathclyde

P < = 0.05

YorkThe University of York

2006

Data provided by: Dimitar Kazakov

Pass/Fail statistics

P < = 0.001

Data provided by: Dimitar Kazakov

The University of York

Middlesex UniversitySecond Experiment

2006

Data provided by: Saeed Dehnadi Middlesex University

Pass/Fail (Quiz 1)

P < = 0.20



Pass/Fail (Quiz 2)

P < = 0.01



Collaborators

• Ireland - School of Computing in DCU• Australia - Charles Stuart University• Mozambique - Instituto Superior de Transportes e

Comunicações, Maputo• UK- Bournemouth University (Royal School of Signals in

Blandford) • UK – University of Birmingham • UK – University of Sheffield• UK – King’s College, London• Canada - University of Toronto• Denmark – University of Aarhus • USA – A-level in Computing, High school• UK - A-level in Computing, North West College• Germany – Computing study, teacher training Institution

Summary

• OriginalityWe are the first to have an aptitude test that works.

• ObjectivityWe have objective test materials that can be used by others.

• SpeculationStudents who can handle meaningless rules learn programming more easily.

To be continue …..

Programming Aptitude Testing Materials, Data analysis Saeed Dehnadi Middlesex University.

Documents

Transcript of Programming Aptitude Testing Materials, Data analysis Saeed Dehnadi Middlesex University.