Programming Aptitude Testing Materials, Data analysis
Saeed Dehnadi
Middlesex University
I talk about:
• Test materials– Questions– Mental Models– Answer Sheet– Mark Sheet– Interpretation Algorithms
• Results– Middlesex University (initial test) - 2005– University of Newcastle (Australia) - 2006– York - 2006– Strathclyde - 2006– Middlesex University (second test) - 2006
• Collaborators• Summary
Questions 1 (single assignment)
(M1) Mental model “move value”
a = b;
• The value of b is given to a and b changes its value to zero.
a b // b 0
Ans ( a = 20 , b = 0 ) 8th Answer
(M2) Mental model“copy value”
a = b;
• The value of b is given to a and b keeps its original value.
a b // b unchanged
Correct conception of LHS RHS
Ans ( a = 20 , b = 20 ) 4th Answer
(M3) Mental model “move value”
a = b;
• The value of a is given to b and a changes its value to zero.
b a // a 0
Ans ( a = 0 , b = 10 ) 3rd Answer
(M4) Mental model“copy value”
a = b;
• The value of a is given to b and a keeps its original value.
b a // a unchanged
Ans ( a = 10 , b = 10 ) 1st Answer
Mental models (M5) (copy + add) value
a = b;
• The sum of a and b is given to a, and b keeps its original value.
a (a + b) // b unchanged
Ans ( a = 30 , b = 20 ) 2nd Answer
Mental models (M6) (move + add) value
a = b;
• The sum of a and b is given to a, and b changes its value to zero.
a (a + b) // b 0
Ans ( a = 30 , b = 0 ) 10th Answer
Mental models (M7) (copy + add) value
a = b;
• The sum of a and b is given to b, and a keeps its original value.
b (a + b) // a unchanged
Ans ( a = 10 , b = 30 ) 9th Answer
Mental models (M8) (move + add) value
a = b;
• The sum of a and b is given to b, and a changes its value to zero.
b (a + b) // a 0
Ans ( a = 0 , b = 30 ) 5th Answer
Mental models (M9) “no changes”
a = b;
• a and b keep their original values.
a unchanged // b unchanged
Ans ( a = 10 , b = 20 ) 6th Answer
Mental models (M10) Equality
a = b;
• Assignment is a simple equation, and then all equal values of a and b are acceptable.
Ans ( a = 10 , b = 10 ) and
Ans ( a = 20 , b = 20 )
both 1st and 4th answers
Mental models (M11) “Swap values”
a = b;
• a and b swap their values simultaneously.
a b a gets b’s value
b a b gets a’s value
Ans ( a = 20 , b = 10 ) 7th Answer
Models for a single assignment int a = 10; int b = 20;
a = b;
Answer Sheet (Question1)
Questions 2 (single assignment)
Questions 3 (single assignment)
Question 4 (two assignments)
Mental model“M1 + Sequence”
a = b;b = a;
Model is M1 that applies sequentially through both statements:
• L1) a b and b 0 then a = 20 and b = 0 • L2) b a and a 0 then b = 20 and a = 0
Single answer: a = 0 , b = 20
Mental model“M1 + Independent (M1 I)”
a = b;b = a;
• Model is M1 that applies independently for each individual line.
• L1) a b and b 0 • L2) b a and a 0
Multiple answers:a = 20 and b = 0a = 0 and b = 10
Mental model“M1 + Simultaneous-single (M1 Ss)”
a = b;b = a;
• M1 Ss Same as (M1 I), subjects only interested on Left-hand-side values of statements and ignores the right-hand-side values.
• L1) a b and b ignores • L2) b a and a ignores
Single answer: a = 20 , b = 10
Answer Sheet (question4)
Question 5 (two assignments)
Question 6 (two assignments)
Answer Sheet (question6)
Question 7 (three assignments)
Answer Sheet (question7)
Mark Sheet
Mark Sheet Interpretation(Tree Structure)
Mark Sheet Interpretation (Mode Structure)
Data sample
Middlesex University
Barnet CollegeFirst Experiment
2005
Data provided by: Saeed DehnadiMiddlesex University
Experiment
• Subjects:– 30 students from Middlesex University
Introductory to programming (Java)– 31 students from Barnet College Further
Education Programming Course
• Time:– Week 0 of the course
• Questions– 12 questions
CategoriesWeek 0
• Consistent (45%)
• Inconsistent (39%)
• Blank (16%)
CategoriesWeek 3
• Consistent (67%)
• Inconsistent (33%)
• Blank (0%)
Stable group membership(Week3)
Data provided by: Saeed Dehnadi
Middlesex University
Pass/Fail statistics(Initial Result)
P < = 0.01
Data provided by: Saeed Dehnadi
Middlesex University
Two populations(Initial Result)
Australian Data
University of Newcastle2006
Data provided by: SimonSchool of DCIT (Design, Communication, and
Information Technology)
Pass/Fail statistics(First glance)
P < = 0.01
Data provided by: Simon
The University of Newcastle
Pass/Fail statistics Exclude m2
P < = 0.025
Data provided by: Simon
The University of Newcastle
Pass/Fail statistics Exclude m2
P < = 0.001
Data provided by: Simon
The University of Newcastle
Strathclyde Data
The University of Strathclyde
2006
Data provided by: Linxiao Mu
Mental models in week 20
Data provided by: Linxiao Mu
The University of Strathclyde
Value assignment
Reference assignment
Value assignment mental models in final exam
P < = 0.05
Data provided by: Linxiao Mu
The University of Strathclyde
Value assignment mental models in 4 in-course tests
Data provided by: Linxiao Mu The University of Strathclyde
P < = 0.01
Reference assignment mental models in final exam
P < = 0.025
Data provided by: Linxiao Mu
The University of Strathclyde
Reference assignment mental models in 4 in-course tests
Data provided by: Linxiao Mu The University of Strathclyde
P < = 0.05
YorkThe University of York
2006
Data provided by: Dimitar Kazakov
Pass/Fail statistics
P < = 0.001
Data provided by: Dimitar Kazakov
The University of York
Middlesex UniversitySecond Experiment
2006
Data provided by: Saeed Dehnadi Middlesex University
Pass/Fail (Quiz 1)
P < = 0.20
Data provided by: Saeed Dehnadi
Middlesex University
Pass/Fail (Quiz 2)
P < = 0.01
Data provided by: Saeed Dehnadi
Middlesex University
Collaborators
• Ireland - School of Computing in DCU• Australia - Charles Stuart University• Mozambique - Instituto Superior de Transportes e
Comunicações, Maputo• UK- Bournemouth University (Royal School of Signals in
Blandford) • UK – University of Birmingham • UK – University of Sheffield• UK – King’s College, London• Canada - University of Toronto• Denmark – University of Aarhus • USA – A-level in Computing, High school• UK - A-level in Computing, North West College• Germany – Computing study, teacher training Institution
Summary
• OriginalityWe are the first to have an aptitude test that works.
• ObjectivityWe have objective test materials that can be used by others.
• SpeculationStudents who can handle meaningless rules learn programming more easily.
To be continue …..
Top Related