Post on 01-Nov-2014
description
Learning from Online ErroneousExamples
Dimitra Tsovaltzi, Erica Melis, Bruce McLaren,
Ann-Kristin Meyer, Michael Dietrich, Goerge Goguadze
DFKI- University of Saarland - GermanyDimitra.Tsovaltzi@dfki.de
Summary
• Research background
• Research questions
• Studies
• Summary of Results
• Discussion of results
• Conclusion
Research Background
• Erroneous Examples (EE): worked out solutions with errors
• Novel learning opportunities, reflection, inquiry (Borasi ’95, Müller ’03, Oser&Hascher ’97)
• Benefit of self-explaining correct and incorrect solutions (Siegler ’02, Siegler&Chen ’08)
• Evidence for erroneous examples with feedback (Kopp et al ’08)
Research Background
BUT
• May be more beneficial to students with favourable prior knowledge (Grosse&Renkl ’07)
• „why“ self-explanations indispensible (Siegler ’02,
Grosse&Renkl ’08), but to the detriment of principle-based explanations (Grosse&Renkl ’08)
Help for self-explanation of errors and principle-based explanations
Adaptation to counterbalance prior knowledge differences
Summary
• Research background
• Research questions
• Studies
• Summary of Results
• Discussion of results
• Conclusion
Research Questions
Online Learning of Fractions with Erroneous Examples and Self-explanation
When?• Do advanced students, gain more from online erroneous examples?
How?• Can online EE improve:
cognitive skills?conceptual understanding?transfer abilities improve?
• Can online EE improve error detection and error correction? • Does adaptive help play a role?
Research Questions
• Hypothesis 1: Presenting erroneous examples with help to students will lead to deeper, more conceptual learning and better error-detection (i.e., metacogntive) skills, which will help improve their cognitive skills and will promote transfer.
control group (problem solving) vs. erroneous examples and erroneous examples with help
• Hypothesis 2: The learning effect of erroneous examples is conditional on whether students are supported in finding and correcting the error with additional help.
Help vs. no help
Summary
• Research background
• Research questions
• Studies
• Summary of Results
• Discussion of results
• Conclusion
• Similar design: – NOEE (control): standard exercise (standard feedback,
correct answer)
– EEWOH: standard exercise and erroneous examples without extra help (standard feedback, correct answer)
– EEWH: standard exercise and erroneous examples with extra help
• Different levels: 6th vs. 7th-8th vs. 9th-10th
• Presentation of erroneous examples
Studies
• Design – Familiarisation
– Pre-questionnaire
– Pretest
– Intervention
– Posttest
– Post-questionnaire
• Participants– paid volunteers in lab studies - German 6th grade
– EEWH (8), EEWOH (7), NOEE(8)
Study 1: Method
• Familiarisation
• Intervention– Standard exercises
– Erroneous examples
• Sequences (6):
– NOEE: SE – SE - SE
– EEWH: SE – EE – SE help
– EEWOH: SE – EE – SE no help
Study 1: Materials
Please write all individual thinking steps as if youwere thinking aloud. Add more steps whenever youneed to.
Add steps
Results
• Familiarisation
• Intervention– Standard exercises
– Erroneous examples
• Sequences (6):
– NOEE: SE – SE - SE
– EEWH: SE – EE – SE help
– EEWOH: SE – EE – SE no help
Study 1: Materials
Feedback consists of:
• Correct(√) / incorrect(X) (All conditions)
• error-awareness (EEWH): “The result cannot be smaller than one“, if two fractions larger than one are added
• self-explanation (EEWH): “Why is this step wrong?”, “How should Paul add?”
• error-correction (EEWH):“How does one add fraction with like denominators”
• error-specific (EEWH): “You did not expand the fractions”
• Worked-out correct solution (All conditions)
Feedback in ActiveMath
• Pretest and posttest
– Similar exercises plus erroneous examples with conceptual questions: “What did Paul not understand?”
• Questionnaires
motivation, self-efficacy, learning orientation,
cognitive load, error-awareness, critical thinking
Study 1: Materials
Study 1: Results
Condition EEWH N=8 EEWOH N=7 NOEE N=8
Score Subscore mean(sd)% mean(sd)% mean(sd)%
Cognitive
Skills
Pretest 80.2(26.7) 85.7(17.8) 86.5(12.5)
Post-pre-diff -2.1(33.6) 1.2(21.7)^ 2.1(23.9)+
Metacognitive
Skills (EE)
EE-find 91.7(15.4)+ 76.2(31.7)^ 66.5(35.6)
EE-correct 80.2(12.5)+ 75.0(21.0)^ 68.7(25.9)
EE-ConQuest* 64.6(25.5)+ 60.2(33.3)^ 41.7(21.2)
EE-total 75.3(16.8)+ 67.9(27.5)^ 54.7(23.0)
Total-time-on-postEE 16.9(6.2)^ 13.8(5.5)+ 18.0(5.1)
Transfer Transfer 75.0(46.2)+ 71.4(48.8) 75.0(46.3)^
• Metacognitive for finding error: EEWH>NOEE (t(20)=2.37, p<.05 , d=1.06)
• Metacognitive for total EE:
– Main (t(20)=2.34, p<.05 , d=1.01)
– EEWH>NOEE (t(20)=2.96, p<.05 , d=1.32)
• Conceptual questions: Main (t(20)=2.48, p<.05 , d=1.11)
• EEWH lower cog load (F(2,13)=7.76, p=.006, n2=0.54)
• Design as in Study 1, but one modeling exercise 7 sequences
• Participants
– paid volunteers in lab studies – German 7th and 8th
grades
– EEWH (8), EEWOH (8), NOEE (8)
Study 2: Method
Study 2:Materials
“2 groups of students get a pizza each. In thefirst group there are 3 students, 2 of whomare girls. In the second group there are 5students, 4 of which are girls. The pizza issplit equally within every group. Karl is tryingto calculate what part of the pizza the girls ofboth groups got together. His result is ¾ of apizza. Karl has made an error. Find the errorin Karl’s calculations. Choose the firsterroneous step.”
Study 2: Results
Condition EEWH N=8 EEWOH N=7 NOEE N=8
Score Subscore mean(sd)% mean(sd)% mean(sd)%
Cognitive
Skills
Pretest 73.7(26.7) 71.2(19.7) 77.9(12.4)
Post-pre-diff 2.4(24.4)^ -4.3(26.6) 6.9 (17.9)+
Metacognitive
Skills (EE)
EE-find 68.7(34.7) 75.0(13.4)^ 90.6(12.9)+
EE-correct 57.8(26.7)^ 54.7(21.1) 65.6(20.8)+
EE-ConQuest* 55.2(46.5) 62.5(12.6)+ 61.5(19.4)^
EE-total 59.3(37.1) 63.7(11.9)^ 69.8(15.0)+
Total-time-on-postEE 8.1(4.3)+ 11.5(4.2)^ 15.5(4.8)
Transfer Transfer 45.2(45.8)^ 38.0(36.0) 67.3(28.5)+
Conc. Underst. Modelling 36.4(42.2)^ 19.8(35.0) 40.8(48.6)+
• Term-grade sig. covariate for conc. questions (F(1,21)=4.49, p=.047, n2 =.18)
• More students could find the error than correct (t(23)=4.89, p<.05 , d=0.59):– EEWH (t(7)=2.19, p>.05 , d=1.64)– EEWOH (t(7)=4.83, p<.05 , d=1.15)– NOEE (t(7)=4.32, p<.05 , d=1.44)
• NOEE more cog. load drop than EEWOH (t(13)=2.52, p<.05, d=1.9)
• Design as in Study 1 and 2, but – conceptual sequences: “addition as increasing”, “part
of whole”
– Transformation exercises: 3/5+1/4
– More elaborated “how” questions
– Classroom for ecological validity
– Order of sequence: SE – SE- EE
• Participants– German school kinds in 9th and 10th grades
– EEWH (18), EEWOH (20), NOEE(19)
Study 3: Method
Study 3: Materials
Error Detection Phase
Students find the error
Error-awareness feedback“The result, walking distance=5 1/30, cannot be correct. Travel with the bus is already 4/5 of the total distance, so the walking distance
must be less than 1/5”
Step #: walking distance=…path
Study 3: Results - Cognitive Skills
Condition EEWH N=18 EEWOH N=20 NOEE N=19
Type of score Type of Subscore mean(sd)% mean(sd)% mean(sd)%
Cognitive
Skills
Pretest 74.5(14.2) 66.4(21.1) 64.9(17.2)
Diff-post-pre-total 8.9(12.8)+ 1.4(23.5) 4.9(18.8)^
Transform 16.2(23.0)+ 4.9(33.2)^ -10.2(45.4)
[A1]E.i. postEE[A2]In results file was named Add-subtr-total
•Cog. Skills: EEWH vs. EEWOH (t(30)=2.13, p<.05 , d=0.58)
•Transform: •main (t(30)=2.42, p<.05 , d=0.66)
•EEWH vs NOEE: (t(23)=2.87, p<.05 , d=0.97)
•More cog load reduction:•EEWH vs. NOEE (t(30)=2.22, p<.05, d=0.24)
•EEWH vs. EEWOH (t(28)=2.05, p=.05, d=0.14)
Study 3: Results - Metacognitive Skills
Condition EEWH N=18EEWOH
N=20NOEE N=19
Type of score Type of Subscore mean(sd)% mean(sd)% mean(sd)%
Metacognitive
Skills (EE)
EE-find 61.1(28.7)+ 50.0(28.1) 60.5(28.0)^
EE-correct 40.3(28.0)+ 21.3(30.6) 30.3(33.9)^
EE-ConQuest* 50.9(20.7)+ 50.4(24.9)^ 47.8(25.1)
EE-total 50.8(22.1)+ 44.5(24.0) 46.8(24.7)^
Total-time-on-EE 5.9(3.2)+ 4.1(3.1) 5.9(3.9)+
• Significantly less students found the error than could correct it t(56)=, p<.001 , d=0.87
• Also within individual conditions• EEWH t(20)=3.83, p<.05 , d=0.66
• EEWOH t(19)=5.88, p<.001 , d=0.98
• NOEE t(18)=5.75, p<.001, d=0.97
Study 3: Results - Transfer
Condition EEWH N=18EEWOH
N=20NOEE N=19
Type of score Type of Subscore mean(sd)% mean(sd)% mean(sd)%
Transfer
Cog-transf-total 32.0(30.1)+ 20.0(34.3) 29.0(34.6)^
Conc-transf-total* 46.8(34.7)+ 30.4(29.3)^ 29.5(30.30)
Transfer-total 39.4(20.3)+ 24.3(26.8) 26.5(28.6)^
[A1]E.i. postEE[A2]In results file was named Add-subtr-total
• EEWH better, but• No significant results
Study 3: Results – ConceptualKnowledge
Condition EEWH N=18 EEWOH N=20 NOEE N=19
Type of score Type of Subscore mean(sd)% mean(sd)% mean(sd)%
Conceptual
Understanding
Part-of-whole 11.1(47.3)+ -5.0(59.4)^ -9.9(44.6)
Addition-as-incr 65.3(44.7)+ 56.3(48.6)^ 30.5(46.4)
Subtr-as-decreas 52.9(49.9)+ 27.5(44.4) 34.2(47.3)^
Rel-part-of 22.2(42.8)^ 7.5(24.5) 23.7(42.1)+
Modelling-total 54.5(30.5)+ 33.1(24.6) 35.6(27.4)^
• Significant results include: Modelling in generalo EEWH vs EEWOH (t(30)=2.10, p<.05 , d=0.58)
Modelling “addition as increasing” o Main (t(54)=2.32, p<.05 , d=0.63)
o EEWH vs NOEE (t(23)=2.35, p<.05 , d=0.64)
• Problem with “part of a whole”
Study 3: Results – ConceptualKnowledge
Study 3: Results – ConceptualKnowledge
Condition EEWH N=18 EEWOH N=20 NOEE N=19
Type of score Type of Subscore mean(sd)% mean(sd)% mean(sd)%
Conceptual
Understanding
Part-of-whole 11.1(47.3)+ -5.0(59.4)^ -9.9(44.6)
Addition-as-incr 65.3(44.7)+ 56.3(48.6)^ 30.5(46.4)
Subtr-as-decreas 52.9(49.9)+ 27.5(44.4) 34.2(47.3)^
Rel-part-of 22.2(42.8)^ 7.5(24.5) 23.7(42.1)+
Modelling-total 54.5(30.5)+ 33.1(24.6) 35.6(27.4)^
• Significant results include: Modelling in generalo EEWH vs EEWOH (t(30)=2.10, p<.05 , d=0.58)
Modelling “addition as increasing” o Main (t(54)=2.32, p<.05 , d=0.63)
o EEWH vs NOEE (t(23)=2.35, p<.05 , d=0.64)
• Problem with “part of a whole”
Summary
• Research background
• Research questions
• Studies
• Summary of Results
• Discussion of results
• Conclusion
Cognitive Skills
Problem Solving Ceiling effect Ceiling effectEEWH >EEWOH
TransformationMain effect
EEWH >NOEE EEWH >EEWOH
MetacognitiveSkills
Finding error EEWH >NOEE
Correcting error
total-EEMain effect
EEWH>NOEE
Finding vs. correctingAcross conditions
NOEE>EEWOHAcross conditions
NOEE, EEWOH>EEWH
Conceptual
UnderstandingMain
EEWH >NOEE Term grade covariate
Modeling [not done] EEWH >EEWOH
Model. “addition as increasing”
[not done] [not done]Main effect
EEWH >NOEE
Cognitive Skills
Problem Solving Ceiling effect Ceiling effectEEWH >EEWOH
TransformationMain effect
EEWH >NOEE EEWH >EEWOH
MetacognitiveSkills
Finding error EEWH >NOEE
Correcting error
total-EEMain effect
EEWH>NOEE
Finding vs. correctingAcross conditions
NOEE>EEWOHAcross conditions
NOEE, EEWOH>EEWH
Conceptual
UnderstandingMain
EEWH >NOEE Term grade covariate
Modeling [not done] EEWH >EEWOH
Model. “addition as increasing”
[not done] [not done]Main effect
EEWH >NOEE
Cognitive Skills
Problem Solving Ceiling effect Ceiling effectEEWH >EEWOH
TransformationMain effect
EEWH >NOEE EEWH >EEWOH
MetacognitiveSkills
Finding error EEWH >NOEE
Correcting error
total-EEMain effect
EEWH>NOEE
Finding vs. correctingAcross conditions
NOEE>EEWOHAcross conditions
NOEE, EEWOH>EEWH
Conceptual
UnderstandingMain
EEWH >NOEE Term grade covariate
Modeling [not done] EEWH >EEWOH
Model. “addition as increasing”
[not done] [not done]Main effect
EEWH >NOEE
Summary
• Research background
• Research questions
• Studies
• Summary of Results
• Discussion of results
• Conclusion
Hypothesis 1: cog., metacog., transfer, concept. for EEWH
• Maybe conceptual knowledge also promotes cognitive skills
• No effects for other levels but maybe due to ceiling effect, or due to less conceptual material
• No transfer, but maybe basic-concept should be made explicit
• Metacognitive skills dependent on level
• Dissociation between declarative vs. procedural knowledge (Ohlsson ‘96), but students learned
Discussion
Hypothesis 2: Help vs. no Help• Microadaptation: more effects for help• EE with help to self-explain errors take advantage of
learning opportunities (Ohlsson ‘96)• Conceptual, principled-based help is useful (van Gog et
al ‘04)• Correcting the error may not be important lower
costOther results: When• Macroadaptation: EE after practice with SE• Despite adaptive help, class level may be important
Discussion
Summary
• Research background
• Research questions
• Studies
• Summary of Results
• Discussion of results
• Conclusion
• EE can be beneficial
• Previous results on EE and WE in other domains transfer to only EE and in fractions (Siegler ’02; Siegler&Chen ’08; Grosse&Renkl‘07)
• Analogues to aptitude-treatment from Große&Renkle(07), grade-level importance
• Like Kopp et al (08), help better
Conclusion
Learning from ErroneousExamples
Dimitra Tsovaltzi, Erica Melis, Bruce McLaren, Ann-Kristin Meyer, Michael Dietrich, Goerge Goguadze
DFKI- University of Saarland - GermanyDimitra.Tsovaltzi@dfki.de
Thank you!