Chan & Chou’s system

• Chan, T.-W., & Chou, C.-Y. (1997). Exploring the design of computer supports for reciprocal tutoring. International Journal of Artificial Intelligence and Education, 8, 1-29.

• Task domain: Designing recursive Lisp functions• Reciprocal: Yes• Communication: Weird• Expert knowledge: Yes• Evaluation: Underpowered

User interface for tutee role

• Base case vs. recursive case• Syntax handled by GUI • Steps, but no immediate feedback; must submit/ask

User interface for tutor role

• Shows correct code & tutee’s code• User must localize tutee’s bug by descending

through a “fault tree”• If user tries to descend to wrong node, its

blocked by the system• When a leaf is reach, user selects which hint

to give the tutee• Points are taken off for giving too specific a

hint

Evaluation’s conditions• 5 forms of single-user instruction– User is tutor & agent is tutee (teachable agent)– User is tutee & agent is tutor (tutoring system)

most motivating? Especially if mostly tutee early, like model scaffold fade theory.

– User is tutee & agent is tutor (2nd version of tutor)– They switch roles periodically (reciprocal tutoring)– User works without help (no agent) worst gains

• 2 forms of two-user instruction– User1 is tutor, user2 is tutee & agent guides tutor– User1 is tutor & user2 is tutee (no agent) gains

Evaluation results

• 5 students per condition under powered• Teachable agent is worst condition– User is tutor & agent is tutee– Users reported that it was very easy to walk down

the fault tree, but they didn’t learn much• Caution– Giving immediate feedback on tutoring actions

invites gaming and no learning– Did this occur with PAL?

LECOBA

• Ramirez Uresti, J.A. and B. du Boulay (2004). “Expertise, Motivation, and Teaching in Learning by Teaching Systems, International Journal of Artificial Intelligence in Education 14: 67-106.

• Task domain: Boolean Algebra• Reciprocal: user decides who will solve problem• Communication: Editing agent’s knowledge• Evaluation: Yes

Editing the agent’s knowledge

• User can change order of rules & how they are applied.

Evaluation

Motivated vs. free

Results

• Underpowered: 8 per cell• No significant differences between conditions

Findings

• The teachable agent sometimes rejected the user’s suggestions– If the agent thinks it knows a rule & the user

suggests a different one, it will reject the user– This irritated the users

• The teachable agent forgot sometimes– This surprised and irritated the users

Schwartz, Chase, Chin et al.• Pg 6 ff: Do students treat Betty as sentient & take responsibility for teaching her?– 5th graders using Gameshow– Contestant is either Betty or user

• Code attributions of K as self vs. Betty

• When given opporutnity to prepare some more, TA group did and Student group did not

How to do this study better?

• More coding of transcripts for computer talk• Tutoring an agent vs. tutoring a person – Wizard of Oz; menu based communitcation– Turing test in detail

Physiological measures e.g., pupil dialation

Does TA reflect student knowledge?

• High correlation between student answers to all possible questions and Betty’s answers.

• Potential alternative to standard tests

Does the TA make a difference in learning gains?

• Using Betty vs. using just a concept map editor pg. 13 ff

• Students in Betty’s reasoning method in that they became better at answering long inference chain questions

• On simple short chain questions, no difference• On long chain questions, Betty gets better

gradually. • Intact classes

Does SRL Betty help learning?

• 5th grades on river ecosystem for 7 class periods• SRL Betty– Mr. Davis prompts– Betty refuses to take quiz until taught enough

• Betty– Mr. Davis provided direct hints after quiz

• Intelligent Coach– Same as Betty without the cover story

Results

• During training SRL Betty > Betty > Coach• During transfer SRL Betty = Betty > Coach

What did they do differently?

• During training, SRL Betty forced students to do more debugging of their maps, so much more time on that than Betty and Coach groups

• During transfer, SRL Betty group continued to do more debugging.