Art Graesser

Professor, Psychology & Institute for Intelligent Systems, University of Memphis Honorary Research Fellow, University of Oxford

Collabortions through Dialogues and Trialogues with

Conversational Agents

Bill & Melinda Gates Foundation

IIS: Integrating Methodologies Unique interplay of Theoretical

Research (cognitive science/psychology), technology

development (computer sciences/engineering),

educational practice, and empirical evaluation

Development of GIFT: Generalized Intelligent Framework for Tutoring

UM-IIS Learning Technologies

Critical thinking (ARIES: Millis, Forsyth, Butler, Wallace,

Graesser, & Halpern, 2011)

Physics (Again!) (DeepTutor: Rus, D’Mello, Hu,

& Graesser, 2013)

Biology (Guru: Olney, D’Mello, Person, Cade, Hayes,

Williams, Lehman, & Graesser, 2012)

English literacy (Greenberg, Graesser, & Lovett, 2012)

Physics (AutoTutor: Graesser, Chipman,

Haynes, & Olney, 2005)

Computer literacy (AutoTutor: Graesser, Wiemer-Hastings,

Wiemer-Hastings, & Kreuz 1999)

When a car without headrests on the seats is struck from behind, the passengers often suffer neck injuries. Why do passengers get neck injuries in this situation?

Question Head

Simulation

Parameter Controls Describe what

happens

Learning Conceptual Physics VanLehn, Graesser, Jackson, Jordan, Olney, & Rose, 2007)

Four conditions: • Read Nothing • Read Textbook • AutoTutor • Human Tutor

0.5 0.6 0.7 0.8

Adjusted post-test scores

Multiple Choice Tests on Deep Learning (Computer Literacy and Critical Thinking)

Graesser, Lu, Jackson, Mitchell, Ventura, Olney, & Louwerse (2004)

Computer Literacy

0.4 0.5 0.6 0.7

AutoTutor

Read Textbook

Read nothing

Adjusted post-test scores

Storey, Kopp, Wiemer, Chipman, & Graesser (2009)

Critical Thinking

0.4 0.5 0.6 0.7

AutoTutor

Read Textbook

Read nothing

Adjusted post-test scores

Speech Act Hierarchy

Human-Human kappa = .80 Human-Computer kappa = .73

Functions of Conversational Agents

• Help when initiated by the user • Navigational guide • Modeling action, thought, and social

interaction • Adaptive intelligent conversational dialog • Many roles: peers, tutor, mentor

Adaptive Trialogs Dr. Quinn

Expert Glass

Fellow Student

Human Player

Vicarious

Trialogs in Learning Low Ability Vicarious learning Medium Ability Tutorial dialogue High Ability Teachable agent

Trialogs in Assessment Low Ability

Short responses to prompts Inaccurate or irrelevant Little initiative Violation of social norms High Ability Lengthier turns

Accurate contributions Takes initiative Social appropriateness

Center for the Study of Adult Literacy

Emotion Sensors and Channels

Dialogue

Face

Posture

Speech

18

AutoTutor with ALEKS Mathematics ITS

20

Definition of Collaborative Problem Solving for PISA 2015

Collaborative problem solving competency is the capacity of an individual to effectively engage in a process whereby two or more agents attempt to solve a problem by sharing the understanding and effort required to come to a solution and pooling their knowledge, skills and efforts to reach that solution.

Table 1 Matrix of Collaborative Problem Solving skills for PISA 2015

(1) Establlshlng and maintaining shared

understanding

(2) Taking appropriate action to solve the problem

(3) Establlshlng and maintaining team

organisation

(A) Exploring and Understanding

(A 1 ) Discovering perspectives and abilities of team members

(A2) Discovering the type of collaborative interaction to solve the problem, along with

goals

(A3) Understanding roles to solve problem

(B) Representing and Formulating

(B 1) Building a shared representation and

negotiating the meaning of the problem (common ground)

(82) Identifying and describing tasks to be

completed

(83) Describe roles and team organisation (communication

protocol/rules of engagement)

(C) Planning and Executing

(C1) Communicating with team members about the

actions to be/ being performed

(C2) Enacting plans (C3) Following rules of engagement, (e.g., prompting

other team members to perform their tasks.)

(D) Monitoring and Reflecting

(D1) Monitoring and repairing the shared

understanding

(D2) Monitoring results of actions and evaluating

success in solving the problem

(D3) Monitoring, providing feedback and adapting the

team organisation and roles

Note: The 12 skill cells have been labelled with a letter-number combination referring to the rows and columns for ease of cross­referencing later in the document

References VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., &

Rose, C. P. (2007). When are tutorial dialogues more effective than reading? Cognitive Science, 31, 3–62.

Graesser, A.C. (2011). Learning, thinking, and emoting with discourse technologies. American Psychologist, 66, 743-757.

Rus, V., D’Mello, S., Hu, X., & Graesser, A.C. (2013). Recent advances in intelligent systems with conversational dialogue. AI Magazine, 34, 42-54.

D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A.C. (2014). Confusion can be beneficial for learning. Learning and Instruction. 29, 153-170.

Graesser, A. C., Li, H., & Forsyth, C. (2014). Learning by communicating in natural language with conversational agents. Current Directions in Psychological Science, 23, 274-280.

.