Social and Communicative

Factors in Learning

andLightSIDE

Carolyn Penstein Rosé

Carnegie Mellon UniversityLanguage Technologies Institute and Human-Computer Interaction InstituteSchool of Computer Science

Bringing two research communities together….

• Joint work with Lauren Resnick at the Learning Research and Development Center

• Partnership with Pittsburgh Public Schools 9th grade biology• District wide teacher professional development about classroom

discussion facilitation• Student preparation for discussions using computer supported

collaborative learning• Exciting finding: Preparing students with CSCL activities prior to whole

group classroom discussion has a facilitating effect on the teacher (over 1 standard deviation effect!)

2

Computer Supported Collaborative Learning

Classroom Discourse and

Teacher Professional Development

Automated Assessment

3

http://stateimpact.npr.org/ohio/2012/06/08/the-pros-and-cons-of-using-computers-to-teach-students-how-to-write/

• LightSIDE text mining tool bench

– Http://www.cs.cmu.edu/~emayfiel/side.html

• Assessment of essay answers• Assessment of student behavior

– Estimates of self-efficacy (Howley et al., 2011)

– Estimates of effective discussion behavior (Joshi & Rosé, 2007; Rosé et al., 2008; Mu et al., 2012)

– Measures of teacher facilitation behavior

Accountable Talk (O’Connor, Michaels, & Resnick)

Eddie: Well, i don't think it matters what order the numbers are in. You still get the same answer. But three times four and four times three seem like they could be talking about different things.

Teacher: Rebecca, do you agree or disagree with what Eddie is saying?Rebecca: Well, I agree that it doesn't matter which number is first, because

they both give you twelve. But I don't get what Eddie means about them saying different things.

Teacher: Eddie, would you explain what you mean?Eddie: Well, I just think that like three times four can mean three groups of

four things, like three bags of four apples. And four times three means four bags of three apples, and those don't seem like the same thing.

Tiffany: But you still have the same number of apples, so they are the same!

Teacher: OK, so we have two different ideas here to talk about. Eddie says the order does matter, because the two orders can be used to describe different situations. So Tiffany, are you saying that three times four and four times three can't be used to describe two different situations?

VMT-Basilica

** Students learn up to 1.25 standard deviations more when interactive support is provided in the environment.

Core Research Focus

6

• Identify conversational interactions that are valuable for learning

• Automatic conversation analysis – Facilitates learning research– Automates assessment of group

processes– Enables context sensitive

triggering of support

• Interactive support technologies

• Drawn from multiple frameworks valuing sociocognitive engagement– Students explicitly display

their reasoning– Students orient their

contributions towards previous contributions

– Students increase in their ownership of their ideas

• Made precise through constructs from linguistics– Integrative book chapters (Sionti, Ai, Rosé, & Resnick, 2011; Howley, Mayfield,

& Rosé, 2012)

Operationalization of Productive Talk for Learning

7

Learning Results

• Students in accountable talk classrooms (where transactivity is elicited) learn more than students in a traditional classroom (O’Connor, Michaels, & Resnick, in preparation)

• Transactivity correlates with learning (Joshi & Rosé, 2007; Kumar et al., 2007)– Consistent with results in connection with elaborated explanations

(e.g., Webb, Nemer, Zuniga, 2002)

• Collaboration support that increases transactivity increases learning (Wang et al., 2007)

8

Historical Perspective…

• Socratic tutoring: directed lines of reasoning– Evidence that socratic tutoring is more beneficial than

didactic tutoring (Rosé et al., 2001a)

• Socratic style implemented in Knowledge Construction Dialogues (KCDs)– General attempt to model effective human tutoring

– Hierarchical structure: adaptive to student needs

– Used to elicit reflection

– First used to support individual learning in Physics • Atlas-Andes (Rosé et al., 2001)• WHY-Atlas (Rosé et al., 2003; Rosé & VanLehn, 2005)

Empirical Foundation for CycleTalk

• Human tutoring not always better than non-interactive support (VanLehn et al., 2007)– Focus shift to capturing what it is about interaction that is

effective for instruction

• Human tutors guide students towards opportunities for reflection (Rosé & Torrey, 2004)

• Human tutor support by effective tutors is significantly better than hint based support (Rosé et al., 2005)

Towards enriching the interaction…

• Student interaction with dialogue agents lacks elaboration (Rosé et al., 2003)

• Students expect to behave differently with agents (Rosé & Torrey, 2004)

• Students can benefit from working with another student, even in the absence of computer scaffolding (Gweon et al., 2006; Kumar et al., 2007)

11

LightSIDE

Labeled Texts

Unlabeled Texts

Labeled Texts

A Model that can Label More Texts

Time

Beh

avio

r

<Triggered Intervention>

Download tools at:http://www.cs.cmu.edu/~emayfiel/side.html

Monitoring Collaboration with Machine Learning Technology

Using LightSIDE

Keep this picture in mind…

• Machine learning isn’t magic• But it can be useful for

identifying meaningful patterns in your data when used properly

• Proper use requires insight into your data

?

Error Analysis

VMT-Basilica

** Students learn up to 1.25 standard deviations more when interactive support is provided in the environment.

Computer Supported Learning in Large Classroom Studies

• University Level– Freshman Engineering– Calculus– Sophomore Thermodynamics

• High School– 9th Grade Biology– Earth Sciences

• Middle School– Engineering Outreach– 6th grade math

• Sophomore Thermodynamics Course – Rankine Cycle Design Competition

• Part of a Thermodynamic Cycles Lab (1 session)• Students Work in Teams of 2 with Their Partner in Another Part of the Room• Instructed on Effects of Changing System Variables (Temperature,

Pressure) on System Output (Power, Waste Heat)• Goal: Increase Efficiency within Practical Constraints• Software Package is CyclePad (Developed at Northwestern University

(Forbes et. al. 1999) to Allow Students to Construct and Analyze a Variety of Thermodynamic Cycles)

Second-Year Thermodynamics Project

Thermo Collaborative Task

• Goal: Design a power plant based on the Rankine Cycle paradigm– Each pair turns in exactly one design

• Competing Student Goals:– Power: Design a power plant that

achieves maximum power output• Motivated by economic concerns

– Green: Design a power plant that has the minimum impact on the environment

• Motivated by environmental concerns

Increasing Conceptual Depth

• Student: so i think the best thing to go with first is to increase the pressure entering the connector

• Tutor: You might want to review what you saw earlier in the sensitivity analysis investigating the effect of changing P-max on steam quality and efficiency, from the top half of page 13 in your book.

• Tutor: type: HELP WITH CHANGEPMAX if you want to discuss it with me.

• Student: help with changepmax• Tutor: Remember the sensitivity analysis you have

completed where you manipulated P-Max and observed it's effect on the efficiency of the cycle. You can refer back to Figure 11 on page 13 of your booklet, which …

Example Interaction

3 in vivo studies in 9th grade biology Other similar studies in math, freshman engineering, thermodynamics, and chemistry

Online small group activities, support from Conversational Computer Agents

In vivo studies

Pretest Post-Activitytest

Post-Discussiontest

Small GroupActivity(ExperimentalManipulation)

Whole classDiscussion

Example Intervention: Revoicing Agent

Teacher: Accountable TalkComparing Years 1 and 2

Auto

Pre

dic

ted A

T

R = .36

R = .45

Student talkComparing Years 1 and 2

Avera

ge S

tudent

Word

s per

Turn

R = .18

R = .59

Current Directions

• Continuing to investigate social considerations for integrating dialogue agents with groups– Investigating how motivation orientation interacts with

treatment

• Working with groups larger than pairs (Gweon et al., in press)– Monitoring collaboration quality from speech– Challenges of multi-party conversation analysis

• Multiple interwoven threads (Rosé et al., 1995; Wang et al., 2008a,b)

27

28

ThankYou!!!

Funding: The Office of Naval Research and the National Science Foundation