Learning Analytics support for

Just-in-time Teaching

Raquel M. CrespoUniversidad Carlos III de Madrid (Spain)

<rcrespo@it.uc3m.es>

#lasi

#emadridnet

5 July 2013 1LASI-Madrid

Agenda

• Introduction

• Learning analytics for just-in-time teaching

• The classON system

– Functionality

– Implementation

– Experimental results

• Conclusions and future work

5 July 2013 LASI-Madrid 2

Learning Analytics

“The measurement, collection, analysis

and reporting of data about learners and

their contexts, for purposes of

understanding and optimizing learning and

the environments in which it occurs.”

SoLAR

5 July 2013 LASI-Madrid 3

Applications

• Predicting learner performance

• Learner modelling

• Recommendation of learning resources

• Enhance social learning environment

• Detect undesired learner behaviours

• Detect affects of the learner

• Increase learner awareness and reflection

5 July 2013 LASI-Madrid 4

Awareness Applications

• Awareness about student accesses tolearning resources (Mazza & Milani, 2005)

• Visualization techniques to make studentsactivities in the course and inform the self-reflection (Govaerts, Verbert, Klerkx, & Duval, 2010)

• Visualization-based awareness: GLASS (Leony, Pardo, De la Fuente Valentín, De Castro, & Delgado Kloos, 2012)

5 July 2013 LASI-Madrid 5

Mostly in

eLearning

Awareness

5 July 2013 LASI-Madrid 6

• Knowing who is around, what they

do in terms of activities and

interactions - CSCW

(Dourish & Bly, 1992)

• Lamps as distributed awareness

mechanisms in recitations

sections

(Alavi, Dillenbourg, & Kaplan, 2009)

• System that minimizes learning

progress differences in software

teaching classes

(Dong & Hwang, 2012)

Just-in-time teaching

Teaching and learning strategy (Novak, Patterson, Gavrin, & Enger, 1998)

Adaptive Hypermedia & ITS (Brusilovsky & Peylo, 2003)

Personalization,

active learning,

constructivism

5 July 2013 LASI-Madrid 7

Context

• Active learning activities

• Lab sessions using computers (f2f)

• Intensive tutoring support

• Engineering Degrees

• Overcrowded classes

photo: berkeleylab at flickr

5 July 2013 LASI-Madrid 8

The traditional approachStudents work at lab supported by the teacher’s

personalized/general explanations

• Face-to-face

• Direct interaction

• Progress monitoring?

• Questions solving?

• Particularly in overcrowded classes.

photo: vickicaruana.blogspot.com.es/2011/01/are-you-afraid-to-raise-your-hand.html

5 July 2013 LASI-Madrid 9

ProblemsCollected from literature, experts and experiments

• Teacher

– … overwhelmed with questions

– … not efficiently answering them

– … not aware of students learning situation

• Students

– … monitor teacher to get their questions answered

– … not meeting the objectives of the session

photo: vickicaruana.blogspot.com.es/2011/01/are-you-afraid-to-raise-your-hand.html

5 July 2013 LASI-Madrid 10

Requirements

• Personalized monitoring

• Teacher awareness to

decide interventions

• Improve teacher efficiency

• Make students confident

that will be helped so they

make well use of the

session time

• Formative assessment

• Storage of learning traces

for summative assessment

5 July 2013 LASI-Madrid 11

Image source:

http://vizconsult.wordpress.com/2011/03/09/requirements-definition/

classONin-Class Live Analytics for Assessment and Orchestration

• Just-in-time teaching support

• Supporting technologies

– Mobile devices

– Visualization techniques

– Learning analytics

• Web-based solution

– Student interface

– Teacher interface

5 July 2013 LASI-Madrid 12

www.class-on.org

5 July 2013 LASI-Madrid 13

classON architecture

Student UI

5 July 2013 LASI-Madrid 14

Student UI

5 July 2013 LASI-Madrid 15

Navigation

Helper

Assignment

Student UI

5 July 2013 LASI-Madrid 16

Questions

& Answers

Teacher UI

5 July 2013 LASI-Madrid 17

Teacher UI

5 July 2013 LASI-Madrid 18

Teacher UI

5 July 2013 LASI-Madrid 19

Qu

estio

n

Tutoring time

Progress

Stu

de

nts

5 July 2013 LASI-Madrid 20

classON in action

Analytics

about questions

5 July 2013 LASI-Madrid 21

Analytics about learning traces

5 July 2013 LASI-Madrid 22

Experimental results

Multimedia

Applications

Systems

architecture

Multimedia

Applications

Date 2012 2012 2013

Teachers 4 3 1

Students 70 121 30

Group settings Pairs Pairs Pairs

• Data sources– Interviews with teachers

– Students surveys

– Traces collected by the system

5 July 2013 LASI-Madrid 23

Experimental results

0%

10%

20%

30%

40%

50%

60%

70%

80%

Solved quickly

Solved but waiting

Unresolved

Help requests

classON No classON Other

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

classON No classON Other

Fairness in doubt solving

Yes No (time) No (order) No (both)

5 July 2013 LASI-Madrid 24

Lessons learned

5 July 2013 LASI-Madrid 25

Conclusions

• Positive evaluation

& valuable suggestions by

– Students

– Teachers

– TEL experts discussion groups

• Q&A by students expected to

reduce help requests

– Reduce waiting time

– Reduce unresolved doubts

5 July 2013 LASI-Madrid 26

Collected results validate that classON fulfills the expected

requirements, and thus helps teachers and students to make the

most of the learning sessions with scarce resources.

Future Work

• New metrics

– Students comparison with the mean of the class

• Gamification

– of the activities in the assignment, Q&A

• Application to other contexts

– Project-based learning

– Extension to massive face-to-face environemnts

5 July 2013 LASI-Madrid 27

Related publications

• Gutiérrez Rojas, I., & Crespo García, R. M. (2012). Towards efficient

provision of feedback in lab sessions. International Conference on

Advanced Learning Technologies (ICALT) (Vol. 1). Rome.

• Gutiérrez Rojas, I., Crespo García, R. M., & Delgado Kloos, C. (2011).

Orchestration and feedback in lab sessions: improvements in quick

feedback provision. Towards Ubiquitous Learning. 6th European

Conference of Technology Enhanced Learning, EC-TEL 2011

Palermo, Italy, September 20-23, 2011 Proceedings. LNCS, 6964.

doi:10.1007/978-3-642-23985-4

• Gutiérrez Rojas, I., Crespo García, R. M., & Delgado Kloos, C. (2012).

Enhancing orchestration of lab sessions by means of awareness

mechanisms. EC-TEL 2012. Saarbrücken.

5 July 2013 LASI-Madrid 28

5 July 2013 LASI-Madrid 29

Thanks for your

attention!

Questions?

Learning Analytics support for

Just-in-time Teaching

Raquel M. CrespoUniversidad Carlos III de Madrid (Spain)

<rcrespo@it.uc3m.es>

#lasi

#emadridnet

5 July 2013 30LASI-Madrid