Learning Analytics@UC3M Carlos Delgado Kloos, Abelardo Pardo (U. Sydney),

Pedro J. Muñoz-Merino, Israel Gutiérrez, Derick Leony

Universidad Carlos III de Madrid www.it.uc3m.es/cdk

www.emadridnet.org

Education?

  Just knowledge transfer?

  One way process?

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Teaching

K K’>K

Feedback!

  Feedback is needed to   Control that learning occurs   Take adequate measures if not

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

K K’

K’>K?

Assessment

  Simplest form of feedback   Formative: to support learning   Summative: also grading function

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Teaching

Assessment

Analytics

  Another form of feedback   Recollection of educational information   Can be done especially well in the digital world

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Teaching

Analytics

5 Developments

  1. Capturing events (CCOLAB)

  2. Visualization (LearnGlass)

  3. Abstracting out data (Genghis)

  4. Just-in-Time teaching (ClassON)

  5. Gamification (ISCARE)

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

1. Capturing Events

  CCOLAB system   Capture raw events generated   Use virtual machine

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Capture of data

Data

Virtualizing the environment

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Virtual Machine

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Events captured

  Power up and shutdown of virtual machine

  Commands used in the command line interface (bash)

  Execution of the compiler and errors obtained (gcc)

  Execution of the debugger and commands used (gdb)

  Execution of the editor and IDE (kate & kdevelop resp.)

  Execution and outcome of the memory profiler (valgrind)

  Pages visited with the browser (Firefox)

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Experiment

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

References

  A. Pardo, C. Delgado Kloos: “CCOLAB UC3M: Events recorded in an undergraduate collaborative C Programming Course”. In: DataTel 2010

  V.A. Romero, A. Pardo, D. Burgos, C. Delgado Kloos: “Monitoring Student Progress Using Virtual Appliances: A Case Study”. Computers & Education 58: 4, May 2012, 1058–1067

  Lead researcher: Abelardo Pardo

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

2. Visualization

  LearnGlass: Platform to facilitate the creation of visualizations of learning events

  Implements generic requirements:

  User and permission management   Data filters   Visualization dashboard

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Visualization

LearnGlass Dashboard

Berlin, Germany EDUCON, 2013-03-12--15

14

3 variations of the same visualization

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Modular Architecture

  LearnGlass can be extended through the installation of modules

  Each module provides:   Visualization interface   An optional light visualization for the dashboard

  There are currently two modules available:   Activity report   Learners with most and least events

Berlin, Germany EDUCON, 2013-03-12--15

15

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

View of a module interface

Berlin, Germany EDUCON, 2013-03-12--15

16

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

References

  D. Leony, A. Pardo, L. de la Fuente Valentín, D. Sánchez de Castro, C. Delgado Kloos: “GLASS: A Learning Analytics Visualization Tool”. In: International Conference on Learning Analytics and Knowledge. ACM New York, USA, 2012, 162-163

  Lead researchers: Abelardo Pardo, Derick Leony

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

3. Abstracting Out Data

  Genghis project at UC3M

  Khan Academy platform deployed locally Aug 2012

  Flipping the classroom in remedial course in Physics

  Analyse online student behaviour

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Capture of data

Visualization

online

Low-level analytics provided

  Progress summary

  Daily activity report

  Skill progress over time

  Activity by day

  Badges earned

  Activity on exercises

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Higher-level analytics inferred

  Time distribution

  Correct progress on the platform

  Efficiency in learning

  Gamification habits (influence of badges)

  Exercise solving habits (hint abuse, hint avoidance, …)

  Activity habits (explorer or recommendation follower)

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Example: Efficiency in learning

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

References

  P.J. Muñoz-Merino, J.A. Ruipérez, C. Delgado Kloos: “Inferring Higher Level Learning Information from Low Level Data for the Khan Academy Platform”. In: LAK2013: 3rd Conf. Learning Analytics and Knowledge, 8-12 Apr. 2013, Leuven (Belgium)

  Lead researcher: Pedro Muñoz-Merino

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

4. Just-in-Time Teaching

  ClassON system (in-CLass Analytics for aSSessment and OrchestratioN)

  Use of learning analytics in face-to-face sessions   awareness of students activity   information to orchestrate in a better way

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Capture of data

Visualization

face-to-face

Student component

  Web-based

  Integrated in the problem assignment

  Monitor student interactions   progression   questions   timing

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Teacher component

  Web app for tablets or smartphones

  Information attached to the physical space

  Teacher is aware of student’s   location   name and photo   Progression   questions   waiting times

  And (s)he’s more informed to make decisions!

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Teacher component interface

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

25

32

BUSY

References

  I. Gutiérrez Rojas, R.M. Crespo, C. Delgado Kloos: “Orchestration and feedback in lab sessions: improvements in quick feedback provision”. In: EC-TEL 2011: 6th European Conf. of Technology Enhanced Learning, 20-23 Sep. 2011, Palermo (Italy), Proc. LNCS 6964. Springer 2011

  Lead researcher: Israel Gutiérrez

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

5. Gamification

  ISCARE system

  Analytics for motivation and joy: Adaptation of the Swiss-system of competition (used eg. in chess) to education

  Motivating and fair system:   Different rounds. No elimination   For each round, pairings of 2 people that compete   Assign each person an opponent with close scoring

  Active learning: solving exercises

  Evaluation

  Interoperability: IMS QTI

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

ISCARE System

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Analytics in ISCARE

  Evolution of scores for each student in each round

  The difficulty of the opponents the student competed against

  Exercises assigned to each student and if (s)he answered them correctly or not

  Information about each exercise: number of times it was presented or solved correctly

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Analytics in ISCARE

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

References

  M. Fernández Molina, P.J. Muñoz-Merino, M. Muñoz-Organero, C. Delgado Kloos: “Educational Justifications for the Design of the ISCARE Computer Based Competition Assessment Tool”. In: Internat. Conf. Web Based Learning, 2011, 289-294

  P.J. Muñoz-Merino, M. Fernández Molina, M. Muñoz-Organero, C. Delgado Kloos: “An Adaptive and Innovative Question-driven Competition-based Intelligent Tutoring System for Learning”. Expert Systems with Applications, 39(8), 2012, 6932-6948

  Lead researcher: Pedro Muñoz-Merino

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15

Conclusion

  In all important processes, the behaviour is monitored (Quality control)

  Assessment is not enough

  Prerequisite for scaling: MOOCs

  Definitely helps in improving education

  Privacy concerns have to be taken care of

cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15