Post on 18-Nov-2014
description
Learning Analytics@UC3M Carlos Delgado Kloos, Abelardo Pardo (U. Sydney),
Pedro J. Muñoz-Merino, Israel Gutiérrez, Derick Leony
Universidad Carlos III de Madrid www.it.uc3m.es/cdk
www.emadridnet.org
Education?
Just knowledge transfer?
One way process?
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Teaching
K K’>K
Feedback!
Feedback is needed to Control that learning occurs Take adequate measures if not
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
K K’
K’>K?
Assessment
Simplest form of feedback Formative: to support learning Summative: also grading function
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Teaching
Assessment
Analytics
Another form of feedback Recollection of educational information Can be done especially well in the digital world
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Teaching
Analytics
5 Developments
1. Capturing events (CCOLAB)
2. Visualization (LearnGlass)
3. Abstracting out data (Genghis)
4. Just-in-Time teaching (ClassON)
5. Gamification (ISCARE)
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
1. Capturing Events
CCOLAB system Capture raw events generated Use virtual machine
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Capture of data
Data
Virtualizing the environment
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Virtual Machine
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Events captured
Power up and shutdown of virtual machine
Commands used in the command line interface (bash)
Execution of the compiler and errors obtained (gcc)
Execution of the debugger and commands used (gdb)
Execution of the editor and IDE (kate & kdevelop resp.)
Execution and outcome of the memory profiler (valgrind)
Pages visited with the browser (Firefox)
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Experiment
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
References
A. Pardo, C. Delgado Kloos: “CCOLAB UC3M: Events recorded in an undergraduate collaborative C Programming Course”. In: DataTel 2010
V.A. Romero, A. Pardo, D. Burgos, C. Delgado Kloos: “Monitoring Student Progress Using Virtual Appliances: A Case Study”. Computers & Education 58: 4, May 2012, 1058–1067
Lead researcher: Abelardo Pardo
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
2. Visualization
LearnGlass: Platform to facilitate the creation of visualizations of learning events
Implements generic requirements:
User and permission management Data filters Visualization dashboard
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Visualization
LearnGlass Dashboard
Berlin, Germany EDUCON, 2013-03-12--15
14
3 variations of the same visualization
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Modular Architecture
LearnGlass can be extended through the installation of modules
Each module provides: Visualization interface An optional light visualization for the dashboard
There are currently two modules available: Activity report Learners with most and least events
Berlin, Germany EDUCON, 2013-03-12--15
15
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
View of a module interface
Berlin, Germany EDUCON, 2013-03-12--15
16
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
References
D. Leony, A. Pardo, L. de la Fuente Valentín, D. Sánchez de Castro, C. Delgado Kloos: “GLASS: A Learning Analytics Visualization Tool”. In: International Conference on Learning Analytics and Knowledge. ACM New York, USA, 2012, 162-163
Lead researchers: Abelardo Pardo, Derick Leony
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
3. Abstracting Out Data
Genghis project at UC3M
Khan Academy platform deployed locally Aug 2012
Flipping the classroom in remedial course in Physics
Analyse online student behaviour
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Capture of data
Visualization
online
Low-level analytics provided
Progress summary
Daily activity report
Skill progress over time
Activity by day
Badges earned
Activity on exercises
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Higher-level analytics inferred
Time distribution
Correct progress on the platform
Efficiency in learning
Gamification habits (influence of badges)
Exercise solving habits (hint abuse, hint avoidance, …)
Activity habits (explorer or recommendation follower)
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Example: Efficiency in learning
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
References
P.J. Muñoz-Merino, J.A. Ruipérez, C. Delgado Kloos: “Inferring Higher Level Learning Information from Low Level Data for the Khan Academy Platform”. In: LAK2013: 3rd Conf. Learning Analytics and Knowledge, 8-12 Apr. 2013, Leuven (Belgium)
Lead researcher: Pedro Muñoz-Merino
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
4. Just-in-Time Teaching
ClassON system (in-CLass Analytics for aSSessment and OrchestratioN)
Use of learning analytics in face-to-face sessions awareness of students activity information to orchestrate in a better way
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Capture of data
Visualization
face-to-face
Student component
Web-based
Integrated in the problem assignment
Monitor student interactions progression questions timing
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Teacher component
Web app for tablets or smartphones
Information attached to the physical space
Teacher is aware of student’s location name and photo Progression questions waiting times
And (s)he’s more informed to make decisions!
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Teacher component interface
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
25
32
BUSY
References
I. Gutiérrez Rojas, R.M. Crespo, C. Delgado Kloos: “Orchestration and feedback in lab sessions: improvements in quick feedback provision”. In: EC-TEL 2011: 6th European Conf. of Technology Enhanced Learning, 20-23 Sep. 2011, Palermo (Italy), Proc. LNCS 6964. Springer 2011
Lead researcher: Israel Gutiérrez
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
5. Gamification
ISCARE system
Analytics for motivation and joy: Adaptation of the Swiss-system of competition (used eg. in chess) to education
Motivating and fair system: Different rounds. No elimination For each round, pairings of 2 people that compete Assign each person an opponent with close scoring
Active learning: solving exercises
Evaluation
Interoperability: IMS QTI
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
ISCARE System
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Analytics in ISCARE
Evolution of scores for each student in each round
The difficulty of the opponents the student competed against
Exercises assigned to each student and if (s)he answered them correctly or not
Information about each exercise: number of times it was presented or solved correctly
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Analytics in ISCARE
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
References
M. Fernández Molina, P.J. Muñoz-Merino, M. Muñoz-Organero, C. Delgado Kloos: “Educational Justifications for the Design of the ISCARE Computer Based Competition Assessment Tool”. In: Internat. Conf. Web Based Learning, 2011, 289-294
P.J. Muñoz-Merino, M. Fernández Molina, M. Muñoz-Organero, C. Delgado Kloos: “An Adaptive and Innovative Question-driven Competition-based Intelligent Tutoring System for Learning”. Expert Systems with Applications, 39(8), 2012, 6932-6948
Lead researcher: Pedro Muñoz-Merino
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15
Conclusion
In all important processes, the behaviour is monitored (Quality control)
Assessment is not enough
Prerequisite for scaling: MOOCs
Definitely helps in improving education
Privacy concerns have to be taken care of
cdk@it.uc3m.es EDUCON, Berlin, Germany, 2013-03-12--15