JISC RSC London Workshop - Learner analytics
-
Upload
james-ballard -
Category
Education
-
view
487 -
download
1
description
Transcript of JISC RSC London Workshop - Learner analytics
![Page 1: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/1.jpg)
Learning AnalyticsWhat? How? Why?
James Ballard
jameslballard
JamesBallard2
@jameslballard
![Page 2: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/2.jpg)
Overview
What are Learning Analytics?
Learner Engagement - a metric for learning
Preparing institutions – tools and skills
Infinite Rooms
![Page 3: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/3.jpg)
Open Discussion
What are learning analytics?Who are they for?
Activity 1 - Introduction
![Page 4: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/4.jpg)
4
Learning AnalyticsWhat are they?
![Page 5: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/5.jpg)
Not quite big dataIn 2012 we created 2,500,000,000,000,000,000 (2.5
quintillion) bytes of data every day
Annual Moodle log data 5Gb
![Page 6: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/6.jpg)
Learning AnalyticsType of Analytics Level or Object of Analysis Who benefits
Learning Analytics Course-level: social networks, conceptual development, discourse analysis, intelligent curriculum
Learner, faculty
Departmental: predictive modelling, patterns of success/failure
Learners, faculty
Academic Analytics Institutional: learner profiles, performance of academics, knowledge flow
Administrators, funders, marketing
Regional (state/provincial): comparisons between systems
Funders, administrators
National and International National governments, education authorities
Siemens and Long (2011)
![Page 7: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/7.jpg)
Common focus• Identifying learners at-risk of drop-out from
the course• Identifying momentum/crisis points
Retention• Predicting final exam success• Predicting future performance (e.g. school ->
university)Performance
• Quantitative views of activity• What are learners doingActivity• Usually linked to a bench-marking of staff
performance• Learning design patterns
Course• What types of things are learners doing• Learner engagement as a metric/proxyEngagement
![Page 8: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/8.jpg)
Small groupsList some examples of what learning providers are measuring or might want to measure.
Activity 2 - Examples
Retention Performance Activity Course Engagem
ent
![Page 9: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/9.jpg)
9
Learner EngagementA metric for learning
![Page 10: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/10.jpg)
EngagementCommon activities such as checking announcements, viewing grades and uploading assignments represent little time investment from the user and may not be useful indicators of engagement.
MacFadyen and Dawson (2012)
![Page 11: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/11.jpg)
Engagement
Engagement has emerged as an alternative view of the learner experience that can enrich the often reductionist language of performance, skills and competence.
HEA, Trowler and Trowler (2010)
![Page 12: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/12.jpg)
Engagement ProcessEngagement is the new metric that supersedes previous linear metaphors, through a developmental process of discovery, evaluation, use, and affinity.
Haven (2007)
![Page 13: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/13.jpg)
Small groupsTag previous examples within the engagement process.
Activity 2 - Examples
Involvement
• The presence of a learner within the institution including data such as physical or virtual visits
Interaction
• Provides a depth of understanding: where involvement measures touches, interaction measures actions.
Intimacy
• Helps understand sentiment and affection; the most common way to collect this type of data is through interviews or surveys.
Influence
• Determines the likelihood of the individual recommending learning to others and contributing to local culture(s).
![Page 14: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/14.jpg)
InvolvementThe presence of a learner within the institution including data such as physical or virtual visits.
Overall Activity
Locations
Time of day
![Page 15: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/15.jpg)
InvolvementThe presence of a learner within the institution including data such as physical or virtual visits.
Overall Activity
Locations
Time of day
![Page 16: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/16.jpg)
InteractionProvides a depth of understanding: where involvement measures touches, interaction measures actions.
Activity types
Action analysis
Connectivity maps
Conole (2007)
![Page 17: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/17.jpg)
InteractionProvides a depth of understanding: where involvement measures touches, interaction measures actions.
Activity types
Action analysis
Connectivity maps
![Page 18: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/18.jpg)
IntimacyHelps understand sentiment and or affection; the most common way to collect this type of data is through interviews or surveys.
Learning Power
Self-theory
Motivated Strategies for Learning Questionnaire, MSLQ
Self-determination theory
Deakin Crick, Broadfoot, and Claxton (2004)
![Page 19: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/19.jpg)
IntimacyHelps understand sentiment and or affection; the most common way to collect this type of data is through interviews or surveys.
Learning Power
Self-theory
Motivated Strategies for Learning Questionnaire, MSLQ
Self-determination theoryRehearsal Elaboration Organisation Self-Regulation Critical Thinking
0
1
2
3
4
5
6
MSLQ
Pre Post
Pintrich (1990)
![Page 20: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/20.jpg)
InfluenceDetermines the likelihood of the individual recommending learning to others and contributing to local culture(s).
Social Network Analysis
Distributed Cognition
Collective Intelligence
Pathway of ParticipationDawson (2010)
![Page 21: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/21.jpg)
InfluenceDetermines the likelihood of the individual recommending learning to others and contributing to local culture(s).
Social Network Analysis
Distributed Cognition
Collective Intelligence
Pathway of Participation
School Leader Network
Harré (1983)
![Page 22: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/22.jpg)
22
Preparing InstitutionsEmpowering environments for learning
![Page 23: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/23.jpg)
You are what you measure
Measure what you get
Metrics are based on the data that is easiest to extract/access, and what you don’t measure is lost.Get what you measure
Anything you measure will impel a person to optimize his score on that metric. Data is not neutral Don’t be surprised if people find ingenious and destructive ways in how they get there. For example, standardised assessment produce kids who perform well on these tests but can falter when asked to demonstrate their knowledge of the same material in a different way
‘Incremental change is not enough. You have to drive large-scale change by changing the environment in which people work’
– Kevin Bonnett, Deputy Vice Chancellor Student Experience
JISC Report
MMU Review
![Page 24: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/24.jpg)
Open Discussion
What types of skills are required by e-learning teams?Do they already exist?
Activity 1 - Introduction
![Page 25: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/25.jpg)
Analytics Process
Collection Storage Cleaning Integration Analysis Presentati
on
CETIS Analytics Series (2012)
![Page 26: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/26.jpg)
Open Source Tools
Data Storage
MySQL / PostgreSQL Apache Hadoop HP VerticaData Mining
Pentaho Rapid Miner Social Network Analysis Gephi Visualisation Google Visualisation d3.js InfoVis Toolkit
Investing in staff experimentation with low cost components from a range of traditions may be a more prudent initial move, even if the most effective tool subsequently turns out to be a ready-made suite.
![Page 27: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/27.jpg)
Data Mining
Algorithm Usage Purpose
Step regression Used for binary classification (0,1)• Select a
parameter• Assign a weight• Calculate value
Predicts simple binary results such as is a student at-risk?
Logistic regression Used for binary classification (0,1)
Same as above but more conservative
J48/C4.5 Decision trees (Quinlan, 1993)
Tries to find optimal split in variables
Good when data splits into groups
JRip Decision rules Find the “best” path and make this a rule until no sensible paths are left and set these to otherwise.
Good when multi-level interaction are common
K* Instance based classifiers
Predicts data based on neighbouring points.
Good when data is very divergent
Random Forest
Classification is used when one wants to predict something (label) which is categorical and not a number.
![Page 28: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/28.jpg)
28
Infinite RoomsLearner enhanced technology
![Page 29: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/29.jpg)
Research Project
ScopeWeb dashboards based on engagement process accessing a data warehouse model developed from Activity Theory. Utilises new and existing analytics and supports multiple learning design approaches.Objectives1. How can student activity help identify
and promote effective teaching practices?
2. Understand the role that analytics can play in learning design, feedback and assessment.
3. Explore how student contributions can provide dynamic indications of success.
If patterns of nonparticipation (disengagement) are to be disrupted an improved conceptual framework may be necessary.
![Page 30: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/30.jpg)
Activity AnalysisEngeström’s (1987, 1999) approach allows us to overcome oppositions between activity and communication and highlight subject-community relations.
Modelling pedagogy with Activity Theory
Stevenson (2008)
http://goo.gl/vOuiqp
![Page 31: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/31.jpg)
Exposing ActivityThe intention of this is to reveal the nature of the system, allowing designers (e.g. teachers) to evaluate the system in the wider context of their teaching and learning practice.
![Page 32: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/32.jpg)
Data Model
Dimension
Fact
Action Post to forum
Tool Forum
Instance Discussion topic
User Oliver Twist
Role Student
Course Introduction to English
Date 02/10/2013
Time 9:45
System Moodle
Enables multi-dimensional tagging to explore data from different perspectives.
![Page 33: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/33.jpg)
Data Capture
Things a learner doesActions• Submissions• Quiz attempts • Forum posts
Feedback to the learnerIntervention
s• Targets• Grades • Assignment feedback
Recognising learningAchieveme
nts• Course completions• Badges• Certificates
How learning is perceivedSurveys• Attitudes to learning/technology• Satisfaction survey
What types of things can we capture.
![Page 34: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/34.jpg)
CodingOne can then begin to distinguish the possible actions that are generated through the use of tools from the operations needed to access them and code these via learning design theories.
![Page 35: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/35.jpg)
Activity TypesApply learning design models to learner data.
Conole (2007)
![Page 36: JISC RSC London Workshop - Learner analytics](https://reader033.fdocuments.us/reader033/viewer/2022061216/54b313784a795932488b466a/html5/thumbnails/36.jpg)
VisualisationExplore different visualisations of the same data set for different insights.