June 2016The key role of relational, student & other data sources in developing Learning Analytics

Digital & Data – The power of Learning Analytics

Jisc UK Learning Analytics

The Business Case: Improve student retention, achievement & employability

Method: The application of big data techniques such as machine based learning and

data mining to help learners and institutions meet their goals.

Research & Development: Statistical analysis of historical and current data derived from

the learning process to create models that allow for predictions that can be used to

improve learning outcomes. Models are developed by “mining” large amounts of data to

find hidden patterns that correlate to specific outcomes.

Key Outputs: Secure & rich multi-tenanted data warehouse, provision of early alerts to

members of staff - enabling early interventions to students, and further BI from data;

Enabling Services: Intuitive bilingual software tools and apps to automate, facilitate &

engage both student and staff to realise LA benefits;

About the

student

(UDD)

Activity data

(xAPI)

Current Progress – June 2016• Discovery readiness reviews completed/ ongoing

• Deployment of our LA Data Access Agreement (DAA)

• Review latest student data specification (UDD v1.2.4) -

https://github.com/jiscdev/analytics-udd

• Ongoing Technical Trials: set-up Learner Records Warehouse(s), install VLE

plugin(s), perform ETL for UDD student data output

• Predictive Model development – service pilots start Q4 2016

• BlackBoard Learn – activity data building block development/ evaluation

• Student App – Beta v1.0 due for release June/ July 2016 (iOS/ Android)

• Student App evaluations – currently being formulated for 3 HEIs

Progress in numbers…• Expressions of interest: 85

• Engaged in activity: 24

• Discovery to Sept 16: agreed (20), completed (11), reported (6)

• Over 4 million activity (engagement) records collected in real-time

• Moodle Historic Data Transformation: 41 million records

transformed from Moodle log files to xAPI

• Blackboard Data Transformation: 12 million blackboard records

transformed from Blackboard log files to xAPI

UDD – Unified Student Data Definition (v1.2.4)

• Student data will be sourced to enable all LA services – historical & live data collections

• Personal and Professional (academic progression) data, per individual student (HE & FE)

• Peace of mind through all-encompassing Jisc DAA

• Structure similar to HESA - a near-subset of the student return (ER, fields, mappings)

• HEDIIP/ HESA Data Futures – align to evolving structures for future data landscape;

• Interventions need to work properly – therefore be timely

• Correct (quality) data needs to be captured as frequently as possible

• Systems & business processes + data-input efficiency required

Let’s talk about data…• Historical data (12-18 months minimum) for a Learning Analytics implementation

• Predictive Model Creation (12-36months – ideal!) & Pilot

• (Portable) Predictive Model Validation (12months minimum) & Pilot

• Engagement & activity (xAPI) data – VLE (Bb, Moodle), LMS, AMS and more…

• Moodle & Bb xAPI live plug-ins for activity, Moodle (log extract) & Bb (Plugin) for xAPI historical

• No (or not good enough!) historical data?

› Confirm/ refine retention and data recycling measures

› Guidance on data quality, reduce gaps etc.

› Use other predictive models derived from the project – validate now, create later

› Evaluate, influence and co-design other LA service offerings – SSP, Student App, dashboards…

• Individual assignment (coursework) and examination marks required per module – with dates!

• Data transformation: in-house or using off-the-shelf software tool (Kettle, Alteryx…)

A Granular View of Data Sources

Student Insight Overview

Data collection and

Data mining

Create models

Predict and understand

Patterns

RelationshipsTrends

Behaviours

Collect

SIS integration

Student activity data

1

Identify

Outcome influences

Risk prediction

Student tagging

2 Awareness

Monitor cohorts and tags

Student engagement

Understand risk influences

3 Act

Proactively manage progress

Record decisions and actions

Manage student interventions

4

Improve

Assess intervention effectiveness Student feedback

5

Student Insight – 5 Stage Process

Press Coverage – Times Higher (Feb 2016)

Press Coverage – The Independent (June 2016)

Press Coverage – Scottish Herald (May 2016)

Get involved!

• Do you foresee any issues or problems with near-LIVE student data capture

or timely data extraction for Learning Analytics?

• Are there any resource implications for you, in making Learning Analytics

happen at your institution?

• Can student assessment information be easily extracted from your system(s)

per module/ student? Is the information time indexed?

• What other sources/ good indicators of student engagement or activity exist?

• Turnitin, specific attendance monitoring systems…

Demonstration

AgendaRisk prediction

Academic performance

risk prediction

Course withdrawal

risk prediction

Monitor groups

and individual

AgendaIdentify

Tag students

at risk

Monitor risk by

student cohort

See what factors affect

outcomes

AgendaOutcome analysis

See what factors affect

outcomes

AgendaUnderstand risk influences

Prediction trend

View what influenced

the prediction

Compare to student

cohorts

AgendaRecord decisions and actions

Log decisions

and actions

Maintain history of decisions

made

AgendaManage student interventions

Inform student support

team

Cases allocated

to student support

staff

Track cases through to completion

AgendaAssess intervention effectiveness

View impact of

actions made

Take into account student

feedback

Thank you

Rob Wyn Jones – Consultant – JISC Special Projects

Rob.Jones@jisc.ac.uk

Craig Petch – Tribal – Consultant, Student Insight

craig.petch@tribalgroup.com