Open Learning Analytics panel at Open Education Conference 2014
-
Upload
stian-haklev -
Category
Data & Analytics
-
view
940 -
download
0
description
Transcript of Open Learning Analytics panel at Open Education Conference 2014
J O S H B AR O N
S T I AN H Å K L E V
N O R M AN B I E R
H AS H TAG # O P E N L A
Open Learning Analytics Panel
Panel Session Overview
Session Goal: Stimulate discussion around the
importance of open learning analytics to the future of
the larger open education movement
Longer-Term Objective: Form connections between
the OpenEd and Open Learning Analytics networks
Session Format:
Setting the Context – Open Learning Analytics
Examples from the Real World
Discussion/Q&A
What is Learning Analytics?
Academic Analytics Learning AnalyticsA process for providing higher
education institutions with the data
necessary to support operational
and financial decision making*
The use of analytic techniques to help
target instructional, curricular, and
resources to support the achievement
of specific learning goals*
Focused on the business of the
institution
Focused on the student and their
learning behaviors
Management/executives are the
primary audience
Learners and instructors are the
primary audience
* - Source: Analytics in Higher Education: Establishing a Common Language
2014 Open Learning Analytics Summit
Society for Learning Analytics (SoLAR) began
exploring openness in learning analytics in 2011
International OLA Summit held in March 2014
Participants identified OLA “knowledge domains”
as means to organize future work
OLA Knowledge Domains
Open Data and Models
Releasing data sets and models under open licenses
Open Research
Publishing research in open-access journals
Open-Source Software/Platforms
Open software, standards and APIs
Open Strategy and Policy
Open documents on strategy and policy
Open Learning Designs
Combine OER & LA to create new models of learning
N O R M AN B I E R
D I R E C T O R , O P E N L E AR N I N G I N I T I AT I V E
C AR N E G I E M E L L O N U N I V E R S I T Y
Openness = Science
The changing value of content
Changing focus in OER community
Commoditization of content (Wiley: ‘content is
infrastructure’)
Instrumenting content is difficult and expensive
Well-instrumented content and the tools to analyze
student interactions with that content will continue to
increase in importance
Problems with Black-Box Systems
Challenges Opportunities
CC-OLI Research
MOOC Research
UC Davis
Common measures of
outcomes and
achievement
Simon DataLab
Design Analytics
Stanford Outcomes
Analytics Service
Swappable Models
Learner Centered
Examples
S T I AN H Å K L E V
I N S T I T U T I O N AL R E S E AR C H E R , O P E N U T O R O N T O
U N I V E R S I T Y O F TO R O N TO
MOOC RESEARCH AND
REPRODUCIBLE SCIENCE
Supporting three MOOC research projects
(MRI)
Tools to collaborate and document
Connecting database with other data
Clicklog: big data, making it queryable,increasing levels of abstraction
20
J O S H B AR O N
S E N I O R AC AD E M I C T E C H N O L O G Y O F F I C E R
M AR I S T C O L L E G E
Open Data Models & OS
Learning Analytics Platform
OAAI: Overview and Impact
EDUCAUSE Next
Generation Learning
Challenges (NGLC)
Funded by Bill and
Melinda Gates Foundations
$250,000 over a 15 month period
Goal: Leverage Big Data concepts to create an
open-source academic early alert system and
research “scaling factors”
Student Aptitude Data
(SATs, current GPA, etc.)
Student Demographic
Data (Age, gender, etc.)
Sakai Event Log Data
Sakai Gradebook Data
Predictive
Model
Scoring
Identifies
students
“at risk” to
not
complete
course
SIS
Dat
aLM
S D
ata
OAAI Early Alert System Overview
Intervention Deployed
“Awareness” or Online
Academic Support
Environment (OASE)
“Creating an Open Academic Early Alert System”
Model DevelopedUsing Historical Data
Step #1: Developed
model using historical
data
Academic Alert
Report (AAR)
Research Design
Deployed OAAI system to 2200 students across
four institutions
Two Community Colleges
Two Historically Black Colleges and Universities
Design > One instructor teaching 3 sections
One section was control, other 2 were treatment groups
Each instructor received an AAR three times during
the semester:
Intervals were 25%, 50% and 75% into the semester
Intervention Research Findings
Final Course Grades
Analysis showed a
statistically significant
positive impact on final
course grades
No difference between
treatment groups
Saw larger impact in
spring then fall
Similar trend amount low
income students
50
60
70
80
90
100
Awareness OASE Control
Fin
al G
rad
e (%
)
Mean Final Grade for "at Risk" Students
Intervention Research Findings
Content Mastery
Student in intervention
groups were statistically
more likely to “master the
content” then those in
controls.
Content Mastery = Grade
of C or better
Similar for low income
students.
0
200
400
600
800
1000
Yes No Yes No
Content Mastery for "at Risk" Students
Control Intervention
Freq
uen
cy
JAYAPRAKASH , S . M. , MOODY, E . W. , LAURÍA , E . J . ,
REGAN, J . R . , & BARON, J . D . (2014) . EARLY ALERT
OF ACADEMICALLY AT-RISK STUDENTS: AN OPEN
SOURCE ANALYTICS IN IT IATIVE . JOURNAL OF
LEARNING ANALYTICS, 1 (1 ) , 6 -47 .
More Research Findings…
Strategic Vision: Open
Learning Analytics PlatformCollection – Standards-based data capture from any potential source using Experience API and/or IMS Caliper/Senor API
Storage – Single repository for all learning-related data using Learning Record Store (LRS) standard.
Analysis – Flexible Learning Analytics Processor (LAP) that can handle data mining, data processing (ETL), predictive model scoring and reporting.
Communication –Dashboard technology for displaying LAP output.
Action – LAP output can be fed into other systems to trigger alerts, etc.
O AA I P R E D I C T I V E M O D E L D O W N L O AD
H T T P S : / / C O N F L U E N C E . S AK AI P R O J E C T. O R G / X / 8 AW C B
AP E R E O L E AR N I N G AN ALY T I C S P R O C E S S O R D O W N L O AD
H T T P S : / / C O N F L U E N C E . S AK AI P R O J E C T. O R G / X / K W C V B Q
Access to Predictive Model and
related OS Software…
Discussion and Q&A
Discussion Questions
Do you feel LA will be important to OER and Open
Education in the future? How important?
Where do you see connections between the OLA
network and Open Education?
How might we best facilitate making connections
across different “networks”?
[insert more questions]
Additional Resources
European OLA Summit – December 1st (LACE)
http://www.laceproject.eu/
The Asilomar Convention for Learning Research in
Higher Education
http://asilomar-highered.info
Apereo Learning Analytics Initiative
https://confluence.sakaiproject.org/x/rIB_BQ
Society for Learning Analytics and Research
http://solaresearch.org