Post on 19-Dec-2014
description
Open Education 2011:Openness and Learning Analytics
John Rinderle @johnrinderleNorman Bier @normanbier
Open Learning Initiative
Produce and improve scientifically-based courses and course materials which enact instruction and support instructors
Provide open access to these courses and materials
Develop communities of use, research and development that enable evaluation and continuous improvement
Introduction: OutcomesShared understanding of challenges, tensions and possibilities in learning analytics, around the dimensions of:
• Potential of well-used OER in a use-driven design context• Adaptability (Variety)← → Analytics (Coherence)• Analytics Tools and Approach• Data—needs and challenges
Describe community-based analytics plans:• Flexible, long-range planning• Useful, short-term steps
Commit to action• Identify best existing efforts
Driving Feedback Loops
Infinite Points of Light
Infinite Points of Light
Infinite Points of Light
Infinite Points of Light
Infinite Proliferation
The 4 R’s
ReuseRedistributeReviseRemix
Infinite Proliferation
The 4 R’s
ReuseRedistributeReviseRemix
NOT:
Recreate
Add:Evaluate
Proliferation isn’t just OER…
Intro to CS @ CMU Statistics @ everywhere
Core Statistics
Business Statistics
Research Statistics
Medical Statistics
What drives change in these scenarios?
• Data
• Intuition
• Market demand
• Instructor preferences
The problems of variety
• Quality is highly variable
• Much duplication of effort
• Difficult to choose appropriately
• Hard to evaluate
• Impossible to improve
• Hard to scale success up
Effectiveness
is hit or miss
Effectiveness
What is working in open education? Why? And how do you know?
Effectiveness
Demonstrably support students in meeting articulated, measurable learning outcomes in a
given set of contexts
So why don't we do this now?
• It's hard
• It's expensive
• Individual faculty can't do it alone
• It can be threatening to educators
• Disparate systems
• How do we measure it? We need enabling processes and systems
Driving Feedback Loops
Great, but:What does it mean when we get out of the realm of discussion and into the realm of practice?
Learning AnalyticsWhat are they?
How do we create and use them?
What do we mean by learning analytics?
Proxies vs authentic assessment and evaluation
Analytics Definition
Data Collection Reporting Decision Making Intervention Action
Collecting the data is not enough. We also need to make sense of if in ways that are actionable.
Types of analytics
• Educational/Academic Management analytics
• Classroom Management analytics
• Learning Outcomes analytics
The problem of data collection
1. Agreed upon standards
2. Core collection
3. Space for exploration
The problem of data collection
1. Agreed upon standards
2. Core collection
3. Space for exploration
• Ownership
• Privacy
• Policy
Ideal world
•Common data standards
•Analytics-enabled OER
•Commonly accepted ownership and privacy approaches
•Commitment to measuring effectiveness through assessment
Bring Together What Already Works
1) Data Collection Systems Data Schemas
2) Communities of Evidence
3) Analysis Tools
Learning Dashboard
DataShop
Evidence Hub
Learning Registry
Communities of Evidence
And build new things
1) Data Collection Systems Data Schemas
2) Communities of Evidence
3) Analysis Tools
Driven by different types of data
Raw
Interaction
Behavioral
Contextual
Paradata
Semantic
Metadata
Synthetic Data
Share Alike and Share Data
Community Based Approach
A middle ground?
Infinite Variety
The OneTrue Course
Communities Coalesce
Can we put these together?
"Full spectrum" analytics to drive different types of decision making, address different feedback loops
Learning Intelligence Systems
What would we be giving up?
This approach forces us to allow our minds to be changed by evidence.
Conclusion: next steps
• Innovate• Standardize• Scale
Conclusion: next steps
• Innovate• Standardize• Scale
• Commitment to Assessment and Evaluation
• Community Definition of Analytics-enabled OER
• Common approach to data
• Shared and private analytics platforms
“Improvement in Post Secondary Education will require converting teaching from a ‘solo sport’ to a community based research activity.”
—Herbert Simon
Questions
• Do you believe in this approach to analytics-enabled OER?
• Can this better address the pedagogy vs. reuse value curve?
A Virtuous Cycle
Data
Theory
Educational
Technology &
Practice