Managing Data-Driven Learning Projects
-
Upload
aaron-silvers -
Category
Data & Analytics
-
view
635 -
download
0
Transcript of Managing Data-Driven Learning Projects
Managing Data-Driven Learning Projects
Aaron E. Silvers & Megan Torrance,
Learning Solutions 2015 Orlando, FLUSE THE TWITTERS!!!!!
@aaronesilvers @MMTorrance #LScon
“ - Robert DeNiro, RONIN (1998)
“I never walk into a place I don't know how to walk out of.”
#KeepingTheEndInMind
Data-Driven Learning Projects
What doThere are a couple of ways data, like xAPI data, opens new doors for Learning & Development projects.
look like?
Adaptive Learning
There’s a lot of promise for
adaptive learning in the
enterprise. What exactly
does “Adaptive Learning”
mean? Well…
Adaptive Learning works like the story of Goldilocks and the Three Bears
Photo by SlyOwl - http://flic.kr/p/j9fFYy
First, Goldilocks (“online
self-paced eLearning”)
can choose her porridge…
Photo by SlyOwl - http://flic.kr/p/j9fFYy
Online, Self-Paced Learning• Pre-testing preselects content
based on a pre-tests or learner’s history to “test out” of content. This porridge is too hot (arbitrary).
• Linear eLearning delivers the same content to every user. This porridge is too cold (not all that adaptive).
• Micro-adaptive adjusts the sequence of content based on a learner’s reactions in real-time.This porridge is just right.
Photo by SlyOwl - http://flic.kr/p/j9fFYy
\
Next, Goldilocks has to
choose among the Micro-
Adaptive systems…
Photo by SlyOwl - http://flic.kr/p/j9fFYy
Micro-Adaptive Systems• Preference-based systems adapt to
the style or medium preferred by each learnerThis chair is too big (it’s imaginary).
• Rule-based systems are pre-programmed with closed loops of “if… then…” logic. This chair is too small (“rules”).
• Algorithmically sequenced systems calculate the sequence of content dynamically.This chair is just right.
Photo by Viracocha - http://bit.ly/1A6fcdM
\
Next, Goldilocks has to
choose among different
types of Micro-adaptive,
algorithmically
sequenced systems…
Photo by SlyOwl - http://flic.kr/p/j9fFYy
Algorithmically-Sequenced Systems
• Test-driven systems are based on automated test sequencing. This bed is too hard (so testy).
• Memorization systems prioritizes repetitions based on data about the individual, the learning activity and the interaction between the two. This bed is too soft (narrow applications).
• Proficiency-driven systems look broadly at what someone is trying to learn, pulling from a wide range of content. This chair is just right.
Photo by BlueStarMedia - http://bit.ly/1y0IEVZ
Adaptive Learning
Indeed, lots of promise for
adaptive learning but the
way forward has some
bumps.
Challenges
You probably don’t have enough content.
Photo by Tim Ebbs - http://flic.kr/p/aiBFtv
Photo by Brian Corredor - http://flic.kr/p/dLW2aT
The LMS is a fortress for content.
IA
Nav
LLR&A Au T Re
Inv Feed Look G
Sta Weed Find M
Use Need DFor more on Content Strategy:
http://bit.ly/contentWrangling
You need a content strategy.
Photo by woodleywonderworks - http://flic.kr/p/4Lx3mh
Defining competencies
while the nature of work
changes is hard.
Photo by woodleywonderworks - http://flic.kr/p/4Lx3mh
Adaptive Learning
relies on tracking lots of
learning activity data.
This means way more
interaction design and
development when
creating content.
Learning Analytics
The Big Data approach
…i.e. looking for a needle in
haystacks without knowing
what the needle looks like.
Photo by woodleywonderworks - http://flic.kr/p/4Lx3mh
Remember all that
tracking we’re doing?
Where adaptive systems
keep the data mostly out
of view, Analytics are
meant to show you what
the data means.
WHO TAKES WHAT LEARNING PATH? WHEN? WHAT IS THEIR CAREER, AGE, ETC.? WHAT PSYCHOMETRIC TESTING INFORMATION IS AVAILABLE? ITEM ANALYSIS & PRE/POST TESTING? HOW MANY TIMES DID IT TAKE TO “PASS”? WHAT PERFORMANCE DATA DO WE HAVE? WHAT COURSES AND LEARNING PATHS DO THE BEST PERFORMERS TAKE? WHAT KINDS OF COURSES ARE PEOPLE ASKING FOR? WHAT FEEDBACK ARE WE GETTING FROM COURSE EVALUATIONS ACROSS MULTIPLE CURRICULA, TOPICS, INDUSTRY BENCHMARKING, ETC.?
There are many
questions we can ask
right now. Things we
“should” be doing
already, even if we don’t
see immediate benefits.
There’s the stuff we’re already doing…
• We’re collecting data with our LMS databases
• Our CRMs and ERPs and ESNs are capturing a lot of information about employee activity
• HRIS systems and Talent Management have uniform (kinda standardized) ways of describing people.
Making Sense of It All…
• Sentiment Analysis - What do the words people use tell us about their disposition to learn?
• Engagement Analysis - What’s the activity level with learning content?
• Cohort Analysis - Who forms what groups for what reasons?
• Keyword Analysis - How do people seek info & what do they find?
• Conversion Rate - How many people respond (i.e. comment)?
• Amplification Rate - How many times is something shared?
• Applause Rate - How many likes/favorites/bookmarks?
• Economic Value - Short/Long Term Revenue/Cost Savings?
Getting Started While Already in Motion
Matching structured and
unstructured data so you
can actually do something
with it in the real world.
How Our Data Is Structured
STRUCTURED tabular, orderly
UNSTRUCTURED not tabular not tagged
wild wild west
How Our Data Is Structured
STRUCTURED tabular, orderly
UNSTRUCTURED not tabular not tagged
wild wild west
Data-type
metatagging #LSCon
#omg1Distotsawesome
How Our Data Is Structured
STRUCTURED tabular, orderly
UNSTRUCTURED not tabular not tagged
wild wild west
LMS
It Would Be So Simple, Except…
Learning Informatics
The Little Data approach
…i.e. known needles in
small haystacks you make.
Speaking of small haystacks
Speaking of small haystacks
Speaking of small haystacks
…getting bigger
…and less structured
Activity streams are
interesting … for a while.
And data visualizations are
pretty … for a while.
Phase 2 will involve far
more complex interactions
and data requirements to
define.
The RISC - VTA PDF Annotator
app developed by Float
Mobile Learning allows an
employee to use PDF job aids
like they read/highlight/
annotate on Amazon Kindle.
In this case, there are only
four activities that matter:
highlighting, underlining or
noting a word or passage and
leaving a note for the page.
The data collected has
a known pattern to it,
so specific reports can
be created that add
distinct value back to
the organization about
how they’re used.
…and that data can represented
in different ways for different
uses. In this data visualization,
content owners and designers
can see exactly which pages in a
PDF have the most annotations,
and what kind — helpful if they
need to know where to make
edits.
Data-Driven Learning Projects
So…How do we manage that? This is where y’all ask the tough questions and we share what works :)