LearningAnalytics - People · PDF fileVariables Conditionals Help Backwards Mistakes...
Transcript of LearningAnalytics - People · PDF fileVariables Conditionals Help Backwards Mistakes...
Putting the Pieces Together• Consider an intelligent tutoring system designed to assist
the user in programming
• Possible actions (intended to be helpful)?– A1: Auto-‐complete the current programming statement– A2: Suggest several options for completing the current
statement– A3: Provide hints for completing the statement– A4: Ask if user needs help in a pop-‐up dialogue– A5: Keep observing the user (do nothing)
• What system does should depend on how much help the user needs right now
2
Putting the Pieces Together• Consider an intelligent tutoring system designed to assist
the user in programming
• Possible actions (intended to be helpful):– A1: Auto-‐complete the current programming statement– A2: Suggest several options for completing the current
statement– A3: Provide hints for completing the statement– A4: Ask if user needs help in a pop-‐up dialogue– A5: Keep observing the user (do nothing)
• What system does should depend on how much help the user needs right now
3
Putting the Pieces Together• Consider an intelligent tutoring system designed to assist
the user in programming
• Possible actions (intended to be helpful):– A1: Auto-‐complete the current programming statement– A2: Suggest several options for completing the current
statement– A3: Provide hints for completing the statement– A4: Ask if user needs help in a pop-‐up dialogue– A5: Keep observing the user (do nothing)
• What system does should depend on how much help the user needs right now
4
“Your Happiness is My Happiness”
• System acts to help user• If user is happy, system is doing the right thing• Therefore:– System makes actions to keep user happy– Utility function should reflect user’s preferences
5
Images taken from cliparts.co and clipartkid.com
“Your Happiness is My Happiness”
• System acts to help user• If user is happy, system is doing the right thing• Therefore:– System makes actions to keep user happy– Utility function should reflect user’s preferences
6
Images taken from cliparts.co and clipartkid.com
• System’s decision problem:– Which is the best action to take to make user most happy?
– With consideration to how much help the user needs right now
• EU(Action) = Pr(Help) × U(Help,Action)
• .
Utility Function with User Variables
7
Comes from marginal distributioncomputed from Bayes net
Comes from utility defined/elicited and stored in separate file
• System’s decision problem:– Which is the best action to take to make user most happy?
– With consideration to how much help the user needs right now
• EU(Action) = Pr(Help) × U(Help,Action)
• .
Utility Function with User Variables
8
Comes from marginal distributioncomputed from Bayes net
Comes from utility defined/elicited and stored in separate file
Possible Way to Compute Pr(Help)
9
Boolean Expressions Variables Conditionals Help
Backwards Mistakes
Logic Mistakes
Persistence Mistakes
We will come back to this…
Defining U(Help,Action)• For each scenario, assign real number to each action: auto-‐complete, suggest options, hint, ask, do nothing
• If Help is high,– Auto-‐complete, Suggest options, Hint are more appropriate– Note: Should also model quality of suggestion
• If Help is medium, – Ask and Hint is more appropriate
• If Help is low, – Do nothing is more appropriate
10
Reasoning Over Time
• What if model dynamics change over time?
12
Knowledge of Concepts
Knowledge of Concepts
VariousMistakes
VariousMistakes
Knowledge of Concepts
VariousMistakes
. . .
time = 1 2 . . . n
. . .
Dynamic Inference
• Model time-‐steps (snapshots of the world)– Granularity: 1 second, 5 minutes, 1 session, …
• Allow different probability distributions over states at each time step
• Define temporal causal influences– Encode how distributions change over time
13
The Big Picture
• Life as one big stochastic process:– Set of States: S– Stochastic dynamics: Pr(st|st-‐1, …, s0)
– Can be viewed as a Bayes net with one random variable per time step
14
Challenges with a Stochastic Process
• Problems:– Infinitely many variables– Infinitely large conditional probability tables
• Solutions:– Stationary process: dynamics do not change over time
–Markov assumption: current state depends only on a finite history of past states
15
Challenges with a Stochastic Process
• Problems:– Infinitely many variables– Infinitely large conditional probability tables
• Solutions:– Stationary process: dynamics do not change over time
–Markov assumption: current state depends only on a finite history of past states
16
K-‐order Markov Process
• Assumption: last k states sufficient• First-‐order Markov process– Pr(st|st-‐1,…,s0) = Pr(st|st-‐1)
• Second-‐order Markov process– Pr(st|st-‐1,…,s0) = Pr(st|st-‐1,st-‐2)
17
K-‐order Markov Process
• Assumption: last k states sufficient• First-‐order Markov process– Pr(st|st-‐1,…,s0) = Pr(st|st-‐1)
• Second-‐order Markov process– Pr(st|st-‐1,…,s0) = Pr(st|st-‐1,st-‐2)
18
K-‐order Markov Process
• Assumption: last k states sufficient• First-‐order Markov process– Pr(st|st-‐1,…,s0) = Pr(st|st-‐1)
• Second-‐order Markov process– Pr(st|st-‐1,…,s0) = Pr(st|st-‐1,st-‐2)
19
K-‐order Markov Process
• Advantage:– Can specify entire process with finitelymany time steps
• Two time steps sufficient for a first-‐order Markov process– Graph:– Dynamics: Pr(st|st-‐1)– Prior: Pr(s0)
20
Building a 2-‐Stage Dynamic Bayes Net
21
Bools Vars Conds Help
Back-‐wards
LogicPersis-‐tence
Use same CPTs as BN
Building a 2-‐Stage Dynamic Bayes Net
22
Bools Vars Conds Help
Back-‐wards
LogicPersis-‐tence
Bools Vars Conds Help
Back-‐wards
LogicPersis-‐tence
time=t – 1
time=t
Use same CPTs as BN
Use same or differentCPTs as other time step
Building a 2-‐Stage Dynamic Bayes Net
23
Bools Vars Conds Help
Back-‐wards
LogicPersis-‐tence
Bools Vars Conds Help
Back-‐wards
LogicPersis-‐tence
time=t – 1
time=t
For each new relationship,define new CPTs:Pr(St|St–1)
Defining a DBN
• Recall that a BN over variables X = {X1, X2, …, Xn} consists of:– A directed acyclic graph whose nodes are variables
– A set of CPTs Pr(Xi|Parents(Xi)) for each Xi
• Dynamic Bayes Net is an extension of BN• Focus on 2-‐stage DBN
24
DBN Definition
• A DBN over variables X consists of:– A directed acyclic graph whose nodes are variables– S is a set of hidden variables, S ⊂ X– O is a set of observable variables, O ⊂ X– Two time steps: t–1 and t–Model is first order Markov s.t.Pr(Ut|U1:t–1) = Pr(Ut|Ut–1)
25
DBN Definition (cont.)
• Numerical component:– Prior distribution, Pr(X0)– State transition function, Pr(St|St–1)– Observation function, Pr(Ot|St)
• See previous example
26
For BNs: CPTs Pr(Xi|Parents(Xi)) for each Xi
Example
27
Bools Vars Conds Help
Back-‐wards
LogicPersis-‐tence
Bools Vars Conds Help
Back-‐wards
LogicPersis-‐tence
time=t – 1
time=t
Exercise #1
• Changes in skill from t–1 to t?• Context:My son is potty training. When he has to go, he either tells us in advance (yay!), he starts but continues in the potty, or an accident happens. With feedback and practice, his ability to go pottyimproves.
• Does a temporal relationship exist? What might the CPT look like?
28
Exercise #2
• Changes in hint quality from t–1 to t?• Context:You’re tutoring a friend in Math. Rather than solving the exercises for him, you offer to give hints. Sometimes the hints aren’t so great. Depending on the quality, you may or may not give hints each time he is stuck.
• Does a temporal relationship exist? What might the CPT look like?
29
Exercise #3
• Changes in blood type from t–1 to t?• Context:People with blood type A tend to be more cooperative. We have the opportunity to observe a student’s teamwork in class twice each week. We want to infer the student’s blood type.
• Does a temporal relationship exist? What might the CPT look like?
30
Temporal Inference Tasks
• Common tasks:– Belief update: Pr(st|ot, …, o1)
– Prediction: Pr(st+k|ot, …, o1)
– Hindsight: Pr(sk|ot, …, o1) where k < t
–Most likely explanation: argmax Pr(st, …, s1|ot, … o1)
– Fixed interval smoothing:Pr(st|oT, …, o1)
31
st, …, s1
Temporal Inference Tasks
• Common tasks:– Belief update: Pr(st|ot, …, o1)
– Prediction: Pr(st+𝛿|ot, …, o1)
– Hindsight: Pr(sk|ot, …, o1) where k < t
–Most likely explanation: argmax Pr(st, …, s1|ot, … o1)
– Fixed interval smoothing:Pr(st|oT, …, o1)
32
st, …, s1
Temporal Inference Tasks
• Common tasks:– Belief update: Pr(st|ot, …, o1)
– Prediction: Pr(st+𝛿|ot, …, o1)
– Hindsight: Pr(st-‐𝜏|ot, …, o1) where k < t
–Most likely explanation: argmax Pr(st, …, s1|ot, … o1)
– Fixed interval smoothing:Pr(st|oT, …, o1)
33
st, …, s1
Temporal Inference Tasks
• Common tasks:– Belief update: Pr(st|ot, …, o1)
– Prediction: Pr(st+𝛿|ot, …, o1)
– Hindsight: Pr(st-‐𝜏|ot, …, o1) where k < t
–Most likely explanation: argmax Pr(st, …, s1|ot, … o1)
– Fixed interval smoothing:Pr(st|oT, …, o1)
34
st, …, s1
Temporal Inference Tasks
• Common tasks:– Belief update: Pr(st|ot, …, o1)
– Prediction: Pr(st+𝛿|ot, …, o1)
– Hindsight: Pr(st-‐𝜏|ot, …, o1) where k < t
–Most likely explanation: argmax Pr(st, …, s1|ot, … o1)
– Fixed interval smoothing:Pr(st|oT, …, o1)
35
st, …, s1
Inference Over Time
• Same algorithm: clique inference
• Added setup:– Enter evidence in stage t– Take marginal in stage t, keep it aside– Create new copy– Enter marginal into stage t–1– Repeat
36
Inference Over Time
37
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = t–1 t
Setup:
Mimic:
Inference Over Time
38
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 0 1
Start, time=1:
Mimic:
Observe user behaviourEnter evidence
Inference Over Time
39
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 0 1
Start, time=1:
Mimic:
Compute marginal of interestGet: Pr(S1)
Inference Over Time
40
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 0 1
We have:
Mimic:
Pr(S1)
Inference Over Time
41
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 0 1
We have: New copy:
Mimic:
O
S
O
SPr(S1)
time = t–1 t
Pr(S1)
Inference Over Time
42
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 0 1
We have: New copy:
Mimic:
O
S
O
SPr(S1)
time = t–1 t
Pr(S1)
Inference Over Time
43
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 1 2
Continue, time=2:
Mimic:
Pr(S1)
Inference Over Time
44
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 1 2
Continue, time=2:
Mimic:
Pr(S1)
Observe user behaviourEnter evidence
Inference Over Time
45
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 1 2
Continue, time=2:
Mimic:
Pr(S1)Compute marginal of interestGet: Pr(S2)
Inference Over Time
46
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 1 2
We have:
Mimic:
Pr(S1)Pr(S2)
We have: New copy:
Inference Over Time
47
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 1 2
Mimic:
Pr(S1)Pr(S2)
O
S
O
SPr(S2)
time = t–1 t
We have: New copy:
Inference Over Time
48
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 1 2
Mimic:
Pr(S1)Pr(S2)
O
S
O
SPr(S2)
time = t–1 t
Inference Over Time
49
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 2 3
Continue, time=3:
Mimic:
Pr(S2)
Inference Over Time
50
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 2 3
Continue, time=3:
Mimic:
Pr(S2)
Observe user behaviourEnter evidence
Inference Over Time
51
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 2 3
Continue, time=3:
Mimic:
Pr(S2)Compute marginal of interestGet: Pr(S3)
Inference Over Time
52
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 2 3
We have:
Mimic:
Pr(S2)Pr(S3)
We have: New copy:
Inference Over Time
53
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 2 3
Mimic:
Pr(S2)Pr(S3)
O
S
O
SPr(S3)
time = t–1 t
We have: New copy:
Inference Over Time
54
O
S
O
S
O
S
O
S . . .
O
S
O
S
time = 0 1 2 3 . . .
time = 2 3
Mimic:
Pr(S2)Pr(S3)
O
S
O
SPr(S3)
time = t–1 t
Key Ideas
• Main concepts– Reasoning over time considers evolving dynamics in the model
• Representation:– Stationarity assumption: given model, dynamics don’t change over time
–Markov assumption: current state depends only on a finite history
– DBN is an extension of BN with an additional set of temporal relationships (transition functions)