Uncertainty in Expert Systems CPS 4801. Uncertainty Uncertainty is the lack of exact knowledge that...
-
Upload
myra-doyle -
Category
Documents
-
view
235 -
download
0
Transcript of Uncertainty in Expert Systems CPS 4801. Uncertainty Uncertainty is the lack of exact knowledge that...
Uncertainty• Uncertainty is the lack of exact knowledge
that would enable us to reach a fully reliable solution.oClassical logic assumes perfect
knowledge exists:IF A is trueTHEN B is true
• Describing uncertainty:o If A is true, then B is true with
probability P
Sources of uncertainty• Weak implications: Want to be able to
capture associations and correlations, not just cause and effect.
• Imprecise language: o How often is “sometimes”?o Can we quantify “often,” “sometimes,” “always?”
• Unknown data: In real problems, data is often incomplete or missing.
• Differing experts: Experts often disagree, or have different reasons for agreeing.o Solution: attach weight to each expert
Two approaches• Bayesian reasoning
o Bayesian rule (Bayes’ rule) by Thomas Bayeso Bayesian network (Bayes network)
• Certainty factors
Probability Theory
• The probability of an event is the proportion of cases in which the event occurso Numerically ranges from zero to unity (an
absolute certainty) (i.e. 0 to 1)
P(success) + P(failure) = 1
the number of possible outcomes
the number of successessuccess)P(
the number of possible outcomesfailure)P(
the number of failures
Example• Flip a coin• P(head) = ½ P(tail) = ?• P(head) = ¼ P(tail) = ?
• Throw a dice• P(getting a 6) = ?• P(not getting a 6) = ?
• P(A) = p P(¬A) = 1-p
Example• P(head) = ½ P(head head head) = ?
• Xi = result of i-th coin flip Xi = {head, tail}
• P(X1 = X2 = X3 = X4) = ?
• Until now, events are independent and mutually exclusive.
• P(X,Y) = P(X)P(Y) (P(X,Y) is joint probability.)
Conditional Probability
• Suppose events A and B are not mutually exclusive, but occur conditionally on the occurrence of the othero The probability that event A will occur if
event B occurs is called the conditional probability
occurcanBtimesofnumbertheoccurcanBandAtimesofnumberthe
BAp
probability of A given B
Conditional Probability
• The probability that both A and B occur is called the joint probability of A and B, written p(A ∩ B)
BpBAp
BAp
ApABp
ABp
occurcanBtimesofnumbertheoccurcanBandAtimesofnumberthe
BAp
Conditional Probability
• Similarly, the conditional probability that event B will occur if event A occurs can be written as:
BpBAp
BAp
ApABp
ABp Bp
BApBAp
ApABp
ABp
Conditional Probability
BpBAp
BAp
ApABp
ABp
ApABpABp
ApABpBAp
BpBAp
BAp
ApABpABp
ApABpBAp
BpBAp
BAp
ApABpABp
ApABpBAp
BpBAp
BAp
• The Bayesian rule (named after Thomas Bayes, an 18th-century British mathematician):
The Bayesian Rule
Bp
ApABpBAp
Applying Bayes’ rule
• A = disease, B = symptom• P(disease|symptom) = P(symptom|
disease) * P(disease) / P(symptom)
Bp
ApABpBAp
Applying Bayes’ rule• A doctor knows that the disease meningitis
causes the patient to have a stiff neck for 70% of the time.
• The probability that a patient has meningitis is 1/50,000.
• The probability that any patient has a stiff neck is 1%.
• P(s|m) = 0.7• P(m) = 1/50000• P(s) = 0.01
Applying Bayes’ rule• P(s|m) = 0.7• P(m) = 1/50000• P(s) = 0.01• P(m|s) = P(s|m) * P(m) / P(s) • = 0.7 * 1/50000 / 0.01 • = 0.0014• = around 1/714• Conclusion: Less than 1 in 700 patients
with a stiff neck have meningitis.
Example: Coin Flip• P(X1 = H) = ½
1) X1 is H: P(X2 = H | X1 = H) = 0.9
2) X1 is T: P(X2 = T | X1 = T ) = 0.8
P(X2 = H) = ?
What we learned from the example?
• If event A depends on exactly two mutually exclusive events, B and ¬B, we obtain:
• P(¬X|Y) = 1 – P(X|Y)• P(X|¬Y) = 1 – P(X|Y)?
p(A) =p(AB) p(B) +p(AB) p(B)
p(B) =p(BA) p(A) +p(BA) p(A)
ApABpApABp
ApABpBAp
• If event A depends on exactly two mutually exclusive events, B and ¬B, we obtain:
• Similarly, if event B depends on exactly two mutually exclusive events, A and ¬A, we obtain:
Conditional probability
p(A) =p(AB) p(B) +p(AB) p(B)
p(B) =p(BA) p(A) +p(BA) p(A)
ApABpApABp
ApABpBAp
p(A)=p(AB) p(B)+p(AB) p(B)
p(B) =p(BA) p(A) +p(BA) p(A)
ApABpApABp
ApABpBAp
• Substituting p(B) into the Bayesian rule yields:
The Bayesian Rule
p(A)=p(AB) p(B)+p(AB) p(B)
p(B) =p(BA) p(A) +p(BA) p(A)
ApABpApABp
ApABpBAp
Bp
ApABpBAp
p(A)=p(AB) p(B)+p(AB) p(B)
p(B) =p(BA) p(A) +p(BA) p(A)
ApABpApABp
ApABpBAp
• Instead of A and B, consider H (a hypothesis) and E (evidence for that hypothesis).
• Expert systems use the Bayesian rule to rank potentially true hypotheses based on evidences
Bayesian reasoning
HpHEpHpHEp
HpHEpEHp
p(A)=p(AB) p(B)+p(AB) p(B)
p(B) =p(BA) p(A) +p(BA) p(A)
ApABpApABp
ApABpBAp
• If event E occurs, then the probability thatevent H will occur is p(H|E)
IF E (evidence) is trueTHEN H (hypothesis) is true with
probability p
Bayesian reasoning
HpHEpHpHEp
HpHEpEHp
Bayesian reasoning Example: Cancer and
Test • P(C) = 0.01 P(¬C) = 0.99• P(+|C) = 0.9 P(-|C) = 0.1• P(+|¬C) = 0.2 P(-|¬C) = 0.8
• P(C|+) = ?
HpHEpHpHEp
HpHEpEHp
• Expert identifies prior probabilities forhypotheses p(H) and p(¬H)
• Expert identifies conditional probabilities for:o p(E|H): Observing evidence E if hypothesis
H is trueo p(E|¬H): Observing evidence E if
hypothesis H is false
Bayesian reasoning
HpHEpHpHEp
HpHEpEHp
• Experts provide p(H), p(¬H), p(E|H), and p(E|¬H)
• Users describe observed evidence Eo Expert system calculates p(H|E) using
Bayesian ruleo p(H|E) is the posterior probability that
hypothesis H occurs upon observing evidence E
• What about multiple hypotheses and evidences?
Bayesian reasoning
Bayesian reasoning with multiple hypotheses
i
n
ii
n
ii BpBApBAp
11
AB4
B3
B1
B2
p(A)
p(A) =p(AB) p(B) +p(AB) p(B)
p(B) =p(BA) p(A) +p(BA) p(A)
ApABpApABp
ApABpBAp
Bayesian reasoning with multiple hypotheses
• Expand the Bayesian rule to work with multiple hypotheses (H1...Hm)
HpHEpHpHEp
HpHEpEHp
Bayesian reasoning with multiple hypotheses and
evidences• Expand the Bayesian rule to work with
multiple hypotheses (H1...Hm) and evidences (E1...En)
• Expand the Bayesian rule to work with multiple hypotheses (H1...Hm) and evidences (E1...En)
Assuming conditional independence among evidences E1...En
Bayesian reasoning with multiple hypotheses and
evidences
m
kkknkk
iiniini
HpHEp...HEpHEp
HpHEpHEpHEpE...EEHp
121
2121
...
• Expert is given three conditionally independent evidences E1, E2, and E3
o Expert creates three mutually exclusive and exhaustive hypotheses H1, H2, and H3
o Expert provides prior probabilities p(H1), p(H2), p(H3)
o Expert identifies conditional probabilities for observing each evidence Ei for all possible hypotheses Hk
Bayesian reasoning Example
• Expert data:
Bayesian reasoning Example
H ypothesi sProbability
1i 2i 3i
0.40
0.9
0.6
0.3
0.35
0.0
0.7
0.8
0.25
0.7
0.9
0.5
iHp
iHEp 1
iHEp 2
iHEp 3
• user observes E3 H ypothesi sProbability
1i 2i 3i
0.40
0.9
0.6
0.3
0.35
0.0
0.7
0.8
0.25
0.7
0.9
0.5
iHp
iHEp 1
iHEp 2
iHEp 3
Bayesian reasoning Example
32,1,=,3
13
33 i
HpHEp
HpHEpEHp
kkk
iii
0.3425.0.90+35.07.0+0.400.6
0.400.631
EHp
0.3425.0.90+35.07.0+0.400.6
35.07.032
EHp
0.3225.0.90+35.07.0+0.400.6
25.09.033
EHp
32,1,=,3
13
33 i
HpHEp
HpHEpEHp
kkk
iii
0.3425.0.90+35.07.0+0.400.6
0.400.631
EHp
0.3425.0.90+35.07.0+0.400.6
35.07.032
EHp
0.3225.0.90+35.07.0+0.400.6
25.09.033
EHp
32,1,=,3
13
33 i
HpHEp
HpHEpEHp
kkk
iii
0.3425.0.90+35.07.0+0.400.6
0.400.631
EHp
0.3425.0.90+35.07.0+0.400.6
35.07.032
EHp
0.3225.0.90+35.07.0+0.400.6
25.09.033
EHp
32,1,=,3
13
33 i
HpHEp
HpHEpEHp
kkk
iii
0.3425.0.90+35.07.0+0.400.6
0.400.631
EHp
0.3425.0.90+35.07.0+0.400.6
35.07.032
EHp
0.3225.0.90+35.07.0+0.400.6
25.09.033
EHp
expert system computes
posterior probabilities
user observes E3
H ypothesi sProbability
1i 2i 3i
0.40
0.9
0.6
0.3
0.35
0.0
0.7
0.8
0.25
0.7
0.9
0.5
iHp
iHEp 1
iHEp 2
iHEp 3
• user observes E3 E1 H ypothesi sProbability
1i 2i 3i
0.40
0.9
0.6
0.3
0.35
0.0
0.7
0.8
0.25
0.7
0.9
0.5
iHp
iHEp 1
iHEp 2
iHEp 3
32,1,=,3
131
3131 i
HpHEpHEp
HpHEpHEpEEHp
kkkk
iiii
0.1925.00.5+35.07.00.8+0.400.60.3
0.400.60.3311
EEHp
0.5225.00.5+35.07.00.8+0.400.60.3
35.07.00.8312
EEHp
0.2925.00.5+35.07.00.8+0.400.60.3
25.09.00.5313
EEHp
Bayesian reasoning Example
user observes E1
32,1,=,3
131
3131 i
HpHEpHEp
HpHEpHEpEEHp
kkkk
iiii
0.1925.00.5+35.07.00.8+0.400.60.3
0.400.60.3311
EEHp
0.5225.00.5+35.07.00.8+0.400.60.3
35.07.00.8312
EEHp
0.2925.00.5+35.07.00.8+0.400.60.3
25.09.00.5313
EEHp
expert system computes
posterior probabilities
H ypothesi sProbability
1i 2i 3i
0.40
0.9
0.6
0.3
0.35
0.0
0.7
0.8
0.25
0.7
0.9
0.5
iHp
iHEp 1
iHEp 2
iHEp 3
m
kkknkk
iiniini
HpHEp...HEpHEp
HpHEpHEpHEpE...EEHp
121
2121
...
• user observes E3 E1 E2 H ypothesi sProbability
1i 2i 3i
0.40
0.9
0.6
0.3
0.35
0.0
0.7
0.8
0.25
0.7
0.9
0.5
iHp
iHEp 1
iHEp 2
iHEp 3
32,1,=,3
1321
321321 i
HpHEpHEpHEp
HpHEpHEpHEpEEEHp
kkkkk
iiiii
0.4525.09.00.50.7
0.7
0.7
0.5
+.3507.00.00.8+0.400.60.90.30.400.60.90.3
3211
EEEHp
025.09.0+.3507.00.00.8+0.400.60.90.3
35.07.00.00.83212
EEEHp
0.5525.09.00.5+.3507.00.00.8+0.400.60.90.3
25.09.00.70.53213
EEEHp
32,1,=,3
1321
321321 i
HpHEpHEpHEp
HpHEpHEpHEpEEEHp
kkkkk
iiiii
0.4525.09.00.50.7
0.7
0.7
0.5
+.3507.00.00.8+0.400.60.90.30.400.60.90.3
3211
EEEHp
025.09.0+.3507.00.00.8+0.400.60.90.3
35.07.00.00.83212
EEEHp
0.5525.09.00.5+.3507.00.00.8+0.400.60.90.3
25.09.00.70.53213
EEEHp
Bayesian reasoning Example
expert system computesposterior probabilitiesuser observes E2
H ypothesi sProbability
1i 2i 3i
0.40
0.9
0.6
0.3
0.35
0.0
0.7
0.8
0.25
0.7
0.9
0.5
iHp
iHEp 1
iHEp 2
iHEp 3
m
kkknkk
iiniini
HpHEp...HEpHEp
HpHEpHEpHEpE...EEHp
121
2121
...
Bayesian reasoning Example
• Initial expert-based ranking:o p(H1) = 0.40; p(H2) = 0.35; p(H3) = 0.25
• Expert system ranking after observing E1, E2, E3:o p(H1) = 0.45; p(H2) = 0.0; p(H3) = 0.55
H ypothesi sProbability
1i 2i 3i
0.40
0.9
0.6
0.3
0.35
0.0
0.7
0.8
0.25
0.7
0.9
0.5
iHp
iHEp 1
iHEp 2
iHEp 3
Problems with the Bayesianapproach
• Humans are not very good at estimating probability!o In particular, we tend to make different
assumptions when calculating prior and conditional probabilities
• Reliable and complete statistical information often not available.
• Bayesian approach requires evidences to be conditionally independent – often not the case.
• One solution: certainty factors