Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University...
-
Upload
ashlee-mason -
Category
Documents
-
view
219 -
download
0
Transcript of Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University...
![Page 1: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/1.jpg)
Slide 1
UCL JDI Centre for the Forensic Sciences21 March 2012
Norman FentonQueen Mary University of London and Agena Ltd
Bayes and the Law
![Page 2: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/2.jpg)
Slide 2
R vs Levi Bellfield, Sept 07 – Feb 08
![Page 3: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/3.jpg)
Slide 3
R v Gary Dobson 2011
Stephen Lawrence
![Page 4: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/4.jpg)
Slide 4
Questions
• What is 723539016321014567 divided by 9084523963087620508237120424982?
• What is the area of a field whose length is approximately 100 metres and whose width is approximately 50 metres?
![Page 5: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/5.jpg)
Slide 5
Court of Appeal Rulings
• “The task of the jury is to evaluate evidence and reach a conclusion not by means of a formula, mathematical or otherwise, but by the joint application of their individual common sense and knowledge of the world to the evidence before them” (R v Adams, 1995)
• “..no attempt can realistically be made in the generality of cases to use a formula to calculate the probabilities. .. it is quite clear that outside the field of DNA (and possibly other areas where there is a firm statistical base) this court has made it clear that Bayes theorem and likelihood ratios should not be used” (R v T, 2010)
![Page 6: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/6.jpg)
Slide 6
![Page 7: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/7.jpg)
Slide 7
Revising beliefs when you get forensic evidence
• Fred is one of a number of men who were at the scene of the crime. The (prior) probability he committed the crime is the same probability as the other men.
• We discover the criminal’s shoe size was 13 – a size found in only 1 in a 100 men. Fred is size 13. Clearly our belief in Fred’s innocence decreases. But what is the probability now?
![Page 8: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/8.jpg)
Slide 8
Are these statements equivalent?
• the probability of this evidence (matching shoe size) given the defendant is innocent is 1 in 100• the probability the defendant is
innocent given this evidence is 1 in 100
The prosecution fallacy is to treat the second statement as equivalent tothe first
The only rational way to revise the prior probability of a hypothesis when we observe evidence is to apply Bayes theorem.
![Page 9: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/9.jpg)
Slide 9
Fred has size 13
![Page 10: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/10.jpg)
Slide 10
Imagine 1,000other people also at scene
Fred has size 13
![Page 11: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/11.jpg)
Slide 11
About 10 out of the1,000 peoplehave size 13
Fred has size 13
![Page 12: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/12.jpg)
Slide 12
Fred is one of11 withsize 13
So there is a 10/11chance that Fred is NOT guilty
That’s very different fromthe prosecutionclaim of 1%
![Page 13: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/13.jpg)
Slide 13
Ahh.. but DNA evidence is different?
• Very low random match probabilities … but same error
• Low template DNA ‘matches’ have high random match probabilities
• Principle applies to ALL types of forensic match evidence
• Prosecutor fallacy still widespread
![Page 14: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/14.jpg)
Slide 14
Bayes Theorem
E(evidence)
We now get some evidence E.
H (hypothesis)
We have a prior P(H)
We want to know the posterior P(H|E)
P(H|E) = P(E|H)*P(H) P(E)
P(E|H)*P(H)P(E|H)*P(H) + P(E|not H)*P(not H)
=
1*1/1001
1*1/1001+ 1/100*1000/10001P(H|E) = =
0.000999
0.000999 + 0.009990.091
![Page 15: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/15.jpg)
Slide 15
Determining the value of evidence
– Prosecution likelihood (The probability of seeing the evidence if the prosecution hypothesis is true)
(=1 in example)
Posterior Odds = Likelihood ratio x Prior Odds
Bayes Theorem (“Odds Form”)
Likelihood ratio = Prosecutor likelihood Defence likelihood
(=100 in example)
–Defence likelihood (The probability of seeing the evidence if the defence hypothesis is true)
(=1/100 in example)
![Page 16: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/16.jpg)
Slide 16
Likelihood Ratio Examples
Prosecutor 1 100 25 X =Defence 4 1 1
Prior odds Likelihood ratio Posterior Odds
Prosecutor 1 100 1 X =Defence 1000 1 10
LR > 1 supports prosecution; LR <1 supports defenceLR = 1 means evidence has no probative value
![Page 17: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/17.jpg)
Slide 17
Example: Barry George
• H: Hypothesis “Barry George did not fire gun”
• E: small gunpowder trace in coat pocket
• Defence likelihood P(E|H) = 1/100
• …
• …But Prosecution likelihood P(E| not H) = 1/100
• So LR = 1
![Page 18: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/18.jpg)
Slide 18
R v T
• Good: all assumptions need to be revealed
• Bad: Implies LRs and Bayesian probability can only be used where ‘database is sufficiently scientific’ (e.g. DNA)
• Ugly: Lawyers are ‘erring on side of safety’ – avoiding LRs, probabilities altogether in favour of vacuous ‘verbal equivalents’
![Page 19: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/19.jpg)
Slide 19
But things are not so simple…
![Page 20: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/20.jpg)
Slide 20
But things are not so simple…
![Page 21: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/21.jpg)
Slide 21
But things are not so simple…
![Page 22: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/22.jpg)
Slide 22
The problems
• ‘Simple’ Bayes arguments do not scale up in these ‘Bayesian networks’
• Calculations from first principles too complex
• …. But Bayesian network tools provide solution
![Page 23: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/23.jpg)
Slide 23
Assumes perfect test accuracy
(this is a1/1000randommatch probability)
![Page 24: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/24.jpg)
Slide 24
Assumes
false positive rate 0.1
false negative rate 0.01
![Page 25: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/25.jpg)
Slide 25
Bayesian nets in action
• Separates out assumptions from calculations
• Can incorporate subjective, expert judgement
• Can address the standard resistance to using subjective probabilities by using ranges.
• Easily show results from different assumptions
• …but must be seen as the ‘calculator’
![Page 26: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/26.jpg)
Slide 26
Bayesian nets: where we are
• Used to help lawyers in major trials
• Developing methods to make building legal BN arguments easier (with Neil and Lagnado):– set of ‘idioms’ for common argument fragments (accuracy
of evidence, motive/opportunity, alibi evidence)
– simplifying probability elicitation (mutual exclusivity problem etc)
– tailored tool support
• Biggest challenge is to get consensus from other Bayesians
![Page 27: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/27.jpg)
Slide 27
Conclusions and Challenges
• The only rational way to evaluate probabilistic evidence is being avoided because of basic misunderstandings
• But real Bayesian arguments cannot be presented from first principles.
• Use BNs and focus on the calculator analogy (argue about the prior assumptions NOT about the Bayesian calculations)
• How to consider all the evidence in a case and determine (e.g. as CPS must do) the probability of a conviction
![Page 28: Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and.](https://reader035.fdocuments.us/reader035/viewer/2022062409/5697bfad1a28abf838c9c271/html5/thumbnails/28.jpg)
Slide 28
A Call to ArmsBayes and the Law Network
Transforming Legal Reasoning through Effective use of Probability and Bayes
https://sites.google.com/site/bayeslegal/
Contact: [email protected]
Further reading:Fenton, N.E. and Neil, M., 'Avoiding Legal Fallacies in Practice Using Bayesian
Networks', Australian Journal of Legal Philosophy 36, 114-151, 2011 www.eecs.qmul.ac.uk/~norman/papers/fenton_neil_prob_fallacies_June2011web.pdf
Fenton, N.E. and Neil, M., 'On limiting the use of Bayes in presenting forensic evidence'
www.eecs.qmul.ac.uk/~norman/papers/likelihood_ratio.pdf