The Logic of Probability Theory - Union University Theory as Extended Logic • George Boole • The...
21
The Logic of Probability Theory Deriving Bayesian Statistics from Boolean Algebra William Sipes July 14, 2010
Transcript of The Logic of Probability Theory - Union University Theory as Extended Logic • George Boole • The...
The Logic of Probability Theory
Deriving Bayesian Statistics from Boolean Algebra
William Sipes
July 14, 2010
Probability Theory as Extended Logic
• George Boole • The Laws of Thought • Algebraic expression of Aristotelian logical
propositions • Full Title: An Investigation of the Laws of Thought
on Which are Founded the Mathematical Theories of Logic and Probabilities (1854)
Probability Theory as Extended Logic
• Cox and Jaynes • The Algebra of Probable Inference (1961) • Probability Theory: The Logic of Science (2003) • Boolean Logic and Three Desiderata necessitate
Bayesian Probability
Boolean Algebra
• Commutative ring wrt operations of disjunction and conjunction
• Equivalence classes of [0] and [1] (representing FALSE and TRUE)
• Foundation of Computer Science
• Finite Field
Boolean Algebra
Disjunction
Conjunction
Negation
Boolean Algebra
Though there are three distinct operations, it can be shown that any combination of two that includes negation is complete.
Disjunction
Conjunction
Negation
Boolean Algebra
F F F F T F T F F T T T
F F F F T T T F T T T T
F T
T F
Double Negation
Commutativity
De Morgan’s Laws
Associativity
Idempotence
Boolean Algebra
Bayesian Probability
• Conditional Probability • Differs from frequentist approaches • Based on prior distributions • Models can be updated with new information
Bayesian Probability
Conditional: A given B
Conjunction: A and B
The Desiderata
1) Representation of plausibility with real numbers
2) Qualitative correspondence with common sense
3) Structural Consistency
These uniquely determine the allowable operations of all probabilistic theory.
Assumption of twice differentiability in functions
Product Rule
1. Require that any plausibilities obey all of the desiderata simultaneously, unambiguously, and completely
2. Assume that there is a functional relation for conjoined propositions
3. Argue (using the requirements imposed by the desiderata) that there is only one form for this rule
4. Derive the form of the rule using differential equations
Product Rule
• Most basic assumption • Only functional form that does not degenerate
when tested at “extremes” • Key feature of familiar probability theory
Product Rule
Product Rule
Sum Rule
1. Using the product rule, derive a function that relates propositions and their negations
2. Impose the conditions of the desiderata to derive another functional equation
3. Argue analytically about the functional equation 4. Reduce the functional equation to a differential
equation 5. This leaves a functional relation for complementary
plausibilities
Sum Rule
Sum Rule
Sum Rule
Sum Rule
Further Developing the Theory
• Conditional probability and the sum rule gives definition of independence
• Product rule can be used to derive Laplace’s definition of probability (frequentist)
• Demonstrate agreement with Kolmogorov’s axioms of probability
