A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.
-
Upload
jane-obrien -
Category
Documents
-
view
216 -
download
0
Transcript of A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.
![Page 1: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/1.jpg)
A Theory of Theory FormationA Theory of Theory Formation
Simon Colton
Universities of Edinburgh and York
![Page 2: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/2.jpg)
OverviewOverview
What is a theory?Four components of the theory of ATF
– Techniques inside the components
Cycles of theory formation– Case Studies
Applications (briefly)– Of both the theories and the process
![Page 3: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/3.jpg)
What is a Theory?What is a Theory?
Theories are (minimally) a collection of:– Objects of interest– Concepts about the objects– Hypotheses relating the concepts– Explanations which prove the hypotheses
Finite Group Theory:– All cyclic groups are Abelian
Inorganic Chemistry:– Acid + Base Salt + Water
![Page 4: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/4.jpg)
So, We Require:So, We Require:
Object
GeneratorConcept
Generator
Hypothesis Generator
Explanation Generator
![Page 5: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/5.jpg)
In Principle, These Could Be:In Principle, These Could Be:
Database, CAS, CSP, Model Generator
Machine Learning Program
Data Mining Program
ATP System, Pathway Finder, Visualisation
![Page 6: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/6.jpg)
In Practice, In Practice, Current Implementation:Current Implementation:
Database, Model Generator, (CAS,
CSP nearly)
The HR Program
The HR Program
ATP Systems
![Page 7: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/7.jpg)
Object Generation and Object Generation and Explanation GenerationExplanation Generation
Object Generation:– Machine learning – reading a file, database
In Mathematics– CSP (e.g., FINDER, Solver), CAS (e.g., Maple)– Davis Putnam method (e.g., MACE)– Resolution Theorem Proving (e.g., Otter)
HR must be able to communicate– Read models and concepts from MACE’s output– Read proofs and statistics from Otters output
![Page 8: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/8.jpg)
Concept GenerationConcept Generation
Build a new concept from old ones– 10 general production rules (demonstrated later)– Produce both a definition and examples
Throw away concepts using definitions– Tidy definitions up– Repetitions, function conflict, negation conflict
Decide which concepts to use for construction– Plethora of measures of interestingness– Weighted sum of measures
![Page 9: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/9.jpg)
Concept Generation:Concept Generation:Lakatos-inspired TechniquesLakatos-inspired Techniques
Monster Barring– Remove an object of interest from theory
Counterexample Barring– Except a finite subset of objects from a theorem– E.g., all primes except 2 are odd
Concept Barring– Except a concept from a theorem– All integers other than squares have an even number of divisors
Credit to Alison Pease
![Page 10: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/10.jpg)
Hypothesis Generation:Hypothesis Generation:Finding Empirical RelationshipsFinding Empirical RelationshipsEquivalence conjectures
– One concept has the same examples as anotherSubsumption conjectures
– All examples of one concept are examples of otherNon-existence conjectures
– A concept has no examples Assessment of conjectures
– Used to assess the concepts mentioned in them
![Page 11: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/11.jpg)
Hypothesis GenerationHypothesis GenerationExtracting Prime ImplicatesExtracting Prime Implicates
Extract implications, then prime implicatesEquivalence conjectures are split:
– A & B & C D & E & F becomes– A & B & C D, A & B & C E, etc.
Non-existence conjectures are split:– ¬(A & B & C) becomes: A & B ¬C, etc.
Extract Prime implicates:– A & B & C D, try A D, then B D,
C D, then A & B D, etc.
![Page 12: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/12.jpg)
Hypothesis Generation: Hypothesis Generation: Imperfect ConjecturesImperfect Conjectures
User sets a percentage minimum, say 80% Near-subsumption conjectures
– E.g., primes odd (99% true)– Also returns the counterexamples: here, 2
Near-equivalence conjectures– Prime odd (70% true)
Applicability conjectures– A concept has a (small) finite number of examples– E.g., even prime numbers: 2 is only example
![Page 13: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/13.jpg)
Cycles of Theory FormationCycles of Theory Formation
How the individual techniques are employedConcept driven conjecture making
– Finding conjectures to help understand concepts– Exploration techniques
Conjecture driven concept formation– Inventing concepts to fix faulty conjectures– Imperfect conjectures, Lakatos techniques
![Page 14: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/14.jpg)
Concept Driven Cycle (cut-down)Concept Driven Cycle (cut-down)
Invent Concept
EquivalenceNon Existence New Concept
Subsumptions
Implications
Reject
![Page 15: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/15.jpg)
Concept Driven Cycle ContinuedConcept Driven Cycle Continued
Implications
Counterexample Proof
Prime Implicates
Counterexample Proof
![Page 16: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/16.jpg)
Conjecture Driven CycleConjecture Driven CycleInvent Concept Reject
Near Equivalence Applicability Near Subsumption
Concept Barring
Counterex Barring
New/Old Concept
Equivalence
Implications
Monster Barring
New Concept
Counterex Barring
Concept Barring
New/Old Concept
![Page 17: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/17.jpg)
Case Study: GroupsCase Study: Groups
Given: Group theory axioms
![Page 18: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/18.jpg)
Case Study: GroupsCase Study: Groups
MACE model generator finds a model of size 1
Davis Putnam Method
![Page 19: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/19.jpg)
Case Study: GroupsCase Study: Groups
Extracts concepts:Element, Multiplication, Identity, Inverse
HR Reads MACE’s Output
![Page 20: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/20.jpg)
Case Study: GroupsCase Study: Groups
Invents the concept idempotent elements (a*a=a)
Match Production Rule
![Page 21: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/21.jpg)
Case Study: GroupsCase Study: Groups
Makes Conjecture: a*a=a a is the identity element
Equivalence Finding
![Page 22: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/22.jpg)
Case Study: GroupsCase Study: Groups
Otter proves this in less than a second
Resolution Theorem Proving
![Page 23: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/23.jpg)
Case Study: GroupsCase Study: Groups
a*a = a a=identity, a=identity a*a=aEnd of cycle
Extracts Prime Implicates
![Page 24: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/24.jpg)
Case Study: GroupsCase Study: Groups
Later: Invents the concept of triples of elements(a,b,c) for which a*b=c & b*a=c
Compose Production Rule
![Page 25: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/25.jpg)
Case Study: GroupsCase Study: Groups
Invents concept of pairs (a,b) for which thereexists an element c such that: a*b=c & b*a=c
Exists Production Rule
![Page 26: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/26.jpg)
Case Study: GroupsCase Study: Groups
Invents the concept of groups for which all pairsof elements have such a c: Abelian groups
Forall Production Rule
![Page 27: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/27.jpg)
Case Study: GroupsCase Study: Groups
Makes the Conjecture: G is a group if and only if it is Abelian
Equivalence Finding
![Page 28: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/28.jpg)
Case Study: GroupsCase Study: Groups
Otter fails to prove this conjecture
Sorry
![Page 29: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/29.jpg)
Case Study: GroupsCase Study: Groups
MACE finds a counterexample:Dihedral Group of size 6 (non-Abelian)
Davis Putnam Method
![Page 30: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/30.jpg)
Case Study: GroupsCase Study: Groups
Concept of Abelian groups allowed into theoryTheory recalculated in light of new object of interest
Assessment of Concepts
![Page 31: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/31.jpg)
Case Study: GoldbachCase Study: Goldbach
Given: Integers 1 to 100, Concepts: Divisors, Addition
![Page 32: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/32.jpg)
Case Study: GoldbachCase Study: Goldbach
Invents: Even Numbers (divisible by 2)
Split Production Rule
![Page 33: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/33.jpg)
Case Study: GoldbachCase Study: Goldbach
Invents: Number of Divisors (tau function)
Size Production Rule
![Page 34: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/34.jpg)
Case Study: GoldbachCase Study: Goldbach
Invents: Prime numbers (2 divisors)
Split Production Rule
![Page 35: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/35.jpg)
Case Study: GoldbachCase Study: Goldbach
Half an hour later:Invents: Goldbach numbers (sum of 2 primes)
Compose Production Rule
![Page 36: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/36.jpg)
Case Study: GoldbachCase Study: Goldbach
Conjectures: Even numbers are Goldbach numbers(with one exception, the number 2)
Near Equivalence Finding
![Page 37: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/37.jpg)
Case Study: GoldbachCase Study: Goldbach
Forces: Concept of being the number 2
Counterexample Barring (Split)
![Page 38: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/38.jpg)
Case Study: GoldbachCase Study: Goldbach
Forces concept: Even numbers except 2
Counterexample Barring (Negate)
![Page 39: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/39.jpg)
Case Study: GoldbachCase Study: Goldbach
Conjectures: Even numbers except 2 are GoldbachNumbers (Goldbach’s Conjecture)
Subsumption Finding
![Page 40: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/40.jpg)
Case Study: GoldbachCase Study: Goldbach
Passes the conjecture to an inductive theorem prover?
Absolutely No Chance
![Page 41: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/41.jpg)
Applications of TheoriesApplications of Theories
Puzzle generation– Which is the odd one out: 4, 9, 16, 24– Which is the odd one out: 2, 9, 8, 3
Problem generation– TPTP library: find theorem to differentiate Spass & E– See AI and Maths paper
Prediction tests: (e.g., Progol animals file)– P(mammal | has_milk) = 1.0– P(mammal | habitat(water)) = 0.125– Take average over all Bayesian probabilities
![Page 42: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/42.jpg)
Applications of Theory FormationApplications of Theory Formation
Identifying concepts (e.g., Michalski trains)– Forward look ahead mechanism (see ICML-00 paper)
Simplifying problems– Lemma generation for ATP – Constraint generation for CSP (see CP-01 paper)
Identifying outliers– How unique an object of interest is
Inventing concepts– Integer sequences (and conjectures), – See AAAI-00 paper, Journal of Integer Sequences
![Page 43: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/43.jpg)
ConclusionsConclusions
Presented a snapshot of the theory of ATF– Autonomous– Four components, numerous techniques– Uses third party software– Concept driven and conjecture driven cycles
Applies to many machine learning tasks– Concept identification, puzzle generation,– Predictions, problem simplification
![Page 44: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/44.jpg)
Welcome to the Next LevelWelcome to the Next Level
For any of the four components– Substitute a human for interactive ATF– Roy McCasland (hopefully), mathematician– Work on Zariski spaces with HR
For any of the four components– Substitute another agent for multi-agent ATF– Alison Pease’s PhD, cognitive modelling– Lakatos style reasoning and machine creativity
![Page 45: A Theory of Theory Formation Simon Colton Universities of Edinburgh and York.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ec85503460f94bd5c2a/html5/thumbnails/45.jpg)
Theory Formation in Theory Formation in Bioinformatics?Bioinformatics?
Can work with non-maths dataCan form near-conjecturesNeeds to relax notion of equalityMulti-agent approach definitely needed