Representation and Search Techniques - Artificial Intelligence
Data representation techniques for adaptation
description
Transcript of Data representation techniques for adaptation
![Page 1: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/1.jpg)
Data representation techniques for adaptation
Alexandra I. CristeaUSI intensive course “Adaptive Systems” April-May 2003
![Page 2: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/2.jpg)
Overview: Data representation
1. Data or knowledge?2. Subsymbolic vs. symbolic techniques3. Symbolic representation4. Example5. Subsymbolic reprensentation6. Example
![Page 3: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/3.jpg)
Data or knowledge?
• Data for AS becomes often knowledge– data < information < knowledge
• We divide into:– Symbolic– Sub-symbolic knowledge representation
![Page 4: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/4.jpg)
Data representation techniques for adaptation
• Symbolic AI and knowledge representation, such as:– Concept Maps – Probabilistic AI (belief networks)
• see UM course
• Sub-symbolic: Machine learning, such as:– Neural Networks
![Page 5: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/5.jpg)
Symbolic Knowledge Representation
![Page 6: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/6.jpg)
Symbolic AI and knowledge representation
• Static knowledge– Concept mapping– terminological knowledge– concept subsumption (inclusion) inference
• Dynamic Knowledge– ontological engineering, e.g., temporal representation
and reasoning– planning
![Page 7: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/7.jpg)
Concept Maps
Example
![Page 8: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/8.jpg)
Proposition: Without the industrial chemical reduction
of atmospheric nitrogen, starvation would be rampant
in third world countries.
FOOD
Human Healthand Survival
Contains
Required for
and
Requiring more
Essential Amino Acids
Animals
Used for
Such as
Madeby
Plants
Grains Legumes
Required forgrowth of
Symbiotic Bacteria
“Fixed” Nitrogen
Possess
That produce
Agricultural Practices
Population Growth
Politics
Economics
Distribution
Climate
Starvation and Famine
Malthus 1819
Eastern Europe
India
Africa
Deprivation leads to
Can be limited by
and
Such as in
Pesticides HerbicidesGenetics & Breeding
Irrigation
Fertilizer Which significantly supplements naturally
Such as
Predicted by
Can be increased by
NH3Haber
ProcessAtmospheric N2
Protein
Includes
Eatenby
Use
d by
hu m
a ns
a s
![Page 9: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/9.jpg)
Constructing a CM
• Brainstorming Phase: • Organizing Phase: create groups and sub-
groups of related items. • Layout Phase: • Linking Phase: lines with arrows
![Page 10: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/10.jpg)
Reviewing the CM• Accuracy and Thoroughness.
– Are the concepts and relationships correct? Are important concepts missing? Are any misconceptions apparent?
• Organization. – Was the concept map laid out in a way that
higher order relationships are apparent and easy to follow? Does it have a representative title?
• Appearance. – spelling, etc.?
• Creativity.
![Page 11: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/11.jpg)
Sub-symbolic knowledge representation
![Page 12: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/12.jpg)
Subsymbolic systems
• human-like information processing:• learning from examples, • context sensitivity, • generalization, • robustness of behaviour, and • intuitive reasoning
![Page 13: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/13.jpg)
Some notes on NNExample
![Page 14: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/14.jpg)
Why NN?
• To learn how our brain works (!!)• High computation rate technology• Intelligence• User-friendly-ness
![Page 15: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/15.jpg)
Applications
vsvs
Why NNs?Why NNs?
![Page 16: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/16.jpg)
ApplicationsWhy NNs?Why NNs?
![Page 17: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/17.jpg)
Man-machine hardware comparison
![Page 18: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/18.jpg)
Man-machine information processing
![Page 19: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/19.jpg)
What are humans good at and machines not?
• Humans: – pattern recognition– Reasoning with incomplete knowledge
• Computers:– Precise computing– Number crunching
![Page 20: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/20.jpg)
![Page 21: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/21.jpg)
The Biological Neuron
![Page 22: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/22.jpg)
(very small) Biological NN
![Page 23: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/23.jpg)
Purkinje cellPurkinje cell
![Page 24: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/24.jpg)
Spike (width 0.2 – 5ms)
![Page 25: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/25.jpg)
Firing
• Resulting signal– Excitatory:
• encourages firing of the next neuron– Inhibitory:
• Discourages firing of the next neuron
![Page 26: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/26.jpg)
What does a neuron do?
• Sums its inputs• Decides if to fire or not with respect to
a threshold• But: limited capacity:
– Neuron cannot fire all the time– Refractory period: 10ms – min time to
fire again– So: max. firing frequency: 100 spikes/
sec
![Page 27: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/27.jpg)
Hebbian learning rule (1949)
• If neuron A repeatedly and persistently contributes to the firing of neuron B, than the connection between A and B will get stronger.
• If neuron A does not contribute to the firing of neuron B for a long period of time, than the connection between A and B becomes weaker.
![Page 28: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/28.jpg)
Different size synapses
![Page 29: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/29.jpg)
Summarizing
• A neuron doesn’t fire if cumulated activity below threshold
• If the activity is above threshold, neuron fires (produces a spike)
• Firing frequency increases with accumulated activity until max. firing frequency reached
![Page 30: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/30.jpg)
The ANN
![Page 31: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/31.jpg)
The Artificial Neuron
InputInput
OutputOutput
Functions:Functions: InsideInside :: SynapseSynapseOutsideOutside ::f f
==thresholdthreshold
![Page 32: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/32.jpg)
An ANNAn ANNInput
Output
Layer :1
Layer :2
Layer :3
Black BoxBlack Box
![Page 33: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/33.jpg)
• Let’s look in the Black Box!
![Page 34: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/34.jpg)
NEURON LINK
W: weight
neuron 1 neuron 2
V1value
V2=w*v1value
![Page 35: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/35.jpg)
ANN
• Pulse train – average firing frequency 0• Model of synapse (connecting element)
– Real number w0 : excitatory– Real number w0 : inhibitory
• N(i) – set of neurons that have a connection to neuron i– jN(i)– wij – weight of connection of j to i
![Page 36: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/36.jpg)
neuron computation neuron computation
V1
W1
V2
W2 。。。 Vn
Wn
O
S=S= ΣViΣVi **WW i - i - bb i=1..n
internal activation fct
O = f (S)external activation fct
![Page 37: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/37.jpg)
Typical input output relation f
1. Standard sigmoid fct.: f(z)= 1/(1+e-z)2. Discrete neuron: fires at max. speed, or does not fire
xi={0,1}; f(z) = 1, z>0; 0 z0
![Page 38: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/38.jpg)
Other I-O functions f
3. Linear neuron f(z)=z
output xi=zi – = …
4. Stochastic neuron: xi {0,1}; output 0 or 1
input zi = j wij vi – ii
probability that neuron fires f(zi)
probability that it doesn’t fire 1- f(zi)
![Page 39: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/39.jpg)
Feedforward NNs
![Page 40: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/40.jpg)
Recurrent NNs
![Page 41: Data representation techniques for adaptation](https://reader035.fdocuments.us/reader035/viewer/2022062305/56815d37550346895dcb3830/html5/thumbnails/41.jpg)
Summarizing ANNs
• Feedforward network, layered– No connection from the output to the input, at
each layer but also at neuron level• Recurrent network
– Anything is allowed – cycles, etc.