Post on 24-Mar-2020
Evolving “Physical” Intelligence:
physiology, robotics, and
computational biology
By Bradly Alicea
EI Meeting, DevoLab, Michigan State University, Fall 2007
Introduction
Research Question: how do we uncover and represent the adaptive and
phylogenetic processes behind “physical” intelligent behavior (e.g.
movement, kinetics, control)?
* examples focus on autonomous physical
intelligence in vertebrates (lampreys
to humans), may generalize to design of
machines (biomimetics).
* paradigm focuses on motility related to
propulsion and “work”; interaction of
multiple physical elements.
* requires approximating a physiological
control system. Application domains:
biomechatronics, robotics, even micro-
machines.
* look at morphology alone, nervous system
alone, and morphology and nervous system
together.
Introduction (con’t)
Jeff Hawkins (Redwood Neuroscience Institute, Palm Technologies): “On
Intelligence”:
Intelligence is an internal mechanism:
* serves “pattern prediction” function
* memory-based, adaptive, hierarchical
* has an effect on behavior, not behavior
in and of itself.
* his focus is on “neocortex”, which is a
specific physiological system.
* idea can be generalized; formalized as
a control system.
Design principles (or principles of
evolvability)Principle #1 – modeling physical intelligence takes into account:
* physical sensory receptors: proprioceptors, nociceptors, muscle spindles. Capture the
collective activity of excitable cell populations.
* adaptability of morphology (e.g. muscle, bone): hypertrophy, fatigue, stress/strain,
regeneration.
Principle #2 – tetanic stimulation, physical exercise, environmental
training = “triad” of inducing adaptability (e.g. physiological plasticity):
* tetanic stimulation: deliver a tetanus (rapid electrical pulse) to muscle, neuronal tissue.
Results in LTP, “virtual” training
* physical exercise: Kaatsu (restrict blood flow to limb, stress muscles in that limb),
Fartlek (alternate intensity of training).
* alternate and extreme environment training: 0-g, force field adaptation, environmental
switching, H2S respiration (reduced metabolic baseline), ischemic preconditioning.
Design principles (or principles of
evolvability) – (con’t)Principle #3 – structural modular
intelligence:
* custom prosthetics (C-leg, foot-ankle
prosthetics, brain-machine interfaces)
replicate “intelligence” locally.
* adaptive walking, reaching, motility,
even thinking.
* function regulated by nervous system,
other morphological systems, environment.
Andy Clark (Natural-Born Cyborgs, 2004),
transformative potential of prosthetics.
Limbs > cells (e.g. living heart valve).
* due to role of proprioception, induces
locally adaptive changes in cell populations
(Smith et.al, Tissue Engineering, 7(2),
131, 2001.
1) Passive Dynamic Walkers (PDWs).
2) stability enforcement mechanisms for intelligent physical behavior.
3) the intelligence of “physical” intelligence.
Part I: Morphology
by Itself
Morphology by itself: PDWs
Andy Ruina and friends: Passive Dynamic Walking (PDW)
* inverted pendulum model: given a stochastic input (simple oscillator, finite
energetic input), stable gait can be physically approximated.
* bipedal: hindlimbs – human gait,
forelimbs – gibbon brachiation.
* no neuromuscular or cognitive
feedback, no mechanotransduction
(e.g. efference copy).
* when environmental conditions
are variable, gait is not stable.
Morphology by itself: PDWs (con’t)
Honda‟s ASIMO: demonstrates basic application of how bipedal gait is
regulated (also falls down a lot).
* afferent signal (tells legs to move)
* morphology reinforces efficiency
of movement.
* efference copy (feedback from
environment)
No “biological” component (e.g. muscle plasticity, neuroplasticity, learning
and memory).
* what would a “biological” controller look like?
Morphology by itself: PDWs (con’t)
Key feature of PDWs: behavior for
“free”.
* bipedal gait = zero net energy
expenditure given constant
movement (no adaptive adjustments).
Stable state discovery: Sherrington
(Integrative Action of the Nervous
System, 1947):
* amputate one limb, insect finds new 'stable phase' for motility.
* robotics/postural sway work: „internal‟ mechanisms perform relevant
computations.
Morphology by itself: stability
enforcement mechanisms
Mechanism #1: “static” allometry:
*controls the size of limbs relative to one
another and body size.
* basis for metabolic efficiency (cost of
locomotion decreases as body weight
increases in quadrupeds).
Body weight + limb shape +
forces in environment = cost of
transport.
* linear function, true for many varieties of
quadruped (see graph).
* cost of transport ~ muscle power (output)
needed for specific tasks and environments.From: Herr et.al (J. Experimental Biology,
205, 2005)
Morphology by itself: stability
enforcement mechanisms (con’t)
From: Bejan and Marden, J. of
Experimental Biology, 209, 238.
“Constructal” effects across phylogeny
(energy needs during locomotion = strong
positive selection on morphology):
* vary environment (air, water; variables = Reynolds
number, surface reaction forces)
* vary mode of locomotion (running, swimming;
variables embodied in velocity, frequency, force).
* linear scaling for all verts/inverts. Swimming
(fishes), flying (birds, bats, insects), running
(mammals, reptiles) all “cluster” along same trend
line (force production vs. body mass).
Morphology by itself: stability
enforcement mechanisms (con’t)Mechanism #2: matched volumes. MacIver‟s
simulation of Apternotus albifrons (Nelson and
MacIver, J. Experimental Biology, 202(10),
1999):
Weakly electric fish have a
special sensory modality called
electroreception.
* “active” (e.g. field generated around
organism).
* originated from neuromuscular system,
important in navigation.
* “map” at right is the electrosensory
field as it overlaps with “short-time
motor volume”.
Morphology by itself: stability
enforcement mechanisms (con’t)
Active sensing in context of set matching:
* actively sense at time t; at every t, iteratively
create vol(x) based on current environment.
* fill in space vol(x) with form(y); shift ith set
of motor commands towards leading front of
movement and exploration (optimize degree of
isomorphy).
* tail bending behavior (Behrend, Neuroscience,
13, 171-178, 1984); introduces "critical"
exploration points.
* electrodermal potential changes during tail
bending, potentially shifts the phase of short-
time motor volume.
* a "memory" of interaction (sensory inputs|limb size x muscle power); acts as an integrator
mechanism (allometric scaling in development and evolution ensures control).
1) biological A.I. (hybrots = cortical cells for computational environments)
2) neural coding (movement vector) and applicability to A.I. problems.
3) future advances: molecular models.
Part II: Nervous System
by Itself
Nervous system by itself: biological
AI = hybrots
In experiments by Reger et.al (Artificial Life, 10, 2000), hindbrain of
lamprey explanted and connected to Khepera robot.
* artificial photoreceptors from robot body provided input channel to Muller
cells, play the role of sensorimotor integration in lamprey brainstem.
* sensors on the robot's body = inputs to neural system. Resulting control
loop allows for adaptive behavior.
“Brain-in-a-dish”: collective
output, environmental
feedback (simulation).
* at left is an example of an
adaptive flight control
system.
* software is used to find
“taxic” information in neural
output.
* signals “mapped” to degrees
of freedom in the simulation
(roll, pitch, and yaw).
Nervous system by itself: biological
AI = hybrots (con’t)
DeMarse and Dockendorf, IEEE International Joint
Conference on Neural Networks, 3, 1548-1551,
2005.
Nervous system by itself: biological
AI = hybrots (con’t)
Control systems called hybrots have been
used to map neural signals to “skilled”
behaviors, such as drawing on an easel.
* cell culture of cortical neurons that
selectively grow connections between
neurons and show postsynaptic
modification (neuroplasticity).
* systems inform general processes behind
learning and memory in systems where
biology and machines are tightly coupled.
Nervous system by itself: applied
neural codingPole balancing (neural integrator keeps pole
from falling due to inertia or gravity):
* 1 DOF, “toy” problem.
* reinforcement learning methods solve this problem well
(actor-critic model).
* perceptron can be used to calculate and encode information
for movement direction, velocity, etc.
* does not approximate complex physiologically-based
functions (dampening, rate limiting).
See Broussard and Kassardjian, Learning
and Memory, 11, 127, 2004.
In mammals, neurons in premotor and motor cortex (PMC) contribute
to planning and directionality of movement:
* activity onset is 1-2 seconds before actual behavior.
* a "population code" (collective encoding of single behavioral events by
neuronal cell populations) has been found to exist.
* population coding may be important for other functions (memory encoding,
satiety states, etc).
Movement vector: Georgeopoulos et al (Journal of Neuroscience, 2, 1987):
* single cell activity in premotor and motor cortex predicts direction
of movement, mental rotation, force and velocity parameters.
Nervous system by itself: applied
neural coding (con’t)
The collective activity of cells results
in the encoding of desired behavioral
states.
* average activity of a population is greatest
in a certain direction(e.g. 45, 90, 155 degrees
from straight ahead).
* used as the driving
force behind Brain-
Machine Interface
(BMI) technology.
Nervous system by itself: applied
neural coding (con’t)
Nervous System by itself: future
advances -- molecular modelsMechanostimulation:
* activates stress pathway in cell populations.
* within minutes of stimulation, series of genes
upregulated (enhanced expression).
* in preconditioning, low levels of perturbation
increase robustness of system to acute shocks.
* depending on stimulus (environmental setting),
different regulatory patterns should result.
* patterns not well understood: what are the
effects of environmental switching, mutation of
genes involved in stimulus response, long-term
adaptation?
Nervous System by itself: future
advances -- molecular models (con’t)
Signaling pathways in memory-associated plasticity in brain (left - CREB)
and hypertrophy-associated plasticity in muscle (right - IGF):
Activity of pathways change across
training, interaction with environment.
* One emergent property of gene
expression and regulation = change in
morphology and internal state (figures:
http://www.biocarta.com).
Presence of hormone receptors, proteins and mRNAs in specific concentrations
(activity-dependence). Contributes to plasticity outcome (“increased/decreased
capacity” of tissues).
How do we piece together the interesting aspects of morphology and neural
systems into one unified framework/approach?
1) functional allometry/epigenetic matching
2) neurobiological control theory
Part III: Morphology and
Brain Together
Allometry: different anatomical segments are genetically “linked”. Consequences
for growth regulation and function within and between species.
Y = ax + b, Y = axb, Y = -Ax2 + Bx – c
Functional effects of allometry:
Herr et.al (J. Experimental Biology, 205, 2005):
* allometric scaling is a feature of "optimal“
locomotion and goal-directed behavior. Limb
length, circumference, brain size, metabolic
rate ~ body mass.
* provides a mechanism for determining
"optimal" scaling.
* Collins et.al (Science, 307, 1996) have
found that there is an optimal ratio of 1.06
between the length of the shank and thigh
in human bipedalism.
Morphology + Brain: functional
allometry/epigenetic matching
Morphology + Brain: functional
allometry/epigenetic matching (con’t)Epigenetic Matching: motorneuron population ~ target tissue (allometry and
growth regulation of target tissues ~ evolution and adaptability of nervous
system):
Streidter (Principles
of Brain Evolution,
Sinauer, 2006)
* finite pool of
motorneurons,
finite volume
of muscle target
tissue (myocytes).
* if axon from
motorneuron does
not innervates target
tissue = apoptosis.
Morphology + Brain: functional
allometry/epigenetic matching (con’t)
Katz and Lasek (PNAS USA, 75(3), 1977): Type I and Type II evolution.
* Type I: “linkage” (neuron-to-myocyte matching; innervational “linkage”
between two sets of cells).
* conservation via hormone action, high degree of epistasis, high degree of
evolution (no developmental constraint).
Type II: no autonomous preservation of axonally-mediated matches (no
innervational linkage between two sets of cells).
* depends on function of interactome, serves as evolutionary constraint
(unless mutation introduced for both motorneuron pool and muscle mass,
complexity remains low).
Morphology + Brain:
neurobiological control theory
Computational Neurobiology of Reaching and
Pointing: Reza Shadmehr (Johns Hopkins) and
Steven Wise.
* internal states not a black box, play an
important role in regulating behaviors
(normal and pathological).
* internal “model” is a statistical mechanism
(others are more interested in the internal
model as anatomical ROI).
* internal model = memory-based
displacement mechanism. Updates =
incoming physical sensory information,
visual information, and prior states.
Morphology + Brain: neurobiological
control theory (con’t)Reaching involves contributions from
both the CNS and constraints imposed
by limb geometry (230 and 137):
*anatomical stiffness ~ constraints.
Stiffness = stability.
* disease states (e.g. Parkinson‟s):
represents perturbation of neural
mechanisms involved with “normal”
movement (135).
* cerebellar, basal ganglia components
of learning system = nuclei, synapses
mediated by neurotransmitters (456).
Reinforcement learning mechanism.
Morphology + Brain: neurobiological
control theory (con’t)Internal Model: Computational function of
cerebellum:
* internal model is highly
conserved across vertebrates.
* general (innate) and specific
(acquired) internal models.
* innate: general limb
movements, environmental
resistance.
* specific: single and related
sets of objects.
How does evolution of the nervous system and morphology (as a unified
system) proceed phylogenetically?
* what “strategies” (e.g. combination of mutations, adaptations) are used to achieve a
derived form?
* three slides with hypothetical phylogenies only suggestive (focus on locomotive gait --
could have happened many different ways, and actually has in terms of convergent
evolution).
Postscript: “solutions” for
evolving physical
intelligence
Phylogenetic “solutions” application
domain: morpho-functional machines
Defined as the co-evolution of morphology and control unit:
* change functionality by changing control parameters and shape.
* evolve whole system in pieces, or modules (specialized substructures or
distinct behaviors).
* evolve morphology (morphogenesis) semi-independently from neural
controller.
* evolution of both morphology and neural mechanisms define a particular
evolutionary derivation (but multiple evolutionary “strategies”).
Phylogenetic “solutions” to evolving
physical intelligence
At left: how Type I and II
evolution may proceed:
Cladogenesis requires
generalized capacity for
plasticity.
* one mutation, may trigger
endocrine plasticity.
Anagenetic taxa may
require two specialized
mutations.
* morphology and nervous
system specialized but not
evolvable.
Phylogenetic “solutions” to evolving
physical intelligence (con’t)
At left: how static allometry
in hindlimb evolves along
mode of gait.
* gene controlling thigh
plasticity evolves before
common ancestor of C,
D, E, and F.
* bipedalism evolves in F
(requires other associated
mutations).
* genes “unlinked” by thigh
plasticity mutation, “relinked”
when bipedalism arises.
Phylogenetic “solutions” to evolving
physical intelligence (con’t)
At left: how to move from
one physically intelligent
mode to another in
evolution:
* three behavior-related
mutations to go from
specialized quadruped to
a biped (probably more).
* also anatomical changes
(joint morphology, spinal
cord alignment).
* behavioral mutations >
anatomical mutations
(which come first)?
Conclusions“Physically” Intelligent Systems:
1) Consider morphology and physiology together
* provides a mechanism for dynamic behavior
* emergent features of physiological interactions – constrained by morphology
2) Dissociate morphology and physiology for purposes of understanding
phylogeny
* shared derived characters (changes in phylogeny required for behavior,
match phenotype?)
* possible control mechanisms (morphology, genes, regulatory mechanisms)
3) Computational Principles
* What else is needed? What other tools can be deployed?
Additional Notes:
Comparative function and main neural centers:
Each brain center has a
specific computational
function:
* integration, acquisition,
encoding, and recall of
information.
* work together as an
anatomical network
to send feedforward
information to limbs.
* cross-talk between
networks.
Interacting Neural Systems and
Crosstalk: an “inconvenient truth”
Notes on Passive Sensing
(according to me)Passive sensing in the context of moving a limb towards a target:
* uncontrolled manifold hypothesis (Domkin et.al,
Experimental Brain Research, 163(1), 2005). Arm
has many DOFs with which it can potentially
reach an object.
* no finite sensory envelope, dynamic opposition
of forces from environment determine manifold
for movement.
* lots of behavioral variability as compared
with orthogonal manifold (set of solutions
chosen by CNS).
* scaling (geometry) of limbs important to
constrain what functional manifolds look
like in adulthood (also limits mathematical
solutions for SI|LS x MP).
* motor primitives in spinal cord (see Mussa-Ivaldi and Arbib) – combinatorially
put together to drive outputs based on current environmental demands.
Bayesian-Systems Model of
Adaptation via Molecular Pathways
A preliminary “model” of signal
transduction in a cell w.r.t. motor
performance (mechanotransduction
and control).
* expression of genes in tissues ~
properties of tissues. Each set of
relationships for single cell, many of
these in parallel ~ tissue.
* may be able to approximate emergent
changes in tissues ~ changes in
performance, morphological adaptation
(ability to encode adaptive changes).