Learning by imitation:Computational Modeling And Robotics

Aude Billard

Computer Science DepartmentProgram of Neuroscience

University of Southern CaliforniaLos Angeles

Robot Learning By Imitation

Teaching a robot complex motor skills by demonstration

What Does It Take To Imitate?

• What should we imitate?

• Which features of the action are relevant?

• What should we pay attention to?

Finding the goal of the action

Grasping an object

Relevant Features: Hand-Object relationship

To what extend is biological inspiration useful?

Is a model of human imitation useful for robotics?

The imitator robot

Should the robot have the same body configuration?

EPFL, ASL Pygmalion RobotLausanne, Switzerland

Kawato Erato ProjectATR, Kyoto, JapanHumans and robots have different body dynamics

Learning by imitation: Motivations

Robotics: A means of transmitting motor skills - Coordinated behavior, implicit attentional mechanism - Natural means of interaction - No need of explicit programming

Biology: Computational Neuroscience - Abstract model of primate ability to imitate

- Neural mechanisms behind learning by imitation

- Cut down the debate concerning imitation

RoboticsMotion Studies

Computational Modeling

Imitation Learning

Modeling Implementing

From Human To Robot

Motion studies on human imitation

Recording and analysis of kinematics of full body motion

Collaborations:

James Gordon, Department of Biokinesiology & Stefan Schaal, CS dept, USC Steve Boker, Univ. of Notre Dame, Indiana

Visual and motor representation of movements:

Sensitivity to body cues Orientation of body, direction of limb motion Eccentric versus intrinsic space Tracking hand path versus joint angles

Motion studies on human imitation In collaboration with James Gordon, Department of Biokinesiology, USC

Steve Boker, Univ. of Notre Dame, Indiana

Hypotheses:

Imitation is based on a hierarchy of goals.It can be goal-directed, exact, partial.

The metric is task-dependent.

Bekkering, Wholschlager & Prinz, Psycholoquia 2000

Question I : What are the metrics behind imitation?

Question II: What mechanisms are behind the immediate body to body mapping?

Hypothesis

Biological clues: body symmetries, limb orientation

Results:

Bias for mirror imitation. Two transformations of frames of reference only.

Hypothesis:We have a model of the human body kinematics and dynamics

Question III: How do we recognize biological motions from non biological ones?

Results:

Hand path only is too ambiguous an information to reconstruct completely the motion.

Question IV: Which representation of movement?

HypothesesReconstruction is based on a model of natural motion

Basic, primitive patterns of motions: Coupled oscillation of limbs

Results:

No significant effect of display type on performance.

Poor performance on in-phase/anti-phase patterns: bimanual coordination

Non leading arm tends to produce the closest preferred pattern

Motion studies on human imitation

Experiments: 1. Goal-directed: grasping, kicking an object2. Functional: Tying shoes, stacking boxes3. Abstract: dance, highly skilled motion

Imitation Learning

Learning new motions requires both eccentric and intrinsic information, as well as information on amplitude, speed, acceleration.

Task-dependent method of analysis1. Eccentric: End-point trajectories

Principal Component Analysis

2. Egocentric: joint trajectories Cross-correlation, phase shift

RoboticsMotion Studies

Computational Modeling

Imitation Learning

Modeling Implementing

From Human to Robot

Neural mechanisms behind learning by imitation Hypotheses: 1. Common parametrisation to visual and motor systems

- Body-centered reference frame

- Coding of mvt in orientation, amplitude and speed

- Mirror Neurons: visuo-motor mapping

2. Dynamic learning of motor commands

- Coarse coding of information, movement sequence

- Adaptation and combination of basic motor patterns

Frontal Lobe:Decision CenterInhibition of motion

Pre-Motor Cortex / Broca’s area: Visuo-motor transformation / Mirror Neurons

Temporal Lobe (STS):Eccentric – Intrinsic visualRepresentation of movement

Parietal Lobe:Eccentric visual coding

Cerebellum:Timing, Sequencing

SMA:Sequence learning

Spinal Cord + Brain Stem:Basic motor patterns, CPGLocomotion, Reflexes

Motor CortexSomatotopic control

High-Level representation of the brain mechanisms underlying imitation

Functional and abstract model of the brain areas and their connection

Cerebellum:Timing, Sequencing

Frontal Lobe:Decision CenterInhibition of motion

SMA:Sequence learning

Pre-Motor Cortex: Visuo-motor transform

Spinal Cord Basic motor patterns, CPG

Motor CortexSomatotopic control

Temporal Lobe (STS):Eccentric – Intrinsic visual

Parietal Lobe:Eccentric visual coding

High-Level representation of the brain mechanisms underlying imitation

Functional and abstract model of the brain areas and their connection

Brain Stem

Schematic of the model

Segmentation

Joint Angle

Filtering of Small Motions

Neural Output

Visual Processing

Motor Control

Leaky-integrator neurons

Spring and Damper Muscle Model(Lacquaniti & Soechting 1986)

Flexor-extensor pair per degree of freedom (DOF)

41 DOFs simulator, 30 DOFs humanoid robot

Visuo-Motor Transformation

Visual Module

Learning Module

Motor Module

Fixed transformation

First order approximation of inverse dynamics

Visuo-Motor Module

Example: Imitating Human Arm Motions

Imitation of Gestures

Gesture 1 Gesture 2 Gesture 3

•Recurrent NN•Sequence learning•Generalization across movements

DRAMA: Dynamical Recurrent Associative Memory Architecture

- Fully recurrent NN with self connections on each unit

- Time-delay neural network: Learning of complex time series and of spatio-temporal invariance - Hebbian Learning: on-line and on-board robot learning

DRAMA: Unit Activation function

Decay of activityInput

Thresholds

Thresholds

CLMC-LAB

Learning of a dance movement sequence

Human DemonstrationReplay of Recordings

2nd pattern1st pattern

Learning actions: one by one

Joint by joint segmentation

3rd action pattern

CLMC-LAB

Learning of a dance movement sequence

1st pattern

Learned actions lead to the following postures

Learned Posture 1

2nd pattern

Learned Posture 2

Learning sequences of actions

ActionE

ActionC

ActionD

ActionB

ActionA

Learned Motor Programs

ActionA

ActionC

ActionE

ActionB

ActionD

Learning sequences of actions

ActionE

ActionC

ActionD

ActionB

ActionA

ActionA

ActionC

ActionE

ActionB

ActionD

Fully Recurrent NNConnectivity is built on-line

Time-delay neural networkLearn sequencing and timing of the action sequence

Imitate sequences of actions

ActionE

ActionC

ActionD

ActionB

ActionA

ActionA

ActionC

ActionE

ActionB

ActionD

Improvise using the learned sequences of actions

ActionE

ActionC

ActionD

ActionB

ActionA

ActionA

ActionC

ActionE

ActionB

ActionD

Randomly activate or shut down nodes to produce new action sequences

Modeling: Summary

• Data Segmentation: Finding the key features of motion

Change in speed and orientation, joint-based representation

• Common parametrization of movements: visual and motor systems Speed and direction of movement, joint-based representation

• Reconstruction of movements: robustness against perturbation

• Learning of actions: synchronous and sequential activations of limbs

• Recombination of basic movements: improvisation

RoboticsMotion Studies

Computational Modeling

Imitation Learning

Modeling Implementing

From Human to Robot

Robota

Robota Clever Toy and Educational Toy

Robota

First Prototype

Univ of Edinburgh, 1998

Second Prototype

LAMI - EPFL, 1999

In collaboration with Jean-Daniel Nicoud and Andre Guignard

Robota – The Product

DIDEL SA, Switzerland Jean-Daniel Nicoud, director of DIDEL SA

ROBOTA: TECHNICAL SPECIFICATIONS

Power Supply Connector

Left Leg Motor

Head Motor

Right Arm Motor

Left Arm Motor

Right Leg Motor

Head Potentiometer

Right and Left Arm Potentiometers

Fixation points for the stand

ROBOTA: TECHNICAL SPECIFICATIONS

MOTOR BOARD: RS232 Connector to PC

MOTOR BOARD: RS232 connector to Kameleon Board + Jumper Selector

SENSOR BOARD: SPI connector across Motor and Sensor Board

SENSOR BOARD: Spare SPI Connector

SENSOR BOARD: Buzzer

SENSOR BOARD: 3x 2-pin connector for head, arms and legs switches

SENSOR BOARD: Connector for 2x 2-pin analog input (e.g. IR RX)

SENSOR BOARD: Connector for 2x 3-pin analog output (e.g. IR TX)

ROBOTA: TECHNICAL SPECIFICATIONS

Fixation points on Motor Board

Fixation points on Battery board

Fixation points on Kameleon Board

Robota battery set attaches to the back of the Motor Board7.2V,7x1.2NiCd

Robota at the Museum

La Cite des Sciences et de l’Industrie, French National Science Museum

November 2001 – July 2003

Robota: Applications I

French National Science Museum - La Cite des Sciences et de L’industrie

LANGUAGE GAME

Robota: Applications III

Aurora Project Dr. Kerstin Dautenhahn,

Univ. of Hertfordshire

www.aurora.com

Center for Fundamental Infant Development

Drs Demuth, Pena, Bradley, Turman

USC Dept of Biokinesiology and Physical Therapy

USC Premature Infant Follow-Up Pediatric Clinic

Mechatronics: Programming Humanoids Robots

USC – CS499

Undergraduate Computer Science Degree (4th Year)

30 Students. Equipment (Robots + PCs) supported by a «Innovative Teaching » grant from Intel Corp.

ROBOTA Class SyllabusAssignment 2

“Conversational Robot"

 

ROBOTA Class SyllabusAssignment 4

“Imitator Robot"

Robota drives a car

Carbota

 

Robota leg motors are wired to the remote control car motors:

Kameleon 376 Processor Board (K-Team) is attached to the front of Robota

Carbota

          

So pretty! Robota sitting in the remote controlled car.

Final projects

Robota Counting Game Robota Learns to Dress up

Robota Does Cross-Country skiing

ROBOTA UserGuide

Webots-Robota : The SimulatorWebots is a product of Cyberbotics (www.cyberbotics.com)

RoboticsMotion Studies

Computational Modeling

Imitation Learning

Modeling Implementing

From Human to Robot

Summary

Biology Robotics

Take inspiration from Biology and Psychology to design robots

which can learn from interacting with humans and with other robots

Keywords: Learning, human-robot and multi-robot interactions

To use robotic tools (robots and realistic simulations) to build biologically plausible, computational models of animal ability.

Keywords: Computational neurobiology, biologically inspired robotics

Biology Robotics