Download - Jenkins — Modular Perception and Control Brown Computer [email protected] — ROUGH DRAFT (05.19.2005) 1 Workshop Introduction: Modular Perception.

Jenkins — Modular Perception and Control

Brown Computer [email protected] — ROUGH DRAFT (05.19.2005)

1

Workshop Introduction:Modular Perception and Control for Autonomous Robotics

Chad JenkinsAssistant Professor

Computer Science DepartmentBrown University

Robotics 2005 Workshop on Modular Foundations for Perception and Control

June 11, 2005Cambridge, MA USA



2

Common Background

● Brooks’ Subsumption Architecture introduced the use of modular components for autonomous robot perception and control during the mid-1980’s

● Since then, modularity has been a crucial aspect of robotics systems, including:– Behavior-based and hybrid autonomous control

● Trajectory formation, manipulation, robot teams– Parsimonious state spaces

● Reinforcement learning and decision making– Human-robot interaction

● Gesture recognition and imitation learning



3

Common Background

● Modularity is typically cast in terms of behaviors or models at two levels of abstraction:– Primitive modules that encode basic low-level behavior– Compositional modules that compose existing modules

to provide greater functionality● Through modularity, we are able to

– Limit the cost of searching for high-DOF systems– Respond promptly in potentially dynamic environments– Provide scalability over time in developing perception

and control systems



4

A General View for Primitive Modules

Low-level Sensing

Modular Vocabulary of

Primitives

Motor-level Actuation

Set desired motion trajectory

Task-level Controller

Internal Planner

Human Communication

Encode observed motion into vocabulary

Interact through common vocabulary

Plan using modular capabilities

Execute modules to achieve objectivesTask-level

Skill-level

Motor-level

CompositionalModules

Compose existing modules into new functions



5

The Greater Challenge for Modularity

● These issues get to a greater questions:– Why do roboticists continually reinvent modular

architectures and components for practical applications/platforms?

● Programming languages analogy:– Java code is platform independent– Any computer with a virtual machine can run Java code– Libraries are reused to construct greater functionality

● Should this be our goal for robotics?– Obviously, our problem is harder



6

Underlying Issues

● To this point, there is little unified consensus the following four interrelated issues– Principles for modular design and development– General module representation– Common interface structure– Refinement of modules over time



7

Issue 1: Modular Development

● What modules are necessary?– Size, scope, and interaction among modules

● How should the modules be developed?● Robotics has not adopted a general methodology

for designing and developing modular controllers. – Instead, ad hoc decisions or biased domain knowledge

is applied to a specific implementation. – I believe modules should be learned in a data-driven

fashion from the real-world.



8

Issue 2: Module Representation

● Is there a general formulation for encoding a wide spectrum of primitive modules?

● Can compositional operators be defined for sequencing and superposition?

● Can we avoid coding limited-purpose modules or learning overly specific policies?



9

Issue 3: Common Interface Structure● Ideally, modules are used in a wide variety of applications

– Neural prosthesis, humanoid robots, dynamics-based interactive entertainment, human-robot teams, assistive robotics, robotic telepresence, smart spaces

– Involving a variety of problems: low-bandwidth communication, mixed-initiative interaction, user modeling, gesture recognition, distributed inference, exteroceptive information, attention/saliency

● Modules should be generally applicable and not engineered for specific objective

● Is there a general interface structure capable of expressing the output of a wide spectrum of modules?– Avoiding complex data structure for handling many special cases

● Some architectures exist, but are domain specific– Player/Stage (mobile), Orocos (industrial), Yarp (humanoid)



10

Issue 4: Refinement over time

● Aspects of control and perception change over a robot’s useful lifetime.

● How amenable are modules to modification?● Can modules be added or removed without

complication?● Which modules should be refined and when?

– Should primitives be developed and then remain static?● How much to the primitives need to account for

subtle changes to the robot’s dynamics over time?



11

My Approach● Prediction is the key element

– Modules provide the expectation for future states● Learn predictive modules from data

– Categorize and generalize in an unsupervised fashion to a NLDS– Gradient field s.t. \hat{x}_m[n+1] = f_m(x[n],u[n]) ; y[n] = g(x[n])

● Predictions directional vectors in state space– Applications access modules at each time step for control or

perception tasks● Learned modules are exemplar-based

– Modifying an example redefines the behavior of the module● Currently working on composition operators and modules



12

...

Develop training programs

Human performance

Matrix Analogy: Jujitsu

Punch

Stand ready

Kick

Jump

Download to human

Capture data from performance Learn behaviors

Human-likerobot control

Enhanced human performance

Perception of human activity