CS 99k: Digital ActorsCS 99k: Digital ActorsURL: http://robotics.stanford.edu/~latombe/cs99k/2000URL: http://robotics.stanford.edu/~latombe/cs99k/2000

Instructor: Jean-Claude LatombeInstructor: Jean-Claude Latombe

Computer Science DepartmentComputer Science Department

Stanford UniversityStanford University

Computer Animated CharactersComputer Animated Characters

A Bug’s Life (Pixar/Disney) Toy Story (Pixar/Disney)

Tomb Raider 3 (Eidos Interactive) Final Fantasy VIII (SquareOne)The Legend of Zelda (Nintendo)

Antz (Dreamworks)

Autonomous AgentsAutonomous Agents

DefinitionDefinition::

““An An autonomous agentautonomous agent is a system situated within is a system situated within and a part of an environment that senses that and a part of an environment that senses that environment and acts on it, over time, in pursuit of environment and acts on it, over time, in pursuit of its own agenda and so as to effect what it senses in its own agenda and so as to effect what it senses in the future.”the future.”

Franklin & Graesser, 1996“Is it an Agent, or just a Program?” Proc. of 3rd Int. Workshop on Agent Theories

Animated Graphic AgentAnimated Graphic Agent

Basic definition:Basic definition: Animated character whose motion Animated character whose motion does not need to be explicitly defined by a user.does not need to be explicitly defined by a user.

??

Other commonly-used terms:Other commonly-used terms:

Digital Actor, Virtual HumanDigital Actor, Virtual Human

Ultimate Goal:Ultimate Goal: Fully-autonomous Fully-autonomous animated agents able to sense, act, animated agents able to sense, act, and interact in changing virtual and interact in changing virtual environmentsenvironments

Goals vsGoals vs.. Motions Motions

Autonomous Goal Generation:Algorithm to decide what the charactershould do next (specifies high-level goal)

Autonomous Motion Generation:Algorithm to generate underlying motion given a high-level goal

Example: Moving an ObjectExample: Moving an ObjectReachReach

GrabGrab

TransferTransfer

ReleaseRelease

ReturnReturn

Two Classes of MotionTwo Classes of Motion

Primary (Active) Motion: gross body movements, facial animation

Secondary (Passive) Motion: clothes, skin, hair, env. effects

Virtual Clothes (MIRALAB / Univ. of Geneva)

Facial animation (Geri’s Game / Pixar)

Endgame (Stanford)Endgame (Stanford)

Autonomy vs. InteractivityAutonomy vs. Interactivity

Autonomy

Interactivity

User-controlled

CompletelyAutonomous

Off-line Real-time

Key-frameAnimation

Systems

Video GameAdversaries

WebAvatars

DigitalActors

Semi-AutonomousCharacters

Urban / FactorySimulation

Virtual TourGuides

HistoricalNarratives

1999 1999 Interactive Chess (Stanford)Interactive Chess (Stanford)

Playing ChessPlaying Chess

1994 1994 ENDGAME (Stanford)ENDGAME (Stanford) 1997 Geri’s Game (Pixar)

Animation VariablesAnimation Variables

q2

q1

q3

q0

qn

q4

Q(t)

)(

)(

)(0

tq

tq

tQ

n

Tt ,0

Parts DOFL 19 68H 51 118

Motion Generation TechniquesMotion Generation Techniques

Synthetic Characters (MIT)

Blumberg, et. Al.

Reactive BehaviorsBrooks ‘85, Reynolds ‘87, Tu & Terzopoulos ‘94, Reich, et al. ‘94, Blumberg & Galyean ‘95 Marilyn (EPFL/Miralab)

Thalmann, et. Al.

Artificial Fish (U. Toronto)

Tu / Terzopoulos

ScriptingKahn ‘79, Ridsdale ‘87, Haumann & Parent ‘88, Wilhelms ‘90, Strassmann ‘91, Bates, et al. ‘92, Perlin & Goldberg ‘96

OZ project (CMU)

Bates, et. Al.

Kinematic WarpingBruderlin & Calvert ‘89, Witkin & Popovich ‘95, Unuma, et al. ‘95, Gliecher ‘98

Motion Generation TechniquesMotion Generation TechniquesPhysically-Based SimulationArmstrong & Green ‘85, Hahn ‘88, Baraff ‘89, McKenna & Zeltzer ‘90, Raibert & Hodgins ‘91, Mirtich ‘95, Ngo & Marks ‘93, Sims ‘94, Metaxas, et al. ‘97

Motion PlanningLengyel, et at. ‘90, Ching & Badler ‘92, Koga, et al. ‘94, Bandi & Thalmann ‘97, Hsu & Cohen ‘98

Human Dynamics (GaTech)

Hodgkins, et. Al.

HFAP (Microsoft)

Cohen, et. Al.

ENDGAME (Stanford)

Koga, et. Al.

Spacetime ConstraintsIsaacs & Cohen ‘87, Witkin & Kass ‘88, Rose, et al. ‘96, Gleicher ‘98, Popovic & Witkin ‘99

Jack (U Penn)

Badler, et. Al.

Motion Planning for Digital ActorsMotion Planning for Digital ActorsManipulationManipulation

Sensory-based locomotionSensory-based locomotion

ApplicationsApplications

VideogamesVideogames

MoviesMovies

ErgonomicsErgonomics

TrainingTraining

Web page animationWeb page animation

Slide showsSlide shows

E-commerceE-commerce

Course OutlineCourse Outline

Class 1:Class 1: Introduction + Motion Representation Introduction + Motion Representation Class 2:Class 2: Digital Actor Authoring and Runtime Digital Actor Authoring and Runtime

Software (Motivate)Software (Motivate)Guest speaker: Yotto KogaGuest speaker: Yotto Koga

Class 3:Class 3: Geometric/Kinematic/Dynamic modeling Geometric/Kinematic/Dynamic modeling of a Digital Actorof a Digital Actor

Class 4:Class 4: Motion Generation Techniques Motion Generation TechniquesGuest speaker: Chris BreglerGuest speaker: Chris Bregler

Class 5:Class 5: Motion Planning Motion PlanningGuest Speaker: Oussama KhatibGuest Speaker: Oussama Khatib

Course Outline (cont’d)Course Outline (cont’d)

Class 6:Class 6: Animation of Soft Tissues Animation of Soft TissuesGuest speakers: Joel Brown and Stephen SorkinGuest speakers: Joel Brown and Stephen Sorkin

Class 7:Class 7: Digital Actors with Sensing and Memory Digital Actors with Sensing and MemoryGuest speaker: Josh TenenbaumGuest speaker: Josh Tenenbaum

Class 8:Class 8: High-Level Behaviors and Man-Machine High-Level Behaviors and Man-Machine InteractionInteraction

What is a Motion?What is a Motion?

What is a Motion ?What is a Motion ?

Definition of an Object’s ConfigurationDefinition of an Object’s Configuration

The The configurationconfiguration of an object is a specification of of an object is a specification of the positions of all the points in this object relative the positions of all the points in this object relative to a fixed coordinate systemto a fixed coordinate system

Usually it is expressed as a “vector” of position Usually it is expressed as a “vector” of position and orientation parametersand orientation parameters

qq11 qq22

qq33

qqnn

q=(qq=(q11,…,q,…,qnn))

Rigid Object ExampleRigid Object Example

xx

yy

reference pointreference point

robotrobotreference directionreference direction

workspaceworkspace

Object’s configuration is: q = (x,y,Object’s configuration is: q = (x,y,))

In a 3-D workspace q would be of the form (x,y,z,In a 3-D workspace q would be of the form (x,y,z,))

Articulated Object ExampleArticulated Object Example

qq11

qq22

q = (qq = (q11,q,q22,…,q,…,q1010))

Configuration Space of a Moving ObjectConfiguration Space of a Moving Object

Space of all its possible configurationsSpace of all its possible configurations

The topology of this space is usually not that of a The topology of this space is usually not that of a Cartesian spaceCartesian space

C = S1 x S1

Case of a Rigid Robot in 3-D WorkspaceCase of a Rigid Robot in 3-D Workspace q = (x,y,z,q = (x,y,z,))

Other representation: q = (x,y,z,rOther representation: q = (x,y,z,r1111,r,r1212,…,r,…,r3333) where r) where r1111, r, r1212, …, r, …, r3333 are are the nine components of a 3x3 rotation matrixthe nine components of a 3x3 rotation matrix rr1111 r r1212 r r1313

r r2121 r r2222 r r2323

r r3131 r r3232 r r3333

with:with:

– rri1i122+r+ri2i2

22+r+ri3i32 2 = 1= 1

– rri1i1rrj1 j1 + r+ ri2i2rr2j 2j + r+ ri3i3rrj3j3 = 0 = 0– det = +1det = +1

the configuration space is a the configuration space is a 6-D space6-D space (manifold) (manifold) embedded in a 12-D Cartesian space. It is denoted by embedded in a 12-D Cartesian space. It is denoted by RR33xSO(3)xSO(3)

Parameterization of SO(3)Parameterization of SO(3) Euler angles:Euler angles: ( (

Unit quaternion:Unit quaternion: (cos (cos nn11 sin sin /2, n/2, n22 sin sin /2, n/2, n33 sin sin /2)/2)

xxyy

zz

xx

yy

zz

xx

yy

zz

xx

yy

zz

Metric in Configuration SpaceMetric in Configuration Space A A metricmetric or or distancedistance function function dd in C is a map in C is a map

d: (q d: (q11,q,q22) d(q) d(q11,q,q22) ) >> 0 0such that:such that:

– d(qd(q11,q,q22) = 0 if and only if q) = 0 if and only if q11 = q = q2 2

– d(qd(q11,q,q22) = d (q) = d (q22,q,q11))

– d(qd(q11,q,q22) ) << d(q d(q11,q,q33) + d(q) + d(q33,q,q22)) Example:Example: Given an object Given an object AA and a point x of and a point x of AA, let x(q) be , let x(q) be

the point of the workspace occupied by x when the robot is the point of the workspace occupied by x when the robot is at configuration q. at configuration q. A distance d is defined by:A distance d is defined by: d(q,q’) = max ||x(q)-x(q’)|| over all points x ofd(q,q’) = max ||x(q)-x(q’)|| over all points x of A Awhere ||a - b|| denotes the distance between points a and bwhere ||a - b|| denotes the distance between points a and b

A Specific Example: Metric in A Specific Example: Metric in RR22 x S x S11

Let q = (x,y,Let q = (x,y,) and q) and q’’ = (x = (x’’,y,y’’,,’’) with ) with and and ’’ in [0,2 in [0,2))

d(q,qd(q,q’’) = sqrt[(x-x) = sqrt[(x-x’’))22 + (y-y + (y-y’’))22 + + 22]]where where = min{|= min{|--’’| , 2| , 2-|-|’’|}|}

d(q,qd(q,q’’) = sqrt[(x-x) = sqrt[(x-x’’))22 + (y-y + (y-y’’))22 + ( + (]]

where where is the maximal distance between the reference is the maximal distance between the reference point and a robot pointpoint and a robot point

’’

Notion of a Path Notion of a Path

A A pathpath in C is a piece of in C is a piece of continuouscontinuous curve connecting two curve connecting two configurations q and q’:configurations q and q’: : s in [0,1] : s in [0,1] (s) in C(s) in C

Other possible constraints: finite length, smoothness, …Other possible constraints: finite length, smoothness, …

A A trajectorytrajectory is a path parameterized by time is a path parameterized by time

q1

q3

q0

qn

q4

q2

(t)

Homotopic PathHomotopic Path

Two paths with the same endpoints are Two paths with the same endpoints are homotopic homotopic if one can be continuously deformed into the otherif one can be continuously deformed into the other

R x SR x S11 example: example:

Paths Paths 11 and and 22 are homotopic are homotopicPaths Paths 11 and and 33 are not homotopic are not homotopicInfinity of Infinity of homotopic classeshomotopic classes

qq

q’q’

Configuration Space ConnectednessConfiguration Space Connectedness

C is C is connectedconnected, meaning that every two , meaning that every two configurations can be connected by a pathconfigurations can be connected by a path

C is C is simply-connectedsimply-connected if any two paths connecting if any two paths connecting the same endpoints are homotopicthe same endpoints are homotopicExample: Example: RR22 or or RR33

Otherwise C is Otherwise C is multiply-connectedmultiply-connectedExamples: SExamples: S11 and SO(3) are multiply connected and SO(3) are multiply connectedIn SIn S11, infinity of homotopic classes, infinity of homotopic classesIn SO(3), only two homotopic classesIn SO(3), only two homotopic classes

Obstacles in Configuration SpaceObstacles in Configuration Space

A configuration is A configuration is collision-freecollision-free, or , or freefree, if the , if the robot placed at this configuration has no robot placed at this configuration has no intersection with the obstacles in the workspaceintersection with the obstacles in the workspace

The The free spacefree space is the set of all free configurations is the set of all free configurations

A A C-obstacle C-obstacle is the set of all configurations where is the set of all configurations where the robot collides with a given workspace obstaclethe robot collides with a given workspace obstacle

A configuration is A configuration is semi-freesemi-free if the robot at this if the robot at this configuration touches obstacles without overlap configuration touches obstacles without overlap

Disc Robot in 2-D WorkspaceDisc Robot in 2-D Workspace

Rigid Robot Translating in 2-DRigid Robot Translating in 2-D

CB = B A = {b - a | a in A, b in B}CB = B A = {b - a | a in A, b in B}

Linear-Time Computation of C-Obstacle in 2-DLinear-Time Computation of C-Obstacle in 2-D

Rigid Robot Translating and Rotating in 2-DRigid Robot Translating and Rotating in 2-D

C-Obstacle for Articulated RobotC-Obstacle for Articulated Robot

A Remark on the Topology of the Free SpaceA Remark on the Topology of the Free Space

The robot and the obstacles in the workspace are The robot and the obstacles in the workspace are modeled as modeled as closedclosed subsets. This means that they subsets. This means that they contain their boundariescontain their boundaries

One can show that the C-obstacles are closed subsets One can show that the C-obstacles are closed subsets of the configuration space Cof the configuration space C

Consequently, Consequently, the free space F is an open subset of the free space F is an open subset of CC. Hence, . Hence, each free configuration is the center of a each free configuration is the center of a ball of non-zero radius entirely contained in Fball of non-zero radius entirely contained in F

The semi-free space is a closed subset of C. Its The semi-free space is a closed subset of C. Its boundary is a superset of the boundary of F boundary is a superset of the boundary of F

Free and Semi-Free PathsFree and Semi-Free Paths

A free path is one that lies entirely in the free A free path is one that lies entirely in the free spacespace

A semi-free path is one that lies entirely in the A semi-free path is one that lies entirely in the semi-free spacesemi-free space

Classes of Homotopic Free PathsClasses of Homotopic Free Paths