Computer Animation Where we are (overview) Where we are going (perhaps)

Computer Animation

Where we are (overview)

Where we are going (perhaps)

Animation overview

Computer Animation

Popular perception - CGI is animation (full length animations, CGI effects films or computer games).

Animation overview

Off-line/pre-recorded Animation is expensive

Production ´effort´ same as handmade animation

The Fox and the Hound Toy Story (1)

Time 4 years 4 years (1.5 story + 2.5 production)

Frames 110,000 110,064

Time 2.9 hours/frame 45mins-24hrs/frame

Paint 450 gallons 110 SUNs

Animation overview

‘Real-time’ Computer Animation in Games

Animation control (script) in games is:

• pre-recorded (MOCAP) or pre-designed (currently the de facto standard in games)

• calculated in real time

(IK and dynamics)

• a mix of pre-recorded and real time

Game events

MoCap 1

MoCap 2

MoCap n

Animate skeleton skin Render

MoCap X

MoCap Yblend

Animation overview

MOCAP in Games is select and blend

For example a football game will have 200-300 sequences.

Animation overview

Script creation methods

Recording real motion (MOCAP) [1st=]

By ´hand´ using proprietary or in-house software, keyframe animation [1st=]

Posing real motion using a digital input device (DID) (film special effects)

Executing dynamic equations (scientific visualisation, computer games)

Behaviour models (film special effects)

Animación

(portero)

Animation overview

Script creation -Motion quality-best is MOCAP

z

x

y

Hombro = 3 DOFs

Animation overview


time

DOF angle

yxz

xz

yApplying MOCAP

to a skeleton

Animation overview


Animation overview

Motion Capture – quality motion is always perceivable as such (even with stick figures)

Animation overview

MOCAP-bones-skinning is a well-established technology

Animation overview

Script creation

By ´hand´ using proprietary or in-house software. The most popular method is keyframe animation.

Animation overview

Real time dynamics

Executing dynamic equations (computer games, scientific visualisation)

Flong = Ftraction+ Fdrag

a = F/m

v = v + dt*a

p = p + dt*v

Animation overview


High level behavioural models – original was “flocking”

Animation overview


Posing real motion - stop motion animation was used in Jurassic Park (Dinosaur Input Device) to script the computer models

Zajac 1966 Bell Telephone LabFirst Computer animation in science

Animation in Science

Animation overview

Max Born 1935


Animation overview

Animation overview


Max Born 1935 The Restless Universe

Muscle Fibres of the heart


Animation overview

Animation overview

Forensic Animation - ethics?

Animation overview

Forensic Animation – ethics?

Technology blesses the production with veracity?

Who controls the content of simulation?

How can the accuracy be guaranteed?

No cross examination possible

Provides a synthetic view of reality, constructed from a database, which cannot be seen because of, for example, weather conditions. The best example is civil aviation.

Principles used are exactlty the same as games where a view frustum is ‘driven’ through an environment under user control.

Synthetic vision

Animation overview

Synthetic vision in civil aviation

Animation overview

Cockpit view

Landing display


Animation overview

Uses as database

Shuttle Radar Topography Mission (SRTM)

Wide Area Augmentation System (WASS)

Local Area Augmentation System (LASS)

Derives 3D position (Accuracy < 1m) from

GPS + INS

On-board sensors (such as RADAR altimeters)


Animation overview

Animation of an approach


Animation overview

Animation overview

• Off-line -manual

• Combining off-line + event driven

•Event driven – dynamic simulations – walk throughs

Where we are

Animation overview

The future ?

Whats wrong with MOCAP• Although pre-reorded aninimation is of high quality, it is inherently

limited – the more complex the game the more clips are required.

• Cannot MOCAP animals.

• MOCAP transitions – blending is unsatifactory

What we would like• Increase the quality of real-time animation and obtain any motion in

real time accoording to the ‘action demand’ – event driven

• Speech/emotion expression needs to be event driven

Game events

MoCap 1

MoCap 2

MoCap n

Animate skeleton skin Render

MoCap X

MoCap Yblend

Event driven animation for humanoids

What we have now – event driven recorded animation

This model can only react to completely pre-determined actions


What we have now- MOCAP – more general

One generic motion fits all

characters

Event driven animation for humanoidsWhy do we need it?

camera

speech recogn.

visualspeech

expressn

computer vision

NLP

textgeneratn.

emotiongeneratn.

querysystem

game

Important element in an anthropomorphic interface

Event driven animation for humanoidsWhat do we aim for

•Seems sensible to retain MOCAP technology – high quality, well established so increase its flexibility - adaptation

•BUT oranges are not the only fruit. Can we generate animation in real time.


Examples

•Using IK adapted MOCAP in human motion

•‘Total’ IK solution for human motion

•Using MOCAP in visual speech

•‘total’ solution for visual speech


Character adaptation not straight forward

Change scale joint angles change in non-linear manner

From Shin et al 2001

Event driven animation for humanoids Cheating for real-time

Use v.simple skeleton and complex skin.

C.G skeletons – 50 DOFs

human skeletons - >250 DOFs

Motion from skeleton, visual complexity from skin


MOCAP is forward kinematics

X

MOCAP Motion of end effector = f( )

= f(

Event driven animation for humanoids Inverse Kinematics – an old idea

jointspace

Cartesianspace

x

x = f ()

ForwardKinematics

= f-1 (x)

InverseKinematics

use for complete soln.

use to adapt MOCAP

Circa 1985

Event driven animation for humanoids Inverse Kinematics – solutions

• Geometric/Analytical: This class of solvers generate a solution in a single step for a given goal and therefore fast. They can be used as part of a solution in a hybrid method.

• Differential Algorithms: The task is transformed into a linear problem based on small changes using the Jacobian and iteratively refining the system to meet the goal position.

• Cyclic Co-ordinate Descent: An algorithm which again moves towards a solution in small steps. This time, however, the steps are formed heuristically.

• Hybrid Methods: Uses a combination of approaches. Their motivation is usually real-time performance.

Event driven animation for humanoids Differential IK – the Jacobian

The Jacobian is the multidimensional extension to differentiation of a single variable. Given a function:

X = f()

where X is of dimension n and of dimension m, the Jacobian J is the n x m matrix of partial derivatives relating differential changes of , to differential changes in X, written as:

dX = J()d d = J-1()dX

where the (i, j)th element of J is given by:

Jij = fi/j


Event driven animation for humanoids Differential IK - iteration

1. Calculate the incremental step X = Xgoal – X

2. Calculate the Jacobian matrix using the current joint angles

3. Calculate the inverse of the Jacobian – using right-hand generalised inverse if required; J-1 = JT(JJT)-1

4. Check for iterative convergence – i.e. make sure the Jacobian inverse is suitably accurate(a) If ||(I – JJ-1)|| > e, reduce X=X/2 and repeat 4 (where e is a convergence threshold)(b) If ||(I – JJ-1)|| > e after a number of steps then the goal is likely out of reach so terminate

5. Calculate the updated values for the joint angles where = J-1X

6. Using forward kinematics to determine whether the solution is close enough to the goal. If the solution is adequate then terminate iteration else go back to step 1 (as step 4 could have reduced X).


Event driven animation for humanoids Differential IK – example Jacobian

Determining the JacobianConsider: )(FX

)sin(sin),cos(cos( 2121121211 llllX


Event driven animation for humanoids Differential IK – example Jacobian

)cos(cos 21211 llfx

21

21

yy

xx

ff

ff

Jj

iij

fJ

)sin(sin 212111

llfx

)sin( 2122

lfx

2

1

21221211

21221211

)cos()cos()cos(

)sin()sin()sin(

lll

lll )(JX

)sin(sin 21211 llfy

where

)cos(cos 212111

llfy

)cos( 2122

lfy

Event driven animation for humanoids Differential IK – the Jacobian

• For large articulations the complexity of analytically expressing the differentiation is very tedious.

• The Jacobian can be viewed as expressing the velocity of the end of the chain in terms of local angular velocities with respect to a base frame.

• This information is easily extracted from transformation matrices that already exist in the graphics pipeline – i.e. the matrix concatenation of child-parent relationships as the articulation is built up.

• When the Jacobian is not square (whenever the number of DOFs in the chain increase past the dimension of the end-effector), a pseudo-inverse is required, which could lead to numerical error.

• Singularities – a decrease in the rank of the Jacobian can result in the loss of a degree of freedom that usually happens when the chain is fully extended

Event driven animation for humanoids Differential IK- main problem

• Underdetermined System:

The purpose of Inverse kinematics is to produce a set of joint angles that allows an end-effector to be positioned in a given location. This is an underdetermined system therefore many solutions exist.

•

•

Event driven animation for humanoids Differential IK- joint constraints

• Removal of redundant DOFs from the Jacobian

• Angular Constraints – Modification of step 5 of the iterative algorithm to include boundary constraints on specified DOFs

= lower bound if J-1P < lower bound

= upper bound if J-1P > upper bound

= J-1P otherwise

Event driven animation for humanoids Differential IK- demo

Unconstrained IK chain Constrained IK chain

001800190

-30230 -183-18

C:\Documents and Settings\UOS\Desktop\denmark\IKCons.exe

C:\Documents and Settings\UOS\Desktop\denmark\IKUnCons.exe

Event driven animation for humanoids Differential IK- MOCAP adaptation

Change scale joint angles change in non-linear manner

Event driven animation for humanoids Differential IK- MOCAP adaptation

Retargetting by simply scaling

Retargetting using IK constraints to maintain foot plants


Differential IK

Scaled Retargetting IK Retargetting to maintain foot plants

Event driven animation for humanoids Differential IK- total solution-walking

Stage Description

(a) (b)

Starting Configuration Left Foot Both heels and toes are planted on

the floor Toes are planted on the floor

Right Foot Toes are planted on the floor Heel is planted on the floor

Movement 1. Hips move forward, 2. Right heel is advanced

forward through the air, 3. Only the left toes remain

planted.

1. Hips move forward, 2. Right toes are gravitated

towards the floor, 3. Left toes remain planted to

the floor

Foot flight curve

(a) (b) (a)

Event driven animation for humanoids Differential IK- main problem


Facial Animation – two level model

• Apply motion to ‘bones’ which control the skin

• Motivation is identical – script applied to bones and bones control face vertices

• No. of bones 2-3 orders of magnitude less than face vertices


Facial Animation – two level model - muscles


MOCAP – concatenating

phonemes

visemes

Muscle values

Keys and

interpolate

Blahblahblahblahblah…………………………………

text


Facial Animation – muscles –problem?

Interpolating between static targets does NOT produce convincing mouth motion


Facial Animation – two level - bones


Facial Animation – bones –problem?

Incapable of particularly subtle expressions and so unsuitable for expressive speech


Facial Animation – visual speech

MOCAP can be used to:

1) Cure subtlety problem

2) Implement ‘general domain’ speech by concatenation

BUT

1) How do we retarget? Face changes both scale and shape

2) How do we concatenate motion? – will conventional blending work?


MOCAP – mesh control from sparse markers

MOCAP markers

SOFFD mesh from markers


MOCAP – mesh control from sparse markers

1) Position markers on reference mesh to define a control surface - 66+7 for head motion

2) Deform reference mesh to fit target mesh

1 2

3

3) Retargetted control surface

MOCAP – retargetting



Marker

motion

speech


MOCAP – for visual speech

• Can use variable length fragments (sentences, words or syllables)

• Overcomes the co-articulation problem

• Conventional blending seems to work



So is MOCAP speech the answer? NO

Because:

1) The inherent quality advantage derives from using variable length units (sentences, phrases, words) and this would demand masses of data for general domain speech.

2) Expressive speech? E.g combine a smile with an utterance.


Facial Animation – the return of static phonemes

text

phonemes

visemes

Muscle values

Keys and

interpolate

Blahblahblahblahblah…………………………………Phonemes/visemes as static a units are a good solution for general domain speech

Can we do better than interpolation?

• Treat V as a point in 13D space

• Assign a weight/dominance to each V

• For each unit (sentence…) find a global solution – a trajectory through this space

• Solution does NOT interpolate the means exactly


Facial Animation – constraint based global solution



Decreasing the dominance of the 4th segment reduces its effect over the entire trajectory

Acknowledgments/contacts

Mocap/inverse kinematics

[email protected]

Visual speech

[email protected]

mailto:[email protected]

mailto:[email protected]

Computer Animation Where we are (overview) Where we are going (perhaps)

Documents

Transcript of Computer Animation Where we are (overview) Where we are going (perhaps)