Optical Motion Capture

Bobby Bruckart

Ben Heipp

James Martin

Molly Shelestak

Outline• History

– Types of Motion Capture– Development of Motion Capture

• State-of-the-Art– Hardware / Software– Data Collection Algorithms– Pros & Cons

• Future– Possible solutions to current problems– Possible future paths

Types of Motion Capture

• History

– Types of Motion Capture

– Development of Motion Capture

• State-of-the Art

• Future

1. Mechanical Motion Capture

2. Optical Motion Capture

3. Electromagnetic (magnetic) Motion Capture

Mechanical Motion Capture

• performer wears a human-shaped set of straight metal pieces• other types: gloves, mechanical arms, articulated models• pro:

– no interference from light or magnetic fields • con:

– the technology has no awareness of ground– equipment must be calibrated often. – does not know which way the performer's body is pointing– absolute positions are not known but are calculated from the rotations

Optical Motion Capture

• performer wears reflective dots that are followed by several cameras and the information is triangulated

• markers are either reflective, or infra-red emitting• developed primarily for biomedical applications • pro:

– performer feels free to move due to no cables connecting body to the equipment– larger volumes possible– more performers are possible– very clean, detailed data

• con:– it is prone to light interference– occlusion– rotations of body parts must be solved for and are not absolute – performer must wear a suit with dots and balls– information has to be post-processed or 'tracked' before viewing – comparatively high cost

Electromagnetic Motion Capture

• performer wears an array of magnetic receivers which track location with respect to a static magnetic transmitter

• First use: military• often layered• pro:

– positions are absolute, rotations are measure absolutely– can be real-time – relatively cheaper than optical

• con:– magnetic distortion– data prone to noise and interference– performers wear cables connecting them to a computer– relatively low sampling speed

Development

• History

– Types of Motion Capture

– Development of Motion Capture

• State-of-the Art

• Future

Early 1980s – Beginning of optical systems

• MIT Architecture Machine Group & New York Institute of Technology Computer Graphics Lab experimented with optical tracking of the human body

• few cameras and markers with limited success.

• created virtual stick-figure marionette

1985-1988 Jim Henson Productions

• ability to control the position and mouth movements of a low resolution character in real-time

• Waldo C. Graphic

1988: deGraf/Wahrman “Mike the Talking Head”

• shows off the real-time capabilities

• driven by a specially built controller

• The Silicon Graphics hardware provided real-time interpolation between facial expressions and head geometry as controlled by the performer

1989 Kleiser-Walczak

• Dozo• used optically-based solution

from Motion Analysis (multiple cameras to triangulate the images of small pieces of reflective tape placed on the body)

• resulting output is the 3-D trajectory of each reflector in the space

1992 - SimGraphics

• facial tracking system called a "face waldo.“ • could track the most important motions of

the face and map them in real-time onto computer puppets

• Important: one actor could manipulate all the facial expressions of a character by just miming the facial expression himself

SIGGRAPH 1993: Acclaim

• realistic and complex two-character animation done entirely with motion capture

• developed a high-performance optical motion tracking system able to track up to a 100 points simultaneously in real-time

• used the system to generate character motion sequences for video games

Optical Motion Capture

• History

• State-of-the Art

– Hardware / Software

– Data Collection Algorithms

– Pros & Cons

• Future

Data Acquisition

Optical Motion Capture

• History

• State-of-the Art

– Hardware / Software

– Data Collection Algorithms

– Pros & Cons

• Future

Data Translation

Optical Motion Capture

• History

• State-of-the Art

– Hardware / Software

– Data Collection Algorithms

– Pros & Cons

• Future

Data Implementation

Data Collection Algorithms

• History

• State-of-the Art

– Hardware / Software

– Data Collection Algorithms

– Pros & Cons

• Future

• Cameras placed around center of room

• Recording at 320x240 @ 15fps

• Cameras calibrated into common coordinate system

• Voxel-based volume rendered through a space-carving technique

• Also uses color-consistency to enhance quality

Data Collection Algorithms

• History

• State-of-the Art

– Hardware / Software

– Data Collection Algorithms

– Pros & Cons

• Future

• Run a quick test to determine if the point lies on, in, or outside the shape

• Moment analysis is then used to determine other information about the point

• The fitting error is then found

• A split and merge technique is used to recreate the volume

Data Collection Algorithms

• History

• State-of-the Art

– Hardware / Software

– Data Collection Algorithms

– Pros & Cons

• Future

• The whole voxel volume is approximated as one ellipsoid

• It is subdivided into two ellipsoids so D is less than some threshold value

• A new voxel set is created from a pairing of voxel sets and neighboring sets

• A novel ellipsoid is paired with this new set

• D is computed, the smallest D value is used to replace the two ellipsoids its replacing

• This is performed only at the first time step to reduce error in merging

Data Collection Algorithms

• History

• State-of-the Art

– Hardware / Software

– Data Collection Algorithms

– Pros & Cons

• Future

• Problems arise when there are differing numbers of ellipsoids from one time step to the next

• ellipsoid sets are merged

Data Collection Algorithms

• History

• State-of-the Art

– Hardware / Software

– Data Collection Algorithms

– Pros & Cons

• Future

• To identify rigid body structures, it must be known that the mutual Euclidean distance between two points will not change if they are in the same rigid body

• The paths through the time steps are compared in an iterative fashion until all ellipsoids are handled

Data Collection Algorithms

• History

• State-of-the Art

– Hardware / Software

– Data Collection Algorithms

– Pros & Cons

• Future

• Two adjacent rigid bodies found in the previous step are used to sample points along any of the three major axes which lie on this ellipse

• The number of point samples increases along all three axes simultaneously until the average 3D location of these points lies within the adjacent ellipse (or vice versa)

Pros of Optical Motion Capture

• History

• State-of-the Art

– Hardware / Software

– Data Collection Algorithms

– Pros & Cons

• Future

Pros

Cons of Optical Motion Capture

• History

• State-of-the Art

– Hardware / Software

– Data Collection Algorithms

– Pros & Cons

• Future

Cons

Current Issues

• History

• State-of-the Art

• Future

– Possible solutions to current problems

– Possible future paths

 1. Geometric Dissimilarity

Make human-shaped data work on one of these characters, without introducing effects of mismatched proportions

Current Issues

• History

• State-of-the Art

• Future

– Possible solutions to current problems

– Possible future paths

2.  Different movement qualities in performers and cartoon characters

Ability to adjust the speeds of capture to simulate more cartoon-like movements when capturing natural human movement 

Current Issues

• History

• State-of-the Art

• Future

– Possible solutions to current problems

– Possible future paths

 3. Increasing data collection

abilityMake motion capture easier to process with increased technology in the current software

Current Issues

• History

• State-of-the Art

• Future

– Possible solutions to current problems

– Possible future paths

4.  Increased number of performers which can be captured simultaneously

Ability to capture groups of performers with out reducing

the image quality

Future Paths

• History

• State-of-the Art

• Future

– Possible solutions to current problems

– Possible future paths

enhancement of performanceconditions through lack of tethering and simplification of performanceapparel

increased speed of the technology

increased 'volume' or area in which performances can be captured

lower cost, so that consumers and independent artists can have access& experiment / expand the technology

Future Paths

• History

• State-of-the Art

• Future

– Possible solutions to current problems

– Possible future paths

  increased accuracy of the

results, including improved

physical abilities, so that

characters can touch each other

and feet meet solidly on the ground

greater ability to capture data

from multiple characters

combination of virtual reality and

existing motion capture technologies

could aid in technological development

Discussion Questions

1. Should using motion capture for animation be considered art?

2. Could motion capture be used to create realistic human motion in robotics?

3. What would be some other ways to capture motion? Audio?

4. Four