DIY Motion Capture12/5 Volumetric Society of NYC

How does the Kinect work?

Photo by y3rdua

Getting Started: Hardware

You will need:• A Kinect, or generic equivalent such as the Asus Xtion• USB adapter cable, if you have a Kinect that was

bundled with the Xbox (don’t buy the Microsoft adapter, the cheap ones work fine)• Enough space to perform

Getting Started: Software

OpenNI driver bundle:simple-openni.googlecode.com

KinectToPin:kinecttopin.fox-gieg.com

KinectToPinKinect Motion Capture for After Effects

What is KinectToPin?

A standalone motion capture application A UI panel for After Effects

+

What can you do with it?

Rig 2D character models in 3D

Setting up your space to record

6 – 10 feet

Capturing

• Record 15-point 3D skeletal tracking data• Capture in sync to dialogue• Remote control with a

standard presentation remote• Open source and cross-

platform: runs on OpenNI, built in Processing, also works with generic sensors like the Asus Xtion

Acting for Kinect

• The Kinect can’t record what it can’t see. • Keep your entire body in the camera’s range• Face the camera when capturing• Watch out for occluded joints (sitting down, putting your

hands behind your back etc. can cause it to lose the track)

• It’s surprisingly good at picking up subtle posture shifts, but not smaller motions like laughter. • Use a different performer for each character if possible

Importing tracking dataNative import or copy & paste

Getting data into After Effects

Data format options:• 3D Point Controls (CS5.5+)• 2D Point Controls (CS4+)• Puppet pins• Raw XML and JSON

UI Panel

• Create character templates• 2D and 3D setups• Native XML import

(SLOW)• Automatically add

expressions to smooth tracking data• Automate rigging

Building a CharacterRigging with the UI panel

The Puppet ToolTHREE OR MORE PINS ON ONE LAYER: Elbows don’t bend this way. Ow.

TWO PINS EACH, MULTIPLE LAYERS:Rotation with stretch and squash!

AE Puppet best practices

• Work BIG. Start high-res, scale down in the puppet comp, then scale back up in the project comp (enable Collapse Transformations).• Precompose if you’re using vectors or shape layers• Keep multiple angles of the same character in syncHigh-res Character

Layers

Rigging Template

Project Comp

End big.Start big.

Retargeting for arbitrary character shapes

Quick auto-lip sync

1. Convert audio amplitude to keyframes2. Connect to mouth animation3. Size the audio data to fit. Use this expression:

linear(source,sourceMin,sourceMax,targetMin,targetMax)

Beyond After EffectsMotion capture for live production

Embrace the limitations of the

format.

One last thought:

Further non-K2P tools

FREEBrekel Kinect

Open-source capture for 3D animation (Windows-only)www.brekel.com

DuikInverse kinematics plugin for AEwww.duduf.net

COMMERCIALiPi Soft

Uses two Kinects to recordwww.ipisoft.com

NI MateKinect for OSC/MIDI/Blenderwww.ni-mate.com

Questions?

kinecttopin.fox-gieg.comDownload the software!

Contact UsNick Fox-Giegwww.fox-gieg.com@N1ckFG

Victoria Necewww.victorianece.com@FakeGreenDress