Performance Evaluation of Vision-based Real-time Motion Capture
34
Image and Media Understanding Laboratory for Performance Evaluation of Vision-based Real-time Motion Capture Naoto Date, Hiromasa Yoshimoto, Daisaku Arita, Satoshi Yonemoto, Rin-ichiro Taniguchi Kyushu University, Japan
description
Performance Evaluation of Vision-based Real-time Motion Capture. Naoto Date, Hiromasa Yoshimoto, Daisaku Arita, Satoshi Yonemoto, Rin-ichiro Taniguchi Kyushu University, Japan. Background of Research. Motion Capture System Interaction of human and machine in a virtual space - PowerPoint PPT Presentation
Transcript of Performance Evaluation of Vision-based Real-time Motion Capture
Image and Media
Understanding
Laboratory for
Performance Evaluation of Vision-based Real-time Motion Capture
Naoto Date, Hiromasa Yoshimoto, Daisaku Arita, Satoshi Yonemoto, Rin-ichiro Taniguchi
Kyushu University, Japan
Laboratory for Image and Media Understanding
Background of Research
Motion Capture System– Interaction of human and machine in a virtual space– Remote control of humanoid robots– Creating character actions in 3D animations or video
games
Sensor-based Motion Capture System– Using Special Sensors (Magnetic type, Infrared type etc.)– User’s action is restricted by attachment of sensors
Vision-based Motion Capture System– No sensor attachments– Multiple cameras and PC cluster
Laboratory for Image and Media Understanding
Key Issue Available features acquired by vision
process is limited.– Head, faces and feet can be detected
robustly.
How to estimate human postures from the limited visual features– Three kinds of estimation algorithms– Comparative study of them
Laboratory for Image and Media Understanding
System Overview
人物 2CG model
PC
PC
PC
PC
PC
PC
camera
camera
camera
camera
camera
Laboratory for Image and Media Understanding
System Overview
人物 2CG model
PC
PC
PC
camera
camera
camera
camera
camera
PC
Using 10 cameras for robust motion capture
Laboratory for Image and Media Understanding
System Overview
人物 2CG model
1 top-view camera on the ceiling
PC
PC
PC
camera
camera
camera
camera
camera
Laboratory for Image and Media Understanding
System Overview
人物 2CG model
9 side-view cameras around the user
PC
camera
camera
camera
camera
camera
Laboratory for Image and Media Understanding
System Overview
人物 2CG model
Using PC cluster for real-time feature PC
PC
PC
PC
PC
PC
camera
camera
camera
Laboratory for Image and Media Understanding
System Overview
人物 2CG model
First, take images with each camera
PC
PC
PC
PCcamera
camera
camera
camera
camera
Laboratory for Image and Media Understanding
System Overview
人物 2CG model
Extract image-features on the first stage PCs
PC
PC
PC
PC
PC
camera
camera
camera
camera
camera
Laboratory for Image and Media Understanding
System Overview
人物 2CG model
PC
PC
PC
PC
PC
camera
camera
Reconstruct human CG model by feature parameters
in each image
Laboratory for Image and Media Understanding
System Overview
人物 2CG model
Synchronous IEEE1394 cameras: 15fps
PC
PC
PC
camera
camera
camera
camera
camera
Laboratory for Image and Media Understanding
System Overview
人物 2CG model
CPU : Pentium 700MHz Ⅲ ｘ 2OS : LinuxNetwork： Gigabit LAN Myrinet
camera
camera
camera
PC
PC
PC
PC
PC
Laboratory for Image and Media Understanding
Top-view camera process
Background subtraction Opening operation Inertia principal axis Detect body direction
and transfer it
Laboratory for Image and Media Understanding
Top-view camera process
Background subtraction Opening operation Inertia principal axis Detect body direction
and transfer it
Laboratory for Image and Media Understanding
Top-view camera process
Background subtraction Opening operation Inertia principal axis Detect body direction
and transfer it
Laboratory for Image and Media Understanding
Top-view camera process
Background subtraction Opening operation Feature extraction
– Inertia principal axis– Body direction
Laboratory for Image and Media Understanding
Side-view camera process Background subtraction Calculate centroids of skin-color blobs
Laboratory for Image and Media Understanding
Side-view camera process Background subtraction Calculate centroids of skin-color blobs
Laboratory for Image and Media Understanding
Side-view camera process Background subtraction Calculate centroids of skin-color blobs
Laboratory for Image and Media Understanding
From all the combination of cameras and blob centroids, we select all possible pairs of lines of sight. Then we calculate an intersection point of each line pair. Unless the distance of the two lines is smaller than a threshold, we decide there is no intersection point.
Estimate 3D position of skin-color blob
Laboratory for Image and Media Understanding
Estimate 3D position of skin-color blob The calculated points are clustered according
to distances from the feature points (head, hands, feet) of the previous frame.
Select points where feature points are dense as the 3D positions of the true feature points.
Laboratory for Image and Media Understanding
Estimate 3D position of torso
L1
L2
head
right shoulder
V: V is the vector which intersects perpendicularly with a body axis and with a body direction.
V
torso
・ A method based on simple body model
Center point
Laboratory for Image and Media Understanding
Performance evaluation of right hand position
estimation
Laboratory for Image and Media Understanding
Estimate 3D positions of elbows and knees
3 estimation methods – Inverse Kinematics (IK)– Search by Reverse Projection (SRP)– Estimation with Physical Restrictions
(EPR)
Laboratory for Image and Media Understanding
Estimate 3D positions of elbows and knees
IK assumed to be a constant
Laboratory for Image and Media Understanding
Estimate 3D positions of elbows and knees
SRP
Laboratory for Image and Media Understanding
EPR An arm is assumed to be the connected two
spring model. The both ends of a spring are fixed to the position
of the shoulder, and the position of a hand. The position of an elbow is converged to the
position where a spring becomes natural length. (the natural length of springs is the length of the bottom arm and the upper arm which acquired beforehand.)
Estimate 3D positions of elbows and knees
Laboratory for Image and Media Understanding
Accuracy of estimating right elbow position
Laboratory for Image and Media Understanding
Accuracy of posture parameters
Laboratory for Image and Media Understanding
Visual comparison of 3 methods
Laboratory for Image and Media Understanding
Computation time required in each algorithm
Top-view camera processing : 50msSide-view camera processing : 26ms3D blob calculation : 2msIK calculation : 9msSRP calculation : 34msEPR calculation : 22ms
Laboratory for Image and Media Understanding
Online demo movie (EPR)
Laboratory for Image and Media Understanding
We have constructed a Vision-based Real-time Motion Capture System and evaluated its performance
Future works– Improvement of posture estimation
algorithm– Construction of various applications
Man and machine interaction in a virtual space
Humanoid robot remote control system
Conclusions
![A Survey of Advances in Vision-Based Human Motion Capture ...kahlan.eps.surrey.ac.uk/Personal/AdrianHilton/Publications_files/... · Similarly motion capture data-bases [1,2,4] have](https://static.fdocuments.us/doc/165x107/5f409c0dab49ef385346ffbd/a-survey-of-advances-in-vision-based-human-motion-capture-similarly-motion-capture.jpg)
