Slide 1

Proxemics Recognition Yi Yang 1, Simon Baker, Anitha Kannan, Deva Ramanan 1 1 Department of Computer Science, UC Irvine

Slide 2

Proxemics Proxemics: the study of spatial arrangement of people as they interact - anthropologist Edward T. Hall in 1963Edward T. Hall

Slide 3

Slide 4

Slide 5

Slide 6

Slide 7

Slide 8

Brother and Sister Holding Hands Friends Walking Side by Side Husband Hugging and Holding Wifes Hand Mom Holding Baby

Slide 9

Touch Code Hand Touch Hand Hand Touch Shoulder Shoulder Touch Shoulder Arm Touch Torso

Slide 10

Applications Personal Photo Search: Find a specific interesting photo Analysis of TV shows and movies Kinect Web Search Auto-Movie/Auto-Slideshow Locate interesting scenes

Slide 11

Proxemics DataSet 200 training images 150 testing images Collected From Simon, Bing, Google, Gettyimage, Flickr No video data No Kinect 3d depth data

Slide 12

Proxemics DataSet Number of People Complexity

Slide 13

A Nave Approach Input Image

Slide 14

A Nave Approach Input ImageImage Feature

Slide 15

A Nave Approach Input Image Pose Estimation i.e. Find skeletons Image Feature

Slide 16

A Nave Approach Input Image Interaction Recognition Hand touch Hand Pose Estimation i.e. Find skeletons Image Feature

Slide 17

Nave Approach Results Hand touch Hand Hand touch Shoulder

Slide 18

Human Pose Estimation Not bad when no real interaction between people - Y. Yang & D. Ramanan, CVPR 2011

Slide 19

Interactions Hurt Pose Estimation Occlusion + Ambiguous Parts

Slide 20

Our Approach Direct Proxemics Recognition Input ImageImage Feature

Slide 21

Our Approach Direct Proxemics Recognition Input Image Interaction Recognition Image Feature Hand touch Hand

Slide 22

Pictorial Structure Model - P. Felzenszwalb etc., PAMI 2009

Slide 23

Pictorial Structure Model

Slide 24

Slide 25

Two Head Monster Model i.e. Hand-Touching-Hand

Slide 26

Two Head Monster Model i.e. Hand-Touching-Hand

Slide 27

The models Hand touch HandHand touch Shoulder Shoulder touch ShoulderArm touch Torso

Slide 28

Match Model to Image

Slide 29

Refinements + Extensions Sub-categories Because of symmetry, 4 models for hand-hand etc R. Hand L. Hand L. Hand L. Hand R. Hand R. Hand

Slide 30

Refinements + Extensions Sub-categories Because of symmetry, 4 models for hand-hand etc Co-occurrence of proxemics: Reduce redundancy, map Multi-Label -> Multi-Class R. Hand L. Hand L. Hand L. Hand R. Hand R. Hand

Slide 31

Nave Approach Results Hand touch Hand Hand touch Shoulder

Slide 32

Direct Approach Results Hand touch Hand Hand touch Shoulder

Slide 33

Quantitative Results [1] Y. Yang & D. Ramanan, CVPR 2011

Slide 34

Improves Pose Estimation Y & D CVPR 2011 Our Model

Slide 35

Improves Pose Estimation Y & D CVPR 2011 Our Model

Slide 36

Improves Pose Estimation Y & D CVPR 2011 Our Model

Slide 37

Conclusion Proxemics and touch codes for human interaction

Slide 38

Conclusion Proxemics and touch codes for human interaction Directly recognizing proxemics significantly outperforms

Slide 39

Conclusion Proxemics and touch codes for human interaction Directly recognizing proxemics significantly outperforms Recognizing proxemics helps pose estimation

Slide 40

Acknowledgements Thank Simon and MSR for internship

Slide 41

Acknowledgements Thank Anitha for a lot of suggestions

Slide 42

Acknowledgements Thank Anarb for gettyimages

Slide 43

Acknowledgements Thank Eletee for her beautiful smiling

Slide 44

Acknowledgements Thank everybody for not falling asleep

Slide 45

Thank you

Slide 46

Articulated Pose Estimation - Yi Yang & Deva Ramanan, CVPR 2011

Slide 47

Inference & Learning Learning Inference