Post on 27-May-2015
Cross-fertilization of graphics and vision
Aaron HertzmannUniversity of Toronto
Image/Video Manipulation
Image and video manipulation
C. Barnes, E. Shechtman, A. Finkelstein, D. B. Goldman, “PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing, ACM Trans. Graph (SIGGRAPH) 2009
Special effects
Stargate Studios (www.stargate!lms.com)
Special effects
Stargate Studios (www.stargate!lms.com)
Look Effects
coming next year:Computer Vision for Visual EffectsRichard J. RadkeCambridge University Press, Fall 2012
Generalization of vision ideas
Geometry signal processing
Taubin, SIGGRAPH 1995 Jones, Durand, Desbrun, SIGGRAPH 2003
Geometry processing
A. M. Bronstein, M. M. Bronstein, M. Ovsjanikov, L. J. Guibas, "Shape Google: geometric words and expressions for invariant shape retrieval", ACM Trans. Graphics (TOG), 2011.
E. Kalogerakis, A. Hertzmann, K. Singh, "Learning 3D Mesh Segmentation and Labeling", ACM Trans. Graphics / SIGGRAPH 2011
Non-photorealistic rendering
(demo)
Non-photorealistic rendering
Lambertian rendering Contours and Suggestive Contours[DeCarlo et al. 2003]
“Bone” by Jeff Smith © 2007 Our result
“Bone” by Jeff Smith © 2007 Our result
“Bone” by Jeff Smith © 2007 Our result
The Science of Art
Claim: Vision and graphics will be fundamental to the scientific study of art
See: A. Hertzmann, “Non-Photorealistic Rendering and The Science of Art,” NPAR 2010
Fei-Fei et al. 2007
Why vision should continue to look to graphics:
Graphics builds models of precisely the “visual world”
Scene models
Yu et al. SIGGRAPH 2011
Chaudhuri et al.SIGGRAPH 2011
M. Fisher, M. Savva, P. Hanrahan. “Characterizing Structural Relationships in Scenes Using Graph Kernels,” SIGGRAPH 2011
Google 3D Warehouse
Human motion
K. Grochow, S. L. Martin, A. Hertzmann, Z. Popović. “Style-Based Inverse Kinematics”ACM Trans. Graphics / SIGGRAPH 2004.
R. Urtasun, D. J. Fleet, A. Hertzmann, P. Fua. “Priors for People Tracking from Small Training Sets”ICCV 2005
Claim: Understanding people in motion requires physical models
Physics-Based Characters
M. de Lasa, I. Mordatch, A. Hertzmann,“Feature-Based Locomotion Controllers,” SIGGRAPH 2010
Contact
COM
Features
AM Posture
M. de Lasa, I. Mordatch, A. Hertzmann,“Feature-Based Locomotion Controllers,” SIGGRAPH 2010
M. de Lasa, I. Mordatch, A. Hertzmann,“Feature-Based Locomotion Controllers,” SIGGRAPH 2010
M. de Lasa, I. Mordatch, A. Hertzmann,“Feature-Based Locomotion Controllers,” SIGGRAPH 2010
M. Brubaker, D. J. Fleet, A. Hertzmann, “Physics-Based Person Tracking Using the Anthropomorphic Walker”CVPR 2007, IJCV 2010
Summary
Many successes of vision ideas in graphics
Vision and graphics are key to the science of art
Vision should look to graphics for models of the visual world