Sketching Facial Expressions - York University · Sketching Facial Expressions ... Sketches and the...
1
Sketching Facial Expressions Gabriele Nataneli * University of California Los Angeles Petros Faloutsos † University of California Los Angeles Figure 1: Sketches and the corresponding 2D and 3D facial expressions. 1 Introduction We present an innovative sketch-based interface for driving facial expressions. Unlike existing solutions [Chang and Jenkins 2006] our approach relies on recognition and constructs a semantically relevant representation of a sketched face. This representation is parameterized and used to drive a facial model. The main appeal of our method is that the interface is completely decoupled from the underlying facial model that is used. Therefore one single interface is capable of driving a variety of different models both 2D and 3D. The connection between our tool and the face model is defined by a library of template strokes that can be generated with ease. Realistic face models used in production environments are often controlled by a large number of parameters. Our interface hides this complexity and automatically alters the subset of parameters that are relevant to the semantic representation of the sketch. Our solu- tion is therefore ideal in the early stages of production as a tool for prototyping. Our tool is particularly useful when animators would like to portray a storyboard for a character before the final facial model is fully constructed. Our solution gives adequate control over main aspects of a facial expression, but does not focus on the fine grained refinement that is possible in sketch-based interfaces that allow direct manipulation of the face model [Chang and Jenkins 2006]. Nevertheless, the advan- tages of decoupling our tool from a specific model outweigh its lim- itations in many practical situations. Specifically our tool provides a powerful abstraction over lower-level details of the facial model; thus facial expressions can be created more expeditely. Moreover, technologies that rely on recognition are generally at odds with the level of control that is available in traditional tools for facial anima- tion or sketch-based solutions such as [Chang and Jenkins 2006]. Consequently, we devote our efforts to achieve a greater level of ro- bustness and flexibility, which is a key research goal in the domain of sketch-based interfaces. Although we concentrate our attention on facial expressions, our work is also amenable to future research on generic recognition-based interfaces. * e-mail: [email protected] † e-mail:[email protected] 2 Overview and Results Our interface captures strokes as they are drawn by the user. Users tend to draw strokes that are semantically related in rapid succes- sion. We can exploit this fact as a basis for our analysis. In most cases, it is also necessary to group strokes into semantic agglomer- ates based on perceptual organizations as suggested in [Saund et al. 2002]. This way, we can avoid handling the problem of segmenta- tion explicitly which is known to be very hard and could impact the robustness of our approach. We refer to groups of semantically related strokes simply as strokes. To further our analysis we compute a number of shape-attributes that range from the bounding box to special quantifications meant to capture important visual and relational aspects of strokes. We designed shape-attributes on the basis of established concepts of art and psychology [Arnheim 1974]. A correspondence must be established between strokes and com- ponents of the face. This task is accomplished by a classifier that labels each stroke appropriately. We employed a Support Vector Machine (SVM) classifier which is trained using a stable subset of shape-attributes. An interesting result is that a selection of high- level shape-attributes is sufficient to obtain very robust classifica- tion. Every component of the face has a library of templates associated with it. Each template defines a mapping between one or more strokes and a set of parameters on the face model. A template matching algorithm finds the best template that corresponds to any given stroke in the input sketch and sets the parameters in the face model accordingly. This template matching is used to select the parameters that need to be adjusted as well as define the base fa- cial expression. The base facial expression captures key features of the semantics (mood) of the sketch. Additional shape attributes are used to parameterize the strokes and refine the values of parameters on the face model. Results are shown in the included video. References ARNHEIM, R. 1974. Art and Visual Perception: A Psychology of the Creative Eye. CHANG, E., AND J ENKINS, O. C. 2006. Sketching articulation and pose for facial animation. 19–26. SAUND, E., MAHONEY, J., FLEET, D., LARNER, D., AND LANK, E., 2002. Perceptual organization as a foundation for intelligent sketch editing.
Transcript of Sketching Facial Expressions - York University · Sketching Facial Expressions ... Sketches and the...
Sketching Facial Expressions
Gabriele Nataneli∗
University of California Los AngelesPetros Faloutsos†
University of California Los Angeles
Figure 1: Sketches and the corresponding 2D and 3D facial expressions.
1 Introduction
We present an innovative sketch-based interface for driving facialexpressions. Unlike existing solutions [Chang and Jenkins 2006]our approach relies on recognition and constructs a semanticallyrelevant representation of a sketched face. This representation isparameterized and used to drive a facial model. The main appeal ofour method is that the interface is completely decoupled from theunderlying facial model that is used. Therefore one single interfaceis capable of driving a variety of different models both 2D and 3D.The connection between our tool and the face model is defined bya library of template strokes that can be generated with ease.
Realistic face models used in production environments are oftencontrolled by a large number of parameters. Our interface hides thiscomplexity and automatically alters the subset of parameters thatare relevant to the semantic representation of the sketch. Our solu-tion is therefore ideal in the early stages of production as a tool forprototyping. Our tool is particularly useful when animators wouldlike to portray a storyboard for a character before the final facialmodel is fully constructed.
Our solution gives adequate control over main aspects of a facialexpression, but does not focus on the fine grained refinement that ispossible in sketch-based interfaces that allow direct manipulation ofthe face model [Chang and Jenkins 2006]. Nevertheless, the advan-tages of decoupling our tool from a specific model outweigh its lim-itations in many practical situations. Specifically our tool providesa powerful abstraction over lower-level details of the facial model;thus facial expressions can be created more expeditely. Moreover,technologies that rely on recognition are generally at odds with thelevel of control that is available in traditional tools for facial anima-tion or sketch-based solutions such as [Chang and Jenkins 2006].Consequently, we devote our efforts to achieve a greater level of ro-bustness and flexibility, which is a key research goal in the domainof sketch-based interfaces. Although we concentrate our attentionon facial expressions, our work is also amenable to future researchon generic recognition-based interfaces.
∗e-mail: [email protected]†e-mail:[email protected]
2 Overview and Results
Our interface captures strokes as they are drawn by the user. Userstend to draw strokes that are semantically related in rapid succes-sion. We can exploit this fact as a basis for our analysis. In mostcases, it is also necessary to group strokes into semantic agglomer-ates based on perceptual organizations as suggested in [Saund et al.2002]. This way, we can avoid handling the problem of segmenta-tion explicitly which is known to be very hard and could impact therobustness of our approach.
We refer to groups of semantically related strokes simply as strokes.To further our analysis we compute a number of shape-attributesthat range from the bounding box to special quantifications meantto capture important visual and relational aspects of strokes. Wedesigned shape-attributes on the basis of established concepts of artand psychology [Arnheim 1974].
A correspondence must be established between strokes and com-ponents of the face. This task is accomplished by a classifier thatlabels each stroke appropriately. We employed a Support VectorMachine (SVM) classifier which is trained using a stable subset ofshape-attributes. An interesting result is that a selection of high-level shape-attributes is sufficient to obtain very robust classifica-tion.
Every component of the face has a library of templates associatedwith it. Each template defines a mapping between one or morestrokes and a set of parameters on the face model. A templatematching algorithm finds the best template that corresponds to anygiven stroke in the input sketch and sets the parameters in the facemodel accordingly. This template matching is used to select theparameters that need to be adjusted as well as define the base fa-cial expression. The base facial expression captures key features ofthe semantics (mood) of the sketch. Additional shape attributes areused to parameterize the strokes and refine the values of parameterson the face model. Results are shown in the included video.
References
ARNHEIM, R. 1974. Art and Visual Perception: A Psychology ofthe Creative Eye.
CHANG, E., AND JENKINS, O. C. 2006. Sketching articulationand pose for facial animation. 19–26.
SAUND, E., MAHONEY, J., FLEET, D., LARNER, D., AND LANK,E., 2002. Perceptual organization as a foundation for intelligentsketch editing.
