FACIAL SKETCH PLUGIN FOR MAYA 2010 PIC – Porto Interactive Center José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
2 FACIAL SKETCH TUTORIAL
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
2
SUMMARY
We have developed a sketching system that allows easy, rapid and interactive prototyping of facial
expressions by drawing strokes. The sketching process is intuitive and similar to how we draw by hand.
The system allows to create facial poses on the fly. It supports non-hierarchy rigs, which make it very
convenient to control 3D models with complex structures, such as a face. We have implemented two
plugins for Autodesk Maya 2010: 2D SKETCH PLUGIN, which allows sketching on predefined and fixed
2D canvases; 3D SKETCH PLUGIN, which allows sketching directly over the 3D mesh and on a 2.5D
dynamic canvas, from any camera orientation;
SYSTEM FEATURES 2D SKETCH PLUGIN 3D SKETCH PLUGIN
DRAW CURVES: draw curves in fixed and predefined canvas draw curves directly on 3D mesh and on
dynamic canvas
CHANGE CURVES: modify the curves drawn
REFERENCE POINTS: show/hide curve control points
REFERENCE CURVES: show/hide reference curves (curves created from joints
initial position: reset curves – blue color) Not implemented;
3D DEFORMATION
WEIGHT:
to increase/decrease the “3D deformation window” (used to exaggerate certain facial
regions)
CREATE POSE: create a pose based on curves drawn
MODIFY POSE: modify selected pose based on curves drawn
RESET POSE: reset selected pose (to its initial position)
DELETE POSE: delete selected pose
SAVE POSES: save all poses in xml file
OPEN POSES: open xml file with poses previously recorded.
RETARGETING POSES: reuse the same pose (curves) in different characters Not implemented;
KEY RIG: put a key in the controls of the rig
SELECT RIG select the controls of the rig to work with the keys
RESET REGIONS: delete all curves and reset all joints to their initial position
RESET ALL: delete all curves, reset all joints to their initial position and delete all poses
BLEND SHAPE: blend shape interpolation using the Maya blendshape deform
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
3
SYSTEM OVERVIEW
DRAWING PANEL (A). This panel only exists in the 2D Sketch Plugin. Panel with 11 fixed and predefined
canvases to draw curves. By default the curves are painted in reset position (blue color). To paint a new curve, you
just need to select the tool icon and draw a stroke (red color) in each canvas. To reset a canvas click one time with
the mouse button in the canvas. To modify the curve change the reference points position (circle points). You can
hide the reference curves and reference points at any time (see the Sketching Tools on the Tool Settings window)
MODEL PANEL (B). In the 2D Sketch plugin, this panel shows the model deformation based on the strokes
drawn in the predefined canvases. In the 3D Sketch plugin, this panel shows the model deformation based on
the strokes drawn directly on the 3D mesh or drawn on dynamic canvases.
TOOL SETTINGS WINDOW (C). To open the tool settings window you need to press twice in the sketch tool in
the Maya tool box.
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
4
TOOL SETTINGS WINDOW DESCRIPTION
SKETCHING TOOLS:
1 Start Clock: to start the clock before sketching the
pose
2 Reset All: reset canvas, joints and poses created with
Pose Tools.
3, 4, 5 Deformation Weight to increase/decrease
the deformation (can be set for each region).
6 Reference Points Show/Hide curve reference points.
7 Reference Curves Show/Hide reference curves. The
reference curves in blue represent the reset position of the rig controls (only works in the 2D sketch plugin)
POSE TOOLS RIG TOOLS
8 Load Load XML file with poses.
9 Save Save the poses in a XML File
10 Create Create a pose and show the Poses
Window. Also, stops the clock when finish the pose.
11 Modify Modify the pose selected in the Poses
Window.
12 Reset Reset the selected pose.
13 Delete Delete the selected pose.
14 Key Rig Put a key in the controls of the rig.
15 Select Rig Select the controls of the rig to work
with the keys.
16 Reset Canvas Reset only the canvas curves.
KNOWN BUGS
Some errors displayed on the Maya status line don’t affect the system functionality.
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
5
BASIC TUTORIAL – 2D SKETCH PLUGIN
To test the plugin you need to follow the next steps:
1. Load Maya;
2. Load Model; 3. Load the plugin facialSketch2D.mll;
4. Run in the command line the mel script sketching;
Now, the system is ready to sketch facial expressions…..
DEFINITIONS
This is a canvas This is a reference curve (blue) This is a stroke in a canvas (red)
A Canvas is a drawing region. We
defined 11 facial regions: brow
left, brow right, eye left up, eye left down, eye right up, eye right down,
cheek left, cheek right, nose, mouth up, mouth down.
Reference curves are curves
generated automatically by the
system from the joints initial position: reset curves
A Stroke is a free-form curve
drawn on a canvas created by
the artist
The user can draw strokes on the canvas to deform the correspondent facial region. See the video 2Dsketch.mp4
(00:01:38).
SKETCHING A POSE
(i) Sketch the left BROW
Draw a stroke in correspondent canvas
and adjust the curve points to the
desired position
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
6
(ii) Close the left EYE
Change the reference curves to close
the left eye
(iii) Increase eye deformation
Increase eye deformation by editing the
Y Amount value on the Tool Settings
Sketching Tools > Deformation Weight Y drag slider to increase/decrease the deformation
(iv) Pull the left CHEEK
Sketch left cheek and adjust the control
points to the desired position
(v) Increase cheek deformation
Increase cheek deformation by editing
the X Amount value on the Tool Settings
Sketching Tools > Deformation Weight X drag slider to increase/decrease the deformation
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
7
(vi) Reset the cheek to the neutral position
(delete stroke)
One click inside the canvas (outside
the stroke) to delete the stroke and
reset this region.
(vii) Sketch the MOUTH
Draw a stroke on the upper canvas to deform
the upper part of the mouth (S1) and another stroke on the lower canvas to deform the lower
part of the mouth (S2)
(viii) CREATE the pose
Pose Tools > Create Display the poses in the shapes Manager
S1
S2
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
8
(ix) SAVE the pose in a
XML file
Pose Tools > SAVE > pose1.xml
NOTE: Data that is crucial for the experiment evaluation, like the time to create a pose, curve points,
number of clicks, number of curves created, modified and deleted are saved in the XML file.
Please, make sure each participant follows the next steps for each pose he will create:
1. RESET ALL
(to get the neutral pose)
2. START CLOCK (to start the clock before sketching the pose)
3. ….sketching the pose….
4. CREATE POSE (to stop the clock after finishing the pose)
5. SAVE POSE
(to save pose in xml file)
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
9
BASIC TUTORIAL – 3D SKETCH PLUGIN
To test the plugin you need to follow the next steps:
1. Load Maya; 2. Load Model;
3. Load the plugin facialSketch3D.mll; 4. Run in the command line the mel script sketching;
Now, the system is ready to sketch facial expressions…..
The user can draw strokes directly on the 3D mesh to deform a facial region. The region deformed will be the region closest to the first point of the stroke (where you started to draw).
Whenever a stroke comes out the mesh, a canvas is dynamically created to give the user the chance to continue
sketching. See the video 3Dsketch.mp4 (00:02:35).
SKETCHING A POSE
(i) Sketch the BROW
Draw a stroke from A to B;
Adjust the curve points 1, 2 and 3 to the
desired position
(ii) Close the upper part of the EYE
Draw a stroke from A to B;
Adjust the curve point 1 to the desired
position
(iii) Close the lower part of the EYE
Draw a stroke from A to B;
Adjust the curve point 1 to the desired
position
A
B
1
3 2
A
B
1
A B 1
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
10
(iv) Pull the left CHEEK
Draw a stroke from A to B;
(notice that a dynamic canvas is created when the stroke comes out the mesh)
Adjust the curve points 1 and 2 to the
desired position
(v)
Sketch the NOSE
Draw a stroke from A to B;
Adjust the curve points 1 and 3 to the
desired position
(vi) Stretch out the NOSE
Rotate the camera
adjust the middle curve point (2) to
come out the mesh to the desired
position
(vii) Reset the nose to the neutral position
(delete canvas and stroke)
One click inside the canvas (outside
the stroke) to delete it and reset the
nose.
1 3
2
2 A B
2
1 A
B 2
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
11
(viii) Stretch out the NOSE
(a different way)
Draw a stroke from A to B;
(notice that a dynamic canvas is created when the stroke comes out the mesh)
adjust the curve point 1 to the desired
position
(ix) Sketch the MOUTH
Draw a stroke directly on the 3D mesh to
deform the upper part of the mouth (S1) and another stroke to deform the lower part of the
mouth (S2)
(x) Reset the mouth to the neutral position
(delete strokes)
One click on the upper part of the
mouth’s region (outside the stroke S1)
to reset it.
One click on the lower part of the
mouth’s region (outside the stroke S2)
to reset it.
(xi) Pull the MOUTH
To deform all joints of the same region at the same time, hold the <SHIFT> key while:
Draw a stroke from A (on the mesh) to B
(outside the mesh);
Adjust the curve point 1 to the desired
position
S1
1 A
B
S2
1 A
B
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
12
(xii) CREATE the pose
Pose Tools > Create Display the poses in the shapes Manager
(xiii) SAVE the pose in a
XML file
Pose Tools > Save > pose1.xml
FACIAL SKETCH TUTORIAL José Carlos Miranda, Xenxo Alvarez Blanco, Verónica Costa Orvalho
13
NOTE: Data that is crucial for the experiment evaluation, like the time to create a pose, curve points,
joints position, number of clicks, number of curves created, modified and deleted are saved in the XML file.
Please, make sure each participant follows the next steps for each pose he will create:
1. RESET ALL
(to get the neutral pose)
2. START CLOCK (to start the clock before sketching the pose)
3. ….sketching the pose….
4. CREATE POSE (to stop the clock after finishing the pose)
5. SAVE POSE
(to save pose in xml file)
Top Related