Synthetic animation of deaf signing
-
Upload
iola-pratt -
Category
Documents
-
view
12 -
download
0
description
Transcript of Synthetic animation of deaf signing
![Page 1: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/1.jpg)
1
Synthetic animation Synthetic animation ofof
deaf signingdeaf signingRichard KennawayRichard Kennaway
University of East AngliaUniversity of East Anglia
![Page 2: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/2.jpg)
2
VISICASTAutomated deaf signing of
broadcast television
• Translation of text to sign
• Animation of signs
• Transmission over broadcast
channel
• Rendering of avatar
![Page 3: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/3.jpg)
3
Motion capture
• Very lifelike animation
• Time-consuming to set up• Blending of signs• Combining signs from different signers
![Page 4: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/4.jpg)
4
Hand-crafted animation
• Can give good animation
• Time-consuming (1/2 hour per sign)• Blending of signs still required
![Page 5: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/5.jpg)
5
Synthesis from semi-abstract transcription
• Quick to create lexicon (a few minutes to transcribe a sign)
• Instantly retargettable to any avatar with humanoid topology
• Automatic blending
• Low bandwidth
![Page 6: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/6.jpg)
6
HamNoSys: Hamburg Notation System
DGS “GO-TO (by car)”
![Page 7: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/7.jpg)
7
SiGML: an XML-isation of HamNoSys
<sign_manual both="true"><handconfig extfidir="uo" palmor="l"
handshape="finger2" thumbpos="open"/><tgt_motion>
<directed_motion direction="o" curve="u"/><handconfig extfidir="do"/>
</tgt_motion></sign_manual>
![Page 8: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/8.jpg)
8
Animation of HamNoSys
Make explicit everything HamNoSysleaves implicit or fuzzy:
• position• elbows and shoulders• speed• trajectories
![Page 9: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/9.jpg)
9
Naturalistic animation
A hard problem in general (e.g. walking).
Easier for signing: no interaction with environment, ignore gravity.
![Page 10: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/10.jpg)
10
desiredposition
k1
actualposition desired
velocity
error
gain
actualvelocity
gain
acceleration
error
k2
![Page 11: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/11.jpg)
11
desiredposition
k1
actualposition desired
velocity
error
gain
Simulated physics
A simple linear control system
actualvelocity
gain
force/torque
error
k2
![Page 12: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/12.jpg)
12
Controller response
![Page 13: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/13.jpg)
13
Inverse kinematics
Hand position and orientation given by HamNoSys
From these, compute joint angles from clavicle to wrist (inverse kinematics).
3 degrees of freedom per arm left undetermined:respect the limits of the jointsavoid the arm passing through the body
![Page 14: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/14.jpg)
14
Stick-figure avatars
Useful for developing animations:
• easier to render, so more frames per second
• skeleton gives clearer view of motion
• prototyping tool only, not intended for end user!
![Page 15: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/15.jpg)
15
VRML for prototyping
Textual description language for 3D animated scenes.
H-Anim standard for articulated humanoid figures.
H-Anim incorporated into MPEG-4.
Virtual Reality Modelling Language
![Page 16: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/16.jpg)
16
Ambient motion
Mix synthetic animation with motion-captured “ambient motion” for the spine and head.
If only arms, hands, and face are animated, the result is stiff.
![Page 17: Synthetic animation of deaf signing](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813099550346895d96783a/html5/thumbnails/17.jpg)
17
Next steps
Implement the whole of SiGML/HamNoSys
Facial animation:blend motion-capture dataimplement HamnoSys 4
Tests