Language Resources

for Computer Assisted translation from Italian to

Italian sign language of deaf people

Nadeem Ahmad, Davide Barberis, Nicola Garazzino, Paolo Prinetto,

Alessandro Savino, Umar Shoaib, Gabriele Tiotto

Politecnico di Torino (Italy)

Umar.shoaib@polito.it

www.testgroup.polito.it www.atlas.polito.it

Politecnico di Torino

Outline

• Language Resources

• Requirement

• Translation Model

• Technical Details

• Conclusion

Politecnico di Torino

ATLAS : what is it?

•A research project co-funded by Regione Piemonte - Converging Technologies 2007 Call

• Period : 15.01.2009 – 14.01.2012

Politecnico di Torino

ATLAS : Objectives

• To create language resources for automatic Text to LIS translation,

• Visualization 0f Virtual character on several medias.

• Platform independant Sign Language dictionary

Politecnico di Torino

Politecnico di Torino

Politecnico di Torino

Mobile Application

• Railway station /airport possible to load a service for

the LIS visualization on the cell phone or PDA for the deaf traveller for announcements.

Politecnico di Torino

Language Resources

• Availability of Language resources.

• Creation and annotation of the corpora.

• Creating a clearly understandable and representative corpora .

• Integration of Language resources with different platforms.

• Linking two languages having different features & communication Media

Politecnico di Torino

ATLAS CAT

• ATLAS CAT deals with the followings

– Translation

– Visualization using Avatar

Politecnico di Torino

Requirements(1)

• Most of existing translation approaches assume some knowledge & Literacy skills.

• Reading captioning resources could be difficult even for fluent signers.

Politecnico di Torino

Requirements(2)

• Text to Intermediary (AWLIS) conversion.

• Feasible Translation retrieval.

• Linguistics corpora creation.

• Set signs in signing space.

• Proper sign planning.

• Smoothing signs with expressions.

Politecnico di Torino

VIRTUAL ACTOR

COMMANDS

TEXT

VIDEO

TEXT

Virtual Actor Commands

SOURCES Teletext Subtitles Audio Audio/Video

Transformations Transformations Transformations

AEWLIS Sentences

ATLAS Work flow

Politecnico di Torino

Overview

Politecnico di Torino

The ATLAS Platform Text Object

TRANSFORMATION into COMMANDS for Virtual Character

TEXT

Com.

Actor

Virtual

External Transformation bocks

From Multi-Media to TEXT

Politecnico di Torino

Animation Module

• Planner

– Sets signing position accordingly

• Animation Interpreter

– AWLIS & planning of signs and renders smoothly

• Animation Engine

– Modifiers to shape i.e. facial expression , position

– Body movements

Politecnico di Torino

ATLAS Corpus

• More than three thousand signs

– Using dictionary standards

– Custom created

• MultiWordNet Integration

• 40 completely Annotated meteos.

Politecnico di Torino

Reliability

• Scale and scope of signs similar to lexical dictionary

• Native or expert signer generated signs

• Main Focus on weather forecasts.

• Flexibility for the optimization.

• Mostly dictionary standards

• Coherent & clear sign animations

• Platform independent application

Politecnico di Torino

Conclusion

• Animations & Annotations are valuable resource.

• Reference for the Translation

• Disambiguation of words & limitation of annotated results using Multiword Net.

• Covers significant Multiword Words

• Encourage SL recognition, gesture recognition and human activity analysis.

Politecnico di Torino

Future Work

• Extend to other Linguistics domain

• Platform supporting the European languages

• LIS4ALL

• Looking for collaboration.

Politecnico di Torino

Questions

umar.shoaib@polito.it

Politecnico di Torino