TABLE OF CONTENTS - project.sliit.lkproject.sliit.lk/.../download/3168/SRS_udani.docx · Web...

SIGNANS MASTER: SIGN LANGUAGE TRANSLATING

AND LEARNING TOOL

Project Id: 18-109

Software Requirement Specification

S.P.U.K.Weerasinghe

Bachelor of Science (Hons) in Information Technology

Specialization in Software Engineering

Department of Software Engineering

Sri Lanka Institute of Information Technology Sri Lanka

May 15, 2018

SIGNANS MASTER: SIGN LANGUAGE TRANSLATING AND

LEARNING TOOL

Project Id: 18-109

Software Requirement SpecificationOn Translating Sign Language to Text and Audio

S.P.U.K.Weerasinghe – IT15117410

Supervisor: Dr. Pradeepa Samarasinghe

Co-supervisor: Ms.Chamari Silva

Bachelor of Science (Hons) in Information Technology

Specialization in Software Engineering

Department of Software Engineering

Sri Lanka Institute of Information Technology Sri Lanka

May 15, 2018

1

DECLARATION OF THE CANDIDATE

I declare that this is my own work and this Software Requirement Specification document

does not incorporate without acknowledgement any material previously submitted for a

degree or diploma in any other university or Institute of higher learning and to the best of my

knowledge and belief it does not contain any material previously published or written by

another person except where the acknowledgement is made in the text.

Name Student ID Signature

S.P.U.K.Weerasinghe IT15117410

2

TABLE OF CONTENTS

TABLE OF CONTENTS........................................................................................................3

LIST OF TABLES...................................................................................................................6

INTRODUCTION...................................................................................................................7

1.1 Purpose.......................................................................................................................7

1.2 Scope...........................................................................................................................8

1.3 Definitions, Acronyms and Abbreviations..............................................................9

1.4 Overview..................................................................................................................11

2. OVERALL DESCRIPTION.........................................................................................12

2.1 Product Perspective................................................................................................14

2.1.1 System interfaces..................................................................................................17

2.1.2 User interfaces.......................................................................................................18

2.1.3 Hardware interfaces.............................................................................................20

2.1.4 Software interfaces...............................................................................................20

2.1.5 Communication interfaces...................................................................................20

2.1.6 Memory constrains...............................................................................................21

2.1.7 Operations.............................................................................................................21

2.1.8 Site adaptation requirements...............................................................................21

2.2 Product Functions...................................................................................................22

2.2.1 Use case diagrams.................................................................................................23

2.2.2 Use case scenarios.................................................................................................24

2.3 User Characteristics....................................................................................................25

2.4 Constraints...................................................................................................................25

2.5 Assumptions and Dependencies..................................................................................25

3

2.5.1 Assumptions..........................................................................................................26

2.5.2 Dependencies.........................................................................................................26

2.6 Apportioning of Requirements...................................................................................26

3. SPECIFIC REQUIREMENTS.....................................................................................28

3.1 External Interface Requirements..........................................................................28

3.1.1 User interfaces..................................................................................................28

3.1.2 Hardware interfaces........................................................................................28

3.1.3 Software interfaces..........................................................................................29

3.1.4 Communication interfaces..............................................................................29

3.2 Performance Requirements...................................................................................30

3.3 Design Constraints..................................................................................................30

3.4 Software System Attributes....................................................................................31

3.4.1 Reliability..........................................................................................................31

3.4.2 Availability.......................................................................................................31

3.4.3 Maintainability.................................................................................................31

3.4.4 User friendliness...............................................................................................32

4. SUPPORTING INFORMATION................................................................................33

4.1 Index.........................................................................................................................33

4.2 References................................................................................................................34

4

LIST OF FIGURES

Figure 1: User interaction with the component.......................................................................12

Figure 2: High level diagram of the component......................................................................13

Figure 3: User Interface 1........................................................................................................18



Figure 6: Use case diagram of the component.........................................................................23

5

LIST OF TABLES

Table 1: Terms and definitions..................................................................................................9

Table 2: Terms and meanings..................................................................................................10

Table 3: Existing systems comparison....................................................................................16

Table 4: User interface description..........................................................................................19

Table 5: Use case scenario 1....................................................................................................24

Table 6: Use case scenario 2....................................................................................................24

Table 7: User types..................................................................................................................25

Table 8: Software description..................................................................................................29

6

INTRODUCTION

1.1 Purpose

This document provides a detailed description about the requirements and functionalities of

the SIGNANS MASTER: Sign Language Translating and Learning Tool. The main objective

of this SRS document is to make sure that stakeholders have the proper understanding about

the system component. This SRS will be a proper feedback to the clients hence it assures the

development team has certainly understood the requirement document properly. Similarly,

this provides a declaration that the development team will implement the functionalities,

which has been detailed.

This document will also cover each of the intended features, user interfaces, functional and

non-functional requirements, design constraints, project approach and other factors necessary

to provide an in-depth view of the project. This document is planned to provide a guidance to

the requirements team, designers, developers and stakeholders to start working on the

project. Furthermore, it is intended to serve a goal to provide the potential users to discovery

whether the application has accomplished their needs or how the application should be

modified correctly to meet their requirements.

7

1.2 Scope

This document covers the requirements for release of “Translate Sign language to text and

audio” of “SIGNANS MASTER: Sign Language Translating and Learning Tool”.

SIGNANS MASTER is a mobile application that has the capability of translating Sign

Language to other languages and vice versa. In addition, it contains a learning tool where

users can actively learn Sign Language while being closely monitored for distractions. It will

be able to download from the Google paly store and install in any android device such as

smart phones and tablets. It will be able to use via Internet at any time and any place around

the world.

The component “Translate Sign language to text and audio” is designed to translate Sign

Language to Sinhala, English and Tamil languages as a text message or as an audio. User can

capture the Sign language performer as a video through the mobile camera of the application.

After analyzing the hearing-impaired users’ hand movements, gestures in the video, meaning

of those signs will be provided as a text or as an audio due to user’s choice. Since the

application is designed mainly focusing on Sri Lankan hearing impaired people, it can

translate Sri Lankan Sign Language signs only. It will be able to translate about more than

350 SSL signs.

This component includes Image Processing Techniques, Video Processing Techniques,

Machine Learning Algorithms and Data Analysis.

This proposed application is very beneficial specially to hearing impaired people in Sri

Lanka and their families, relatives and helpers. This application tends to collapse the

communication barrier between the hearing-impaired people and the general community.

Since the “SIGNANS MASTER: Sign Language Translating and Learning Tool” can be used

for communication purposes in any place, hearing impaired people can spend a regular,

comfortable life without another’s assistance.

8

1.3 Definitions, Acronyms and Abbreviations

Table 1: Terms and definitions

Term Definition

API Application Programing Interface

GUI Graphical User Interface

OS Operating System

SDK Software Development Kit

SRS Software Requirement Specification

SSL Sri Lankan Sign Language

9

Table 2: Terms and meanings

Term Meaning

3D The quality of being three dimensional

Hearing-impaired Has no ability to hear or speak

OS The operating system used for mobile

devices manufactured by Apple Inc.

Mobile Application Software that runs on mobile devices

Stakeholders A person, group or organization, can

affect or be affected by the organization's

actions, objectives, and policies [1].

10

1.4 Overview

“SIGNANS MASTER: Sign Language Translating and Learning Tool” is a mobile

application that consists with four major components such as the Translate Sign language to

text and audio, Translate text and audio to Sign language, Sign language teaching automate

bot and Facial recognition system to monitor users’ involvement.

This application’s users are specially the hearing-impaired people in Sri Lanka as well as any

user who has a will to learn Sign Language or to communicate with hearing-impaired people.

The main target of “SIGNANS MASTER” is to provide a user-friendly application that will

be supportive to carry out the daily activities for the hearing-impaired people in Sri Lanka.

Through the application they can easily communicate with the general community thus, they

can have a regular and a typical life.

This SRS document contains all the user requirements and it contains a general description of

the types of users who will be using this application, the procedure it works and the

technologies that are required to implement it. This includes the project perspective and

descriptions, different interfaces that the application consists, requirements of the system,

summary of major functionalities, user and characteristics of the application and the

background of the general factors affect the system.

In this document’s section 1, mentions the purpose of the SRS document, identifies the scope

of the component “Translate Sign language to text and audio”, and lists the definitions,

acronyms and abbreviations followed by an overview of the system. The section 2 describes

the application in more detail with Product perspective, Product functions, User

characteristics, Constrains, Assumptions and Apportioning of requirements. The section 3

provides a complete list of the functional requirements and the non-functional requirements

of the application. It is divided into sub sections as External interface requirements,

Classes/Objects, Performance requirements, Design constraints and Software system

11

attributes. Finally, the supporting information is listed as the final section of the SRS

document.

2. OVERALL DESCRIPTION

The component “Translate Sign Language to Text and Audio” of “SIGNANS MASTER” is

to convert Sign Language (SL) to Sinhala, Tamil and English. Users can obtain the translated

output in those languages via text and audio formats.

When a hearing-impaired person performs in SL, an ordinary person can use this component

of “SIGNANS MASTER” to get the meaning of those signs. For that, he/she need to capture

the signer through the mobile camera that is provided in the first user interface (UI) of this

process. After that, the user needs to select the translation mode and language. The mode

should be either text or audio and a language can be selected form Sinhala, Tamil and

English. Finally, user presses on the ok button and after a processing time about five seconds

user gets the translated output.

When the user inputs the video, it is processed through some video processing algorithms

and fragmented in to image frames. Using image processing techniques the hand gestures,

orientation of figures are clearly visualized. Then those frames are matched with the sign

database to get the corresponding English word. Then that word set is send through API

gateways to create a grammatically correct sentence. If the user needs the output as a English

then that sentence is straightly displayed in the UI. If the user needs it to be a English audio

then that sentence is gone through text to speech translator API. In some situations user may 12

Figure 1: User interaction with the component

want to have the output in Sinhala or Tamil. Then the English text is processed in a language

translator API and provided or convert to an audio and provided as the output to the user.

If

the user need to communicate with the hearing-impaired person, he/she can use the next

component in “SIGNANS MASTER” which translates Text and audio to SL. Therefor

“SIGNANS MASTER” provides a simple and easy platform to communicate with hearing-

impaired people. Since it is a mobile application, any user can use it around the world at any

time.

13

Figure 2: High level diagram of the component

2.1 Product Perspective

Kinect Sign Language Translator

“Kinect Sign Language Translator” [2] was conducted mainly focusing on the Chinese

community, which contains more than 20 million people who are hard of hearing.

Furthermore, they encounter this project will be beneficial for the estimated 360 million such

people around the world. In addition, they specifically mention this project has immense

potential to generate positive social impact worldwide. They have used language modeling,

translation, computer vision, speech recognition, 3D modeling, and special education to build

the system.

Kintrans – Automatic two way Sign Language translator

“Kintrans” is a smart tech capable of translating sign language into voice and text, and voice

taken to text or sign language automatically [3]. The results allow signers and non-signers to

communicate in their own languages easily. The system relies on a 3D camera and which

track the hand gestures of the signer. It provides the output in English and Arabic. Vice

versa of this process happens through a 3D avatar.

A wearable Device that translates Sign Language to English

Some researchers at Texas A&M University, America are working on this project and it is

still in a prototype stage [4]. The system uses two sensors. One is a motion sensor with an

accelerometer and a gyroscope, which measures the user’s hand and arm speed and angle. By

sensing where a user’s hands and arms are, it can begin to guess what word they might be

signing. Then there is an electromyography sensor, which measures the electrical potential of

muscle movement. Up to now, this device can recognize 40 ASL signs and translate them

with 96 percent accuracy.

14

Vision-based sign language translation device

This is a mobile application which translates Indian Sign Language to speech in English [5].

It is an interactive application developed using “LABVIEW” software and incorporated into

a mobile phone. The sign language gesture images are acquired using the inbuilt camera of

the mobile phone. Vision analysis functions are performed in the operating system and

provide speech output through the inbuilt audio device.

ProDeaf Translator

ProDeaf Translator is a mobile application that translate English and Portuguese words and

phrases to American Sign Language (ASL) and Brazilian Sign Language [6]. Although it

translate both text and audio to SL, it does not translate SL to English.

Mimix Sign Language Translator

Mimix is a mobile application which translate English text and speech in to ASL by using a

3D avatar [7]. Moreover, it translates text into spoken words, so the deaf person can write

text using keyboard or sign language keyboard.

15

Table 3:

Existing systems comparison

16

Device or

Application

name

Translate SL

To English

Translate Text

and speech to

SL

Ability to use Sinhala and Tamil

Can user as a

mobile

application

Kinect Sign

Language

Translator

Kintrans

Automatic two

way Sign

Language

translator

A wearable

Device

Vision-based

sign language

translation

device

ProDeaf

Translator

Mimix Sign

Language

Translator

SIGNANS

MASTER

2.1.1 System interfaces

Some several APIs should be used in order to get the correct output.

Cloud Text-to-Speech API

Cloud Text-to-Speech API allows developers to include natural-sounding, synthetic human

speech as playable audio in their applications [8].

Google Cloud Translation API

Using state-of-the-art Neural Machine Translation, API provides a simple programmatic

interface for translating an arbitrary string into any supported language. It is highly

responsive, thus websites and applications can integrate with Translation API for firm,

dynamic translation of source text from the source language to a target language. [9]

TextGears grammar checking API

It is available over HTTP and HTTPS. The output data format is JSON [10].

Bing Spell Check API

This API offers suggestions for misspelled words, and contextual spell checking for text

provided by a third party application. This is Microsoft's third generation spell-checker relies

on machine learning and statistical machine translation [11].

17

2.1.2 User interfaces

18

Figure 3: User Interface 1


Table 4: User interface description

User Interface 1 User Interface 2 User Interface 3

User can open the mobile

camera to capture a video

User can select the

translation mode

User can select the

translation language

19


2.1.3 Hardware interfaces

The mobile application requires few hardware requirements like,

• Mobile device with minimum resolution 480 x 640.

• Mobile Device rear camera with minimum 2MP camera.

• Which should consist minimum processor speed of 1 GHz, with a RAM of at least

1GB android OS 4.2 above.

2.1.4 Software interfaces

To develop the component “Translate Sign language to text and audio” following software

are needed.

Visual Studio 2015

Android Studio 3.1.0

TensorFlow 1.8

MySQL

2.1.5 Communication interfaces

The communication between the different parts of the “Translate Sign language to text and

audio “component is significant since they share functionalities on each other. To connect

with the cloud database and APIs an Internet connection must be available.

20

2.1.6 Memory constrains

“SIGNANS MASTER: Sign Language Translating and Learning Tool” application will

typically utilize 1GB of device memory to perform efficiently.

2.1.7 Operations

Following are the operations required to be maintained by the user when using the “Translate

Sign language to text and audio” component.

Capture the Sign Language performer as a video through the camera of the

application.

Input the how the translated output should be provided such as text or audio.

Input in what language the translation should be provided such as Sinhala, Tamil or

English.

2.1.8 Site adaptation requirements

“SIGNANS MASTER” mobile application can be downloaded from the Google Play store

and installed it in any Android smart phone.

Although the component “Translate Sign language to text and audio “ provides the output in

three different languages, all the user interfaces should be in English.

Since “SIGNANS MASTER” is designed for Android OS mobile phones, it will be

developed and provided for IOS mobile phones in near future.

21

2.2 Product Functions

This SRS document will focus on the feature, “Translate Sign language to text and audio”

that is being implemented in the “SIGNANS MASTER” mobile application. This component

can be divided into fucnctions to define its functionalities and features.

Divide the video into image frames.

Enhance the figures and hand gestures of each frame.

Match signs with the database and get appropriate English words.

Create a grammatically correct sentence.

Translate to another language.

Convert the text into an audio.

22

2.2.1 Use case diagrams

23

Figure 6: Use case diagram of the component

2.2.2 Use case scenarios

Following are some use case scenarios of the “Translating Sign Language to Text and

Audio” component.

Table 5: Use case scenario 1

Use case no 01

Use case Get the translation output in Sinhala text

Actors Ordinary-person

Pre-condition User should capture the Sign Language performer in a video

Flow of events 1. Select the translation mode as text

2. Select the translation language as Sinhala

3. Click ‘OK’ button

Post-condition Display a message to wait until processing.

Alternatives Display retry message.

Table 6: Use case scenario 2

Use case no 02

Use case Get the translation output in English audio



Flow of events 1. Select the translation mode as audio

2. Select the translation language as English




24

2.3 User Characteristics

Any user who has the basic knowledge to operate a smart phone with common English terms

is eligible to use “SIGNANS MASTER”. No prior practice is needed since the application is

designed with simple, user friendly GUIs, thus any novice user can manage the

functionalities of the application easily.

User characteristics in detail for the component "Translate Sign language to text and audio"

as follows.

Table 7: User types

User type Description

hearing-impaired people People who cannot hear or speak

Typical users Family members, relatives, helpers or

whoever interacts with hearing-impaired

people

2.4 Constraints

There are some limitations when using the applications and they are listed below.

For the complete functionality of the SIGNANS MASTER’s all four components, an

Internet connection must be available.

Android phones OS should be above 4.2 for an efficient performance of the

application.

Server should be able to perform its normal operations without exceeding 1GB of

RAM

Language Translation may take about five seconds to provide the output correctly.

Buttery percentage of the mobile should be a sufficient amount.

2.5 Assumptions and Dependencies

25

The assumptions and dependencies that should be taken in to consideration when using

“Translate Sign language to text and audio “component of “SIGNANS MASTER” as

follows.

2.5.1 Assumptions

If the mobile device does not have enough devise memory available for the

application, for instance, if the user has installed other applications that results a low

memory space, there may be scenarios where the application does not work as

intended.

The connection between the server and the android devices does not disconnect

during the run time of the application.

No data will be lost or corrupted during the communication between the server and

the application.

Future versions of Android OS which will be embedded in the android devices will

allow to run this application using the existing android SDK version.

2.5.2 Dependencies

“SIGNANS MASTER” depends on the OS of the mobile phone. Since it is, an

Android application it cannot be installed in devises which has IOS.

Performance of the functionalities of the application depend on the signal strength of

the Internet connection.

2.6 Apportioning of Requirements

The requirements described in sections 1 and 2 of this document are referred to as primary

specifications; those in section 3 are referred to as functional specifications. Section 2 lists

down the requirements in product perspective. Section 3 describes the most important

requirements that will be implemented in the system in its first release. 26

‘Specific requirements’ and ‘Performance requirements’ referred to in section 3 are to be

implemented for this version of “SIGNANS MASTER”. 'Desirable requirements' are to be

implemented in this release if possible, but are not committed to by the developers. It is

anticipated that they will be part of future release. 'Optional requirements' will be

implemented at the discretion of developers.

27

3. SPECIFIC REQUIREMENTS

3.1 External Interface Requirements

3.1.A User interfaces

Name of the component : "Translate Sign language to text and audio"

Description of purpose: Translate Sign Language to Sinhala, Tamil or English as

a text or audio depending on user input.

Source of input: Video clip

Valid range, accuracy and/or tolerance :More than 350 SSL signs can be

translated correctly

Units of measure: Seconds

Timing: Translation takes about five seconds

Relationships to other inputs/outputs: Specified component acts unaccompanied.

Screen formats/organization: Works under both portrait and landscape.

3.1.B Hardware interfaces

Mobile device with screen size 3.5 - 6 inches.

Which should consist minimum processor speed of 1 GHz, with a RAM of at least

1GB android OS 4.2 above.

For the back-end of the application, an amazon web server will be required with

server space with minimum 1GB. Furthermore, as the database server MySQL must

installed in the server.

28

3.1.C Software interfaces

Table 8: Software description

Name Version Purpose

Visual Studio 2015 As a platform to perform Video and

Image processing techniques

Android Studio 3.1.0 For mobile application

development

TensorFlow 1.8 To train the classifier to identify

similar signs

3.1.D Communication interfaces

To connect with the cloud database and APIs an Internet connection with a strong signal

strength should be available.

29

3.2 Performance Requirements

The mobile application should have the following performance requirements.

The delay between the request from the application and the response from the

server should be less than 5 seconds. Sign language will be translated and

given in five second after processing and querying the database.

Back end should be able to handle multiple requests at a time.

3.3 Design Constraints

Most of the time "Translate Sign language to text and audio" component of “SIGNANS

MASTER”, is used by the typical users who try to communicate with hearing impaired

people by translating Sign Language into Sinhala, Tamil or English.

For that, Sign language performer must be capture through a video. Therefore, mobile

camera icon that opens the camera by a single tough, should be visible clearly in the

interface.

Then the output language user would like to receive the translation and the output

mode such as text or audio should be clearly provided in the user interface for the

user to select.

30

3.4 Software System Attributes

3.4.A Reliability

Reliability is the probability of failure-free software operation for a specified period

of time in a specified environment. Software Reliability is also an important factor

affecting system features. This application will run through many test runs and

modify accordingly to ensure its reliability.

3.4.B Availability

The application is available 24 hours anywhere around the world where there is an Android

smart phone with the constraints mentioned in section 2 and an Internet connection is

possible.

3.4.C Maintainability

Standard coding practices will be followed throughout application implementation. The

system will be implemented with minimum bugs as much as possible.

31

3.4.D User friendliness

The application will consist with easy-to-use user interfaces (UI) with appropriate

amount of UI elements. The flow of the application will be clearly visible to any

consumer, even a novel user can easily execute the application without any others

assistance.

32

4. SUPPORTING INFORMATION

4.1 Index

33

4.2 References

[1] BusinessDictionary.com. (2018). What comes after those ellipses?. [online] Available at:

http://www.businessdictionary.com/definition/stakeholder.html [Accessed 6 May 2018].

[2] "Kinect Sign Language Translator - part 1 - Microsoft Research", Microsoft Research,

2018. [Online]. Available: https://www.microsoft.com/en-us/research/blog/kinect-sign-

language-translator-part-1/. [Accessed: 25- Mar- 2018].

[3] Kintrans.com. (2018). KinTrans. [online] Available at: http://www.kintrans.com

[Accessed 8 Apr. 2018].

[6] APKPure.com. (2018). ProDeaf Translator APK Download - Free Education APP for

Android | APKPure.com. [online] Available at:

https://apkpure.com/prodeaf-translator/com.Proativa.ProDeafMovel [Accessed 6 May 2018].

[7] Translator, M. and Translator, M. (2018). Mimix Sign Language Translator 2.8.15 Free

Download. [online] Mimix-speech-to-sign-language.soft112.com. Available at:

https://mimix-speech-to-sign-language.soft112.com [Accessed 12 May 2018].

[4] @EmilyMatchar, F. (2018). This Wearable Device Translates Sign Language To English.

[online] Smithsonian. Available at: https://www.smithsonianmag.com/innovation/wearable-

device-translates-sign-language-english-180956827/ [Accessed 13 May 2018].

[5] Ieeexplore.ieee.org. (2018). Vision-based sign language translation device - IEEE

Conference Publication. [online] Available at: https://ieeexplore.ieee.org/document/6508395/

[Accessed 14 May 2018].

34

[8] Google Cloud. (2018). Cloud Text-to-Speech API Basics | Cloud Text-to-Speech API |

Google Cloud. [online] Available at: https://cloud.google.com/text-to-speech/docs/basics

[Accessed 9 May 2018].

[9] Google Cloud. (2018). Cloud Translation API - Dynamic Translation | Google Cloud.

[online] Available at: https://cloud.google.com/translate/ [Accessed 12 May 2018].

[10] TextGears. (2018). Grammar check API - TextGears English spelling and grammar

service. [online] Available at: https://textgears.com/api/ [Accessed 12 May 2018].

[11] ProgrammableWeb. (2018). Bing Spell Check. [online] Available at:

https://www.programmableweb.com/api/bing-spell-check [Accessed 11 May 2018].

35

Use case no 02

Use case Get the translation output in English audio



Flow of events 1. Select the translation mode as audio

2. Select the translation language as English




42

TABLE OF CONTENTS - project.sliit.lkproject.sliit.lk/.../download/3168/SRS_udani.docx · Web...

Documents

Transcript of TABLE OF CONTENTS - project.sliit.lkproject.sliit.lk/.../download/3168/SRS_udani.docx · Web...