LYU0203Smart Traveller with Visual Translatorfor OCR and Face Recognition

Supervised by Prof. LYU, Rung Tsong Michael

Prepared by: Wong Chi Hang

Tsang Siu Fung

Department of Computer Science & Engineering

The Chinese University of Hong Kong

Outline

Introduction System Architecture Korean OCR Friend Reminder Conclusion Acknowledgement

Introduction – What is VTT?

Smart Traveller with Visual Translator (VTT)Mobile Device which is convenient for a

traveller to carry Mobile Phone, Pocket PC, Palm, etc.

Recognize and translate the foreign text into native language

Detect and recognize the face into name

Introduction – Objective

Two main features: Korean to English

Visual Translation

Remind Somebody’s Information with Face Image

Introduction – Objective (Cont.)

Real Life ExamplesSometimes we lose the way, we need to know

where we are.Sometimes we forget somebody we met

before.

System Architecture

GUI

Camera API

Camera

Korean OCR Face Recognizer

Face DatabaseStroke Database

&Dictionary

Request

RequestData

Data

RequestOutput

User

QueryResult Query UpdateResult

Request Response

Request Response Request Response

Korean OCR (KOCR) Usage

Visual Translator from Korean to English Procedure for using KOCR

Text Area DetectionCharacter IdentificationTranslation

KOCR – Program FlowInitialization

Capture Image

Text Segmentation

Recognition

Translation

KOCR – Text Area Detection

Edge Detection using Sobel Filter Horizontal Projection and Vertical Projection Find Potential Text Area by threshold

-1 -2 -1

0 0 0

1 2 1

-1 0 1

-2 0 2

-1 0 1

Hor

izon

tal

Pro

jec

tion

Threshold

Vertical Projection

KOCR – Text Area Detection (Cont.)

KOCR – Character Identification

Features on Stroke Extracted by Labeling Connected Component

algorithm

Proposed Feature Extraction Five rays each side Difference of adjacent rays (-1 or 0 or 1) Has holes (0 or 1) Dimension ratio of Stroke (width/height) (-1 or 0 or 1)

KOCR – Character Identification (Cont.)

KOCR – Translation

Dictionary Korean to English About 1000 Korean Words

Matching Longest Match from left to right

KOCR – Translation (Cont.)

KOCR – Evaluations

OCR CorrectnessTraining Set (3327 – 30% of all Character)Testing Set (7845 – Others)Result (64%)Suggestion

Train all Korean characters

KOCR – Evaluations (Cont.)

Text Segmentation Correctness45 Captured Images99 CharactersResult

Segment 83% characters correctly Segment 71% image correctly

Acceptable Result

KOCR – Evaluations (Cont.)

OCR Correctness45 Captured Images99 CharactersResult

79% Characters correctly Recognized 69% Images correctly Recognized

Friend Reminder – Program FlowInitialization

Capture Image

Face Segmentation

Recognition

Show Profile

Friend Reminder (FR)

UsageShow the Profile of Friend by capturing a

photo Procedure for using FR

Face SegmentationFace IdentificationFriend’s Profile

FR – Face Segmentation

Eye DetectionAlgorithm

Gabor Wavelet Feature Log-Polar Sampling

Manual Selected (Suggest) Selected Eyes and Mouth Positions

FR – Face Segmentation

FR – Face Identification

EigenFaceBy using Principal Component Analysis (PCA)Project the input face into the eigenvectors th

at pre-learnedFind the difference between the projection an

d the faces in databaseFace determined to be ‘NEW’ if the difference

is larger than a threshold

FR – Friend’s Profile

FR – Evaluations

Eye Detection Correctness40 ImagesResult

22.5% Image Successfully Detected

Non-acceptableSuggestion

Manually Select Eyes and Mouth Positions

FR – Evaluations Face Identification

Evaluation Information 26 Test Persons’ Faces

16 faces is in database 10 faces is not in database

3 faces Trained per person 8 persons in face database

Result 77% Successfully Identified

63% Successfully Identified as person in database 100% Successfully Identified as person not in database

Conclusion

Combined Modern Equipments Digital camera Personal Data Assistant (PDA)

Techniques Learned Image Processing Optical Character Recognition Face Recognition Techniques

VTT Integrated VTT for Korean to English OCR VTT for Friend Reminder

Acknowledgement

Thanks Professor Michael Lyu,Project SupervisorGive us valuable adviceProvide us necessary equipments

Thanks Edward Yau,Technical Manager of VIEW projectGive us many ideas

~The End~