Smart Traveller with Visual Smart Traveller with Visual TranslatorTranslator

What is Smart Traveller?What is Smart Traveller?

• Mobile Device which is convenience for a Mobile Device which is convenience for a traveller to carrytraveller to carry

• E.g. Pocket PC, Mobile PhoneE.g. Pocket PC, Mobile Phone

What is Visual Translator?What is Visual Translator?

• Recognize the foreign text and Recognize the foreign text and translate it into native languagetranslate it into native language

• Detect the face and recognize it into Detect the face and recognize it into namename

RequirementsRequirements

• Simple (Computational low power)Simple (Computational low power)

• Lightweight (Low Storage)Lightweight (Low Storage)

• User FriendlyUser Friendly

Core Pattern Recognition Core Pattern Recognition ModelModel

ImageSegmentation

Feature Extraction

Classification

Input Image Object Image Feature Vector Object Type

Find Each Object from the Image

Quantify the object by some characteristics

Assign Label for each object

Character RecognitionCharacter Recognition

• Language: KoreanLanguage: Korean

• Target: Sign, GuidepostTarget: Sign, Guidepost– Contrast in ColorContrast in Color– Printed CharacterPrinted Character

Image SegmentationImage Segmentation

• BinarizationBinarization– Using Color Histogram to binarize the image for the Using Color Histogram to binarize the image for the

background and the characterbackground and the character• Text Region SegmentationText Region Segmentation

– User Define MethodUser Define Method– Edge Detection with horizontal and vertical projectiEdge Detection with horizontal and vertical projecti

onsons• Stroke ExtractionStroke Extraction

– Labeling of connected component AlgorithmLabeling of connected component Algorithm

Feature ExtractionFeature Extraction

• Stroke FeaturesStroke Features– Number of Junctions, CornersNumber of Junctions, Corners– Any HoleAny Hole

• Gabor FeaturesGabor Features

RecognitionRecognition

• Minimum Euclid DistanceMinimum Euclid Distance

• Learn the Decision Tree by training Learn the Decision Tree by training examplesexamples

DemoDemo

Face DetectionFace Detection

OutlineOutline•Find Face RegionFind Face Region

•Find the potential eye regionFind the potential eye region

•Locate the iris and eyelidsLocate the iris and eyelids

Find Face Region - Color-based Find Face Region - Color-based modelmodel

• We used this method because of its We used this method because of its simplicity and robustness.simplicity and robustness.

• Usually RGB color model will be Usually RGB color model will be transformed to other color modes transformed to other color modes such as YUV (luminance-such as YUV (luminance-chrominance) and HSB (hue, chrominance) and HSB (hue, saturation and brightness)saturation and brightness)

YUVYUV

• We use YUV or YCWe use YUV or YCbbCCr r color model.color model.• Y component is used to represent the intY component is used to represent the int

ensity of the imageensity of the image• CCb b and Cand Cr r are used to represent the blue aare used to represent the blue a

nd red component respectively.nd red component respectively.

YCYCbbCCr r ImageImage

• Y, Cb ,Cr component imageY, Cb ,Cr component image

Y Cb Cr

Representation of skin colorRepresentation of skin color

• We just use a We just use a simple ellipse simple ellipse equation to equation to model skin color.model skin color.

Cb

Cr

Representation of skin colorRepresentation of skin color

• The white regions represent the skin color pixels

Color segmentationColor segmentation

• We distribute some agent in the image We distribute some agent in the image uniformly.uniformly.

• Then each agent will check whether the Then each agent will check whether the pixel is a skin-like pixel and not visited by pixel is a skin-like pixel and not visited by the other agent.the other agent.

• If yes, it will produce 4 more agents at its If yes, it will produce 4 more agents at its four neighboring points.four neighboring points.

• If no, it will move to one of four neighboring If no, it will move to one of four neighboring points randomly and decrease its lifespan points randomly and decrease its lifespan by 1. When its lifespan becomes zero, it will by 1. When its lifespan becomes zero, it will be removed from the image.be removed from the image.

Color segmentationColor segmentation• This agent produce 4

more agents

Color segmentationColor segmentation

• The advantage of this algorithm is The advantage of this algorithm is that we need not to search the whole that we need not to search the whole image.image.

• Therefore, it is fast.Therefore, it is fast.

Color segmentationColor segmentation

• 19270 of 102900 19270 of 102900 pixels is searched pixels is searched (about 18.7%)(about 18.7%)

• There are 37 There are 37 regionsregions

• Each color regions Each color regions represent each represent each regions searched regions searched by a father agentby a father agent

Eye detectionEye detection

• After the segmentation of face After the segmentation of face region, we have some parts which region, we have some parts which are not regarded as skin color.are not regarded as skin color.

• They are probably the region of eye They are probably the region of eye and mouthand mouth

• We only consider the red component We only consider the red component of these regions because it usually of these regions because it usually includes the most information about includes the most information about faces.faces.

Eye detectionEye detection

• We extraction such We extraction such regions.regions.

• The red region The red region represent the represent the region which is not region which is not skin color.skin color.

Eye detectionEye detection

We do the following on the regions of We do the following on the regions of potential eye regionpotential eye region

1.1. Histogram equalizationHistogram equalization

2.2. ThresholdThreshold

3.3. Template matchingTemplate matching

Eye detectionEye detection

Histogram equalizationHistogram equalization

Threshold with < 49Threshold with < 49

Template MatchingTemplate Matching

Locating the iris and eyelidsLocating the iris and eyelids

We plan to use the following methods to iWe plan to use the following methods to improve the face detectionmprove the face detection

We can use these methods to locate the iriWe can use these methods to locate the iris and eyelid precisely.s and eyelid precisely.

Template matchingTemplate matching– Correlation variance filterCorrelation variance filter– Deformable templateDeformable template

ENDEND