SEGMENTATION FOR HANDWRITTEN DOCUMENTS

Omar Alaql

Fab. 20, 2014

Outline

• Optical Character Recognition (OCR).• OCR for the Historical Documents.• Text Lines Segmentation Approaches.

Profile Projection. Hough Transform. Level Set Method. Affinity Propagation. Steerable Directional Technique.

Optical Character Recognition (OCR)

• The electronic translation of images into machine-editable text.

Input Image Text

Optical Character Recognition (OCR)

• There are four major stages which must be done in any optical characters recognition:

1) Preprocessing.2) Segmentation.3) Feature extraction.4) Recognition.

Optical Character Recognition (OCR)

• Preprocessing:– Noise reduction.– Binarization or Gray scale image. – Compression in the amount of data to be analyzed.

• Segmentation:– The isolation of various writing units, such as paragraphs,

sentences, words, or letters.

Optical Character Recognition (OCR)

Text Lines Segmentation

• Representation:– Extracts the most relevant information from the text

image which helps the recognition stage to recognize the text.

– This information is the features of each symbol that is needed to distinguish it from other symbols.

Optical Character Recognition (OCR)

• Recognition:– Recognition stage is the last and the main decision

making stage. – It is a classification process that identifies each unknown

symbol and assigns it into a predefined class. – This classification is based on the extracted features

which are the output of the previous stage.

Optical Character Recognition (OCR)

• Historical documents processing is a challenging task for various reasons:1) Lack of standard alphabets and presence of unknown fonts.2) Low quality.

OCR for the Historical Documents

OCR for the Historical Documents3) The lack of constraints on page layout.

OCR for the Historical Documents4) The complexity of handwriting.5) The variability of skew between the different text-lines and within the same text-line.

6) Spaces between lines are narrow and variable.7) The existence of small components.

8) Distinguishing noise from text.

OCR for the Historical Documents

Narrow lines

Small Components

Noise

Text Lines Segmentation Approaches

• There are many techniques for text lines segmentation: Profile Projection. Hough Transform. Level Set Method. Affinity Propagation. Steerable Directional Technique.

Projection Profile

• Summing pixel values along the horizontal axis for each y value.

Horizontal Projection

Projection Profile

• Example: Input image.

Projection Profile

• Example: Skew Correction.

Projection Profile

• Example: Horizontal Projection.

Projection Profile

• Example: Peaks detection

• Example: Positions for segmentation.

Projection Profile

• Example: Image for each text line.

Projection Profile

• For skewed or fluctuating text lines, the image may be divided into vertical strips .

• Subdivision the page into columns. • Determination of the minimal values of the

histograms resulting from horizontal projections for all the columns.

• Drawing horizontal stroke by means of each minimal value inside a column.

• The link between these strokes allows the separation of two adjacent lines.

Projection Profile

Partial Projection Method

Projection Profile

Hough Transform

• The Hough transform is used for locating straight lines in images.

• Text line is best align matches the black pixels.• Any black pixel has an infinite number of lines that

could pass through this pixel.

• There are two ways to represent the lines :– y = mx + c– x cos θ + y sin θ = ρ

Each line has a unique value (m , c) or (ρ, θ) which is called accumulator.

There is a vote for the accumulator when the line passes through a black pixel.

The text line is the line that has the maximum accumulator.

Hough Transform

Level-set Method• Instead of directly segmenting on a binary image, it is converted

to a probability map, where each element represents the probability of this pixel belonging to a text line.

Input image Probability Map

Level-set Method• The probability map is analyzed using the level set method to

segment text lines by determining the boundary of neighboring text lines.

• The zero value for the boundary, automatically grows, merges, and stops to the final text line boundary.

Initial estimate of text lines Result after 10 iterations

By Level Set, text lines are

horizontally elongated

Connected Components Clustering

• Grouping many connected components in a cluster by using grouping algorithms, each cluster represents a separate text line.

Connected Components

Affinity Propagation

• The algorithm first estimates local orientation at each primary component of a word to build a sparse similarity graph.

At each point, the region is divided into five regions.

The Breadth-First Search algorithm is applied to find disjoint sets in the similarity graph.

There exist a path from each element to every other element in the set.

Steerable Directional Local Profile Technique

• One of the connected components based approaches is steerable directional technique.

• Adaptive local connectivity map (ALCM) is generated using a steerable directional filter.

Steerable Directional Local Profile Technique

• Firstly, a steerable filter is used to determine foreground intensity along multiple directions at each pixel while generating the ALCM.

Text image ALCM

Steerable Filter

Steerable Directional Local Profile Technique

• The ALCM is then binarized using an adaptive thresholding algorithm to get a rough estimate of the location of the text lines.

ALCM Text Line Location

Binarization

• This approach has difficulties and limitations when it comes to the binarization of the ALCM images.

• Especially when text lines in the document are very close to each other.

Steerable Directional Technique

• To solve the problem: 1) Steerable dynamic directional filter is applied. Angle value is taken instead of the density value.

Steerable Directional Technique

Input image

Text Direction Map

2) apply a mode filter to extract each paragraph in the document and its orientation.

Steerable Directional Technique

Paragraph Map

Steerable Directional Technique

Input Image

Paragraph Map

Steerable Directional Technique

3) a steerable static directional filter is applied.- the direction of the kernel is taken from the paragraph map.

Steerable Directional Technique

4) Thresholding

Text lines patterns

Horizontal Projection Technique• To use Projection Technique:– First : paragraph segmentation.

Paragraph Map

Paragraphs Segmented

Horizontal Projection Technique• To use Projection Technique:– Second: Skew Correction.

After skew correction

Paragraphs Segmented

Horizontal Projection Technique• To use Projection Technique:– Third: Horizontal Projection.

Horizontal Projection Technique• To use Projection Technique:

– Fourth: Profile Analysis.• There are some drawbacks makes finding he maximum and the minimum

in the profile more complicated.

Short line will provide low peak that might be ignored

very narrow lines, or the lines that including many overlapping components will not produce significant peaks

Horizontal Projection Technique• To use Projection Technique:– Fourth: Profile Analysis.• To solve this problem, the profile should be smoothed.