editDist.pdf

Transcript of editDist.pdf

Spelling Correction: Edit Distance

Pawan Goyal

CSE, IITKGP

July 25, 2014

Pawan Goyal (IIT Kharagpur) Spelling Correction July 25, 2014 1 / 26
Spelling Correction

I am writing this email on behaf of ...The user typed behaf.

Which are some close words?behalf

behave

....

Isolated word error correctionPick the one that is closest to behaf

How to define closest?

Need a distance metric

The simplest metric: edit distance

Spelling Correction

I am writing this email on behaf of ...

The user typed behaf.


behave

....





Spelling Correction


Which are some close words?

behalf

behave

....





Spelling Correction



behave

....





Spelling Correction



behave

....





Spelling Correction



behave

....





Spelling Correction



behave

....





Spelling Correction



behave

....





Edit Distance

The minimum edit distance between two strings

Is the minimum number of editing operations

I InsertionI DeletionI Substitution

Edit Distance




Edit Distance




Minimum Edit Distance

ExampleEdit distance from intention to execution


ExampleEdit distance from intention to execution


If each operation has a cost of 1I Distance between these is 5

If substitution costs 2 (Levenshtein)I Distance between these is 8

How to find the Minimum Edit Distance?

Searching for a path (sequence of edits) from the start string to the final string:

Initial state: the word we are transforming

Operators: insert, delete, substitute

Goal state: the word we are trying to get to

Path cost: what we want to minimize: the number of edits

Minimum Edit as Search

How to navigate?The space of all edit sequences is huge

Lot of distinct paths end up at the same state

Dont have to keep track of all of them

Keep track of the shortest path to each state

Defining Minimum Edit Distance Matrix

For two stringsX of length n

Y of length m

We define D(i, j)

the edit distance between X[1..i] and Y[1..j]

i.e., the first i characters of X and the first j characters of Y

Thus, the edit distance between X and Y is D(n,m)



Y of length m

We define D(i, j)






Y of length m

We define D(i, j)




Computing Minimum Edit Distance

Dynamic Programming

A tabular computation of D(n,m)

Solving problems by combining solutions to subproblemsBottom-up

I Compute D(i, j) for small i, jI Compute larger D(i, j) based on previously computed smaller valuesI Compute D(i, j) for all i and j till you get to D(n,m)


Dynamic Programming


Solving problems by combining solutions to subproblems

Bottom-upI Compute D(i, j) for small i, jI Compute larger D(i, j) based on previously computed smaller valuesI Compute D(i, j) for all i and j till you get to D(n,m)


Dynamic Programming





Dynamic Programming



I Compute D(i, j) for small i, j

I Compute larger D(i, j) based on previously computed smaller valuesI Compute D(i, j) for all i and j till you get to D(n,m)


Dynamic Programming



I Compute D(i, j) for small i, jI Compute larger D(i, j) based on previously computed smaller values

I Compute D(i, j) for all i and j till you get to D(n,m)


Dynamic Programming




Dynamic Programming Algorithm

The Edit Distance Table

Computing Alignments

Computing edit distance may not be sufficient for some applicationsI We often need to align characters of the two strings to each other

We do this by keeping a backtrace

Every time we enter a cell, remember where we came fromWhen we reach the end,

I Trace back the path from the upper right corner to read off the alignment


Computing edit distance may not be sufficient for some applications

I We often need to align characters of the two strings to each other







Every time we enter a cell, remember where we came from

When we reach the end,I Trace back the path from the upper right corner to read off the alignment

Minimum Edit with Backtrace

Adding Backtrace to Minimum Edit

The distance matrix

Every non-decreasing pathfrom (0,0) to (M,N)corresponds to an alignmentof two sequences.

An optimal alignment iscomposed of optimalsub-alignments.

The distance matrix



The distance matrix



Result of Backtrace

Performance

Time

O(nm)

Space

O(nm)

BacktraceO(n+m)

Performance

TimeO(nm)

Space

O(nm)

BacktraceO(n+m)

Performance

TimeO(nm)

Space

O(nm)

Backtrace

O(n+m)

Performance

TimeO(nm)

Space

O(nm)

BacktraceO(n+m)

Weighted Edit Distance

Why to add weights to the computation?Some letters are more likely to be mistyped.

Confusion Matrix for Spelling Errors

Keyboard Design

Weighted Minimum Edit Distance

Another Application: Cognate Words

Cognate WordsIn linguistics, cognates are words that have a common etymological origin.

Cognate words for night (English)nuit (French), Nacht (German), nacht (Dutch), nag (afrikaans), nakta(Sanskrit), noch (Russian), notte (Italian), noite (Portugese)


Replacement from gh to k or g should have lesser cost (weight) than fromgh to t, for instance.


Replacement from gh to k or g should have lesser cost (weight) than fromgh to t, for instance.

editDist.pdf

Documents

Transcript of editDist.pdf