Efficient Viterbi algorithms for lexical tree based models

IntroductionAlgorithms for expanded tree models

Error-Correcting Viterbi for DAGsExperimental results and conclusions

Efficient Viterbi algorithms for lexical treebased models

S. España Boquera, M.J. Castro Bleda, F. ZamoraMartínez, J. Gorbe Moya

Departamento de Sistemas Informáticos y ComputaciónUniversidad Politécnica de Valencia, Spain

22-25 May 2007, Paris, France

S. España, M.J. Castro, F. Zamora, J. Gorbe NOLISP 2007

1 Introduction

2 Algorithms for expanded tree modelsLeft-to-right algorithmExtension to across-word context dependent unitsLeft-to-right with skips algorithm

3 Error-Correcting Viterbi for DAGsVERTEX-STEP algorithmEDGE-STEP algorithm

4 Experimental results and conclusionsExperimental resultsConclusionsConclusions

Viterbi algorithms for Automatic Speech Recognition

Hidden Markov Models (HMM) and Finite State Automata(FSA) are the most widespread used models for AutomaticSpeech Recognition (ASR). The Viterbi algorithm is usedfor decoding.Most of large vocabulary Viterbi based decoders make useof a lexicon tree organization together with pruningtechniques such as beam search which only maintainactive the best hypothesis.

Why to use a tree lexicon?

The lexical tree organization is a very good tradeoff betweencompact space representation and adequacy for decoding:

A lexicon tree organization (which has many advantagesover a linear lexicon representation) reduces the size of themodel.More compact representations are possible (lexiconnetwork), but the gain in space is accompanied with amore complex Viterbi decoder.

How to use a tree lexicon?

Large vocabulary one-step decoders usually keep a set oflexical tree based Viterbi parsers in parallel. Two commonapproaches are:

time-start copies All hypothesis competing in a treeparsing share the same word start time.language model history copies When a trigramlanguage model is used, the language model historycopies approach maintains a tree parsing for every bigramhistory (w1, w2).

The second approach has a loss of optimality which is knownas word-pair approximation.

Contributions of this work

In both time-start and language model history copies, fasterViterbi algorithms for the lexical tree model have a significantimpact in the overall performance of the ASR system.

This work proposes Viterbi algorithms specialized for lexicaltree based FSA and HMM acoustic models in order to:

Decode a tree lexicon with left-to-right models VITERBI-M

Decode a tree lexicon with left-to-right models with skipsVITERBI-MS

Parse a directed acyclic graph (DAG) (e.g. a phone graphor a word lattice) with error correcting edition operations(insertion, substitution, deletion) VITERBI-MEC-DAG

Left-to-right algorithmExtension to across-word context dependent unitsLeft-to-right with skips algorithm

1 Introduction

Observations about the Expanded Tree Models

If a lexicon tree is expanded with left-to-right acoustic HMMmodels without skips, we can observe that

Every state has at most two predecessors: itself andpossibly his parent.If we ignore the loops, the expanded model is acyclic.Therefore, a topological order is possible in general.A level traversal of the tree provides a topological orderwith some additional features:

Children occupy contiguous positions.If a subset of states is stored in topological order and wegenerate the children of every active state following thatorder, the resulting list is also ordered with respect to thetopological order.

Model representation

A tree model T of n states is represented with three vectors ofsize n and one of size n + 1 as follows:

loop_prob stores the loop transition probabilities.from_prob stores the parent incoming transitionprobabilities.e_index stores the index of the associated emissionprobability class associated to the acoustic frame to beobserved.first_child stores the index of the first child.

VITERBI-M algorithm

Extension to across-word context dependent units

Coarticulated word transitions: First phone(s) of wordsdepends on the last phone(s) of the previous words.A model which resembles a tree lexicon excepting the rootis needed.

phonetic contextsDifferent

Tree lexicon HMM models

The VITERBI-M can be easily adapted to this structure bytreating specially the general topology at the root.

Left-to-right with skips algorithm (VITERBI-MS)

Left-to-right units with skips are known as Bakis topology.

The states of the expanded tree lexicon can have one, twoor three predecessors.The model representation is similar to that of VITERBI-M

algorithm but another vector skip_prob is needed to store,for every state, the incoming skip transition probabilities.The algorithm uses another auxiliary queue to store theactive states with scores computed by means of the skiptransitions.

Left-to-right with skips algorithm (VITERBI-MS)

α (t)

generate children

generate grandchildren

>= beam threshold?

*loop probability

*emission probabilityupdate best probability

aux_child

aux_gchild

VERTEX-STEP algorithmEDGE-STEP algorithm

1 Introduction

Error-Correcting Viterbi for DAGs

The input is a directed acyclic graph (phone graphs, wordlattices, etc.).Uses error correcting decoding with the following editionoperations: insertions, substitutions and deletions.The VITERBI-MEC-DAG algorithm is a combination of:

VERTEX-STEP algorithm,EDGE-STEP algorithm.

following the input DAG topologic order:

VERTEX-STEP algorithm

Is applied for every vertex when all edges arriving to ithave been processedThis procedure only considers the cost of insertions.Several consecutive insertions are possible.A sole active state at the root could, in principle, activate allthe states of the model. Beam search can be used tocontrol this growth.

EDGE-STEP algorithm

Is applied for every edge after the origin vertex has beenupdated with VERTEX-STEP algorithm.Destination vertex is updatedThis procedure only considers the cost of

Deletions (the edge of input DAG is traversed but edgelabel is deleted).Substitutions (including a symbol by itself or a correcttransition).

EDGE-STEP algorithm

Experimental resultsConclusionsConclusions

1 Introduction

Experimental results Viterbi-M

In order to compare the performance of our Viterbi-M algorithm,three algorithms have been implemented in C++ and testedwith the same data structures to represent tree based HMMmodels:

VITERBI-M described in this work.A conventional Viterbi. algorithm based on two hash tableswith chaining to store and to look up the active states.The active envelope algorithm: uses a single linked list tostore the set of states in topologic order. Since thisalgorithm modifies the original set of active states toperform a Viterbi step, it is restricted to sequential inputdata and cannot be used when the input data is a DAG

Experimental results

The lexical trees used in the experiments were obtained byexpanding 3-state left-to-right without skips hybridneural/HMM acoustic models the tree lexicon.Only the Viterbi decoding time has been measured (theemission scores calculation and other preprocessing stepswere not taken into account).The results are shown in millions of active states orhypothesis updated per second

Num. states Hash swap A. Envelope Viterbi-M9 571 3.001 16.082 29.350

76 189 2.761 12.924 28.036310 888 1.922 6.442 24.534

Conclusions

The proposed algorithms are based on contiguousmemory queues which contain the set of active statestopologically sorted.Less operations in the internal loops and more locality ofdata (cache friendly algorithm) are expected.Although the asymptotic cost of these algorithms is thesame as any reasonable implementation, VITERBI-M isapproximately 10 times faster than the hash-tableswapping implementation and from 2 to 4 times faster thanthe active envelope algorithm.

Conclusions

More experimentation is needed in order to betterunderstand a decrease in speed with the size of themodels and the effect of other parameters such as thebeam width of the pruning during the search. We arepreparing experiments using HMM models trained withHTK toolkit and the the Wall Street Journal corpus.Two word graphs algorithms are being implemented:

Using a tree lexicon with the time-start and the VITERBI-Malgorithm.Using a phone graph generator algorithm and theVITERBI-MEC-DAG algorithm afterwards to obtain thedesired word graph.

Efficient Viterbi algorithms for lexical tree based models

Technology

Transcript of Efficient Viterbi algorithms for lexical tree based models