Yalin Zhang [email protected]. Huffman Coding Variations of Huffman Coding Length-limited coding...

17
A Generic Top-Down Dynamic-Programming Approach to Prefix- Free coding Yalin Zhang [email protected]
  • date post

    23-Jan-2016
  • Category

    Documents

  • view

    229
  • download

    0

Transcript of Yalin Zhang [email protected]. Huffman Coding Variations of Huffman Coding Length-limited coding...

Page 1: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

A Generic Top-Down Dynamic-Programming Approach to

Prefix-Free coding

Yalin [email protected]

Page 2: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Huffman CodingGiven a probability distribution over a set of

n words, find a minimal-cost prefix-free coding for these words.

Input: alphabet , probability Output: code Cost =

Page 3: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Huffman CodingComplexity greedy algorithm for binary case.Complexity P is sorted, r-ary case.

What about extra constraints or generalization?

Page 4: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Variations of Huffman CodingLength-limited codingUnequal-cost codingMixed-radix codingReserved-length codingOne-ended coding

Page 5: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Variations of Huffman Coding

Standard binary tree

Mix-radix tree Reserved-length tree

General approach: Dynamic Programming to build an optimal coding tree.

Page 6: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Top-Down DP for GMR codingGeneralized-Mixed-Radix problemBoth the arity of an internal node and the

length of an edge depends on its level.Given a sequence of arities R = (r1, r2, . . .)

and a sequence of edge lengths C = (c1, c2 . . .), a generalized mixed radix (GMR) tree T satisfying R, C is a tree in which internal node v at level i − 1 has at most ri children and and the length of an edge from v to any of its children is ci

Page 7: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Top-Down DP for GMR codingThe depth of a node on level i of the tree will

be thesum of the lengths of the edges on the path from the root to level i.

Cost(T) =

The problem to be solved is, given P and R,C, find a min-cost tree T∗ with n leaves Cost(T∗) = min {Cost(T) | T has n leaves}

Page 8: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Top-Down Dynamic ProgramIn , ≥ ≥ … ≥ ≥ 0;For i > n, = 0;There is a full tree T∗ with the same cost as

the optimal tree for n leaves. T∗ has n′ leaves where

max(n, r (T∗)) ≤ n′ ≤ n + r (T∗) − 1ℓ ℓThe cost of a tree is fully determined by the

number of leaves on each level.

Page 9: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Top-Down Dynamic ProgramBuild optimal trees top-down, starting with a

root;Expand tree level-by-level by making some

nodes on the bottom level internal.Until a tree whose bottom level contains no

internal nodes.

Page 10: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Top-Down Dynamic Programi-level signature is an ordered pair

in which m = b = |{}| Partial cost of T is

Page 11: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Top-Down Dynamic Program

P={1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}

Page 12: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Top-Down Dynamic Program

Let T′ be an (i − 1)-level tree with. Expand T′ to a full i-level tree by adding all nodes on level i.(m’, b’)->(m,b), where m = m’ + b’ri – b(0, 1)->…->(n’, 0)

Page 13: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Top-Down Dynamic Program

Objective: Complexity? Speedup by batching approach

Page 14: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Batching for Speed up

Page 15: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Batching for Speed up

If

Else

Page 16: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Batching for Speed up For fixed i, we can fill all valid signatures (m, b), where m+b=d, in time.d is bounded by n.We can fill all valid signatures in level i in

time.The overall time complexity is

Page 17: Yalin Zhang zhangyl@tamu.edu. Huffman Coding Variations of Huffman Coding Length-limited coding Unequal-cost coding Mixed-radix coding Reserved-length.

Results