MTH 3-13a Linear Patterns Linear Patterns Simple Linear Patterns Complicated Linear Patterns Simple…
Chapter 3 Linear Block Codessite.iugaza.edu.ps/mtastal/files/ch3new2016Full.pdfSystematic codes •...
Transcript of Chapter 3 Linear Block Codessite.iugaza.edu.ps/mtastal/files/ch3new2016Full.pdfSystematic codes •...
Chapter 3
Linear Block CodesDr.Mohammed Taha El Astal
Outline
• Introduction – Definition of Linear, block and systemic codes.
• Encoding process– Generator matrix and parity-check matrix
– Encoding circuit
• Decoding process– Standard Arrays decoding.
– Syndrome decoding.
• Error detecting and correcting capabilities.
• Probability of decoding error over BSC.
• Examples: Single-Parity-check codes, repetition codes, and self-dual codes
2
Introduction
• As u is k-tuple, there are a total of 2k distinct
message.
• v is referred as codeword/ code vector of the
message u.
• Therefore, corresponding to the 2k possible
messages, there are 2k codewords.
• A total of 2k n-bit codewords (called block code)
constitute (n, k) code. 3
Sequence of binary digits
01010111101100….
Segmentation (i.e. k=4) u
(0101)(0111)……
Transformation: According to certain
rules, each u transforms in to block of n-tuple (v)
Cont.
• There should be a total of 2k distinct
codewords, why?
• Unless certain special structure was
used, the encoding will be complex
for a large k and n, why?
• What do you propose to avoid this?
How?
4
To ease the recovery (decoding). Otherwise, it is impossible to
recover data
Need to store 2k(n+k) digits
Definition of Linear Codes
• Linear code: The sum of any two codewords
is a codeword.
• Observation: The all-zero sequence is a
codeword in every linear block code.
5
Linear Code Example
6
• Check if linear? By definition: Its
codes words form 4-dimenstional
subspace of V7 over GF(2)yes
• By properties :
• The sum of any two codewords is a
code word yes
• The all-zero sequence is a code word
yes
• Result: These two properties are
results of the definition conditions.
Systematic codes
• Any linear block code can be put
in systematic form
• It is referred as linear systematic
block codes
• Table 3.1: 3 left-most bit for
parity-check digits and 4 right-
most bits for message digits
7
n-k
check bits
k
information bits
Generator Matrix
• All 2k codewords of an (n,k) linear code C can be
generated from a set of k linearly independent codewords,
why and why k?
• Let g0, g1, …, gk-1 be a set of k independent codewords.
8
Because an (n,k) linear code C is a k-dimensional subspace of the vector space Vn of all the
binary n-tuples, it is possible to find k linearly independent codewords, g0,g1,…gk-1 in C
such that every codeword v in c is alinear combination of these k codewords
Generator Matrix (cont)
• Encoding:
• Clearly, the rows of G generate (or span) the (n,k) linearcode C. This is it called Generator Matrix
• It can be noted that an (n,k) code C is completely specifiedby the k rows of a G matrix. Why it is big feature?
9
Storage requirement reduced from 2k(n+k) to k(n-k).
Example 3.1
10
• Find G for the code shown in Table 3.1?
• Using G, find v if u =1101 (check with Table 3.1)
• N.B. The simplest choice of this set is the k codewords
corresponding to the information sequences that have a
single nonzero element.
Generator Matrix (cont)
Systematic linear block codes
11
Generate identical as the
message information u ( k-tuple)
Generate the parity-check bits
(n-k)-tuple
• P matrix is created using the parity-check equation of the code
and vice versa.
Generator Matrix (cont’d)
• As shown in Example 3.2, G follows the systematic structure of
G
12
G P | Ik
1 1 0
0 1 1
1 1 1
1 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
k n k k k( )
Example 3.2
13
• Using G of Example 3.1, find the parity-check
equations?
• Using these equation, find v if u=1011, check
using table 3.1
Parity-Check Matrix
14
• Hence, any (n,k) linear block code can be represented in alternative way as follow:
– An n-tuple v is a codeword in the code C generated by G iff v. HT =0
• The code is said to be the null space of H.
• This matrix H is called a parity-check matrix of the code C.
Parity-Check Matrix
Example: Find H for thee code given in Example 3.1?
• Check that GHT = 0? Generalize?
• Using H, find the parity-check equations?15
H
1 0 0 1 0 1 1
0 1 0 1 1 1 0
0 0 1 0 1 1 1
1000101
0100111
0010110
0001011
G
Encoding Using H Matrix
(Parity Check Equations)
16
3212
2101
3200
6542
5431
6530
6542
5431
6530
6543210
uuu=v
uuu=v
uuu=v
vvv=v
vvv=v
vvv=v
0vvv+v
0vvv+v
0vvv+v
101
111
110
011
100
010
001
vvvvvvv
0
information
Encoding Circuit
17
Example : Draw the encoding circuit of the code
given in Example 3.1
18
The Dual Code
• The H matrix, formed from G, has (n-k) rows
of length n that are linearly independent.
• The H matrix can therefore be used to generate
an (n, n-k) linear code Cd.
• Cd is called the dual code of the code C
generated by G.
19
Syndrome and error detection
• Because of the channel noise, r may be different
from v. The vector sum:
is referred as error vector/ error pattern.
20
v=(v0,v1,….vn-1) Noisy
channelr=(r0,r1,….rn-1)
Syndrome and error detection (cont.)
• Define the syndrome of r to be
– Note that r=v+e.
– When s≠ 0, the presence of error has been detected.
– When s= 0, the rx. assign r as the transmitted v.
• Its possible that the errors are not detectable (undetectable error
pattern). When?How many in (n, k) code?
• The syndrome depends only on the error pattern but not on the
transmitted codeword, show?
21
If e is identical to a nonzero codeword
Because there are 2k-1 nonzero codeword, there are 2k-1 undetectable error pattern
Syndrome and error detection (cont.)
22
Syndrome circuit
23
Example: Draw syndrome circuit of code given in
Table 3.1
24
Example 3.5: For the (7, 4) code, let v=(1001011) and
r=(1001001). Decode r?
• s = rHT=111.
25
H
1 0 0 1 0 1 1
0 1 0 1 1 1 0
0 0 1 0 1 1 1
More on the Minimum Distance of Linear Codes
• DF: The Hamming weight of a codeword v , denoted by w(v),
is the number of the nonzero elements in the codeword.
• Example: Let v={1001011}, then w(v)=?
• DF: Hamming distance between v and w, denoted by d(v,w),
is the number of locations where they differ.
• Example:Let v={1001011}and w={0100011}then d(v,w)=?
26
4
3
N.B. Hamming distance satisfies the triangle inequality. Let v,w and x be three n-tuples then d(v,w)+d(w,x) ≥ d(v,x)
N.B. d(v,w)=w (v+w), why?Example?
This is result from the definition of d and the definition of module-2 addition
v={1001011}, w={1110010}, check?
More on the Minimum Distance of Linear Codes
• DF: The minimum weight of a code, wmin, is the smallest
weight of the nonzero codewords in the code.wmin = min {w(v): for every v in C; v ≠0}.
• DF: The minimum distance of the code dmin = min {d(v,w): v,w in C, v ≠ w}
• TH3.1: In any linear code, dmin = wmin, why?
27
If C is a linear block
code, the sum of two
codewords is also a
codeword.
Minimum Distance, Cont.
• Theorem 3.2: For each codeword of Hamming weight ℓ there
exists ℓ columns of H such that the vector sum of these
columns is zero. Conversely, if there exist ℓ columns of H
whose vector sum is zero, there exists a codeword of weight ℓ.
• Corollary 3.2.2: dmin of C is equal to the minimum numbers of
columns in H that sum to zero.
• Example:
Given H matrix of (7,4), find dmin ?
28
1110100
0111010
1101001
H
Error detecting capability
• A block code with dmin is capable of detecting all error patterns of dmin -1 or
fewer errors. Show?
• However, it cannot detect all the error patterns of dmin or more errors,
why?
29
v=(v0,v1,….vn-1) Noisy
channelr=(r0,r1,….rn-1) with ℓerror pattern d(v,r)=ℓ
• Given that the code employed is with dmin, this mean any two codewords
of C differs in at least dmin places. For this code, no error pattern of dmin -1or
fewer can change one codeword into another.
Therefore, any error pattern of dmin -1 or fewer will result in a received vector
r that is not a codeword of C, we say that errors are detected.
• Because there exists at least one pair of codewords that differ in dmin
places and there is an error pattern of dmin errors that will carry one into
the other
Cont.
• How many error patterns can be detected? show?
• For large n, small fraction of error patterns pass through the decoder
without being detected.
30
In an (n, k) linear code, the detecting capability is 2𝑛 − 2𝑘 error
patterns of length n.
• All error patterns =2n-1
• 2k-1 are identical to codewords and hence if added will results a
codeword and then syndrome will be zero (no errors are
detected)
• Total=2n-1-(2k-1)= 2n-2k
Cont.
• Let 𝐴𝑖 the number of codeword of weight i in C
(weight distribution).
• If 𝑃𝑢(𝐸) denotes the probability of an undetected
error over BSC,
𝑃𝑢 𝐸 =
𝑖=1
𝑛
𝐴𝑖𝑝𝑖 1 − 𝑝 𝑛−𝑖
where 𝑝𝑖 is the transition probability of the BSC.
• Example: Given that 𝑷 = 𝟏𝟎−𝟐, determine 𝑷𝒖 𝑬of C shown?
31
𝐴0 = 1, 𝐴1 = 𝐴2 = 0, 𝐴3 = 7, 𝐴4 = 7, 𝐴5 = 𝐴6= 0, 𝑎𝑛𝑑 𝐴7 = 1⇒ 𝑃𝑢 𝐸 = 7𝑝3 1 − 𝑝 4 + 7𝑝4 1 − 𝑝 3 + 𝑝7
If 𝑃 = 10−2 ⇒ 𝑃𝑢 𝐸 = 7 × 10−6.This mean 7 codewords out of 1 million can’t be
detected if errors involved
Error Correcting Capability
• dmin is either odd or even
2𝑡 + 1 ≤ 𝑑𝑚𝑖𝑛 ≤ 2𝑡 + 2 (t be + integer)
• Next, we show that the code C is capable of correcting all error
patterns of t or fewers errors:
• Let v and r be the Tx. and Rx. codewords, respectively
– d(v,r)+d(w,r) ≥ d(v,w)….3.21 (Traingle inequality).
– Suppose error pattern of t’ errors occur: d(v,r)=t’
– Because v and w are codewords in C, we have
d(v,w) ≥ dmin ≥ 2t+1……(3.22)
32
Error Correcting Capability– Combining (3.21) and (3.22) and using the fact of d(v,r)=t’ :
d(w,r) ≥2t+1-t’
– assuming t ≥t’ d(w,r)>t
• This says that if there is an error pattern of t or less , r is still close to
the transmitted codeword v it self. (The decoder is still capable od
recovering the trasnmitted code word.
• In contrast, the code is not capable of correcting all the error patterns
of ℓ errors with ℓ > 𝑡. (see textbook page for details).
• In summary, # of errors that can be corrected t =
33
2
1min
d
ExampleConsider the code given in Table 3.1, Find the
detecting and correcting capabilities of the
code shown
• dmin=3
• Any error patterns with 2 errors can be
detectd
• Any single error can corrected
34
Hard vs. Soft decision decoding• Hard-decision : for each received symbol, Rx. make hard decision (0 or 1).
• Soft-decision: Rx. can be designed to declare a symbol erased when it
received unreliably (0 or 1 or erasures).
• A code with dmin is capable of correcting any pattern of v errors and e erasures
provided the following condition is satisfied
• Show?
35
N.B: dmin≥e+1 mean that all codewords are different in e+1 digits and hence
there is at least a single digit in the rx vector (which also is correct for sure)
can determine the corresponding tx vector
Decoding schemes
• Decoding mission is to find a rule to
partition the 2n possible received
vectors in to 2k disjoint subset D1,
D2,...D2k such that the codeword vi is
contained in the subset Di for 1≤i≤2k.
• Thus, each subset Di is one-to-one
correspondence to a codeword.
• Decoding is correct iff the r is in the
subset Di that corresponds to the
codeword transmitted.
v=(v0,v1,….vn-1) Noisy
channelr∈2n n-tuples over
GF(2)
1. Standard Array
37
Standard Array Construction
1. List the 2k codewords in a row, starting with the all-zero
codeword v1.
2. From the remaining 2n-2k , select an error pattern e2 and
place it below v1. The selected pattern should:
(i) has the smallest weight possible (most probable error) why?
(ii) has not appeared before in the array.
3. Complete the row entries by finding the corresponding
addition results {e2+v2, e2+v3,…. e2+v2k} and place them.
4. Repeat step 2-3 until all the possible error patterns have
been accounted for.
38
Standard Array Construction
39
kknknknkn
k
k
k
22322212
23332313
22322212
2321
veveveve
vev+ev+ev+e
vev+ev+ev+e
vvv0v
- - - -
TH3.3 No two n-tuples in the same row of a standard array are
identical. Every n-tuple appears in one and only one row. Proof?
(Result): There are 2n/2k=2n-k disjoint rows in the standard array and
each row consist of 2k distinct elements.
Any element in a coset can be used as its coset leader. This does not
change the element of the coset, it simply permutes them.
Coset leader
(representative)
40
Standard Array Construction
Example: Consider the (6,3) linear code generated by
G, find the standard array?
41
Decoding using Standard Array
• Still we didn’t show how rx vector can be decoded
using standard array, can you guess?
• Example r=110100, then e=000010 and hence v*=110110
Decoding using Standard Array
• N.B. The decoding is correct iff the error pattern caused
by the channel is a coset leader. Why?
• Lets the experienced error pattern through the channel is x and it is not
a coset leader (Say it is in 𝑙th coset and under vi)
• This mean x=e𝑙+vi
• Suppose that vj is transmitted.
• r=vj+x
• and using x=e𝑙+vi this results that
• r=vj+e𝑙+vi
=e𝑙+vs and hence the Rx. will specify vs as the transmitted codeword
• It should be now clear why the error pattern with the
minimum weight should be placed as coset leaders (place
the most probable (most probable=minimum weight)
patterns in positions that can be corrected). In otherwords,
this achieve the maximum likelihood decoding. Proof?
Proof:
Be noted that the el has been chosen to be the vector with the minimum weight in the coset
Decoding Error Probability
• Let α denote the # of coset leaders of weight i.
• α0, α1, α2…. αn are called the weight distribution of the
coset leaders.
• Because a decoding error occurs iff the error pattern is not
a coset leader, the error probability for a BSC is
Useful Properties
Proof:?
Proof:?
• Using the one-to-one correspondence that shown by Theorem
3.6, a decoding table which is simpler that standard array can be
formed
• This table consist of the 2n-k coset leaders and its corresponding
symdrome
2. Syndrome Decoding (table-lookup)
Decoding Procedure:
1. For the received vector r, compute the syndrome s = rHT.
2. Using the decoding table, identify the error pattern e.
3. Add e to r to recover the transmitted codeword c.
47
2. Syndrome Decoding (table-lookup)
48
2n-k=8
[dmin-1/2] or less= 1 or less
1. How many correctable error pattern?
2. What is the weight that can be corrected?
3. Find the decoding table?
4. Given that v=1001011 is transmitted and
then r=100111 is received, decode r using
the lookup table
s=v*HT=(011) and hence e=0000100 then
v*=100011
Hardware Implementation
• Let r = r0r1r2r3..rn
• By the H matrix , find s =s0s1….sn-k
• From the table of syndromes, a truth
table can be constructed and hence a
combinational logic circuit with s
as input and e as outputs can be
designed.
• Then, added (XOR) to the r
49
Example 3.9: Draw the decoding circuit of the code has the
following decoding table
50
Hardware Implementation
Standard Array Decoding ()• For an (n,k) linear code, standard array decoding is
able to correct exactly 2n-k error patterns, including
the all-zero error pattern.
• Illustration 1: The (7,4) Hamming code
# of correctable error patterns = 23 = 8
# of single-error patterns = 7
Therefore, all single-error patterns, and only single-
error patterns can be corrected.
EELE 6338 51
Probability of undetected error • Two new formulas will be presented with extra advantages.
• Let 𝐴0, 𝐴1, … , 𝐴𝑛 to be the weight distribution of the code (n, k)
and 𝐵0, 𝐵1, … , 𝐵𝑛 to be the weight distribution of the dual code 𝐶𝑑.
• Weight distribution polynomials (weight enumerator of C and Cd)
𝐴 𝑧 = 𝐴0 + 𝐴1𝑧 +⋯+ 𝐴𝑛𝑧𝑛, 𝐵 𝑧 = 𝐵0 + 𝐵1𝑧 +⋯+ 𝐵𝑛𝑧
𝑛
• Then, A(z) and B(z) are related by the following identity
(McWilliams identity) 𝐴 𝑧 = 2− 𝑛−𝑘 1 + 𝑧 𝑛𝐵 1−𝑧
1+𝑧
• Previously was shown that the probability of undetected error 𝑃 𝐸
= σ𝑖=1𝑛 𝐴𝑖𝑝
𝑖 1 − 𝑝 𝑛−𝑖 = 1 − 𝑝 𝑛 σ𝑖=1𝑛 𝐴𝑖
𝑝
1−𝑝
𝑖
• Substituting z = Τ𝑝 1 − 𝑝 in A(z) and noting that 𝐴0 = 1
𝐴𝑝
1 − 𝑝− 1 =
𝑖=1
𝑛
𝐴𝑖𝑝
1−𝑝
𝑖(3.34)
52
Cont.
• Then, by combining this result to 𝑃𝑢 𝐸 leads that 𝑃𝑢 𝐸 = (1
53
(7,4) C has (24) but Cd of (7,3) has (23) codewords
54
Simple Upper Bound on Pu
• Theoretically, the weight distribution can be
determined by examining its 2k or 2n-k (Cd) codewords.
• For large n, k, n-k, this would be impractical. Actually,
the weight distribution is unknown for many known
linear codes and hence is difficult (if not impossible)
to compute their Pu(E)
• However, the upper bound on average probability of
undetected errors can be determined:
• Derivation :
)(22
12 kn
n
k
uP
55
Linear Code Examples
Single-Parity-Check (SPC) codes:
1. (k+1,k) or (n,n-1) linear block code
2. With
3. Because all codewords have
even weight, SPC called even parity check codes.
4. SPC codes often used for simpler error detection
Repetition codes:
1. (n,1) linear block code
2. Only two codewords (all zeros or all ones).
3. It is Cd of SPC, show?
56
N.B. :
1. SPC and repetition codes are often used as components for constructing
long, powerful codes.
2. A linear block codes C that is equal to its dual code Cd is called self-dual
code
Cont.Self-Dual code:
• n must be even and k must equal n/2 (mean R=1/2)
• G.GT=0 (assuming G is systematic then P.PT=In/2)
Next lecture
• We will cover several classes of important
linear block codes.
58