Post on 22-Feb-2016
description
exercise in the previous class (Apr. 26)
For s = 110010111110001000110100101011101100 (|s|=36),compute 2-values of s for block length with 1, 2, 3 and 4.
n = 1: 0 and 1 are expected to appear 18 times each.s contains |0| =17, |1| = 19, 2 = 12/18+ 12/18=1/9
n = 2: four patterns × 4.5 times.s contains |00| =5, |01| = 1, |10| =6, |11| = 6,2 = 0.52/4.5 + 3.52/4.5 + 1.52/4.5 + 1.52/4.5 = 3.78
n = 3: 2 = 2.67, n=4: 2 = 14.11
Implement the linear congruent method as computer program. see the excel file on http://apal.naist.jp/~kaji/lecture/
1
chapter 3:coding for noisy communication
2
about this chapter
coding techniques for finding and correcting errors
coding in the Chapter 2 ... source coding ( 情報源符号化 )work next to the information sourcegive a compact representation
coding in this chapter ... channel coding ( 通信路符号化 )work next to the communication channelgive a representation which is robust against errors
3
4
two codings
channel
source and receiver sender receiver
source coding encode decode
protection (optional) encrypt decrypt
channel coding encode decode
today’s class
motivation and overviewrephrase of the first day introduction
the models of communication channelsbinary symmetric channel (BSC)
elementary components for linear codes(even) parity check codehorizontal and vertical parity check code
5
communication is erroneous
real communicationno guarantee of “sent information = received information”
radio (wireless) communicationnoise alters the signal waveform
optical disk medium (CD, DVD...)dirt and scratch obstructs correct reading
the difference between sent and received information = error
Errors may be reduced, but cannot be eliminated.
6
error correction in daily life
Three types of errors in our daily life:correctable: “take a train for Ikomo” “... Ikoma”detectable: “He is from Naga city” “Nara” or “Naha”undetectable: “the zip code is 6300102” ???
What makes the difference?
7
IkomaNara
Naha
IkomoNaga
names of places... sparse ( 疎 ) in the set
zipcodes... densely ( 密 ) packed
630010163001026300103
:
trick of error correction and detection
For the error correction, we need to create sparseness artificially.( 人工的に )
phonetic code: Alpha, Bravo, Charlie...あさひの「あ」,いろはの「い」 ...
8
A B CD E FG H I
characters... densely packed
phonetic codes... sparse
Alpha
Bravo
To create the sparseness, we...add redundant sequences,enlarge the space, andkeep representations apart.
in a binary world
assume that you want to send two binary bitswithout redundancy...
9
with redundancy...
Even if you receive “01”, you cannot say if...“01” is the original data, orresult of the modification by errors.
00 01
10 11
00 → 0000001 → 0101110 → 1010111 → 11110
00000
01011
01010
0001100010 encode: to add the redundancy
codewords: results of encodingcode: the set of codewords
the principle of error correction
the sender and the receiver agree the codesender: choose one codeword c and send itreceiver: try to estimate the sent codeword c’
from a received data r which can contain errors
assumption: Errors do not occur so frequently.
with this assumption,error correction ≈ search of the codeword c’ nearest to r
10
00000
0101100010
is it really good?
assume that a symbol is delivered correctly with p=0.9
11
00 → 0000001 → 0101110 → 1010111 → 11110
without codingcorrectincorrect
0.92 = 0.811 – 0.81 = 0.19
with codingcorrect = 0 or 1 error in the five bits
0.95 + 5C10.94 ・ 0.1 = 0.9072
incorrect 1 – 0.9072 = 0.0928
Good for this case, but not always... construction of good codes is the main subject of this chapter
the model of channels
communication channelprobabilistic model with one input and one outputthe inputs and outputs are symbols (discrete channel)an output symbol is generated for each input symbol
no lost, no delaythe output symbol is chosen probabilistically
12
channelinput(sender)
output(receiver)
example of channels: 1
binary symmetric channel (BSC, 二元対称通信路 )inputs = outputs = {0, 1}input = 0 output = 0 with prob. 1 – p, 1 with prob. p.input = 1 output = 0 with prob. p, 1 with prob. 1 – p.p is said to be a bit error probability of the channel
13
0
1
0
1
p
1 – p
p
1 – p
input
output
BSC is ...memoryless ( 記憶がない )
errors occur independentlystable ( 定常 )
the probability p does not change
example of channels: 2
binary erasure channel (BEC, 二元消失通信路 )inputs = {0, 1}, outputs = {0, 1, X}input = 0 output = 0 with prob. 1 – p, X with prob. p.input = 1 output = 1 with prob. 1 – p, X with prob. p.X is a “place holder” of the erased symbol
14
0
1
0
1
p
1 – p
p
1 – p
input
output
X
0
1
0
1
p
1 – p – qp
1 – p – q
Xqq
(another variation)
example of channels: 3
channels with memorythere is correlation between occurrences of errorsa channel with “burst errors”
15
011010101010010101011010110101010
011010101011011010101010110101010
unstable channelsthe probabilistic behavior changes according to timelong-range radio communication, etc.
channel coding: preliminary
We will consider channel coding such that...a binary sequence (vector) of length k is encoded intoa binary sequence (vector, codeword) of length n.
k < n, code C Vn
a sequence is sometimeswritten as a tupleb1b2...bm = (b1, b2, ..., bm)
computation is binary,component-wise:001 + 101 = 100
16
Vk
Vn
good code?
A code C is a subset of Vn
Vn contains 2n vectorsC contains 2k vectors
Given k and n, there are codes.
Which choice of C is good?powerful enough for error correctioneasy encoding and easy decoding
17
Vk Vn
00011011
000011101110 C
the class of linear codes ( 線形符号 )
linear codes
easy encoding(relatively) easy decoding (error detection/correction)
mathematics help constructing good codesthe performance evaluation is not too difficult
Most codes used today are linear codes.
We study some examples of linear codes (today),and learn the general definition of linear codes (next).
18
(even) parity code
the encoding of an (even) parity code ( 偶パリティ符号 ):given a vector (a1, a2, ..., ak) ∈ Vk
compute p = a1 + a2 + ... + ak, and let (a1, a2, ..., ak , p) be the codeword of (a1, a2, ..., ak)
19
when k = 3... 000001010011100101110111
00000011010101101001101011001111
p = 0 + 1 + 1 = 0
code C
basic property of the parity code
code length: n = k + 1a codeword consists of...
the original data itself (information symbols)added redundant symbol (parity symbol) systematic code ( 組織符号 )
20
00000011010101101001101011001111
code C
1010information symbols
(bits)parity symbol
(bit)
a vector v of length n is codeword v contains even number of 1s.
parity codes and errors
Even parity codes cannot correct errors.Even parity codes can detect odd number of errors.
21
#errors = #differences between the sent and received vectors
0000sent vector(codeword)
0101received vector
we have two (2) errors
0000 0011
0101
00011001
#errors = even the received vector contains even1s
#errors = odd the received vector contains odd 1s
horizontal and vertical parity check code
horizontal and vertical parity check code (2D parity code)place information symbols in a rectangular form ( 長方形 )add parity symbols in the horizontal and vertical directionsreorder all symbols into a vector (codeword)
22
k = 9, encode (a1, a2, ..., a9)a1 a2 a3
a4 a5 a6
a7 a8 a9
p1
p2
p3
q1 q2 q3 r codeword: (a1, a2, ..., a9, p1, p2, p3, q1, q2, q3, r)
p1 = a1 + a2 + a3
p2 = a4 + a5 + a6
p3 = a7 + a8 + a9
q1 = a1 + a4 + a7
q2 = a2 + a5 + a8
q3 = a3 + a6 + a9
r = a1 + a2 + ... + a9
example of encoding
encode 011100101, and the codeword is 0111001010100101
23
2D codes are systematic codesif k = ab, then the code length is n = ab + a + b + 1
0 1 1 01 0 0 11 0 1 00 1 0 1
0111001010100101information
symbolsparity
symbols
2D codes and errors
A 2D code can correct one-bit error in a codeword.place the symbols in a received vectorcount the numbers of “1” in each row/column
if there is no error... all rows and columns contain even 1sif there is an error...
there is one row and one column with odd number of 1sthe intersecting point is affected by an error
24
010110101 0
0
1 1
1
1
0
0 1even
odd
odd eveneven
even
011110101
(received)
correct the third bit
two-bit errors
What happens if two-bits are affected by errors?
25
real errors
two strange rowstwo strange columns
we cannot decide which has happened...
We know something wrong, but cannot spot the errors.
two-bit errors, another case
26
real errors
no strange rowtwo strange columns
We know something wrong, but cannot spot the errors.
additional remark
Do we need the parity of parity?
27
a1 a2 a3
a4 a5 a6
a7 a8 a9
p1
p2
p3
q1 q2 q3 rcodeword: (a1, a2, ..., a9, p1, p2, p3, q1, q2, q3, r)
Even if we don’t have r,we can correct any one-bit error, but...some two-bit errors cause a problem.
additional remark (cnt’d)
We expect that 2D codes detect all two-bit errors.If we don’t use the parity of parity, then...
28
0 0 00 0 00 0 0
0 0 00 1 10 1 0
codeword codeword0 0 00 1 00 0 0
0 0 00 1 00 1 0
1-bit err.
to thenearestcodeword
1-bit err.
some two-bit errors are not detected,instead, they are decoded to a wrong codeword.
0 0 00 1 00 0 0
to the nearestcodeword
summary of today’s class
motivation and overviewrephrase of the first day introduction
the models of communication channelsbinary symmetric channel (BSC)
elementary components for linear codes(even) parity check codehorizontal and vertical parity check code
29
excersise
In the example of page 11, determine the range of the probability p under which the encoding results in better performance.
30