Post on 04-Jun-2018
8/13/2019 8b Error Correction
1/129
DIGITAL COMMUNICATION
Error - Correction
November 15, 2011
A.J. Han Vinck
8/13/2019 8b Error Correction
2/129
Error correction is needed in high tech devices
A.J. Han Vinck, rondheim, 2011 2
8/13/2019 8b Error Correction
3/129
fundamental limits set by Shannon (1948)
Bell Labs, 1955.
A.J. Han Vinck, rondheim, 2011 !
Fundamental problem: reproducing at the receiver a message selected at the
transmitter
Shannons contribution: - bound on the efficiency (capacity)
- ho to achieve the bound
The Bell System Technical Journal, !ol. "#, pp. $#9%&"$, '"$%'5',
ctober, 19&.
8/13/2019 8b Error Correction
4/129
A.J. Han Vinck 4
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain
' )ome channe% mode%s
' *&rst error mode%s
' +etection r&%es' ntro to the coding (rob%em
' (er$ormance
' he c%ass o$ %inear codes
' *o&nds on the minim&m distance' Eam(%es o$ codes
' h/ error correction
8/13/2019 8b Error Correction
5/129
A.J. Han Vinck 5
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain
' )ome channe% mode%s
' *&rst error mode%s
' +etection r&%es' ntro to the coding (rob%em
' (er$ormance
' he c%ass o$ %inear codes
' *o&nds on the minim&m distance' Eam(%es o$ codes
' h/ error correction
8/13/2019 8b Error Correction
6/129
A.J. Han Vinck 6
he comm&nication mode%
so&rce
datared&ction
com(ression
data(rotection
sink
essageconstr&ction
decoder
k
n
k
34
34
8/13/2019 8b Error Correction
7/129
A.J. Han Vinck 7
oint-to-(oint
transmitter receiverchanne%
(h/sica%modem modem
message
bits
)igna%
generator
)igna%
(rocessor
bits
message
8/13/2019 8b Error Correction
8/129
A.J. Han Vinck 8
osition o$ Error Contro% Coding
signal
genera!rc"annel #eec!r
k in$%
&is
k !%$%
&is
c"annelk
c"annelk in$%
&is
k !%$%
&is
k in$%
&is
signal
genera!r
c!#e# signal genera!r
#eec!r
#eec!r'#ec!#er
n in$%
&isn
(CC (CC
c!#ing
c!#e# )!#%lai!n
8/13/2019 8b Error Correction
9/129
A.J. Han Vinck *
transmission model (OSI)
Data Link
Control
Data Link
Control
Physial Physial
link
6nre%iab%e trans-mission o$ bits
ransmission o$re%iab%e (ackets
8/13/2019 8b Error Correction
10/129
A.J. Han Vinck +,
)omething to think abo&t
message
bits
bits
message
message
com(ression (rotection o$ bits
correction o$ incorrect bits decom(ression message
Error correctionE7, JE7, etc
Com(ression red&ces bit rate
rotection increases bit rate
8/13/2019 8b Error Correction
11/129
A.J. Han Vinck ++
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain' Some hannel models' *&rst error mode%s
' +etection r&%es' ntro to the coding (rob%em' he c%ass o$ %inear codes' *o&nds on the minim&m distance' Eam(%es o$ codes
' h/ error correction
8/13/2019 8b Error Correction
12/129
A.J. Han Vinck +-
emor/%ess channe%
In$% /0123 !%$%
ransii!n $r!&a&iliies
memor/%ess8
- o&t(&t at time i de(ends on%/ on in(&t at time i
- in(&t and o&t(&t a%(habet $inite
8/13/2019 8b Error Correction
13/129
A.J. Han Vinck +
Eam(%e8 binar/ s/mmetric channe%
9*)C:
(rr!r !%rce
E
X
O%$%In$%
EXY =
E is the binar/ error se;&ences.t. 91: < 1-90: < (
= is the binar/ in$ormationse;&ence
> is the binar/ o&t(&tse;&ence
+9$
, , $
: +
+9$
8/13/2019 8b Error Correction
14/129
A.J. Han Vinck +4
$rom A7N to *)C
{ }EX + '
-
-
-
--+0
n
N enp=
Home#ork8 ca%c&%ate the ca(acit/ as a $&nction o$ A and 2
(
7a&ssian noise #ith (rob. densit/ $&nction
= < ?- A > < = ? N
+ecide8 ? or -
---
-
--
+
A
A
edxepx
=
8/13/2019 8b Error Correction
15/129
A.J. Han Vinck +5
"ther mode%s
,
+
, 0lig" !n
+ 0lig" !;;
$
+9$
/0
8/13/2019 8b Error Correction
16/129
A.J. Han Vinck +6
the eras&re channe%
A((%ication8 cdma detection, disk arra/s
,
+
,
+
+9e
e
e
3 1
/0 3 < , < + = /0 3 < + < /,
+isk 1
+isk 2
+isk !
+isk
+isk 5
3no#n(osition o$error
8/13/2019 8b Error Correction
17/129
A.J. Han Vinck +7
Brom 7a&ssian to binar/ to eras&re
e
xi= +/-
O%$%In$%
yi= xi+ e?
- -
?
?-
?
- -
?
?-EEE
8/13/2019 8b Error Correction
18/129
A.J. Han Vinck +8
Eras&re #ith errors
,
+
,
+(
( e
e
1-(-e
1-(-e
8/13/2019 8b Error Correction
19/129
A.J. Han Vinck +*
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain
' )ome channe% mode%s
' !urst error models
' +etection r&%es' ntro to the coding (rob%em
' (er$ormance
' he c%ass o$ %inear codes
' *o&nds on the minim&m distance' Eam(%es o$ codes
' h/ error correction
8/13/2019 8b Error Correction
20/129
A.J. Han Vinck -,
ode%ing8 net#orking
AckNack
- 1-error ca&ses retransmission
- %ong (ackets a%#a/s have an error
- short (ackets #ith ECC give %o#er e$$icienc/
(acket
)&((ose that a (acket arrives correct%/ #ith (robabi%it/ .
hat is then the thro&gh(&t as a $&ntion o$
8/13/2019 8b Error Correction
21/129
A.J. Han Vinck -+
A )im(%e code
' Bor %o# (acket %oss rates 9e.g. 5D:, sending d&(%icates ise(ensive 9#astes band#idth:
' =" code
F =" a gro&( o$ data (kts together to (rod&ce re(air (ktF ransmit data ? ="8 can recover 1 %ost (kt
10101 11100 00111 11000 10110
10101 11100 0011111000 10110
8/13/2019 8b Error Correction
22/129
A.J. Han Vinck --
CD DVD Blue Laser
Density inreases sensiti"ity
8/13/2019 8b Error Correction
23/129
A.J. Han Vinck -
)!#eling
Ho# to mode%scratches on a
C+
Ans#er isim(ortant $orthe design o$ECC
8/13/2019 8b Error Correction
24/129
A.J. Han Vinck -4
M!#eling> &inar1 rans)issi!n
channe%test
se;&ence
error
se;&ence
rob%em8 0 1 0 0 0 1 0 1 0 0 0 0 1 1 1 '''
+etermination o$ b&rst and g&ard s(ace
b&rst g&ard b&rst
8/13/2019 8b Error Correction
25/129
A.J. Han Vinck -5
A sim(%e error detection method
r!? $ari1
@ill r!? ?ise
Trans)i c!l%)n ?ise
RESULT:RESULT:an1 &%rs !; leng"
L can &e #eece#
L
, + + , , + + , , + +, , , + + , , + + , ,
, , , , , , , , , , ,
, + + , , + + , , , ,, , , + + , , + + + +
, , , , , , , , , + +
hatha((ens #ith b&rsts o$ %ength %arger than G
8/13/2019 8b Error Correction
26/129
A.J. Han Vinck -6
b&rst error mode% 97i%bert-E%%iot:
(rr!r !%rce
#andom#andomerror channe% o&t(&ts inde(endent
/0, < +9 /0+
!urst!ursterror channe% o&t(&ts de(endent
(rr!r !%rce/0, 2 sae < &a# < /0+2sae < &a# < +'-
/0, 2 sae < g!!# < + 9 /0+2sae < g!!# < ,.***
)tate in$o8good or bad
g!!# &a#
transition (robabi%it//g&
/&g
/gg/&&
8/13/2019 8b Error Correction
27/129
A.J. Han Vinck -7
;&estion
g!!# &a#
/g&
/&g
/gg/&&
/0, 2 sae < &a# < /0+2sae < &a# < +'-
/0, 2 sae < g!!# < + 9 /0+2sae < g!!# < ,.**
B"a is "e aerage ;!r /0, ;!r> /gg < ,.* /g& < ,.+ /&g < ,.** /&& < ,.,+ E
In#icae"!? 1!% can 1!% e3en# "e )!#elE
8/13/2019 8b Error Correction
28/129
A.J. Han Vinck -8
Britchman mode%$or mobi%e radio channe%s
multi$le states % and only one state !
Closer to an atual real&'orld hannel
Gn F
+9$
(rr!r $r!&a&ili1 , (rr!r $r!&a&ili1 h
G+
8/13/2019 8b Error Correction
29/129
A.J. Han Vinck -*
Eam(%e 9$rom imo 3orhonen, He%sinki:
' n $ading channe%s received data can e(erience b&rst errors that destro/%arge n&mber o$ consec&tive bits. his is harm$&% $or channe% coding
' nter%eaving distrib&tes burst errorsa%ong data stream
' A (rob%em o$ inter%eaving is
introd&ced etra de%a/' Eam(%e be%o# sho#s b%ock
inter%eaving8time
received
power
Reception after
fading channel
1 0 0 0 1 1 10 1 0 1 1 1 0
0 0 1 1 0 0 1
1 0 0 0 1 1 10 1 0 1 1 1 0 0 0 1 1 0 0 1
1 0 0 0 1 0 0 0 1 0 1 1 11 0 11 0 10 1
Received interleaved data:
Block deinterleaving :
Recovered data:
8/13/2019 8b Error Correction
30/129
A.J. Han Vinck ,
eam(%e
F Consider the code C < I 000, 111
F A b&rst error o$ %ength ! can not be corrected.
F Get &s &se a b%ock inter%eaver !=!
A+ A- A F+ F- F C+ C- C
2 errors
A+ A- A F+ F- F C+ C- C
Interleaver
A+ F+ C+ A- F- C- A F C
A+ F+ C+ A- F- C- A F C
Deinterleaver
A+ A- A F+ F- F C+ C- C
1 error 1 error 1 error
8/13/2019 8b Error Correction
31/129
A.J. Han Vinck +
idd%eton t/(e o$ b&rst channe% mode%
)e%ect channe% k#ith (robabi%it/9k:
ransition(robabi%it/ 90:
0
1
0
1
channe% 1
channe% 2
channe% k hastransition(robabi%it/ (9k:
8/13/2019 8b Error Correction
32/129
A.J. Han Vinck -
m(&%sive Noise C%assi$ication
0a ingle ransien )!#el
arameters o$ sing%e transient 8
-(eak am(%it&de- (se&do $re;&enc/ $0
8/13/2019 8b Error Correction
33/129
A.J. Han Vinck
nter%eaving8 $rom b&rst/ to random
essage inter%eaver channe% inter%eaver -1 message
encoder decoder
b&rst/
Lrandom errorM
oteinter%eaving brings encoding and decoding de%a/
Home#ork8com(are the b%ock and convo%&tiona% inter%eaving #.r.t. de%a/
8/13/2019 8b Error Correction
34/129
A.J. Han Vinck 4
nter%eaving8 b%ock
Channe% mode%s are di$$ic&%t to derive8
- b&rst de$inition
- random and b&rst errors $or (ractica% reasons8 convert b&rst into random error
read in ro# #ise
transmitco%&mn #ise
1
0
0
1
1
0
1
0
0
1
1
0
0
0
0
0
0
1
1
0
1
0
0
1
1
8/13/2019 8b Error Correction
35/129
A.J. Han Vinck 5
+e-nter%eaving8 b%ock
read in co%&mn
#isethis ro# contains 1 error
1
0
0
1
1
0
1
0
0
1
1
e
e
e
e
e
e
1
1
0
1
0
0
1
1
read o&tro# #ise
8/13/2019 8b Error Correction
36/129
A.J. Han Vinck 6
nter%eaving8 convo%&tiona%in(&t se;&ence 0
in(&t se;&ence 1 de%a/ o$ b e%ements
in(&t se;&ence m-1 de%a/ o$ 9m-1:b e%ements
Eam(%e8 b < 5, m < !
in
o&t
8/13/2019 8b Error Correction
37/129
A.J. Han Vinck 7
(3a)$leUMT T%r&! (nc!#er
' Brom E) ) 125 212 v!..0 92000-0:, 6) &%ti(%eing and channe% coding' +ata is segmented into b%ocks o$ G bits, #here 0 G 511
U$$er
C
(nc!#er
L!?er
C
(nc!#er
Inerleaer
1se)aic
O%$%k
Uninerleae#
/ari1
Kk
Inerleae#
/ari1
Kk
In$%k
Inerleae#
In$%
k
O%$%
8/13/2019 8b Error Correction
38/129
A.J. Han Vinck 8
6) nter%eaver8nserting +ata into atri
' +ata is $ed ro#-#ise into a b/ C matri. < 5, 10, or 20, O C 25KF $ G P C then matri is (added #ith d&mm/ characters.
+ - 4 5 6 7 8
* +, ++ +- + +4 +5 +6
+7 +8 +* -, -+ -- - -4
-5 -6 -7 -8 -* , + -
4 5 6 7 8 * 4,
- 6 5 7 4 + 8
+, +- ++ +5 + +4 * +6
+8 -- -+ - +* -, +7 -4
-6 -8 -7 + -* , -5 -
4, 6 5 * 7 8 4Daa is $er)%e# withineac" r!?.
4, 6 5 * 7 8 4
-6 -8 -7 + -* , -5 -
+8 -- -+ - +* -, +7 -4
+, +- ++ +5 + +4 * +6
- 6 5 7 4 + 8
!?s are $er)%e#
4, 6 5 * 7 8 4
-6 -8 -7 + -* , -5 -
+8 -- -+ - +* -, +7 -4
+, +- ++ +5 + +4 * +6
- 6 5 7 4 + 8
Daa is rea# ;r!) )ari3 c!l%)n9?ise
8/13/2019 8b Error Correction
39/129
A.J. Han Vinck *
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain
' )ome channe% mode%s
' *&rst error mode%s
' Detetion rules' ntro to the coding (rob%em
' (er$ormance
' he c%ass o$ %inear codes
' *o&nds on the minim&m distance' Eam(%es o$ codes
' h/ error correction
8/13/2019 8b Error Correction
40/129
A channe% transition mode%
A.J. Han Vinck 4,
n(&t = channe% transition "&t(&t >
1 /1
* /*
+++
9/1Q1:
9/*Q1:9/Q1:
9/Q,:
9/1Q,:
8/13/2019 8b Error Correction
41/129
A.J. Han Vinck 4+
-rror $robability (,.P)
)&((ose $or a received vector / #e assign the message i as being transmitted.
hen, the (robabi%it/ o$ a correct decision
8/13/2019 8b Error Correction
42/129
A.J. Han Vinck 4-
aim&m Gike%ihood 9G: receiver
$ind i that maimiSes
9 iQ /: < 9 i, / : 9 / :
< 9 / Qi: 9 i: 9 / :
$or e;&a%%/ %ike%/ ithis is e;&iva%ent to $ind
maim&m 9 / Q i:
8/13/2019 8b Error Correction
43/129
A.J. Han Vinck 4
eam(%e
Bor ( < 0.1 and
=1< 9 0 0 : 9 =1 < 1! :=2< 9 1 1 : 9 =2 < 2!:
7ive /o&r A and G decision $or > < 9 0 1 :
8/13/2019 8b Error Correction
44/129
A.J. Han Vinck 44
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain
' )ome channe% mode%s
' *&rst error mode%s
' +etection r&%es' Intro to the odin0 $roblem
' (er$ormance
' he c%ass o$ %inear codes
' *o&nds on the minim&m distance' Eam(%es o$ codes
' h/ error correction
8/13/2019 8b Error Correction
45/129
A.J. Han Vinck 45
*it (rotection
' "btained b/ Error Contro% Codes 9ECC:
F Bor#ard Error Correction 9BEC:
F Error +etection and $eedback 9A:
' er$ormance de(ends on error statisticsT
F Error mode%s are ver/ im(ortant
8/13/2019 8b Error Correction
46/129
A.J. Han Vinck 46
eam(%e
ransmit8 0 0 0 or 1 1 1
Ho# man/ errors can #e correct Ho# man/ errors can #e detect
ransmit8
A < 00000 * < 01011 C < 10101 + < 11110
Ho# man/ errors can #e correct Ho# man/ errors can #e detect
hat is the di$$erence
8/13/2019 8b Error Correction
47/129
ractica% comm&nication s/stem design
messageestimate
channe% decoder
n
Code#ord in
receive
here are 2k code #ords o$ %ength nk is the n&mber o$ in$ormation bits transmitted in n channe% &ses
2k
Code book
Code book#ith errors
8/13/2019 8b Error Correction
48/129
Channe% ca(acit/
+e$inition8
he rate o$ a code is the ratio kn, #here
k is the n&mber o$ in$ormation bits transmitted in n channe% &ses
)hannon sho#ed that8 8
$or Ca(acit/
an encoding methods eist#ith decoding error (robabi%it/ 0
8/13/2019 8b Error Correction
49/129
Encoding and decoding according to )hannon
Code82kbinar/ code#ords #here (90: < 91: < U
Channe% errors8 90 1: < 91 0: < (
i.e. error se;&ences 2nh9(:
+ecoder8search aro&nd received se;&ence $or code#ord
#ith n( di$$erences
s(ace o$ 2n binar/ se;&ences
8/13/2019 8b Error Correction
50/129
decoding error (robabi%it/
1. Bor t errors8 Qtn-(QW X
0 $or n 9%a# o$ %arge n&mbers:
2. W 1 code #ord in region 9code#ords random:
nan#
4$0"+n
kJ;!r
,--
-
-4+-04+0/
4JF7CC0n4J4$0"+0n
n
4$0n"k
8/13/2019 8b Error Correction
51/129
A.J. Han Vinck 5+
A $ic!rial ie?
-nec!rs -kc!#e ?!r#s
8/13/2019 8b Error Correction
52/129
A.J. Han Vinck 5-
decoder
' Com(are received #ord #ith a%% (ossib%e code#ords
code #ords
received
:Dec!#e "e c!#e ?!r# ?i" )ini)%) !; #i;;erences 0M!s Likel1
8/13/2019 8b Error Correction
53/129
A.J. Han Vinck 5
e3a)$le
code #ords8 0 0 0 0 0 0 1 0 1 1 1 0 1 0 1 1 1 1 1 0
received8 0 0 0 1 1
di$$erence8 0 0 0 1 1 0 1 0 0 0 1 0 1 1 0 1 1 1 0 1
best g&ess8 0 1 0 1 1!nl1 + #i;;erence
8/13/2019 8b Error Correction
54/129
A.J. Han Vinck 54
?e "ae s!)e $r!&le)s
' a((ing $rom in$ormation to code #ordsF generation o$ code #ords 9m&t&a%%/ $ar a(art:F storing o$ code book 92k code #ords, %ength n:
' +ecodingF Com(are a received #ord #ith a%% (ossib%e code #ords
8/13/2019 8b Error Correction
55/129
A.J. Han Vinck, rondheim, 2011 55
hat are the c%assica% research (rob%ems
*odeboo+ maimie - # of codewords of
length n
- minimum distance
dmin
*hannel characteriation-/ypes of errors0 memory in the noise behavior -*hannel *apacity
2ecoder-2esign of a decoding3decision algorithm-4inimie complexity
8/13/2019 8b Error Correction
56/129
A.J. Han Vinck 56
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain
' )ome channe% mode%s
' *&rst error mode%s
' +etection r&%es' ntro to the coding (rob%em
' $erformane
' he c%ass o$ %inear codes
' *o&nds on the minim&m distance' Eam(%es o$ codes
' h/ error correction
8/13/2019 8b Error Correction
57/129
A.J. Han Vinck 57
Definitions
: Ha))ing #isance &e?een 3 an# 1 is
#H>< #03 1 is "e !; $!sii!ns ?"ere 3
i1
i
: T"e )ini)%) #isance !; a c!#e C is
= #)in
< )in #03 1 2 3 C 1 C 3 1P
: Ha))ing ?eig" !; a ec!r 3 is
9 ?03>< #03 , is "e !; $!sii!ns ?"ere 3
i,
8/13/2019 8b Error Correction
58/129
A.J. Han Vinck 58
e3a)$le
: Hamming distance #0 +,,+ ,+++ <
: Minimum distance 0+,+ ,++ ++, < -
: Hamming weight ?0,++,+,+ < 4
Ha))ing ?as a ;a)!%s scienis ;r!) Fell9la& an#
inener !; "e Ha))ing c!#e.
8/13/2019 8b Error Correction
59/129
A.J. Han Vinck 5*
Performance
A c!#e ?i" )ini)%) #isance #)in
is ca$a&le !;
c!rrecing err!rs i;
#)in
- +.
/r!!;> I; err!rs !cc%r "en since #)in
- +
an inc!rrec c!#e ?!r# "as a leas + #i;;erences?i" "e receie# ?!r#.
8/13/2019 8b Error Correction
60/129
A.J. Han Vinck 6,
$ic%re
-+ #i;;erences
A F
#i;;erences ;r!) A #i;;erences ;r!) F
8/13/2019 8b Error Correction
61/129
A.J. Han Vinck 6+
Performane
A code #ith minim&m distance dmin is ca(ab%e o$ correcting E eras&res i$
dmin W E.
roo$8
$ E P dmineras&res occ&r, then at %east 1 (ositionis %e$t to disting&ish bet#een an/ t#o code#ords.
Note8 an eras&re is a (osition #here the receiver kno#s that an error occ&red.
8/13/2019 8b Error Correction
62/129
A.J. Han Vinck 6-
Performane
A code #ith minim&m distance dmin is ca(ab%e o$ correcting E eras&res
and t errors i$
dmin W 2t ?E.
roo$8he minim&m distance is red&ced at maim&m b/ the va%&e E.Hence, i$
dmin - E W 2t
#e can sti%% correct t errors
8/13/2019 8b Error Correction
63/129
A.J. Han Vinck 6
Performane for the %ausian hannel
?-
?
- -
?
E
9eras&re:9error:
(E
(E
'Bor an error correcting code8 2t < 2 n 9(
2
?(!
: error correcting
'ith eras&res8 dmin W n 9(2?(1: ? 2 n (!
')ince (2 P (1 error correction #itho&t eras&res is a%#a/s betterT
(!
(!
(1(2
(!
8/13/2019 8b Error Correction
64/129
A.J. Han Vinck 64
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain
' )ome channe% mode%s
' *&rst error mode%s
' +etection r&%es' ntro to the coding (rob%em
' (er$ormance
' 1he lass of linear odes
' *o&nds on the minim&m distance
' Eam(%es o$ codes
' h/ error correction
8/13/2019 8b Error Correction
65/129
A.J. Han Vinck 65
GNEA C"+E)
*inar/codes are ca%%ed %inear i$$
Z the com(onent #ise mod&%o-2 s&m o$ t#o code#ords is again a code #ord.
Conse;&ent%/, the all 2ero 'ord is acode #ord.
8/13/2019 8b Error Correction
66/129
A.J. Han Vinck 66
GNEA C"+E 7ENEA"
T"e c!#e ?!r#s are
9 linear c!)&inai!ns !; "e r!?s !; a &inar1 genera!r )ari3 G?i"#i)ensi!ns k n
9 G )%s "ae rank kQ
(3a)$le> C!nsi#er k < n < 6.
+ , , + + ,
genera!r )ari3 G = + + , , + +
+ , + + , +
0+,+G < 0 , , + , + +
8/13/2019 8b Error Correction
67/129
A.J. Han Vinck 67
1se)aic c!#es
Le in general "e )ari3 G &e ?rien as
+ , , 2 + + ,
G < R Ik/ S G< , + , 2 + , +, , + 2 , + +
k < n < 6
T"e c!#e generae# is= linear s1se)aic
= "as )ini)%) #isance .
= "e e;;icienc1 !; "e c!#e is '6.
8/13/2019 8b Error Correction
68/129
A.J. Han Vinck 68
Eam(%e 9o(tim&m:
' )ing%e arit/ check code dmin< 2, k < n-1
100 0 1
0100 0 1
7 < R In9+
/ S
8/13/2019 8b Error Correction
69/129
A.J. Han Vinck 6*
Eam(%e 9o(tim&m:
' e(etition code8 dmin< n, k < 1
' 7 < [ 1 1 1 \
8/13/2019 8b Error Correction
70/129
A.J. Han Vinck 7,
E;&iva%ent codes
' An/ %inear code generator can be bro&ght in Ms/stematic $orm]
' 7s/sDeer)ine "e )ini)%) #isance
8/13/2019 8b Error Correction
74/129
A.J. Han Vinck 74
Linear c!#e generator
9=: re(resents the k bit in$o vector 9 i0, i1, ..., ik-1 :
g9=: is a binar/ (o%/nomia% o$ degree 9 n-k :
3-
the code vector C o$ %ength n can be described b/
C9=: < 9=: g9=: a%% o(erations mod&%o-2.
8/13/2019 8b Error Correction
75/129
A.J. Han Vinck 75
EX: ! "# n ! $ and %&X' ! 1 ( X ( X)
: @!r "e in;!r)ai!n ec!r 0+,+,
C0 < 0+ - 0+ 6
< + - 50+++ , ,+ ,.
: "e enc!#ing $r!ce#%re in 0k 3 n )ari3 ;!r)>
+ + , + , , ,G < , + + , + , , c < I G
, , + + , + ,
, , , + + , +
8/13/2019 8b Error Correction
76/129
A.J. Han Vinck 76
I)$le)enai!n ?i" a s"i;9regiser
T"e ;!ll!?ing s"i; regiser can &e %se#> g0 < 0+
ik9+
...i-i+i,
H!)e?!rk>
gie a #escri$i!n !; "e s"i; c!nr!l ! !&ain "e res%l
8/13/2019 8b Error Correction
77/129
A.J. Han Vinck 77
)ome remarks
' 7enerators $or di$$erent k and n
F are constr&cted &sing mathematics
F %isted in man/ tet books
hat remains is the decodingT
8/13/2019 8b Error Correction
78/129
A.J. Han Vinck 78
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain
' )ome channe% mode%s
' *&rst error mode%s
' +etection r&%es' ntro to the coding (rob%em
' (er$ormance
' he c%ass o$ %inear codes
' !ounds on the minimum distane
' Eam(%es o$ codes
' h/ error correction
8/13/2019 8b Error Correction
79/129
A.J. Han Vinck 7*
*o&nds on minim&m distance 9Hamming:
' Ginear codes have a s/stematic e;&iva%ent 7
F inim&m Hamming #eight n F k ? 1' 9)ing%eton bo&nd:
' code #ords _ correctab%e error (atterns 2n
,
-
- A ' -
+l! g + 0 4
t p nn
i
nM t n
i
R M h pn
=
=
=
8/13/2019 8b Error Correction
80/129
A.J. Han Vinck 8,
Hamming bo&nd eam(%e
' rob%em87ive the &((er bo&nd o$ the siSe o$ a %inear code C o$ %ength n < K and dmin< !
' )o%&tion8the code corrects &( to one error and th&s QCQ 2K 9 1 ? K: < K ^ QCQ
)ince $or a %inear code QCQ < 2k #e have QCQ O
eam(%e8 1 0 0 0 1 1
7 < 0 1 0 1 0 1
0 0 1 1 1 0
8/13/2019 8b Error Correction
81/129
A.J. Han Vinck 8+
*o&nds on minim&m distance 9%otkin:
- Gist a%% code#ords o$ a binar/ %inar code n
2k code#ords
- Ever/ co%&mn m&st have 2k2 ones 9(roo$:9no a%% Sero co%&mn eists:
Conc%&sion8he tota% n&mber o$ ones in the code#ords is n 2k2
8/13/2019 8b Error Correction
82/129
A.J. Han Vinck 8-
*o&nds on minim&m distance 97i%bert:
' )tart8 )e%ect code#ord $rom 2n(ossib%e #ords
' 1. emove a%% #ords at distance P dmin$rom se%ected code#ord
' 2. )e%ect one o$ the remaining as net code#ord
' !. 7oto 1. &n%ess no (ossibi%ities %e$t.
' E)6G8
' home#ork8 sho# that %ogn 1 F h92(: $or dmin-1 < 2t 2(n ( P `
) i n +
,
- n
d
i
M
ni
=
8/13/2019 8b Error Correction
83/129
A.J. Han Vinck 8
(%ot
1
0.5
< %og2n
( tn
1-h9(:
1-h92(:
0
)ing%eton
%otkin
8/13/2019 8b Error Correction
84/129
A.J. Han Vinck 84
Bor Additive hite 7a&ssian Noise Channe%s;&antiSed as binar/ s/mmetric channe%
' Error (robabi%it/ ( e-EsNoF #here Es is energ/ (er transmitted s/mbo%F No the one-sided noise (o#er s(ectra% densit/
' Bor an &ncoded s/stem ( e-EbNo
' Bor a coded s/stem #ith minim&m distance dF decoding errors occ&r #hen b/ 2t?1 W dF nEs < kEb and th&s
(ce-d2 kn EbNo
' COCL6SIO make the fator C 7d* kn 1
8/13/2019 8b Error Correction
85/129
A.J. Han Vinck 85
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain
' )ome channe% mode%s
' *&rst error mode%s
' +etection r&%es' ntro to the coding (rob%em
' (er$ormance
' he c%ass o$ %inear codes
' *o&nds on the minim&m distance
' -:am$les of odes
' h/ error correction
8/13/2019 8b Error Correction
86/129
A.J. Han Vinck 86
ichard Hamming 9115-1O:
n 1^, Hamming #as one o$ the ear%iest &sers o$ (rimitive com(&ters at *e%% Gaboratories. He #as $r&strated b/ their%ack o$ $&ndamenta% re%iabi%it/. He there$ore (&SS%ed over the (rob%em o$ ho# a com(&ter might check and correct itso#n res&%ts. ithin severa% months Hamming discovered that etra bits co&%d be added to the interna% binar/ n&mbers o$the com(&ter to red&ndant%/ encode n&merica% ;&antities. his red&ndanc/ enab%ed re%ative%/ sim(%e circ&itr/ to identi$/and correct an/ sing%e bit that #as bad #ithin the encoded b%ock o$ bits 9t/(ica%%/ one #ord o$ data:. his encodingscheme, no# kno#n as Hamming Code, a%so detects the condition o$ an/ t#o bits in the encoded b%ock that $ai%sim&%taneo&s%/.n 150, Hamming (&b%ished his #ork as Error +etecting and Error Correcting Codes in the *e%% )/stem echnica%Jo&rna%, vo%. 2, ((. 1^-1K0.
5 K
^
1 2!
1, 2, ..., ^ are binar/ va%&es 90 or 1:
1, 2, !, are in$ormation bits
5 makes 1?!??5 even
K makes 2?!??K even
^ makes 1?!?2?^ even
1, 2, ..., ^ are transmitted 9stored:
.ny sin0le bit han0e an be orreted
8/13/2019 8b Error Correction
87/129
A.J. Han Vinck 87
)im(%er code, same (rinci(%e
5
K
1
2
!
in$o check
1 ; ;
1 2 ! 5 K
in$o check
1 0 0 0 1 1 transmit1 2 ! 5 K
+ecode received
1 0 0 0 ; 1
8 ransmission e$$icienc/
8/13/2019 8b Error Correction
88/129
A.J. Han Vinck 88
he net code has %ength 15 and 11 in$ormation bits
10
5
2
K
!
O
8 Check that 1 error can be corrected
8 ransmission e$$icienc/
^
11
(
12
!
1
8/13/2019 8b Error Correction
89/129
A.J. Han Vinck 8*
h/ making things com(%icated
' he 9!,1: code
/+ /-+
8/13/2019 8b Error Correction
90/129
A.J. Han Vinck *,
' Can #e do a genera% (resentation >E)
0+-6
8/13/2019 8b Error Correction
91/129
A.J. Han Vinck *+
Hamming codes
: Mini)%) #isance
: C!nsr%ci!n
=G < Ik
All )9%$les !; Ha))ing ?eig" W +
= ?"ere k < -)
= ) 9 +
: *+ec t+at t+e minim,m distance is )-
: .ive t+e efficiency of t+e code
8/13/2019 8b Error Correction
92/129
A.J. Han Vinck *-
Eam(%e k < , n < ^, m < !
+,,, ++,
,+,, +,+G c"!!se $ara)eers "a can &e i)$le)ene#
8/13/2019 8b Error Correction
96/129
A.J. Han Vinck *6
he (arit/ check matri
' e sa# that C < 7 7 is k n7H< 0His n n-k
ro(osition8 "n%/ $or code#ords CH< 0
roo$8 take the s/stematic encoder (resentation
- there are 2n-kdi$$erent s/ndromes
- 2k
di$$erent vectors ca&se the same s/ndromenH< ) mH< ) 9n m:H< 0 i.e. the/ di$$er b/ a code#ord
- the 2k code#ords give the s/ndrome < 0
8/13/2019 8b Error Correction
97/129
A.J. Han Vinck *7
he (arit/ check matri
' A (ro(ert/ to be &sed %ater8
F An/ dmin-1 ro#s o$ Hare %inear%/ inde(endent
eason8 $ not, then %ess than dminro#s give the s/ndrome 0
"n%/ Code#ords give s/ndrome 0 and have minim&m #eight dmin
8/13/2019 8b Error Correction
98/129
A.J. Han Vinck *8
Varshamov-*o&ndLet us onstrut a matri: 3'ith dimensions n : n&k for a 0i"en minimum distane d
)
k in;!r)ai!n s1)&!ls n9k c"eck s1)&!ls
Properties: )ini)%) #isance < n9k+ 0s1)&!ls
leng" -) 9+
8/13/2019 8b Error Correction
101/129
A.J. Han Vinck +,+
7enera% remarks on ) codes
' he genera% (rob%em is the decoding
F ) codes can be decoded &sing' E&c%ids a%gorithm
' *er%ekam( asse/ a%gorithm
8/13/2019 8b Error Correction
102/129
A.J. Han Vinck +,-
eed-&%%er based codes 96):
' )tarting code m < 1 C < I 00,01,10,11 F has minimum distane *m&1
F *m=1ode 'ords 6 of len0th *m
' NE code #ith 2m?2 code #ords o$ %ength 2m?1F 96,6: and 96,6:
distance [ 96,6:, 96,6:\ < 2m #h/ Convince /o&rse%$T
distance [ 96,6:, 9V,V:\ < 2 _ 2m-1 < 2m
distance [ 96,6:, 9V,V:\ < 2 _ 2m-1 < 2m 9&se com(%.(ro(.:
8/13/2019 8b Error Correction
103/129
A.J. Han Vinck +,
eed-&%%er based codes 96):
)&((ose #e have a generator 7 $or certain m
he constr&ction o$ 7 $or m?1 is sim(%e
Eam(%e8m < 1 m
8/13/2019 8b Error Correction
104/129
A.J. Han Vinck +,4
eed-&%%er based codes 96):
' *asis $or constr&ction
' ake 2 %inear code generators 71 and 72
' o$ %ength n' and minim&m distance +1 and +2, res(ective%/
' hen, 71 717 < has d
min
< minI2+1,+2
0 72' roo$T
8/13/2019 8b Error Correction
105/129
A.J. Han Vinck +,5
)*N n&mbering code
' $or the )*N n&mbers 9a1, a2, , a10: #e have
#here
' e &se a s(ecia% s/mbo% = in case a10< 10.
' A sing%e transmission error gives as a res&%t
since 11 is a (rime n&mber.
otecheck that the code can a%so detect a do&b%e trans(osition 9interchange o$ 2 n&mbers:.
++)!#,ia+,
+i
i==
.++)!#,aiaa+,ia +,
*
+i
i+,
*
+i
i ==+ ==
D++)!#,XeXeia XX
+,
+i
i =+=
8/13/2019 8b Error Correction
106/129
A.J. Han Vinck +,6
Varshamov-enengo%S codes
' $or the binar/ n&mbers 91, 2, , n: #e have
' A sing%e transmission error gives as a res&%t
' Check that a%% n&mbers are di$$erent and th&s #e can correct 1 errorT
' he cardina%it/ o$ the code
' 3ome'orkcom(are the rate o$ the code #ith the rate o$ the Hamming code.
4+n-)!#0ai3n
+i
i +==
+n-)!#0X'aXei3 X
n
+ii ++=+
=
+n-
-2C2
n
+
8/13/2019 8b Error Correction
107/129
A.J. Han Vinck +,7
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain
' )ome channe% mode%s
' *&rst error mode%s
' +etection r&%es' ntro to the coding (rob%em
' (er$ormance
' he c%ass o$ %inear codes
' *o&nds on the minim&m distance
' Eam(%es o$ codes' >hy error orretion?
8/13/2019 8b Error Correction
108/129
A.J. Han Vinck +,8
h/ error correction
' )/stems #ith errors can be madea%most error $ree
F C+, +v+ #o&%d not #ork #itho&t ) codes
8/13/2019 8b Error Correction
109/129
A )%li %ser a$$licai!n
: (NCODING> ?! %sers rans)i + !r = +
9 %$$!se !ne %ser %ses an err!r c!rrecing c!#e a rae Y.
9 T"e sec!n# %ser X%s sen#s "is in;!r)ai!n a rae +.
: CHANN(L> a##s "e al%es an# "%s "e receier !&seres - , !r 9-
: D(COD(> #ec!#es "e in;!r)ai!n ;!r %ser + %sing an eras%re #ec!#ing
$r!ce#%re A;er #ec!#ing "e in;!r)ai!n ;r!) %ser - can &e calc%lae#
: UM AT( < +.5 &i'rans)issi!n
: NOT(> Ti)e Diisi!n gies a rae +Q
A.J. Han Vinck +,*
8/13/2019 8b Error Correction
110/129
A )%li %ser a$$licai!n
A.J. Han Vinck ++,
?- 1
?- 1
?- 2 0add
6ser 1 < 12
6ser 2
8/13/2019 8b Error Correction
111/129
A.J. Han Vinck +++
h/ error correction
8/13/2019 8b Error Correction
112/129
A.J. Han Vinck ++-
h/ error correction
' )&((ose8 transmit 9n-k bits: receiver
> < = N9oise: >H NH N = < > add =H add =
= = = =
transmit n bits
Bor n-k < nh9(: #e transmitted n ? nh9(: bits instead o$ 2n
' e$erence8 )%e(ian and o%$
8/13/2019 8b Error Correction
113/129
A.J. Han Vinck ++
h/ error correction
' n A s/stems s/stems co%%a(s can be (ost(onedT
tro&gh(&t
100D
kn D
Channe% error (robabi%it/0 1
8/13/2019 8b Error Correction
114/129
A.J. Han Vinck ++4
Combining error detection-correction
' 719=: 729=: generates a code #ith minim&m distance +2' 719=: generates a code #ith minim&m distance +1 P +2' C9=: < 9=: 719=: 729=: < 9=:719=:
decoding8ste( 18 correct a maim&m o$ 9 +1 F 1 :2errors #ith 719=:ste( 28 detect the remaining errors #ith 719=:729=:
(ro(erties8
1. t9 +1 F 1 :2
errors correctab%e2. t +2 - 1 - 9 +1 F 1 :2 errors detectab%e
Home#ork8 constr&ct some eam(%es
8/13/2019 8b Error Correction
115/129
A.J. Han Vinck ++5
Combining error detection-correction
' Eam(%e8
n k dmin generator (o%/nomia% 9octa%:
K! 5^ ! 71 < 10! 9octa%:
51 5 72 71 < 71 912^:
5 ^ 7! 72 71< 72 71 91^:
8/13/2019 8b Error Correction
116/129
A.J. Han Vinck ++6
Eam(%e8 b%&etooth
' hree error correction schemes are de$ined $or *%&etooth8F 1! rate BEC, a sim(%e re(etitionF 2! rate BEC, a shortened Hamming Code 910,15:F A scheme $or data 9a&tomatic retransmission re;&est:
' BEC schemes red&ce the n&mber o$ retransmissions
' A CC code decides #hether a (acketheader contains errors, i.e.
' transmit C9=: < 9=: g9=:' receive 9=: < C9=: ? E9=:' check 9=: mod&%o g9=: < 0
8/13/2019 8b Error Correction
117/129
Code shortening
' )&((ose that #e have a ) code in s/stematic $orm n4
1 0 ... 0
0 1 ... 0 he generator takes the $irst k ? n4-k co%&mns
arit/ (art minim&m distance is n4-k?1
... 0 0 ... 1
k n4-k
k4 n-k
take the %ast k4 ro#s o$ %ength k4 ? n - k
100...0 arit/ (art minim&m distance is n4 F k4 ? 1
00... 1++7
Case a
Case b
8/13/2019 8b Error Correction
118/129
A.J. Han Vinck ++8
Channe% #ith insertions and de%etions
' *ad s/nchroniSation or c%ock recover/ at receiver8
F insertion
''' 1 0 0 0 1 1 1 0 0 1 0 ''' ''' 1 0 0 10 1 1 1 0 0 1 0 '''
F de%etion
''' 1 0 0 01 1 1 0 0 1 0 ''' ''' 1 0 0 1 1 1 0 0 1 0 '''
rob%em8 $inding start and end o$ messages
8/13/2019 8b Error Correction
119/129
A.J. Han Vinck ++*
Channe% #ith insertions and de%etions
' Eam(%e8
F the $o%%o#ing code corrects' a sing%e inversion error
' or a sing%e de%etioninsertion error
' C < I 0 0 0 0 0 0, 1 1 1 1 1 1, 0 0 1 0 1 1, 1 1 0 1 0 0
F dmin< !F An insertionde%etion makes the #ord %ongershorter
8/13/2019 8b Error Correction
120/129
A.J. Han Vinck +-,
Channe% #ith insertions and de%etions
' +&e to errors in bit (attern
$%ag < 1 1 1 1 1 0, avoid 1 1 1 1 1 in $rame
''' 0 1 1 1 1 1 0 0 1 1 0 1 ''' ''' 0 1 1 1 1 01 0 0 1 1 0 1 '''
''' 0 1 1 0 1 01 0 0 1 1 0 1 ''' insertion
''' 0 1 1 1 0 0 0 0 1 1 0 1 ''' ''' 0 1 1 1 10 0 0 1 1 0 1 '''
''' 0 1 1 1 1 0 0 1 1 0 1 ''' de%etion
8/13/2019 8b Error Correction
121/129
A.J. Han Vinck +-+
Channe%s #ith inter$erence
' Eam(%e 9o(tica% channe%:
Error (robabi%it/ de(ends on s/mbo%s in neighboring s%ots
8/13/2019 8b Error Correction
122/129
A.J. Han Vinck +--
Channe%s #ith memor/ 9e8recording:
: (3a)$le> i<
i
i9+
i + 9+ P
Xi Xi-1
Yi
{ +2, 0, -2
8/13/2019 8b Error Correction
123/129
,5.,-.+4 +-
he (ass#ord (rob%em
serverPass'ordPass'ord< Honar/Hash9Honar/:
memor/ Hash9Honar/:
Ca%c&%ateHash9(ass#ord:
com(are
server
am8 *ahram
Hash9 :
memor/ Hash9 :
Ca%c&%ateHash9 :
com(are
am8*ahram
biometribiometri
8/13/2019 8b Error Correction
124/129
,5.,-.+4 +-4
rob%em in a&thentication8
(ass#ords8 need to be eact
biometrics8 are on%/ ver/ simi%ar
8/13/2019 8b Error Correction
125/129
,5.,-.+4 +-5
he (rob%em A8 sec&re sketch
** )tore *H)tore *H
9s/ndrome:9s/ndrome:
*< **< * NNca%c&%ateca%c&%ate
9*9* *:H*:H < NH< NH
*H*H
+EC"+E N+EC"+E N**NN
*H*H
@rom NHNH *k$ossible "etors ! e:$
8/13/2019 8b Error Correction
126/129
,5.,-.+4 +-6
A&thentication8 sec&re storage, #ith errors
-nrollment-nrollment
)< * E9:
random
Condition8
given ) and Hash9:
it is hard
to estimate *,
store )
Hash9:
*
E9:
authentiationauthentiation )Hash9:
*4 )*4 < * E9:*4 < E9: N
8/13/2019 8b Error Correction
127/129
,5.,-.+4 +-7
A&thentication8 sec&re storage, #ith errors
attakattak
)Hash9:
7&ess or *
*4 < )
E94: < *
E9:
E94:or
) H< [* E9:\ H< *4 H
-nrollment-nrollment
)< * E9:
random
Condition8
given ) and Hash9:
it is hard
to estimate *,
store )
Hash9:
*
E9:
*i t
8/13/2019 8b Error Correction
128/129
12!
*inar/ entro(/
inter(retation8%et a binaryse;&ence contain (n ones, then#e can s(eci$/ each se;&ence #ith
%og22nh9(:
< n h9(: bits
0 4-nh pn
pn
4$0"
$n
nl!g
n
+li) -n
=
Home#ork8rove the a((roimation &sing %nNT N %nN $or N %arge.
6se a%so %oga < / %ogb < / %ogba
he )tir%ing a((roimation Q - N NN N N e
T+e /inary Entropy> "0$ < $l!g $ 0+ $ l!g 0+ $
8/13/2019 8b Error Correction
129/129
0. 1
0. 2
0. 3
0. 4
0. 5
0. 6
0. 7
0. 8
0. 9
1
h
T+e /inary Entropy> "0$ < 9$l!g-$ = 0+9$ l!g-0+9$
N!e>
"0$ < "0+9$