Post on 27-Oct-2014
LOW DENSITY PARITY CHECK CODES
BLOCK DIAGRAM OF A GENERAL COMMUNICATION SYSTEM
WHAT IS CODING?
Coding is the conversion of information to another form for some purpose.
Source Coding : The purpose is lowering the redundancy in the information. (e.g. ZIP, JPEG, MPEG2)
Channel Coding : The purpose is to defeat channel noise.
CHANNEL CODING
Channel encoding : The application of redundant symbols to correct data errors.
Modulation : Conversion of symbols to a waveform for transmission.
Demodulation : Conversion of the waveform back to symbols, usually one at a time.
Decoding: Using the redundant symbols to correct errors.
WHY CHANNEL CODING?
Trade-off between Bandwidth, Energy and Complexity.
Coding provides the means of patterning signals so as to reduce their energy or bandwidth consumption for a given error performance.
CHANNELS The Binary Symmetric Channel(BSC) The Binary Erasure Channel (BEC)
HOW TO EVALUATE CODE PERFORMANCE? Need to consider Code Rate (R), SNR (Eb/No),
and Bit Error Rate (BER). Coding Gain is the saving in Eb/No required
to achieve a given BER when coding is used vs. that with no coding.
Generally the lower the code rate, the higher the coding gain.
Better Codes provides better coding gains. Better Codes usually are more complicated
and have higher complexity.
SHANNON’S CODING THEOREMS
8
• If C is a code with rate R>C, then the probability of error in decoding this code is bounded away from 0. (In other words, at any rate R>C, reliable communication is not possible.)
• It Tells the maximum rate at which information can be transmitted over a communications channel of a specified bandwidth in the presence of noise.
(x))H – (H(x)Max C y
SHANNON’S CODING THEOREMS
STATEMENT OF THE THEOREM
Where, C is the channel capacity in bits per second. B is the bandwidth of the channel in hertz. S is the average received signal power over the
bandwidth N is the average noise or interference power over
the bandwidth S/N is the signal-to-noise ratio (SNR) or the
carrier-to-noise ratio (CNR) of the communication signal.
COMMON FORWARD ERROR CORRECTION CODES
Convolutional Codes Block Codes (e.g. Reed-Solomon Code) Trellis-Coded-Modulation (TCM) Concatenated Codes
LINEAR BLOCK CODESThe parity bits of linear block codes are
linear combination of the message. Therefore, we can represent the encoder by a linear system described by matrices.
BASIC DEFINITIONS Linearity:
where m is a k-bit information sequence c is an n-bit codeword.
is a bit-by-bit mod-2 addition without carry
Linear code: The sum of any two codewords is a codeword.
Observation: The all-zero sequence is a codeword in every Linear Block Code.
then
and If
2121
2211
ccmm
cmcm
BASIC DEFINITIONS (CONT’D)
Def: The weight of a codeword ci , denoted by w(ci), is the
number of of nonzero elements in the codeword.
Def: The minimum weight of a code, wmin, is the smallest
weight of the nonzero codewords in the code.
Theorem: In any linear code, dmin = wmin
Systematic codes
Any linear block code can be put in systematic form
n-kcheck bits
kinformation bits
THE ERROR-CONTROL PARADIGMNoisy channels give rise to data errors: transmission or storage systems
Need powerful error-control coding (ECC) schemes: linear or non-linear
Linear EC Codes: Generated through simple generator or parity-check matrix
Binary information vector (length k)
Code vector (word): (length n)
Key property: “Minimum distance of the code”, , smallest separation between two codewords.
Rate of the code R= k/n
1 0 0 0 1 0 11 1 1 0 1 0 0
0 1 0 0 1 1 00 1 1 1 0 1 0
0 0 1 0 1 1 11 0 1 1 0 0 1
0 0 0 1 0 1 1
G H
mind
1 2 3 4( , , , )u u u u u
, 0Tx uG Hx
Binary linear codes:
LDPC CODESMore than 40 years of research (1948-1994) centered around
Weights of errors that a code is guaranteed to correct
“Bounded distance decoding” cannot achieve Shannon limit
Trade-off minimum distance for efficient decoding
Low-Density Parity-Check (LDPC) Codes
Gallager 1963, Tanner 1984, MacKay 1996
1. Linear block codes with sparse (small fraction of ones) parity-check matrix
2. Have natural representation in terms of bipartite graphs
3. Simple and efficient iterative decoding in the form of belief propagation (Pearl, 1980-1990)
mind
min( 1)/ 2d
THE CODE GRAPH AND ITERATIVE DECODING
Variable nodes Check nodes
(Irregular degrees/codes)1 1 1 0 1 0 0
0 1 1 1 0 1 0
1 0 1 1 0 0 1
H
Most important consequence of graphical description: efficient iterative decoding
Message passing:
Variable nodes: communicate to check nodes their reliability (log-likelihoods)
Check nodes: decide which variables are not reliable and “suppress” their inputs
Small number of edges in graph = low complexity
Nodes on left/right with constant degree: regular code
Otherwise, codes termed irregular
Can adjust “degree distribution” of variables/checks
Best performance over standard channels: long, irregular, random-like LDPC codes
Have proportional to length of code, but correct many more errorsmind
CONSTRUCTION LDPC codes are defined by a sparse
parity-check matrix. This sparse matrix is often randomly generated, subject to the sparsity constraints. These codes were first designed by Gallager in 1962.
In this graph, n variable nodes in the top of the graph are connected to (n−k) constraint nodes in the bottom of the graph. This is a popular way of graphically representing an (n, k) LDPC code.
Specifically, all lines connecting to a variable node (box with an '=' sign) have the same value, and all values connecting to a factor node (box with a '+' sign) must sum, modulo two, to zero (in other words, they must sum to an even number).
CONTD., This LDPC code fragment
represents a 3-bit message encoded as six bits. Redundancy is used, here, to increase the chance of recovering from channel errors. This is a (6, 3) linear code, with n = 6 and k = 3.
In this matrix, each row represents one of the three parity-check constraints, while each column represents one of the six bits in the received codeword.
CONT.,
• In this example, the eight codewords can be obtained by putting the parity-check matrix H into this form through basic row operations.
• From this, the generator matrix G can be obtained as (noting that in the special case of this being a binary code ), or specifically:
• Finally, by multiplying all eight possible 3-bit strings by G, all eight valid codewords are obtained. For example, the code word for the bit-string '101' is obtained by: