incompletepreliminaryversion - TU Wien · 292 Chapter 7. LDPC Codes 7.1 Introduction and Outline...
Transcript of incompletepreliminaryversion - TU Wien · 292 Chapter 7. LDPC Codes 7.1 Introduction and Outline...
7 LDPC Codes
incomplete preliminary version
You do not have to study this chapter for the exam.
Contents
7.1 Introduction and Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
7.2 Definition and Properties of LDPC Codes . . . . . . . . . . . . . . . . . . 292
7.2.1 Definition of LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
7.2.2 Regular LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
7.2.3 Tanner Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
7.2.4 Irregular LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
7.2.5 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
7.2.6 Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
7.2.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
7.3 Some Constructions of LDPC Codes . . . . . . . . . . . . . . . . . . . . . . 306
7.3.1 A Simple Random Construction (MacKay–Neal Construction) . . . . . . . . . 306
7.3.2 Gallager’s Random Construction . . . . . . . . . . . . . . . . . . . . . . . . . . 306
7.3.3 Random Construction Based on Permutation Matrices . . . . . . . . . . . . . . 307
7.3.4 Algebraic Construction Based on Cyclic Shift Matrices . . . . . . . . . . . . . . 308
7.3.5 Algebraic Constructions Based on Finite Geometries . . . . . . . . . . . . . . . 308
7.3.6 Repeat-Accumulate Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
7.3.7 Further Design Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
7.3.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
7.4 Iterative Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
7.4.1 Bit-flipping Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
7.4.2 Belief Propagation (Sum-Product) Algorithm . . . . . . . . . . . . . . . . . . . 316
7.4.3 EXIT Charts and Density Evolution . . . . . . . . . . . . . . . . . . . . . . . . 316
7.4.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
291
292 Chapter 7. LDPC Codes
7.1 Introduction and Outline
Besides turbo codes, low-density parity check (LDPC) codes are a class of linear block codes that
allow communication close to channel capacity and are decoded iteratively. LDPC codes are among
the most powerful codes currently available. The sparse structure of their check matrix potentially
leads to excellent distance properties and enables suboptimum iterative decoding with a complexity
that grows only linearly with the blocklength.
LDPC codes, along with an iterative decoding algorithm, were introduced by Robert Gallager
already in the early 1960s. The method was then more or less ignored during more than 30 years.
This changed only in the mid-1990s, after the advent of turbo codes, when researchers rediscovered
the advantages of codes with sparse (low-density) check matrices and became interested in codes on
graphs and iterative decoding.
In this chapter, we discuss some basic aspects of LDPC codes and present two iterative decoding
algorithms with different performance and complexity. The chapter is organized as follows.
• In Section 7.2, we present the definition of regular and irregular LDPC codes and their description
in terms of row and column weights, degree distributions, and Tanner graphs. We also discuss
basic properties and performance aspects of LDPC codes and describe an efficient encoding
method.
• Section 7.3 presents some systematic random and algebraic constructions of LDPC codes, in-
cluding constructions based on finite geometries and repeat-accumulate codes.
• In Section 7.4, we discuss two iterative decoding algorithms for LDPC codes, the bit-flipping
algorithm and the belief propagation algorithm. We also briefly consider EXIT charts and the
density evolution scheme, which can be used to analyze the performance of iteratively decoded
LDPC codes.
7.2 Definition and Properties of LDPC Codes
Roughly speaking, an LDPC code is a linear block code for which there exists a check matrix H
that is “sparse” in a specific sense. To present the basic idea of LDPC codes, we temporarily assume
transmission of a codeword c ∈ C over a HIHO channel, at whose output a senseword v is received.
Then, the essential implication of the sparse check matrix H is that each senseword symbol vn is
involved in only very few parity check equations, and each parity check equation involves only very
few senseword symbols vn. For a sufficiently large blocklength N , this sparsity structure potentially
leads to extremely good distance properties; in particular, the code’s minimum distance increases
linearly with the blocklength. Furthermore, a sparse H enables an iterative probabilistic decoding
scheme—the belief propagation or sum-product algorithm—whose complexity scales only linearly
with the blocklength. It has been demonstrated that appropriately designed LDPC codes with very
7.2 Definition and Properties of LDPC Codes 293
large blocklength N , when decoded iteratively using the belief propagation algorithm, perform only a
fraction of a decibel away from the channel capacity.
LDPC codes have certain advantages over turbo codes:
• They tend to have a better block error performance, and a better performance on bursty channels.
• They are more amenable to high rates, and in fact can be designed for almost any rate and
blocklength. (In contrast, the rate of turbo codes is usually adjusted by means of a puncturing
scheme, which necessitates an additional design step.)
• Their error floor tends to occur at a lower BER.
• The encoder and decoder do not require interleavers.
• A single LDPC code can be universally good for a collection of channels.
• There exist iterative LDPC decoding algorithms that are easy to implement, have moderate
complexity (which scales linearly with the blocklength), and are parallelizable in hardware. In
particular, LDPC decoding using the belief propagation (sum-product) algorithm tends to be
less complex than turbo decoding using the BCJR algorithm.
• LDPC decoders inherently check if a codeword satisfying the check equations has been found,
and otherwise declare a decoding failure. (In contrast, turbo decoders usually need to perform
additional operations to compute a stopping criterion, and even then it is not clear if the decoding
result corresponds to a codeword satisfying the check equations.)
LDPC codes also have certain disadvantages relative to turbo codes:
• In general, the encoding complexity is higher than for turbo codes. (However, there are special
LDPC code constructions with low encoding complexity.)
• Iterative LDPC decoding typically requires many more iterations than iterative turbo decoding,
which may lead to a higher latency. (The complexity per iteration is much lower, though.)
7.2.1 Definition of LDPC Codes
Check matrix and parity checks. The central object in the definition, analysis, and design of
LDPC codes is the check matrix H. Consider a linear (N,K) block code C with a check matrix H.
According to Section 3.3.2, H has N rows rn, m = 0, . . . , N−1 and M = N−K linearly independent
columns hm, m = 0, . . . , N−K−1. The columns hm span the orthogonal space C⊥, which equals the
dual code; the code C itself is the linear space that is orthogonal to the linear space spanned by all
the columns hm. Equivalently, C is the null space of the matrix HT, i.e., the set of all the vectors
v ∈ [GF(q)]N for which HTv = 0. Furthermore, the rank of H is equal to N−K.
We recall from Equation (3.58) that the syndrome associated with a received senseword v is the
vector of length M = N−K defined as s = HTv. Furthermore, v ∈ C if and only if s = 0. The mth
syndrome component is
294 Chapter 7. LDPC Codes
sm = hTmv =
N−1∑
n=0
(hm)nvn =
N−1∑
n=0
Hn,mvn , m = 0, . . . ,M−1 , (7.1)
where hm is the mth column of H (note that we index the columns from 0 to M−1). Thus, we have
the following necessary and sufficient condition for v being a codeword:
v ∈ C ⇔ sm = hTmv = 0 , for all m = 0, . . . ,M−1. (7.2)
Therefore, we will call the mth syndrome component sm = hTmv the mth (parity) check.
In the context of LDPC codes, it is convenient to allow the possibility of linearly dependent columns
hm. If H contains Q additional columns that are linearly dependent on the N−K linearly independent
columns, then the size of H is N×M with
M = N−K+Q ≥ N−K .
We now have M ≥ N−K checks sm = hTmv and as many check equations (7.2), although only N−K of
them are linearly independent. Note that C is still the null space of HT, because if we add to a given
set of vectors a set of vectors that is linearly dependent on the first set, then the linear space spanned
by the vectors is unchanged. Also, the rank of H is still equal to N−K. In fact, the dimension K of
the code defined by H is now determined not via the dimension N×M of H but via the rank of H.
We furthermore recall from Section 3.3.2 the following obvious but important facts, which are a
straightforward consequence of (7.1):
• The weight of the mth column hm (i.e., the number of nonzero elements of hm) equals the
number of senseword symbols vn that are involved in the mth check sm. The positions of the
nonzero elements of hm equal the indices n of those vn that are involved in check sm = hTmv.
• The weight of the nth row rn (i.e., the number of nonzero elements of rn) equals the number of
checks sm = hTmv involving the nth senseword symbol vn. The positions of the nonzero elements
of rn equal the indices m of those checks sm that involve vn.
Note that, in the sense discussed above, the mth check sm is related to the mth column hm and the
nth senseword symbol vn is related to the nth row rn.
Definition of an LDPC code. An LDPC code is, essentially, a linear block code for which there
is a very sparse check matrix H. Here, “very sparse” means more specifically that the weight of each
row rn and of each column hm is much smaller than the matrix dimensions N and M . That is, each
check sm involves only very few senseword symbols vn, and each senseword symbol vn is involved in
only very few checks sm. The sparse check matrix H is called a low-density parity check matrix. (The
density of H is defined as the number of nonzero elements Hn,m 6= 0 divided by the total number of
elements, NM .) For reasons that will become clear in Section 7.2.3, we additionally require that any
two rows rn, rn′ with n 6= n′ have at most one “nonzero position” in common, i.e., there is at most
one index m ∈ {0, . . . ,M−1} such that the mth elements of rn and rn′ are both nonzero. This means
that there are not two senseword symbols vn , vn′ that are both involved in two different checks sm1
and sm2 simultaneously.
7.2 Definition and Properties of LDPC Codes 295
The LDPC code is then formally defined—consistent with Section 3.3.2—as the linear space that
is orthogonal to the linear space spanned by all the columns hm. That is, the LDPC code is the null
space of the matrix HT. Note that the columns hm may be linearly dependent.
The sparse check matrix H is typically nonsystematic. As any linear code, an LDPC code has
many check matrices besides H, including a systematic check matrix; these will not be sparse in
general. Similarly, the generator matrices G are not sparse in general.
7.2.2 Regular LDPC Codes
Column weight and row weight. An LDPC code whose sparse check matrix H is such that all
its columns hm have the same weight wc and all its rows rn have the same weight wr is called a regular
LDPC code with column weight wc and row weight1 wr. More formally, the sparse check matrix H of
a regular LDPC code satisfies
wH(hm) = wc , m = 0, . . . ,M−1and
wH(rn) = wr , n = 0, . . . , N−1 .
Thus, each check sm involves wc senseword symbols vn, and each senseword symbol vn is involved in
wr checks sm. For later use, we define the set Nm ⊆ {0, . . . , N−1} whose elements are the indices n
of those senseword symbols vn that are involved in the mth check sm. These indices correspond to
the positions of the nonzero elements of hm, i.e.,
Nm ,{n : (hm)n 6= 0
}={n : Hn,m 6= 0
}.
Note that if sm = hTmv 6= 0, then at least one of the senseword symbols vn with n ∈ Nm must have
been changed by the channel. Similarly, we define the set Mn ⊆ {0, . . . ,M−1} whose elements are
the indices m of those checks sm that involve the nth senseword symbol vn. These indices correspond
to the positions of the nonzero elements of rn, i.e.,
Mn ,{m : (rn)m 6= 0
}={m : Hn,m 6= 0
}.
Evidently, the cardinality of the set Nm is equal to the column weight wc, i.e.,
|Nm| = wc , m = 0, . . . ,M−1 ,
and the cardinality of the set Mn is equal to the row weight wr, i.e.,
|Mn| = wr , n = 0, . . . , N−1 .
A schematic illustration is given in Figure 7.1. Note that in the binary case (where Hn,m ∈ {0, 1}),the mth check sm = hTmv =
∑N−1n=0 Hn,mvn can be written as
sm =∑
n∈Nm
vn . (7.3)
1In the literature, a check matrix H is sometimes defined as the transpose of H, and the syndrome is expressed as
s = vHT (where s = sT and v = vT are row vectors) rather than s = HTv. Within this definition, the meanings of
column and row are interchanged, and so are the meanings of column weight wc and row weight wr, i.e., the column
(row) weight is our row (column) weight.
296 Chapter 7. LDPC Codes
Figure 7.1: S hemati representation of H, hm, rn, Nm, and Mn for a regular binary LDPC ode with
N= 6, M= 4, wc = 3, and wr = 2. (The representation is only s hemati be ause the density of H is too
high for a true LDPC ode.)
The sparsity of H helps spread out the senseword symbols vn into checks sm such that two different
checks are unlikely to involve similar sets of senseword symbols (i.e., Nm and Nm′ tend to be very
different for m 6= m′) and two different senseword symbols are unlikely to be involved in similar sets
of checks (i.e., Mn and Mn′ tend to be very different for n 6= n′).
Consider a regular (N,K) LDPC code with a sparse check matrix H of size N×M , where M =
N−K+Q. Let E denote the total number of nonzero elements of H. Clearly
E = Nwr = Mwc . (7.4)
The second equation is a relation between the parameters N ,M , wr, and wc. Thus, e.g., if blocklength
N , column weight wc, and row weight wr have been chosen, the number of columns of H follows as
M = Nwrwc
.
Since typically M ≤ N , this implies wr ≤ wc. Furthermore, note that
wrwc
=M
N. (7.5)
The density of H is defined as the number of nonzero elements divided by the total number of elements,
i.e.,
ρ ,E
NM.
Using (7.4), we obtain
ρ =wrM
=wcN. (7.6)
Rate. Let us next consider the rate of a regular LDPC code, R = K/N . If Q = 0 (i.e., all the
columns of H are linearly independent), then M = N−K, or equivalently K = N−M . That is, the
dimension K equals the difference of the two matrix dimensions of H, N−M . Formally replacing in
R = K/N the dimension K by N−M even if Q> 0 yields the design rate
7.2 Definition and Properties of LDPC Codes 297
R′ ,N−MN
= 1− M
N= 1− wr
wc, for Q ≥ 0 . (7.7)
Because M = N−K+Q ≥ N−K, we have N−M ≤ K, and hence
R′ ≤ R ,
with equality iff Q = 0. Thus, the actual rate R = K/N may be higher than the design rate R′ =
(N−M)/N . Determination of R requires determination of the rank of H (which equals N−K): we
have
R =K
N=N − (N−K)
N=N − rank{H}
N= 1− rank{H}
N.
In many cases, N is chosen quite large, from thousands to hundreds of thousands, while wr is
chosen as 3 or 4. It has been shown that wr = 2 leads to poor performance, whereas for wr ≥ 3 the
minimum distance of a “typical” regular LDPC code increases linearly with the blocklength N . In
other words, each senseword symbol should be involved in at least three checks.
7.2.3 Tanner Graphs
Any binary linear code C—note that binary implies Hn,m ∈ {0, 1}—can be represented by a Tanner
graph (TG). There are many different TGs for a code C, each one corresponding in a one-to-one fashion
to a check matrix H for C.A graph consists of a set of nodes (or vertices) and a set of edges connecting two nodes. Two
nodes connected by an edge are said to be adjacent. The number of edges connected to (or incident
on) a given node is called the degree of that node. The state diagram and trellis diagram previously
encountered in Sections 5.5.1 and 5.5.2 are two examples of graphs.
Definition and interpretation of TGs. The TG corresponding to a given N ×M check matrix
H contains two types of nodes: N bit nodes and M check sum nodes or briefly check nodes. Each bit
node corresponds to one of the N code bits cn or senseword bits vn, and each check node corresponds
to one of the M checks sm. Bit node n is adjacent to check node m—i.e., there is an edge between
these two nodes— iff Hn,m = 1, i.e., iff bit vn is involved in check sm. Thus, the degree of bit node n
equals the number of checks sm involving senseword bit vn, and the degree of check node m equals the
number of senseword bits vn involved in check sm. While there may be an edge between a bit node
and a check node, there are no edges between two bit nodes, nor between two check nodes. Thus, TGs
are bipartite graphs.2
Evidently, the TG describes which bits are involved in which checks, and thus it provides a graphical
description of the check matrix H to which it corresponds. More specifically, each bit node in the TG
corresponds to a row of H, each check node in the TG corresponds to a column of H, and each edge
in the TG corresponds to a nonzero element of H. Since the code C is the null space of H, it follows
that a TG provides a complete (though nonunique) description of C.2A bipartite graph is a graph with the following two characteristics: (i) it contains two different types of nodes, and
(ii) edges may only exist between two nodes of different type.
298 Chapter 7. LDPC Codes
Figure 7.2: TG orresponding to the he k matrix in (7.8), depi ted in two alternative but equivalent
graphi al representations.
Example. We consider the linear binary (7, 4) block code defined by the check matrix
H =
1 1 11 0 01 0 00 1 00 1 00 0 10 0 1
. (7.8)
There are seven code bits c0, . . . , c6 or senseword bits v0, . . . , v6 and three checks s0, s1, s2. For example,
check s0 corresponds to the first column of H. Thus, it involves the senseword bits v0, v1, v2 and equals
s0 = hT0v = v0 + v1 + v2. Furthermore, bit c0 or v0 corresponds to the first row of H and is involved
in all three checks s0, s1, and s2. Note that this code cannot be considered an LDPC code since the
density ρ is not very low (due to the small matrix dimensions). Note also that this is an irregular code
since the weight of the first row is different from the other row weights.
The TG corresponding to this check matrix is shown in Figure 7.2 in two different but equivalent
graphical representations. The first representation emphasizes the bipartite nature of the TG, while the
second emphasizes its tree structure (further discussed below). For example, check node s0 is adjacent
to bit nodes v0, v1, v2; bit node v0 is adjacent to check nodes s0, s1, s2; and the edge connecting check
node s0 and bit node v0 corresponds to the check matrix element H0,0 = 1. �
TGs can be established for any binary linear code. However, the TG description is most efficient
for LDPC codes, because there are only few edges. For a regular binary LDPC code, evidently, the
bit node degrees (= numbers of checks in which the various bits are involved) are all equal and given
by the row weight wr, and the check node degrees (= numbers of bits involved in the various checks)
are all equal and given by the column weight wc. This means that each bit node is adjacent to wr
check nodes and each check node is adjacent to wc bit nodes.
Cycles. A finite sequence of connected edges starting and ending in the same node, and not con-
taining a node more than once, is called a cycle (or loop). The length of a cycle is the number of
edges it contains. TGs may or may not have cycles. The TG of Figure 7.2 does not have cycles; it is
7.2 Definition and Properties of LDPC Codes 299
Figure 7.3: TG orresponding to the he k matrix in (7.9), depi ted in two alternative but equivalent
graphi al representations. One y le of length 4 is emphasized by boldfa e edges.
thus an acyclic or cycle-free graph. Any acyclic graph is a tree. The tree structure is visualized by
the right-hand part of Figure 7.2; note that each one of the nodes can be interpreted as the root of
the tree. The next two examples present codes whose TGs have cycles.
Example. We consider the linear binary (7, 4) block code defined by the check matrix3
H =
1 1 11 0 00 0 10 1 01 0 11 1 00 1 1
. (7.9)
As in the previous example, there are seven code bits c0, . . . , c6 or senseword bits v0, . . . , v6 and three
checks s0, s1, s2. Again, this is an irregular code. The TG corresponding to this check matrix is shown
in Figure 7.3. This TG contains several cycles of length 4 and 6. One cycle of length 4 is emphasized
in the graph by boldface edges; the elements in H corresponding to these edges are emphasized in
(7.9) by boldface 1s. �
Evidently, a TG—just as any bipartite graph—cannot have cycles of length less than 4, and it
cannot have cycles of odd length. The minimum length of cycles in a graph is referred to as the girth
of the graph. Thus, the girth of a TG is at least 4. Formally, the girth of an acyclic TG is infinite.
TGs are of interest not only because they provide an intuitively appealing graphical description of
an LDPC code (or, more specifically, of a check matrix). More importantly, they provide a basis for
iterative belief propagation decoding algorithms, to be discussed in Section 7.4.2. The performance of
these algorithms depends on the presence and length of the cycles in the TG. For good performance,
short cycles—especially cycles of length 4 or 6—should be avoided. This is an important criterion in
the design of LDPC codes. In particular, as demonstrated by our previous example, a cycle of length
4 in the TG corresponds to four 1s in H that lie on the corners of a submatrix of H. This means
3This code is equivalent to the (7, 4)2 Hamming code presented in Section 3.4.3. That is, H is a row-permuted version
of the check matrix in (3.73); the row permutation results in a simpler graphical representation of the associated TG.
300 Chapter 7. LDPC Codes
Figure 7.4: TG orresponding to the he k matrix in (7.10). One y le of length 6 is emphasized by
boldfa e edges.
that two different rows rn, rn′ have more than one 1-position in common. Note that our definition of
LDPC codes in Section 7.2.1 postulated that any two rows have at most one 1-position in common,
which excludes the presence of cycles of length 4 in the TG. Next, we give an example of the TG of a
regular code with girth 6.
Example. We consider the linear binary block code of length N = 6 that is defined by the check
matrix
H =
1 0 1 01 1 0 00 1 0 11 0 0 10 1 1 00 0 1 1
. (7.10)
There are six code bits c0, . . . , c5 or senseword bits v0, . . . , v5 and four checks s0, . . . , s3. However,
one check is redundant in the sense that the rank of H is only 3; indeed, it is easily verified that the
last column of H is the sum of the first three columns. Hence, while M = 4, the dimension is only
K= 3 (because rank{H} = N−K = 6−K). This is a regular code with wr=2 and wc=3. The TG
corresponding to H is shown in Figure 7.4. This TG contains several cycles of length 6 but no cycle
of length 4. Hence, the girth is 6. One cycle of length 6 is emphasized in the TG by boldface edges,
and in H by boldface 1s. �
Within the set of TGs associated with the various check matrices for the same code, it is possible
that some contain cycles whereas others are acyclic; furthermore, those TGs that contain cycles may
have different girths. Unfortunately, the minimum distance dmin of binary codes that possess an acyclic
TG is poor: it has been shown that if R ≥ 1/2, then dmin ≤ 2, and if R < 1/2, then dmin ≤ 2/R. For
growing blocklength N , short cycles become increasingly unlikely. In fact, asymptotically as N→∞,
the cycle-free assumption can be made. This fact is exploited by the density evolution technique
considered in Section 7.4.3.
Besides the girth of the TG, a related concept relevant to the performance of iterative belief
propagation decoders is the expansion of the TG. A TG with a good expansion is such that a given
subset of bit nodes is adjacent to a relatively large subset of check nodes (even though each individual
bit node is adjacent only to few check nodes), which in turn is adjacent to a large subset of other
7.2 Definition and Properties of LDPC Codes 301
bit nodes (even though each individual check node is adjacent only to few bit nodes). Thus, a given
set of erroneous bits is checked by many check nodes, which in turn receive information from a large
number of correct bits.
7.2.4 Irregular LDPC Codes
An LDPC code that is not regular is called irregular. Irregular LDPC codes have a very sparse check
matrix H in which the column weight may vary from column to column and the row weight may vary
from row to row. The ability to allocate the column and row weights flexibly provides a useful design
freedom. In fact, properly designed irregular LDPC codes tend to outperform regular LDPC codes for
large block lengths; gains of up to 0.5 dB can be obtained. In the following description, we consider
binary irregular LDPC codes for simplicity.
Degree distribution. We recall that the nth row of H corresponds to code bit cn and an associated
bit node in the TG, and the weight of the row equals the degree (number of incident edges) of that
bit node. Similarly, the mth column of H corresponds to check sm and an associated check node in
the TG, and the weight of the column equals the degree of that check node. In this sense, the terms
weight and degree can be used interchangeably. Furthermore, we recall that each 1 in H corresponds
to an edge in the TG.
Let Ni denote the number of rows of H with weight i (number of bit nodes in the TG with degree
i), and let Mi denote the number of columns of H with weight i (number of check nodes in the TG
with degree i). For the special case of a regular LDPC code with column weight wc and row weight
wr, we have Ni = Nδi,wr and Mi =Mδi,wc. Furthermore, let wr,max denote the maximum row weight
(maximum bit node degree), and let wc,max denote the maximum column weight (maximum check
node degree). Thenwr,max∑
i=1
Ni = N ,
wc,max∑
i=1
Mi = M. (7.11)
Moreover, the total number of 1s in H (total number of edges in the TG) can be expressed as
E =
wr,max∑
i=1
iNi =
wc,max∑
i=1
iMi . (7.12)
Note that this generalizes relation (7.4), which is reobtained for a regular LDPC code. We also consider
the fraction of rows of H with weight i (fraction of bit nodes with degree i),
νi ,Ni
N, (7.13)
and the fraction of columns of H with weight i (fraction of check nodes with degree i),
µi ,Mi
M.
Note that∑wr,max
i=1 νi =∑wc,max
i=1 µi = 1. The pair (ν,µ) of vectors ν , (ν1 · · · νwr,max)T and µ ,
(µ1 · · · µwc,max)T is called the degree distribution of the LDPC code. It is an important characteristic
of an irregular LDPC code. For a regular code with column weight wc and row weight wr, we have
νi = δi,wr and µi = δi,wc .
302 Chapter 7. LDPC Codes
Edge-related degree distribution. An alternative—but equivalent—definition of degree distri-
bution is related to the edges in the TG. Let Er,i denote the total number of 1s in H that occur in all
the rows with weight i (total number of edges in the TG that are incident on all the bit nodes with
degree i, and also total number of check nodes adjacent to all the bit nodes with degree i). Similarly,
let Ec,i denote the total number of 1s in H that occur in all the columns with weight i (total number
of edges in the TG that are incident on all the check nodes with degree i, and also total number of
bit nodes adjacent to all the check nodes with degree i). For the special case of a regular LDPC code
with column weight wc and row weight wr, we have Er,i = Eδi,wr and Ec,i = Eδi,wc . We note that
wr,max∑
i=1
Er,i = E ,
wc,max∑
i=1
Ec,i = E . (7.14)
Evidently, Er,i is equal to i multiplied by the number of rows with weight i (bit nodes with degree i),
i.e.,Er,i = iNi , (7.15)
and similarly
Ec,i = iMi .
Furthermore, we consider the fraction of 1s in H that occur in all the rows with weight i (fraction of
edges incident on all the bit nodes with degree i),
ǫr,i ,Er,iE
, (7.16)
and the fraction of 1s in H that occur in all the columns with weight i (fraction of edges incident on
all the check nodes with degree i),
ǫc,i ,Ec,iE
.
Note that∑wr,max
i=1 ǫr,i =∑wc,max
i=1 ǫc,i = 1. The pair (ǫr, ǫc) of vectors ǫr , (ǫr,1 · · · ǫr,wr,max)T and
ǫc , (ǫc,1 · · · ǫc,wc,max)T is an alternative version of the degree distribution of the LDPC code. It is
sometimes convenient to express the vectors ǫr and ǫc by the polynomials
ǫr(x) ,
wr,max∑
i=1
ǫr,i xi−1, ǫc(x) ,
wc,max∑
i=1
ǫc,i xi−1.
For a regular code with column weight wc and row weight wr, we have ǫr,i = δi,wr and ǫc,i = δi,wc or,
equivalently, ǫr(x) = xwr−1 and ǫc(x) = xwc−1.
The parameters N and M and, in turn, the design rate R′ (see (7.7)) can be expressed in terms
of the degree distribution (ǫr, ǫc). First, using (7.11), (7.15), and (7.16), we obtain
N =
wr,max∑
i=1
Ni =
wr,max∑
i=1
Er,ii
= E
wr,max∑
i=1
ǫr,ii, (7.17)
and similarly
M = E
wc,max∑
i=1
ǫc,ii. (7.18)
7.2 Definition and Properties of LDPC Codes 303
Then, inserting (7.17) and (7.18) into (7.7) yields for the design rate
R′ = 1− M
N= 1−
∑wc,max
i=1 ǫc,i/i∑wr,max
j=1 ǫr,j/j.
It is easily verified that this can also be expressed as
R′ = 1−∫ 10 ǫc(x) dx∫ 10 ǫr(x) dx
.
The two definitions of degree distribution (ν,µ) and (ǫr, ǫc) are equivalent in that one can be
obtained from the other. Indeed, using (7.13), (7.15), (7.16), and (7.17), we obtain
νi =Ni
N=
Er,i/i
N=
ǫr,i/i∑wr,max
j=1 ǫr,j/j,
and similarly
µi =ǫc,i/i∑wc,max
j=1 ǫc,j/j.
Conversely, carrying out the above steps in reverse order yields
ǫr,i =iνi∑wr,max
j=1 jνj, ǫc,i =
iµi∑wc,max
j=1 jµj.
The “edge-related” degree distribution (ǫr, ǫc) is closely related to the density evolution technique
for irregular LDPC codes, which will be sketched in Section 7.4.3. The density evolution technique is
an important design tool that allows the determination of an “optimal” degree distribution. From the
ensemble of LDPC codes having the optimal degree distribution, a specific LDPC code is then chosen.
Using this approach, an irregular LDPC code of blocklength N = 107 and rate R = 1/2 has been
designed whose simulated performance was within 0.045 dB of the capacity limit for a binary-input
AWGN channel.
7.2.5 Performance
Suitably designed LDPC codes with large blocklength provide near-capacity performance on many
data transmission and storage channels while at the same time admitting implementable decoders.
LDPC codes can be roughly categorized as random and algebraic constructions. The performance of
randomly constructed LDPC codes is good (with high probability) if the blocklength N is large enough,
but may not be sufficiently good for moderate values of N . On the other hand, randomly constructed
LDPC codes typically do not have a sufficient structure to allow efficient encoding. (Note that with
LDPC codes, the complexity of decoding is a lesser issue because iterative message passing algorithms
allow efficient decoding with a complexity that is only linear in N , see Section 7.4.) Algebraic LDPC
codes, due to their strong structure, can usually be encoded more efficiently than random LDPC codes,
and for moderate values of N their performance is often better. However, they tend to perform less
well than random LDPC codes when N is very large.
The BER-versus-SNR performance of LDPC codes for the AWGN channel exhibits an error floor
that depends on the minimum distance dmin; it is low if dmin is large (cf. Section 6.2.3). LDPC codes
tend to have a low error floor— in particular, lower than turbo codes. This is because for a very large
304 Chapter 7. LDPC Codes
blocklength N and a very sparse check matrix H, it is relatively easy to pseudorandomly construct a
code with a large dmin. The sparsity of H guarantees the linear growth of dmin with N for suitably
constructed LDPC code ensembles. In particular, it has been shown that for N sufficiently large, there
exists an LDPC code with rate R ≥ 1− r and fractional minimum distance δ = dmin/N ≥ δ0, for any
δ0 < 1/2 satisfying H2(δ0) < r. (Here, as in Section 2.4.3, H2(·) is the binary entropy function, i.e.,
H2(p) = −p ld p − (1−p) ld (1−p).) In addition, the sparsity of H is essential for a low complexity
of iterative message-passing decoding (see Section 7.4). In fact, the sparsity of H enables both a
minimum distance that grows linearly with the blocklength and a decoding complexity that grows
only linearly with the blocklength.
For very large blocklengths, randomly constructed irregular LDPC codes can perform a small
fraction of a decibel away from the channel capacity. However, such codes tend to have a higher
error floor. Conversely, LDPC codes with an extremely low error floor tend to perform far away from
capacity. Thus, there is a tradeoff between capacity-approaching performance and a low error floor.
For small and medium blocklengths, algebraic constructions can outperform random ones, and
irregular codes are not necessarily better than regular ones. In addition, the structure inherent to
algebraic constructions often leads to much simpler implementations, especially for encoding, and to
minimum distance and girth properties that are difficult to achieve with a random construction for
short and medium blocklengths. In particular, for reasonably small blocklengths, algebraic construc-
tions with a relatively high row weight are able to achieve very low error floors, the downside being a
larger gap to capacity. Some random and algebraic constructions of LDPC codes will be presented in
Section 7.3.
7.2.6 Encoding
LDPC codes are usually designed by first constructing a sparse check matrix H and then determining
a corresponding generator matrix G. Because G is not sparse, the complexity of encoding via the
generic relation c = Gu is quadratic in the blocklength N . However, there exist various classes
of LDPC codes that possess a deterministic structure allowing encoding with reduced complexity.
Examples are certain LDPC code designs based on finite geometries, which lead to cyclic or quasi-
cyclic LDPC codes that can be encoded using shift register circuits (see Section 7.3.5), and the class
of repeat-accumulate codes, which use encoders composed of simple blocks (see Section 7.3.6). On
the other hand, the efficient encoding enabled by a deterministic structure may come at the cost of a
poorer error performance for large blocklengths.
Low-complexity encoding for unstructured LDPC codes. In the absence of a deterministic
structure, it is still possible to significantly reduce the encoding complexity by an encoding method
that is based not on the generator matrix G but directly on the check matrix H. The codeword is
constructed in the systematic form (3.33), i.e., c = (uT pT )T. Thus, encoding a dataword u amounts
to determining the corresponding parity word p. This can be done by solving the check equation
HTc = 0, i.e.,
HT
(up
)= 0 , (7.19)
7.2 Definition and Properties of LDPC Codes 305
for p. Here, H is a check matrix that is consistent with the systematic form of c. We also assume for
simplicity that M = N−K, i.e., all columns of H are linearly independent.
In general, the complexity of solving (7.19) is still quadratic in the blocklength N . However, using
elementary column operations and row permutations, it is possible to transform H into an equivalent
check matrix H in block form, where one of the blocks is zero and a further block is triangular.
Thereby, the complexity of solving (7.19) (suitably transformed) is reduced significantly. Since many
of the steps in the transformation H → H are merely column and row permutations, which do not
change the sparsity of H, the resulting check matrix H preserves the sparsity of H to a large extent.
More specifically, as a first step, column and row permutations are used to obtain the transpose
of an equivalent check matrix HD as
HTD =
(A B TC′ D′ E
),
where A is of size (M−g)×K (for some g<M ; note also that M =N−K), B is of size (M−g) × g,
T is a lower triangular (M−g) × (M−g) matrix with 1s on its diagonal (hence, T−1 exists), C′ is of
size g ×K, D′ is of size g × g, and E is of size g × (M−g). Note that HD is exactly as sparse as H.
Next, HTD is multiplied from left by the matrix
(IM−g 0
−ET−1 Ig
). This amounts to performing elementary
row operations and column permutations in HTD and leaves the blocks A, B, and T unchanged, while
the block E is cleared. Thus, the result of this multiplication is of the form
HT =
(A B TC D 0
),
where C = −ET−1A+C′, D = −ET−1B+D′, and 0 is the g × (M−g) zero matrix.
The equivalent check matrix H is now used instead of H in (7.19). Let us partition the codeword
as c = (uT pT1 pT2 )T, where p1 and p2 are of length g and M−g, respectively. Then the new check
equation HTc = 0 is equivalent to the following two linear equations in p1 and p2:
Au+Bp1 +Tp2 = 0 (7.20)
Cu+Dp1 = 0 . (7.21)
From (7.21), we readily obtain
p1 = −D−1Cu . (7.22)
(If D is singular, then the columns of HT can be permuted until D becomes nonsingular.) Next, (7.20)
gives
p2 = −T−1(Au+Bp1) . (7.23)
Thus, the parity blocks p1 and p2 have been calculated from the dataword u. We note that the
complexity of the matrix-vector multiplication in (7.22) is of order gK, and hence it is desirable to
keep g as small as possible. Furthermore, because T is lower triangular, (7.23) can be calculated by
first calculating d , −(Au+Bp1) and then using back-substitution to solve the equation Tp2 = d.
The complexity of this calculation is low due to the sparsity of A, B, and T.
306 Chapter 7. LDPC Codes
7.2.7 Problems
to be done
7.3 Some Constructions of LDPC Codes
The most obvious way to design an LDPC code is to construct a sparse check matrix H with suit-
able properties. Important design criteria include efficient encoding, near-capacity performance, and
low error floor. For long codes, randomly choosing a check matrix almost always produces a good
code. This is guaranteed by the concentration property, which states that the behavior of long codes
randomly drawn from an ensemble concentrates around the ensemble average. However, in practical
applications the block lengths may not be large enough, possibly due to latency constraints. Most
LDPC codes are constructed pseudorandomly, where the basic construction is random but also features
some structure, and certain bad configurations—such as 4-cycles—are avoided by the construction or
removed afterwards.
Many (pseudo)random and algebraic constructions of LDPC codes have been proposed. In this
section, we present or sketch some of them. For simplicity and practical relevance, we consider only
binary codes.
7.3.1 A Simple Random Construction (MacKay–Neal Construction)
A rather straightforward algorithm for randomly constructing an N×M check matrix H with column
weight wc and row weight wr (where M = Nwr/wc) proceeds as follows. The rows r0, r1, etc. are
randomly generated one after the other. More precisely, for generating the nth row, a binary candidate
vector of length M and weight wr is randomly drawn. It is then checked whether (i) the weight of
each partial column up to this point is at most wc and (ii) any two rows up to this point have at most
one “1-position” in common. If these two properties are satisfied, the candidate vector is accepted
and thus becomes the nth row, and the algorithm proceeds to generate the next row. Otherwise, the
candidate vector is rejected and a new candidate vector for the nth row is generated and checked, etc.
Note that this construction avoids cycles of length 4 in the TG (cf. Section 7.2.3).
With this algorithm, in the final check matrix H, the weight of each column is either wc or smaller
than wc. Thus, in general, an irregular LDPC code is obtained. If a regular code is required, the
algorithm has to be modified in a suitable way. A disadvantage of this code construction is that it
does not offer any specific structure that facilitates efficient encoding.
7.3.2 Gallager’s Random Construction
Gallager proposed the following random construction of a regular LDPC code. The code parameters
are chosen such that N = pwc and M = pwr with an integer factor p, so that N/wc = M/wr = p.
(Note that condition (7.4) is satisfied.) The density of H follows from (7.6) as ρ = wr/M = 1/p.
Thus, the code is an LDPC code if p is chosen sufficiently large. Then, the N×M check matrix H is
7.3 Some Constructions of LDPC Codes 307
constructed as a block-row matrix composed of wr ≥ 3 blocks Hi, i = 1, . . . , wr of dimension N × p
each, i.e.,H =
(H1 · · · Hwr
). (7.24)
Here, the blocks Hi are constructed such that each row of Hi has one 1 and each column has wc 1s, all
other elements being 0. This construction guarantees that H in (7.24) has row weight wr and column
weight wc, as desired.
More specifically, the first block H1 is chosen as
H1 =
1 0 0 · · · 00 1 0 · · · 0.........
...0 0 0 · · · 1
,
where 1 is the all-one vector of length wc, 0 is the zero vector of length wc, and there are p block rows
and p columns. It is seen that, as desired, H1 is an N×p matrix in which each row has one 1 and each
column has wc 1s (in the mth column of H1, these 1s occur at positions n = mwc, mwc +1, . . . , (m+
1)wc − 1; note that m ∈ {0, . . . , p−1} and n ∈ {0, . . . , N−1}). Furthermore, no two columns of H1
have any “1-position” in common, and no two rows of H1 have more than one “1-position” in common.
It is easily verified that the latter two properties remain true if the N rows of H1 are permuted in
an arbitrary manner. Therefore, the remaining blocks Hi, i = 2, . . . , wr are derived from H1 by row
permutations, i.e., certain rows are swapped. These row permutations are chosen randomly. Gallager
showed that if each permutation is chosen randomly from among the N ! possible permutations, then
the average minimum distance (averaged over the entire code ensemble) increases linearly with the
blocklength N provided that wr ≥ 3 and wc > wr. This can be viewed as a constructive realization of
Shannon’s idea of a random code. Furthermore, these codes allow for low-complexity encoding since
parity bits can be calculated from the data bits via the check matrix.
For different specific choices of row permutations, the weight distributions and, in particular,
minimum distances of the resulting codes will generally be different. Also, the property that any two
rows rn, rn′ of H have at most one “1-position” in common (i.e., cycles of length 4 in the TG are
avoided) is only satisfied for certain choices of row permutations. In practice, a computer search is
required to find good row permutations.
7.3.3 Random Construction Based on Permutation Matrices
Another random construction starts with the N0×M0 check matrix H0 of a good “core” regular LDPC
code. Let wc (wr) be the column (row) weight of H0. A longer regular LDPC code is now obtained
by replacing each 1 in H0 with a randomly chosen p1 × p1 permutation matrix4 and replacing each
0 in H0 with the p1 × p1 zero matrix. The result is a new check matrix H1 of increased dimension
N1 ×M1, where N1 = p1N0 and M1 = p1M0. It is easily verified that the column (row) weight of
H1 is unchanged, i.e., still wc (wr). Next, each 1 in H1 is replaced with a randomly chosen p2 × p2
permutation matrix and each 0 with the p2 × p2 zero matrix. This yields a new check matrix H2 of
4A permutation matrix is a square matrix that has a single 1 in each row and a single 1 in each column, all other
elements being zero.
308 Chapter 7. LDPC Codes
increased dimension N2×M2, where N2 = p2N1 = p1p2N0 andM2 = p2M1 = p1p2M0, and of the same
column (row) weight wc (wr) as previously. Evidently, this “inflation” procedure can be repeated an
arbitrary number of times. It has been shown that this construction preserves the girth and minimum
distance of the core code.
7.3.4 Algebraic Construction Based on Cyclic Shift Matrices
A simple algebraic construction of a check matrix H with column weight wc and row weight wr is as
follows. For an arbitrary p ≥ (wc−1)(wr−1) + 1 and an arbitrary l ∈ {0, . . . , p−1}, consider the p× p
cyclic shift matrix Jl that is obtained by cyclically shifting each column of the p × p identity matrix
I down by l positions.5 For example, for l= 1,
J1 =
0 0 0 · · · 0 11 0 0 · · · 0 00 1 0 · · · 0 00 0 1 · · · 0 0...
......
......
0 0 0 · · · 1 0
.
Note that Jl = Jl1 and J0 = I. Then, the check matrix constructed as
H =
I I I · · · I
I J1 J2 · · · Jwr−1
I J2 J4 · · · J2(wr−1)
I J3 J6 · · · J3(wr−1)...
......
...
I Jwc−1 J2(wc−1) · · · J(wc−1)(wr−1)
has N = pwc rows and M = pwr columns; furthermore, each row (column) has wr (wc) 1s. It can
also be shown that two different rows have at most one “1-position” in common, i.e., the TG does not
have cycles of length 4.
7.3.5 Algebraic Constructions Based on Finite Geometries
A finite geometry G is a mathematical object that consists of NP points pi, i ∈ {0, . . . , NP−1} and NL
lines lj, j ∈ {0, . . . , NL−1}. Each line lj is a set of some of the points pi, and the following structural
properties can be postulated:
1. Each line consists of nP points.
2. Each point belongs to nL lines.
3. Two points simultaneously belong to exactly one line.
4. Two lines have either no or one point in common.5We note that Jl is a special case of a permutation matrix. Multiplying a p-dimensional vector by Jl produces a
cyclic shift of the vector elements by l positions.
7.3 Some Constructions of LDPC Codes 309
The incidence vector of line lj is defined as the NP-dimensional binary vector ψL,j ∈ {0, 1}NP with
elements(ψL,j
)i=
{1 , if line lj contains point pi
0 , else ,
for i ∈ {0, . . . , NP−1}. Thus, ψL,j indicates which points belong to line lj . Due to Property 1 above,
the weight of ψL,j is nP.
Similarly, the incidence vector of point pi is defined as the NL-dimensional binary vector ψP,i ∈{0, 1}NL with elements
(ψP,i
)j=
{1 , if point pi belongs to line lj
0 , else ,
for j ∈ {0, . . . , NL−1}. Thus, ψP,i indicates which lines lj contain point pi. Due to Property 2 above,
the weight of ψP,i is nL. Note that the set {ψP,i}NP−1i=0 contains exactly the same information as the
set {ψL,j}NL−1j=0 .
For a given finite geometry G, the check matrix HI of the type-I geometry-G LDPC code has the
incidence vector ψL,j as its jth column and the incidence vector ψP,i as its ith row. Thus, we obtain
a regular LDPC code with N = NP (there are NP rows) and M = NL (there are NL columns). With
this type-I construction, the nth senseword bit vn (or, the nth row) corresponds to the point pn, and
the mth check sm (or, the mth column) corresponds to the line lm. Furthermore, the above properties
1–4 translate into the following properties of the check matrix HI:
1. The column weight is wc = nP (because each ψL,j has weight nP).
2. The row weight is wr = nL (because each ψP,i has weight nL).
3. Two rows have one “1-position” in common (because the two rows correspond to two points,
which simultaneously belong to one line).
4. Two columns have at most one “1-position” in common (because the two columns correspond
to two lines, which have at most one point in common).
By (7.6), the density of HI is ρ = wr/M = nL/NL and simultaneously ρ = wc/N = nP/NP; thus, for
an LDPC code, nL and nP must be sufficiently small. The Tanner graph has no cycles of length 4
(due to Property 3); however, it can be shown to have cycles of length 6. The minimum distance of
the code can be shown to be at least nL + 1.
Similarly, the check matrix HII of the type-II geometry-G LDPC code has ψL,j as its jth row and
ψP,i as its ith column. This results in a regular LDPC code with N = NL and M = NP. Here, the
nth senseword bit vn (or, the nth row) corresponds to the line ln, and the mth check sm (or, the mth
column) corresponds to the point pm. The above properties 1–4 remain valid with the roles of column
and row interchanged and with wc and wr interchanged. We have HII = HTI , and thus, the density
and rank of HII equal those of HI. Since the ranks of HI and HII are equal, the numbers of linearly
independent check equations for the type-I and type-II geometry-G codes are equal too. Furthermore,
the TGs associated with HI and HII are dual graphs in the sense that the bit nodes (check nodes)
of one graph become the check nodes (bit nodes) of the other graph. The minimum distance of the
type-II code is at least nP + 1.
310 Chapter 7. LDPC Codes
Two standard constructions of finite geometries (not discussed here) are Euclidean geometries
(EGs) and projective geometries (PGs). For the m-dimensional EG over GF(2s),
NL =2(m−1)s(2ms−1)
2s−1, NP = 2ms, nP = 2s, nL =
2ms−1
2s−1.
For the m-dimensional PG over GF(2s),
NL =ξs(m)ξs(m−1)
2s+1, NP =
2(m+1)s−1
2s−1, nP = 2s+1, nL =
2ms−1
2s−1,
where ξs(m) ,∑m
µ=0 2µs = 1 + 2s + . . . + 2ms. We now obtain four classes of finite-geometry LDPC
codes: type-I EG-LDPC codes, type-II EG-LDPC codes, type-I PG-LDPC codes, and type-II PG-
LDPC codes. The parameters N , M , wc, wr, and ρ of these code classes follow easily from our
discussion above. Note that although the EG and PG are “over GF(2s),” the resulting LDPC codes
are over GF(2), since the incidence vectors are still binary vectors.
A suitable modification of the type-I EG-LDPC codes results in cyclic codes, and a suitable modi-
fication of the type-II EG-LDPC codes results in quasi-cyclic codes. (For a quasi-cyclic code, a cyclic
shift of a codeword by any integer multiple of a certain number l of positions is another codeword.
Thus, in particular, cyclic codes are quasi-cyclic codes with l= 1.) These codes can be easily encoded
with moderate complexity—linear in the blocklength N—using one or several shift-register circuits.
Their parameters are somewhat different from those of the original EG-LDPC codes. The type-I PG-
LDPC codes are themselves cyclic codes, and a suitable modification of the type-II PG-LDPC codes
yields quasi-cyclic codes.
For small blocklength N , these finite-geometry LDPC codes perform generally better than LDPC
codes based on random constructions. They have relatively good minimum distances, and their TGs
do not contain short cycles. Their performance can be further improved by row splitting (see Section
7.3.7). They can be decoded in various ways ranging from low to high complexity and from reasonably
good to very good error performance. They can be encoded with linear complexity. On the other hand,
there are certain restrictions in the choice of blocklength and rate. Furthermore, the values of wc and
wr tend to be relatively high, which is undesirable since the complexity of iterative message-passing
decoders is proportional to these values.
7.3.6 Repeat-Accumulate Codes
A special type of irregular binary LDPC codes is the class of repeat-accumulate codes (RA codes),
which have characteristics of both LDPC codes and serial turbo codes. A major advantage of RA
codes is their efficient encoding, whose complexity is linear in the blocklength N . In spite of their
simplicity and linear encoding complexity, RA codes perform very well over the AWGN channel and
are capable of operation near capacity. However, a potential drawback is their low rate (typically 1/2
or lower).
Encoding. For a systematic (q, a)-regular RA code, the codeword is constructed in the systematic
form (3.33), i.e., c = (uT pT )T. The encoder is shown in Figure 7.5. We see that the parity word p =
7.3 Some Constructions of LDPC Codes 311
Figure 7.5: Blo k diagram of an en oder for a (q, a)-regular RA ode.
(p0 · · · pM−1)T of length M = N −K is calculated from the length-K dataword u = (u0 · · · uK−1)
T by
the serial concatenation of a binary repetition encoder of blocklength q (or rate 1/q), a pseudorandom
interleaver of length Kq, a combiner summing over blocks of length a, and an accumulator. These
four components are discussed in the following.
1. The repetition encoder produces q copies of each data bit uk. Thus, its output corresponding to
the entire length-K dataword u = (u0 u1 · · · uK−1)T is the length-qK word
b = (b0 · · · bqK−1)T = (u0 · · · u0︸ ︷︷ ︸
q times
u1 · · · u1︸ ︷︷ ︸q times
· · · uK−1 · · · uK−1︸ ︷︷ ︸q times
)T .
Typical values of q are 2 or 3.
2. The pseudorandom interleaver permutes the bits in b, resulting in a new word d = (d0 · · ·dqK−1)
T of the same length qK.
3. In the combiner, the bits in each successive length-a block of d are summed (modulo 2), which
produces one bit for each block of a bits. Thus, the output of the combiner is a binary word of
length6 qK/a, r = (r0 · · · rqK/a−1)T, whose bits are given by the mod-2 sums
rm =a−1∑
i=0
dma+i , m = 0, . . . ,qK
a−1 .
4. Finally, the accumulator produces the parity word p = (p0 · · · pM−1)T of length M = N − K.
The mth parity bit pm is the mod-2 sum of the previous parity bit pm−1 and the mth combiner
output bit rm, i.e.,
pm = pm−1 + rm , m = 0, . . . ,qK
a−1 .
This recursion is initialized (for m = 0) with p−1 = 0. Note that pm =∑m
m′=0 rm′ . The
accumulator can be viewed as a rate-1 recursive convolutional encoder described by the 1×1
generator matrix G(x) =(g(0,0)(x)
)with g(0,0)(x) = 1
1+x .
Parameters. From the above construction of the encoder for a systematic (q, a)-regular RA code,
it is clear that the number of parity bits pm is given by qK/a, i.e., we have
M =qK
a.
6The parameters K, q, and a are chosen such that qK/a is integer.
312 Chapter 7. LDPC Codes
Furthermore, the blocklength of a (q, a)-regular RA code is obtained as
N = K+M = K +qK
a= K
a+ q
a,
and the rate follows as
R =a
a+ q.
There are also nonsystematic RA codes, whose codewords do not include the data bits uk. In other
words, the codewords are simply given as c = p. Here, the blocklength is N = qK/a and the rate is
R = a/q. In what follows, we will only consider systematic RA codes.
Check matrix. It can be shown that for a systematic (q, a)-regular RA code, a sparse check matrix
is given by the N×M matrix
H =
(H1
H2
),
where H1 is an (N−M)×M = K×M matrix with (small) column weight a and row weight q, and
H2 is an upper triangular M×M matrix of the following special form:
H2 =
1 1 0 0 · · · 0 0 00 1 1 0 · · · 0 0 00 0 1 1 · · · 0 0 00 0 0 1 · · · 0 0 0...
......
......
......
0 0 0 0 · · · 1 1 00 0 0 0 · · · 0 1 10 0 0 0 · · · 0 0 1
.
Note that although the RA code is encoded systematically, H is not in systematic form (cf. (3.48)).
Inspection of H also shows that a (q, a)-regular RA code is actually an irregular LDPC code— in
particular, the rows in the H2 block have weight 2 except the last row, which has only weight 1, and
the rows in the H1 block have weight q. Also, the columns of H have weight a + 2 except the first
column, which has only weight a+1.
Because of the systematic encoding, one can distinguish two types of code bits: K data (message)
bits and M parity bits. The upper K rows of H—which make up the upper matrix block H1 —
correspond to the data bits whereas the remaining lower M rows of H—which make up the lower
matrix block H2 —correspond to the parity bits. Note also that H1 is related to the joint operation
of repetition encoder, interleaver, and combiner, whereas the special structure of H2 corresponds to
the operation of the accumulator.
There are also irregular RA codes. Their check matrix H is constructed as above, except that
the row weights of the matrix block H1 are not all equal. This means that instead of using a single
repetition encoder on all data bits uk, different subsets of data bits are repeated different numbers of
times, i.e., some bits are repeated more often than others. Irregular RA codes are capable of operation
even closer to capacity, and they permit higher rates.
7.3 Some Constructions of LDPC Codes 313
Figure 7.6: TG of a parti ular (3, 2)-regular RA ode of blo klength N = 10, orresponding to the he k
matrix in (7.25).
Tanner graph. The TG of a systematic (q, a)-regular RA code has a special structure. The N bit
nodes consist of K data (message) bit nodes and M parity bit nodes. These nodes are connected as
follows:
• Each data bit node is adjacent to q check nodes (this is due to the length-q repetition encoder).
The choice of these q check nodes depends on the interleaver and the combiner.
• Each parity bit node is adjacent to a corresponding check node and, except for the last parity
bit node, to the respective next check node (this is due to the accumulator).
• Each check node is adjacent to a corresponding parity bit node and, except the first check node,
to the respective previous parity bit node (this is due to the accumulator). In addition, each
check node is adjacent to a data bit nodes (due to the combiner), whose choice depends on the
repetition encoder and the interleaver.
This structure of the TG will be made more concrete by considering a simple example.
Example. Consider a (3, 2)-regular RA code (i.e., q = 3 and a = 2) of blocklength N = 10. The
rate of this code is R = a/(a+ q) = 2/5, the dimension is K = RN = 4, and the number of checks is
M = N −K = 6. A possible check matrix is
H =
1 0 1 0 1 00 1 1 0 0 11 0 0 1 1 00 1 0 1 0 1− − − − − −1 1 0 0 0 00 1 1 0 0 00 0 1 1 0 00 0 0 1 1 00 0 0 0 1 10 0 0 0 0 1
. (7.25)
The corresponding TG is shown in Figure 7.6. For easier interpretation of the TG, the K = 4 data bit
nodes and M = 6 parity bit nodes are depicted above and below the M = 6 check nodes, respectively.
314 Chapter 7. LDPC Codes
For example, the first (leftmost) check node in the TG corresponds to the first column of H, and
thus the first check involves the first and third data bit and the first parity bit. Accordingly, in the
TG, the first check node is adjacent to the first and third data bit nodes and the first parity bit node.
We can now verify the general rules stated further above: Each data bit node is adjacent to q = 3
check nodes; each parity bit node is adjacent to a corresponding check node and, except for the last
parity bit node, to the respective next check node; and each check node is adjacent to a corresponding
parity bit node, to the respective previous parity bit node (except for the first check node), and to
a= 2 data bit nodes. �
7.3.7 Further Design Techniques
Many further designs of LDPC codes have been proposed. We briefly describe two classes of construc-
tions.
Graph-based constructions. Cycles and other undesired configurations in the TG can be removed
by row splitting and/or column splitting. In row splitting, a row of H is replaced by two or more rows,
such that each 1 of the original row is allocated to one of the new rows. Thereby, the blocklength N
(number of bits) is increased, and H is made more sparse. Similarly, in column splitting, a column of
H is replaced by two or more columns, such that each 1 of the original column is allocated to one of
the new columns. Here, the number of checks is increased, and H is again made more sparse. Both
row splitting and column splitting reduce the degrees of certain nodes in the TG and thus tend to
reduce the number of cycles.
In the bit filling construction, bit nodes are added to the TG one at a time. The edges incident
on a new bit node are chosen with the aim of avoiding cycles of length l or less. More specifically, the
wc check nodes adjacent to the new bit node are chosen such that each check node is (i) a “distance”
of at least l/2 edges away from all the check nodes already chosen, and (ii) the check node least used
so far (i.e., with the lowest degree). Similarly, in the progressive edge growth construction, edges are
added to the TG one at a time. Here, the goal is to maximize the local girth at the current bit node.
These graph-based techniques can also be used to modify an existing TG that was devised using
some structured design, e.g., the TG of a finite-geometry LDPC code or of an RA code.
Combinatorial constructions. Good LDPC codes may be designed by using concepts and tech-
niques from combinatorial mathematics. A combinatorial design is an assignment of a set of entities
(bits) to subsets (which define checks) subject to certain constraints such as the number of bits involved
in each check or the absence of cycles of specified lengths. The incidence matrix of the combinatorial
design indicates which bits belong to which checks, and is thus used as the check matrix H or the
transposed check matrix HT of the LDPC code. The finite geometry based LDPC codes discussed in
Section 7.3.5 are examples of combinatorial constructions. Other designs are based on combinatorial
objects known as Steiner systems and Kirkman systems.
LDPC codes based on combinatorial designs offer a deterministic (algebraic) construction and
guaranteed lower bounds on girth and minimum distance. They may also allow for efficient encoding.
7.4 Iterative Decoding 315
For example, the EG-LDPC and PG-LDPC code designs can produce cyclic or quasi-cyclic codes as
mentioned in Section 7.3.5. Efficient encoders can also be obtained in other cases by exploiting the
structure of the design. Combinatorial designs lead to check matrices that are both sparse and highly
redundant in the sense that they contain many linearly dependent columns, i.e.,M is much larger than
N−K. The linearly dependent check equations help achieve good performance of iterative decoding
even for moderate blocklengths.
7.3.8 Problems
to be done
7.4 Iterative Decoding
For LDPC codes, several decoding methods with different performance and complexity are available.
We will discuss two methods, which are both iterative: the bit-flipping algorithm and the belief prop-
agation algorithm (or sum-product algorithm). The former is a nonprobabilistic hard-input decoding
algorithm, the latter is a probabilistic soft-input decoding algorithm. The belief propagation algorithm
has a better performance than the bit-flipping algorithm but also a higher complexity. In general, the
complexity of iterative decoding is lower for a sparser check matrix.
7.4.1 Bit-flipping Algorithm
Consider a binary regular LDPC code. A codeword c ∈ C is transmitted over a HIHO channel, at
whose output a binary senseword v is received. Hard-decoding of v by means of bit flipping is based
on the M checks sm = hTmv =∑
n∈Nmvn , m= 0, . . . ,M−1 (see (7.3)). If v is a codeword, then all the
sm are zero. However, if a detectable error occurred, then v is not a codeword and some of the checks
sm are 1. Such checks will be called “parity failures.” According to Section 7.2.2, a given senseword
bit vn is involved in the checks sm for m ∈ Mn. Some of these checks may be parity failures. In
particular, if vn is the only incorrect senseword bit, then all the checks sm, m ∈ Mn are parity failures.
The bit-flipping algorithm is based on this observation; its principle is that a senseword bit vn that is
involved in many parity failures is most likely incorrect and should therefore be flipped (inverted). Of
course, flipping the bit vn will change the number of parity failures for some of the other senseword
bits. Therefore, this selective bit flipping is repeated in an iterative manner.
More specifically, consider the checks involving senseword bit vn, i.e., sm for m ∈ Mn, and denote
by Pn (with 0 ≤ Pn ≤ |Mn|) the number of parity failures among these checks. In the bit-flipping
algorithm, the Pn are calculated for all n ∈ {0, . . . , N−1}, and those bits vn for which Pn is above
a predefined threshold T are flipped.7 This results in a modified senseword v′. The checks are
recomputed for v′, yielding s′m = hTmv′ =
∑n∈Nm
v′n and new numbers of parity failures P ′n. These are
7In other versions of the algorithm, the bit(s) vn for which Pn is maximum is/are flipped, or the bits for which more
checks are 1 than 0 (i.e., Pn > |Mn| − Pn) are flipped.
316 Chapter 7. LDPC Codes
again compared to T , and those bits v′n for which P ′n > T are flipped. This procedure is continued
in an iterative manner. If after a predefined number of iterations there are no more parity failures,
i.e., all the M checks are zero, then the modified senseword is a codeword, and this codeword is
used as the result of the decoding algorithm. On the other hand, if there are still some parity failures,
there are two possible ways to proceed: if additional processing time is available, then further decoding
iterations may be performed with a reduced threshold T ; if no more time is available, then the decoding
procedure is terminated and a decoding failure (uncorrectable error) is declared.
This bit-flipping algorithm can be interpreted as message passing on the TG of the code. Each
bit node passes its bit value as a binary message to each adjacent check node, and each check node
passes its check value as a binary message to each adjacent bit node.
Significant overlap between two rows of H or, equivalently, short cycles in the TG reduce the effec-
tiveness of the bit-flipping algorithm and may even prevent the algorithm from converging. For exam-
ple, assume that two rows of H have two “1-positions” in common. This means that two senseword
bits are simultaneously involved in two checks—thus, when both of the checks are 1, it is impossible to
decide which bit is in error—and, also, that there is a cycle of length 4 in the TG. It is then possible
that in each iteration of the bit-flipping algorithm, the two bits are flipped simultaneously and, over
the iterations, each of them is alternatingly correct and incorrect. Hence, in that case, the algorithm
fails to converge.
7.4.2 Belief Propagation (Sum-Product) Algorithm
Presentation and discussion of the belief propagation (BP) decoding algorithm: to be done
Cycles in the TG adversely affect the performance and convergence of iterative BP decoders. In
fact, cycles lead to correlations in the probabilities passed by the BP decoder; the smaller the cycles the
fewer the number of iterations that are correlation-free. Large improvements in decoding performance
can be achieved by removing cycles of length 4 and 6. However, as the the girth of the TG is increased
further, the additional improvements tend to diminish. For LDPC codes with very large blocklength,
the TG can be assumed to be acyclic.
The number of iterations needed to achieve convergence of BP decoding increases when the channel
SNR is reduced. Below a threshold SNR, the BP decoder does not converge at all (cf. Section 7.4.3).
7.4.3 EXIT Charts and Density Evolution
EXIT charts. The extrinsic information transfer (EXIT) chart was originally developed for turbo
codes, as described in Section 6.3.9. However, an EXIT chart can also be established for iteratively
decoded LDPC codes. In an iterative message passing decoder, in each iteration j, there are messages
from check nodes to bit nodes (“check→bit messages”) and messages from bit nodes to check nodes
(“bit→ check messages”). Accordingly, the two types of mutual information used in the EXIT chart
are (i) the mutual information between a check→bit message for a fixed bit and the value of that bit,
denoted IC→B[j], and (ii) the mutual information between a bit→ check message for a fixed bit and
the value of that bit, denoted IB→C[j]. These mutual informations are estimated based on histogram
7.4 Iterative Decoding 317
estimates of the pdf’s of the involved messages. The EXIT chart is then obtained by plotting IC→B[j]
and IB→C[j] for growing iteration index j in a 2-D diagram, similarly to the mutual informations
I(i)a [j] and I
(i)e [j] in Section 6.3.9. The interpretation of this diagram and the conclusions drawn from
it are analogous to those discussed in Section 6.3.9. Again, EXIT chart analysis reveals the existence
of an SNR threshold and provides a rough characterization of the speed of convergence of iterative
BP message passing decoding for different SNRs above the threshold.
Density evolution. Besides EXIT charts, density evolution is another technique for analyzing the
performance of an iterative message passing decoder. For a given LDPC code described by a TG,
we would like to know for which channel noise levels the decoder performs successfully in the sense
of converging and correcting the errors. This question can be difficult to answer analytically for a
specific LDPC code. However, density evolution is able to answer it for classes (ensembles) of regular
LDPC codes with a given column weight wc and row weight wr, under the simplifying assumptions
of infinite blocklength (N→∞) and, hence, a cycle-free TG. By tracking the evolution of pdf’s over
the message passing iterations and studying the asymptotic behavior of these pdf’s as the number
of iterations tends to infinity, density evolution characterizes the maximum noise level (equivalently,
minimum SNR in the case of an AWGN channel) such that iterative decoding is likely to succeed.
This “decoding threshold” noise level is a clear dividing line between successful and unsuccessful de-
coding. Unfortunately, empirical evidence suggests the existence of a tradeoff between a high decoding
threshold noise level and a low error floor.
The decoding threshold noise level is a convenient characterization of performance in terms of a
single parameter. This characterization allows a designer to search for the column weight wc and row
weight wr maximizing the decoding threshold noise level. From the class of all LDPC codes with the
optimum wc and wr, a specific code can then be selected based on other criteria.
Density evolution can also be used for irregular LDPC codes. Here, classes of codes with a given
degree distribution (ǫr, ǫc) as defined in Section 7.2.4 are considered, and the goal is to determine the
degree distribution maximizing the decoding threshold noise level. Using this approach, an irregular
LDPC code of blocklength N = 107 and rate R = 1/2 has been designed whose simulated performance
was within 0.045 dB of the capacity limit for a binary-input AWGN channel.
The design of LDPC codes using the density evolution technique is best suited to codes with large
blocklength (due to the underlying N→∞ assumption). Applying the optimum degree distribution
(ǫr, ǫc) to short or medium-length codes tends to result in a high error floor.
7.4.4 Problems
to be done
318 Chapter 7. LDPC Codes