Homogeneous Recurrence Relation
-
Upload
basualokrediffmailcom -
Category
Documents
-
view
224 -
download
0
Transcript of Homogeneous Recurrence Relation
-
7/28/2019 Homogeneous Recurrence Relation
1/30
Solving Recurrence Relations
Yen (NTUEE) Discrete Mathematics 2012 1 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
2/30
Recurrence Relations
Definition
A recurrence relation for {an} is an equation that expresses an in termsof a0, . . . , an1. A sequence is called a solution of a recurrence relation ifits terms satisfy the recurrence relation.
For instance, the relation fn = fn1 + fn2 of Fibonacci numbers is a
recurrence relation.
Yen (NTUEE) Discrete Mathematics 2012 2 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
3/30
Tower of Hanoi
Example
(The Tower of Hanoi) Consider moving a stack of disks with different sizeswith three pegs. Initially, all disks are sorted and placed on the first peg.One can only move the topmost disk from a peg to another peg. At anytime, disks on each peg must be sorted as well. How many steps does it
take to move all disks from one peg to another?
Solution: Let Hn be the number of steps to solve the problem of n disks.Clearly, H1 = 1.
Now consider moving n disks from peg 1 to peg 2. If we can move thetopmost n 1 disks from peg 1 to peg 3, we can solve the puzzle bymoving the bottom disk from peg 1 to peg 2 and then the n 1 disksfrom peg 3 to peg 2. In other words, Hn = Hn1 + 1 + Hn1. We haveHn = 2Hn1 + 1.
Yen (NTUEE) Discrete Mathematics 2012 3 / 38
http://goforward/http://find/http://goback/ -
7/28/2019 Homogeneous Recurrence Relation
4/30
Tower of Hanoi
Hence
Hn = 2Hn1 + 1= 2(2Hn2 + 1) + 1= 22Hn2 + 21 + 20
= 22(2Hn
3 + 1) + 21 + 20
= 23Hn3 + 22 + 21 + 20
= 2n1H1 + 2n2 + + 21 + 20
=
n
1
i=0
2i
=2n 12 1 = 2
n 1
Yen (NTUEE) Discrete Mathematics 2012 4 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
5/30
Catalan numbers
Example
Find the number of ways to parenthesize the product of n + 1 numbersx0, x1, . . . , xn.
Solution: Let Cn denote the number of ways to parenthesize the productof n + 1 numbers. For instance, C3 = 5 because
x0 (x1 (x2 x3)) x0 ((x1 x2) x3) (x0 (x1 x2)) x3 ((x0 x1) x2) x
are all the possible ways of multiplying x0 x1 x3.Clearly C0 = 1. Consider x0 x1 xn. We can compute it by(x0 xk) (xk+1 xn) for any k = 0, . . . , n. Therefore
Cn = C0Cn1 + C1Cn2 + + Cn1C0.
The sequence {Cn}, Cn = n1k=0 CkCnk1, is called Catalan numbers.
Well give a closed form for Catalan numbers later.Yen (NTUEE) Discrete Mathematics 2012 5 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
6/30
Homogeneous Linear Recurrence Relations with Constant
Coefficients
DefinitionA linear homogeneous recurrence relation of degree k with constantcoefficients is of the form
an = c1an
1 + c2an
2 +
+ ckan
k,
where c1, c2, . . . , ck R with ck = 0.For instance, the recurrence for Fibonacci numbers fn = fn1 + fn2 is alinear homogeneous recurrence relation of degree two.
Theorem
Let c1, c2 R. Suppose x2 = c1x + c2 has two distinct roots r1 and r2.Then {an} is a solution of the recurrence relation an = c1an1 + c2an2 ifand only if an = 1r
n1 + 2r
n2 for n
N and some constants1, 2.
Yen (NTUEE) Discrete Mathematics 2012 6 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
7/30
Solving Linear Recurrence Relations
Proof.() Suppose an = 1rn1 + 2rn2 . We want to verify an = c1an1 + c2an2.
c1an1 = c11rn11 + c12rn12
c2an2 = c21rn
2
1 + c22rn
2
2c1an1 + c2an2 = 1rn21 (c1r1 + c2) + 2r
n22 (c1r2 + c2)
= 1rn21 r
21 + 2r
n22 r
22
= 1rn1 + 2r
n2
= an
(Contd on Next Page)
Yen (NTUEE) Discrete Mathematics 2012 7 / 38
http://find/http://goback/ -
7/28/2019 Homogeneous Recurrence Relation
8/30
Solving Linear Recurrence Relations
Proof.
(Contd) () From the first part of proof, we know an = 1rn
1 + 2rn
2satisfies the recurrence relation an = c1an1 + c2an2. It remains to showan = 1r
n1 + 2r
n2 satisfies initial conditions for some 1, 2. The theorem
then follows from the uniqueness of solution to linear homogeneousrecurrence relation.
To see an = 1rn1 + 2rn2 satisfies the initial conditions for some 1, 2.Consider a1 = 1r1 + 2r2 and a0 = 1 + 2.This is a linear system of two variables 1 and 2. The solutions are
1 =a1 a0r2
r1 r2,
and
2 =a0r1 a1
r1 r2 .
Yen (NTUEE) Discrete Mathematics 2012 8 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
9/30
Fibonacci numbers
Example
Recall the recurrence relation for Fibonacci numbers fn = fn1 + fn2 withf0 = 0 and f1 = 1. Find an explicit formula for the Fibonacci numbers.
Solution: The solutions to x2 = x + 1 are 1
52 . Hence
fn = 1
1 + 52
n+ 2
1 5
2
n
for some 1 and 2. Solving 0 = 1 + 2, and 1 = 11+
52 + 2
1
52 , we
have 1 =1
5and 2 =
1
5. Therefore
fn =1
5
1 +
5
2
n 1
5
1 5
2
n.
Yen (NTUEE) Discrete Mathematics 2012 9 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
10/30
Characteristic Equations with Multiple Roots
Theorem
Let c1, c2 R with c2 = 0. Suppose x2
= c1x + c2 has only one root r.Then {an} is a solution of the recurrence relation an = c1an1 + c2an2 ifand only if an = 1r
n + 2nrn for n N and some constants1, 2.
Proof.
() 2r = c1 and c1r + 2c2 = 0. Let an = 1rn + 2nrn. Thenc1an1 = c11rn1 + c12(n 1)rn1c2an2 = c21rn2 + c22(n 2)rn2
c1an1 + c2an2 = 1rn
2
(c1r + c2) + 2rn
2
(c1(n 1)r + c2(n 2))= 1r
n2r2 + 2rn2(c1nr c1r + c2n 2c2)= 1r
n + 2rn2(c1nr + c2n)
= 1rn + 2r
n2n(c1r + c2)
= 1rn
+ 2nrn
2
r2
n nYen (NTUEE) Discrete Mathematics 2012 10 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
11/30
Characteristic Equations with Multiple Roots
Proof.
(Contd)(
) It remains to show there are 1 and 2 satisfying the initial
conditions. We have a0 = 1. Therefore
2 =a1 a0r
r.
Yen (NTUEE) Discrete Mathematics 2012 11 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
12/30
An Example
Example
What is the solution of the recurrence relation
an = 6an
1
9an
2
with a0 = 1 and a1 = 6?
Solution: Since 3 is the multiple root of x2 = 6x 9, we havean = 13
n + 2n3n. Moreover, 1 = a0 = 1 and 2 = (6
1
3)/3 = 1.
We have an = 3n + n3n.
Yen (NTUEE) Discrete Mathematics 2012 12 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
13/30
Characteristic Equations with Distinct Roots
TheoremLet c1, c2, . . . , ck R. Suppose the characteristic equation
rk c1rk1 ck = 0
has k distinct roots r1, . . . , rk. Then {an} is a solution of the recurrencerelation
an = c1an1 + c2an2 + + ckankif and only if
an = 1rn1 + 2rn2 + + krnkfor n N and constants1, 2, . . . , k.
Yen (NTUEE) Discrete Mathematics 2012 13 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
14/30
Characteristic Equations with Multiple Roots
Theorem
Let c1, c2, . . . , ck R. Suppose the characteristic equationrk c1rk1 ck = 0
has t distinct roots r1, . . . , rt with multiplicities m1, . . . ,mt respectively
such that mi 1 and m1 + m2 + + mk = k. Then {an} is a solution of the recurrence relation an = c1an1 + c2an2 + + ckank if and only if
an = (1,0 + 1,1n + + 1,m11nm11)rn1+ (2,0 + 2,1n +
+ 2,m2
1n
m21)rn2+ + (t,0 + t,1n + + t,mt1nmt1)rnt
for n N and constantsi,j.
Yen (NTUEE) Discrete Mathematics 2012 14 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
15/30
Solving Nonhomogeneous Linear Recurrence Relations
Definition
A linear nonhomogeneous recurrence relation with constantcoefficients is of the form
an = c1an1 + c2an2 + + ckank + F(n),where c1, c2, . . . , ck R and F : N Z. The recurrence relation
an = c1an1 + c2an2 + + ckankis called the associated homogeneous recurrence relation.
Yen (NTUEE) Discrete Mathematics 2012 15 / 38
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
16/30
Solving Linear Recurrence Relations
Theorem
Let{a(p)n } be a particular solution of
an = c1an1 + c2an2 + + ckank + F(n),Then every solution is of the form {a(p)n + a(h)n }, where{a(h)n } is a solutionof
an = c1an1 + c2an2 + + ckank.
(Contd on Next Page)
Yen (NTUEE) Discrete Mathematics 2012 16 / 38
S l i N h Li R R l i
http://goforward/http://find/http://goback/ -
7/28/2019 Homogeneous Recurrence Relation
17/30
Solving Nonhomogeneous Linear Recurrence Relations
Proof.
Since {a(p)n } is a particular solution, we have
a(p)n = c1a
(p)n1 + c2a
(p)n2 + + cka(p)nk + F(n).
Suppose{
bn}
be any solution of the nonhomogeneous recurrence relationsuch that
bn = c1bn1 + c2bn2 + + ckbnk + F(n).Then
bn a(p)n = c1(bn1 a(p)n1) + c2(bn2 a(p)n2) + + ck(bnk a(p)nk).
That is, {a(p)n bn} is a solution of the associated homogeneousrecurrence relation. Thus, bn = a
(p)n + a
(h)n .
Yen (NTUEE) Discrete Mathematics 2012 17 / 38
A E l
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
18/30
An Example
Example
Find all solutions of an = 3an1 + 2n with a1 = 3.
Solution: Let us guess a(p)n = cn + d. Then
cn + d = 3(c(n 1) + d) + 2n. We have cn + d = (3c+ 2)n + (3d 3c).Solve c = 3c + 2 and d = 3d 3c. We obtain c = 1 and d =
3
2 .Therefore a
(p)n = n 32 is a particular solution.
By Theorem 9, we have a(h)n = 3n as solutions to the associated
homogeneous recurrence relation. Therefore all solutions are of the forman =
n
3
2
+
3n where is a constant.Finally, 3 = a1 = 1 32 + 3. = 116 . Therefore
an = n 32
+11
63n.
Yen (NTUEE) Discrete Mathematics 2012 18 / 38
A E l
http://goforward/http://find/http://goback/ -
7/28/2019 Homogeneous Recurrence Relation
19/30
An Example
ExampleFind solutions of an = 5an1 6an2 + 7n.
Solution: Let us guess a(p)n = c 7n. Then
c
7n = 5c
7n1
6c
7n2 + 7n. Hence 49c = 35c
6c + 49, c = 4920 .
a(p)n =4920 7n is a particular solution.
By Theorem 9, we have a(h)n = 12
n + 23n as solutions of the associated
homogeneous recurrence relation. Therefore
an = 12n + 23n + 4920 7n
are all solutions.
Yen (NTUEE) Discrete Mathematics 2012 19 / 38
S l i P ti l S l ti
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
20/30
Solving Particular Solutions
Theorem
Suppose{
an}
satisfies an
= c1
an1
+ c2
an2
+
+ ck
ank
+ F(n), wherec1, c2, . . . , ck R and
F(n) = (btnt + bt1nt1 + + b1n + b0)sn,
where s, b0, b1, . . . , btR.
When s is not a root of the characteristic equation of the associated linear
homogeneous recurrence relation, there is a particular solution of the form
(ptnt + pt1nt1 + + p1n + p0)sn.
When s is a root of the characteristic equation and its multiplicity is m,
there is a particular solution of the form
nm(ptnt + pt1nt1 + + p1n + p0)sn.
Yen (NTUEE) Discrete Mathematics 2012 20 / 38
S l i N h Li R R l ti
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
21/30
Solving Nonhomogeneous Linear Recurrence Relations
Example
Solve an = an1 + 3n2
3n + 1. with a0 = 0.Solution: Since an = an1 + 3n2 3n + 1, F(n) = (3n2 3n + 1)1n.By Theorem 9, we have a
(h)n = 1n = as homogeneous solutions.
By Theorem 15, we let a(p)n = n(p2n
2 + p1n + p0) as a particular solution.
Substitute a(p)n in the recurrence relation, we have
n(p2n2 + p1n + p0) = (n 1)(p2(n 1)2 + p1(n 1) + p0) + 3n2 3n + 1.
Simplify both sides of the equation and compare their coefficients:
p2 = p2
p1 = 3p2 + p1 + 3p0 = 3p2 2p1 + p0 3
0 = p2 + p1 p0 + 1
Yen (NTUEE) Discrete Mathematics 2012 21 / 38
Solving Nonhomogeneous Linear Recurrence Relations
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
22/30
Solving Nonhomogeneous Linear Recurrence Relations
(Contd)Solve the linear system and get p0 = 0, p1 = 0, p2 = 1. Hence
a(p)n = n(1 n2 + 0 n + 0) = n3 is a particular solution.
Thus, an = n3 + are all solutions. Since a0 = 0, we have = 0. And we
conclude an = n3.
Notice that n3 (n 1)3 = 3n2 3n + 1. Hencenk=1 3k
2 3k + 1 = n3. Alternatively, note that
an =
nk=1 3k
2
3k + 1,
you can solve it by computing the summation.
Yen (NTUEE) Discrete Mathematics 2012 22 / 38
Solving Recurrence Relations by Domain Transformation
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
23/30
Solving Recurrence Relations by Domain Transformation
Example
T(n) = 2T(n/2) + nlog2n, n > 1
where n is a power of 2.
Solutions: Replace n by 2k, we have
T(2k) = 2T(2k1) + k2k
Let tk = T(2k). The above can be rewritten as
tk = 2tk
1 + k2k
Hence,tk = c12
k + c2k2k + c3k22k
T(n) = c1n + c2nlogn + c3nlog2n
Yen (NTUEE) Discrete Mathematics 2012 23 / 38
Solving Recurrence Relations by Range Transformation
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
24/30
Solving Recurrence Relations by Range Transformation
Example
an = 3a2n1 subject to a0 = 1
Solution: Let bn = log an. Then the above can be rewritten as
bn = 2bn1 + log 3 subject to b0 = 0.
Hencebm = (2
n
1)log 3
an = 2(2n1)log3 = 32
n1
Yen (NTUEE) Discrete Mathematics 2012 24 / 38
Divide and Conquer Algorithms and Recurrence Relations
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
25/30
Divide-and-Conquer Algorithms and Recurrence Relations
Recall the merge sort algorithm. The algorithm divides the sequence inhalves, solves each half recursively, and then merges sorted results to getthe solution. This type of algorithms are called divide-and-conqueralgorithms. In the complexity analysis of divide-and-conquer algorithms,
we obtain the recurrence relation of the form
f(n) = af(n
b) + g(n),
where f(n) is the number of steps needed in solving an instance of size n.
We will prove a theorem to help us solve the recurrence relation.
Yen (NTUEE) Discrete Mathematics 2012 25 / 38
Divide-and-Conquer Algorithms and Recurrence Relations
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
26/30
Divide-and-Conquer Algorithms and Recurrence Relations
Theorem
Let f be a non-decreasing function satisfying
f(n) = af(n
b) + c
whenever b
|n, where a
1, b
Z
+, b> 1, and cR
+. Then
f(n) =
O(nlogba) if a > 1O(lg n) if a = 1
Furthermore, when n = bk and a > 1, kZ
+,
f(n) = C1nlogba + C2,
where C1 = c/(a 1) + f(1) and C2 = c/(a 1).
Yen (NTUEE) Discrete Mathematics 2012 26 / 38
Divide-and-Conquer Algorithms and Recurrence Relations
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
27/30
Divide-and-Conquer Algorithms and Recurrence Relations
Proof: We first consider n = bk. Then
f(n) = af(bk
b ) + c
= af(bk1) + c= a(af(bk2) + c) + c= a2f(bk2) + ac + c= = akf(1) + ak1c + ak2c + + ac + c
= akf(1) + ck1
i=0
ai
When a = 1, we have f(n) = f(1) + ck = f(1) + clogbn. Furthermore, ifbk < n < bk+1, we have f(bk) f(n) f(bk+1) and bk+1 < nb. Thus,f(n) f(bk+1) = f(1)+c(k+1) < f(1)+c(1+logbn) = f(1)+c+clogbn.
Therefore f(n) = O(lg n) when a = 1.Yen (NTUEE) Discrete Mathematics 2012 27 / 38
Divide-and-Conquer Algorithms and Recurrence Relations
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
28/30
Divide and Conquer Algorithms and Recurrence Relations
(Contd)When a > 1 and n = bk, then nlogba = bklogba = ak. We have
f(n) = akf(1) + cak 1a 1
= ak
f(1) +c
a
1
c
a
1
= nlogbaC1 + C2
For bk < n < bk+1, we have
f(n) f(bk+1)= C1ak+1 + C2
= aC1ak + C2
aC1nlogba + C2
Hence, f(n) = O(n
logba
).Yen (NTUEE) Discrete Mathematics 2012 28 / 38
An Example
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
29/30
An Example
Example
Let f(n) = 5f(n/2) + 3. Estimate f(n).
Solution: By Theorem 19, we have f(n) = O(n
log2 5
) = O(n
lg 5
) sincea = 5 and b = 2.
For merge sort algorithm, we have T(n) = 2T(n/2) + (n). Theorem 19is not applicable. The following theorem will be proved in your algorithm
class:
Yen (NTUEE) Discrete Mathematics 2012 29 / 38
Divide-and-Conquer Algorithms and Recurrence Relations
http://find/ -
7/28/2019 Homogeneous Recurrence Relation
30/30
Divide and Conquer Algorithms and Recurrence Relations
Theorem
Let f be a non-decreasing function satisfying
f(n) = af(n
b) + cnd
whenever n = bk, k Z+, b Z+ with a 1, b> 1, and c, d R withc> 0, d 0. Then
f(n) =
O(nd) if a < bd
O(nd lg n) if a = bd
O(nlogba) if a > bd
Therefore, the complexity of merge sort algorithm is O(n lg n).
Yen (NTUEE) Discrete Mathematics 2012 30 / 38
http://find/