Prof. Busch - LSU1 Undecidable Problems (unsolvable problems)
Advanced Counting Techniques CSC-2259 Discrete Structures Konstantin Busch - LSU1.
-
Upload
arline-ball -
Category
Documents
-
view
233 -
download
1
Transcript of Advanced Counting Techniques CSC-2259 Discrete Structures Konstantin Busch - LSU1.
Advanced Counting Techniques
CSC-2259 Discrete Structures
Konstantin Busch - LSU 1
Recurrence Relations
Konstantin Busch - LSU 2
nn aaaa ,,,:}{ 10 Sequence
),,,( 110 nn aaafa Recurrence relation:
For any 0nn
Konstantin Busch - LSU 3
Example:
212 nnn aaa 2n
nan 3Solutions to recurrence relation:
5na
Recurrence relation
Konstantin Busch - LSU 4
$10,000 bank deposit
%11 interest
nP :amount after yearsn
111 11.111.0 nnnn PPPP
000,100 P
022
1 )11.1()11.1(11.1 PPPP nnnn
97.922,228$000,10)11.1( 3030 P
Example:
Konstantin Busch - LSU 5
Fibonacci sequence
21 nnn fff
1,0 10 ff
Example:
Konstantin Busch - LSU 6
Towers of HanoiExample:
bar1 bar2 bar3
Goal: move all discs to bar3
Rule: not allowed to put larger discs on top of smaller discs
ndiscs
Konstantin Busch - LSU 7
bar1 bar2 bar3
1n
move recursively discs to bar21nStep 1:
Konstantin Busch - LSU 8
bar1 bar2 bar3
1n
move largest disc to bar3Step 2:
Konstantin Busch - LSU 9
bar1 bar2 bar3
move recursively discs to bar31nStep 3:
n
Konstantin Busch - LSU 10
11 H
12 1 nn HH
nH :total disc moves
2 recursive callswith discs(steps 1&3)
1nmovement oflargest disc(step 2)
one move for one disc
Konstantin Busch - LSU 11
11 H
12
1222
1222
122212122
1221)12(2
12
21
21
1
23
33
2
22
2
1
n
nn-
nn-
nn
nn
nn
H
H)H(
HH
HH
12 1 nn HH
Solving Linear Recurrence Relations
Konstantin Busch - LSU 12
Linear homogeneous recurrence relationof degree :
knknnn acacaca 2211
0kc
k
Rci Constant coefficients:
Konstantin Busch - LSU 13
A sequence (solution) satisfying the relationis uniquely determined by the initial values:
11
11
00
kk Ca
Ca
Ca
k
(these are different constants than the coefficients)
Konstantin Busch - LSU 14
nn ra
if an only if
knknnn acacaca 2211
knk
nnn rcrcrcr 22
11
Solution to recurrence relation:
012
21
1
kkkkk crcrcrcr
divide both sides withknr
characteristic equation
Konstantin Busch - LSU 15
012
21
1
kkkkk crcrcrcr
characteristic equation:
factorize with roots
0)())(( 21 krrrrrr
characteristic roots: krrr ,,, 21
nn ra 1
Multiple possible solutions:n
n ra 2 nkn ra
Konstantin Busch - LSU 16
nn ra 1
The solutionsn
n ra 2 nkn ra
may not satisfy the initial conditions
11
11
00
kk Ca
Ca
Ca
Konstantin Busch - LSU 17
Theorem: Recurrence relation of degree 2
2211 nnn acaca
has unique solutionnn
n rra 2211
where are solutions to the characteristic equation,and are constants thatdepend on initial conditions
21, rr
21,
Konstantin Busch - LSU 18
0212 crcr
Characteristic Equation
21, rrRoots:
02112
1 crcr 2112
1 crcr
02212
2 crcr 2212
2 crcr
Proof:
Konstantin Busch - LSU 19
2211122
1111 rrrra
First compute from initial conditions
210
220
110 rra
21,10 ,aa
Konstantin Busch - LSU 20
22111 rra
210 a 201 a
221201 )( rraa
21
1102 rr
ara
21
201201 rr
raaa
Konstantin Busch - LSU 21
Prove by induction that nnn rra 2211
Basis cases: 210 a
22111 rra
true for the specific choices of 21,
Konstantin Busch - LSU 22
Inductive hypothesis:kk
k rra 2211
for all nk 0
kkk rra 2211
Inductive step:
for nk
prove that
assume that
Konstantin Busch - LSU 23
By inductive hypothesis:
122
1111
nnn rra
222
2112
nnn rra
Konstantin Busch - LSU 24
)()(
)()(
2212
222112
11
222
2112
122
1111
2211
crcrcrcr
rrcrrc
acaca
nn
nnnn
nnn
By recurrence relation definition
Inductive hypothesis
2112
1 crcr 2212
2 crcr
nn
nnn
rr
rrrra
2211
22
222
21
211 )()(
End of Proof
Konstantin Busch - LSU 25
Fibonacci sequence
21 nnn fff 1,0 10 ff
Example:
nnn rrf 2211 Has solution:
2
511
r
2
512
rCharacteristic roots:
Konstantin Busch - LSU 26
5
1
21
1102
rr
frf
5
1
21
2011
rr
rff
Konstantin Busch - LSU 27
nn
nnn rrf
2
51
5
1
2
51
5
1
2211
Konstantin Busch - LSU 28
Degree recurrence relation:
has unique solution
nkk
nnn rrra 2211
where are solutions to the characteristic equation,and are constants thatdepend on initial conditions
krrr ,,, 21
k ,,, 21
knknnn acacaca 2211
k
Konstantin Busch - LSU 29
Example: 321 6116 nnnn aaaa
15,5,2 210 aaa
Solution:
Characteristic equation: 06116 23 rrr
Roots: 3,2,1 321 rrr
nnnnnnn rrra 321 321332211
Konstantin Busch - LSU 30
nnnna 321 321
23
22
212
13
12
111
03
02
010
32115
3215
3212
a
a
a
2,1,1 321
nnnnnna 3221322111
Final solution:
Recurrence Relations for Divide and Conquer Algorithms
Konstantin Busch - LSU 31
Typical divide and conquer algorithm:
•Input of size n
•Divide into sub-problems each of size bn /
a
)(ng•Combine sub-problems with cost
Konstantin Busch - LSU 32
)()/()( ngbnfanf
Divide an conquer recurrence relation:
Cost of subproblemof size bn /
Konstantin Busch - LSU 33
Examples:
Binary search: 2)2/()( nfnf
Merge Sort: nnfnf )2/(2)(
Fast Matrix Multiplication (Stassen’s Alg.):
4/15)2/(7)( 2nnfnf
Konstantin Busch - LSU 34
)()/()( ngbnfanf
Zkbnkbbn k ,,,0,1,
1
0
)/()1()(k
j
jjk bngafanf
Theorem: if
then
Konstantin Busch - LSU 35
Proof:
)()/()/(
)())/()/((
)()/()(
22
2
ngbngabnfa
ngbngbnfaa
ngbnfanf
Konstantin Busch - LSU 36
)()/()/()/(
)()/())/()/((
)()/()/(
)())/()/((
)()/()(
2233
232
22
2
ngbngabngabnfa
ngbngabngbnfaa
ngbngabnfa
ngbngbnfaa
ngbnfanf
Konstantin Busch - LSU 37
1
0
1
0
2233
232
22
2
)/()1(
)/()/(
)()/()/()/(
)()/())/()/((
)()/()/(
)())/()/((
)()/()(
k
j
jjk
k
j
jjkk
bngafa
bngabnfa
ngbngabngabnfa
ngbngabngbnfaa
ngbngabnfa
ngbngbnfaa
ngbnfanf
End of Proof
Konstantin Busch - LSU 38
cbnfanf )/()(Theorem: if
Rcaca
Zkbnkbbn k
,,0,1
,,,0,1,
)(nf)( log abnO
)(log nO b
1a
1athen
Konstantin Busch - LSU 39
Proof:
1
0
1
0
)1(
)1(
)/()(
k
j
jk
k
j
jk
acfa
cafa
cbnfanf
From previous theorem
Konstantin Busch - LSU 40
)(log
log)1(
)1(
)1()(1
0
nO
ncfa
ckfa
acfanf
b
bk
k
k
j
jk
1aCase:
kbn nk blog
Konstantin Busch - LSU 41
)(
11)1(
1
1)1(
)1()(
log
2log
1
1
0
a
a
k
kk
k
j
jk
b
b
nO
CnC
a
c
a
cfa
a
acfa
acfanf
1aCase:
End of Proof
Konstantin Busch - LSU 42
Example:
Binary search: 2)2/()( nfnf
2,2,1 cba
)(log)(log)(log)( 2 nOnOnOnf b
Konstantin Busch - LSU 43
dcnbnfanf )/()(Master Theorem:
if
Rdcadca
Zkbnkbbn k
,,,0,0,1
,,,1,1,
)(nf )log( nnO d
)( dnO
dba
dba
then
)( log abnO dba
Konstantin Busch - LSU 44
Example:
Merge Sort: nnfnf )2/(2)(
1,1,2,2 dcba
dba 122
)log()log()( nnOnnOnf d
Generating Functions
Konstantin Busch - LSU 45
17321 eee
Find number of solutions for:
3
20
3
1317
0,, 321 eee
Answer:
Konstantin Busch - LSU 46
Alternative solution
17321 eee
choices for 1e choices for 2e choices for 3e)1)(1)(1( 173217321732 xxxxxxxxxxxx
Konstantin Busch - LSU 47
Alternative solution
17321 eee
5151
1717
23
2210 xaxaxaxaxaa
17a is the total numberof solutions to equation
)1)(1)(1( 173217321732 xxxxxxxxxxxx
Konstantin Busch - LSU 48
Another problem: 17321 eee
52 1 e
63 2 e
Find total number of solutions which satisfy:
74 3 e
Konstantin Busch - LSU 49
Alternative solution
17321 eee
choices for 1e choices for 2e choices for 3e
))()(( 765465435432 xxxxxxxxxxxx
52 1 e 63 2 e 74 3 e
Konstantin Busch - LSU 50
Alternative solution
17321 eee
))()(( 765465435432 xxxxxxxxxxxx
52 1 e 63 2 e 74 3 e
1818
1717
1010
99 xaxaxaxa
17a is the total numberof solutions to equation
Konstantin Busch - LSU 51
Generating function:
0
33
2210
)(
k
kk
kk
xa
xaxaxaxaaxG
generating function for sequence
,,,,,, 3210 kaaaaa
Konstantin Busch - LSU 52
Solve recurrence relation
13 kk aa 20 a
Generating functions can also be used tosolve recurrence relations
Example:
Konstantin Busch - LSU 53
13 kk aa
Let be the generating function for sequence ,,,,, 3210 kaaaaa
)(xG
20 a
0
)(k
kk xaxG
Konstantin Busch - LSU 54
11
0
1
0
)(
k
kk
k
kk
k
kk
xa
xa
xaxxGx
Konstantin Busch - LSU 55
2
)3(
)3(
3
3)(3)(
0
110
110
11
10
11
0
a
xaaa
xaxaa
xaxaa
xaxaxxGxG
k
kkk
k
kk
kk
k
kk
k
kk
k
kk
k
kk
13 kk aa 20 a
0
Konstantin Busch - LSU 56
2)(3)( xxGxG
xxG
31
2)(
Konstantin Busch - LSU 57
xk
k
xx
1
1
0
xxG
31
12)(
0
32)(k
kk xxG
0
32)(k
kk xxG
Konstantin Busch - LSU 58
0
32)(k
kk xxG
0
)(k
kk xaxG
kka 32
13 kk aa 20 a
Solution to recurrence relation
Inclusion-Exclusion
Konstantin Busch - LSU 59
A BBA
|||||||| BABABA
Konstantin Busch - LSU 60
ABA
||
||||||
||||||||
CBA
CBCABA
CBACBA
B
C
CA CB
CBA
Konstantin Busch - LSU 61
||)1(
||
||
||||
211
1
1
121
nn
nkjikji
njiji
niin
AAA
AAA
AA
AAAA
Principle of Inclusion-Exclusion:
Konstantin Busch - LSU 62
Proof:
We want to prove that:
an arbitrary element is counted exactly one time in the expression of the theorem
x
Konstantin Busch - LSU 63
Suppose is a member of exactly sets: rx
riii AAAx 21
rjjjjj
r
nkjikji
njiji
nii
r
rAAA
AAA
AA
A
1
211
1
1
1
1
||)1(
||
||
||
Then is counted in the terms:x
is counted times
Konstantin Busch - LSU 64
ni
iA1
||
)1,(rCr
In sum:
x
(since belongs exactly to sets)x r
is counted times
Konstantin Busch - LSU 65
)2,(rC
In sum:
x
(since belongs exactly to sets)x r
nji
ji AA1
||
is counted times
Konstantin Busch - LSU 66
)3,(rC
In sum:
x
(since belongs exactly to sets)x r
nkji
kji AAA1
||
is counted times
Konstantin Busch - LSU 67
),(1 rrC
In sum:
x
(since belongs exactly to sets)x r
rjj
jjj
r
rAAA
1
211
||
Konstantin Busch - LSU 68
),()1()3,()2,()1,( 1 rrCrCrCrC r
Thus, in the expression of the theorem is counted so many times:x
ni
iA1
||
nji
ji AA1
||
nkji
kji AAA1
||
rjj
jjj
r
rAAA
1
211
||
Konstantin Busch - LSU 69
),()1()2,()1,()0,()11(0 rrCrCrCrC rr
),()1()2,()1,()0,(1 1 rrCrCrCrC r
End of Proof
From binomial expansion we have that:
Thus, is counted exactly one timex
Konstantin Busch - LSU 70
Example: Find the number of primes between 1…100
If a number is composite and between1…100 then it must be divided by a primewhich is at most :10010
2, 3, 5, 7
Konstantin Busch - LSU 71
:the set of primes between 1…100P
2N :composites between 1…100 divided by 2
3N :composites between 1…100 divided by 3
:the set of composites between 1…100N
5N :composites between 1…100 divided by 5
7N :composites between 1…100 divided by 7
Konstantin Busch - LSU 72
From the principle of inclusion-exclusion:
||
||||||||
||||||||||||
||||||||||
7532
753752732532
757353725232
5532
NNNN
NNNNNNNNNNNN
NNNNNNNNNNNN
NNNNN
Konstantin Busch - LSU 73
78
001232467101614203350
7532
100
753
100
752
100
732
100
532
100
75
100
73
100
53
100
72
100
52
100
32
100
7
100
5
100
3
100
2
100||
N
||
||||||||
||||||||||||
||||||||||
7532
753752732532
757353725232
5532
NNNN
NNNNNNNNNNNN
NNNNNNNNNNNN
NNNNN
Konstantin Busch - LSU 74
Number of primes between 1…100:
21
7899
||99||
NP