COMP250:Mid-termExamReview

Lecture19JérômeWaldispühlSchoolofComputerScience

McGillUniversity

Review-Recursivealgorithms

•  Towritearecursivealgorithm:–  Findhowtheproblemcanbebrokenupinoneormoresmallerproblemsofthesamenature

–  Rememberthebasecase!

•  Usually,beUerrunningVmesareobtainedwhenthesizeofthesubproblemsareapproximatelyequal–  power(a,n)=a*power(a,n-1)⇒O(n)

–  power(a,n)=(power(a,n/2))2⇒O(logn)

•  Fibonnaci,BinarySearch,MergeSort…

MergeSort

4 3 2 1

4 3 2 1

4 3 2 1

3 4 1 2

1 2 3 4

Divide

Merge

Recursivealgorithms&inducVons

•  YoucanseeanalgorithmasafuncVon.

•  Toprovethecorrectnessofarecursivealgorithm,wecanuseasimilartechniquetotheoneusedtoprovemathemaVcalinducVons.

•  Butweshouldalsoprovethatouralgorithmterminates!

InducVonproofs

•  ToprovethataproposiVonP(n)holdsforalln≥a:–  Basecase:ProvethatP(a)holds

–  Induc0onsteponn:InducVonHypothesis:AssumeP(n)holdsProvethatI.H.impliesthatP(n+1)holds

GeneralizedinducVonproofs

ToprovethataproposiVonP(n)holdsforalln≥a:•  Basecase:ProvethatP(a),P(a+1)...(asmanyasneeded)hold

•  Induc0onsteponn:Induc&onHypothesis:AssumeP(k)holdsforallk≤nProvethatinducVonhypothesisimpliesthatP(n+1)holds

InducVonproofs:ExampleClaim:forallposiVveintegersk,n,wehave(1+k)n≥1+kn.

Proof:•  Basecase:n=1,(1+k)1≥1+k•  InducVonhypothesis:forallkandagivenn,wehavewe

have(1+k)n≥1+kn•  InducVonstep(showthatthenitistrueforn+1):(1+k)n+1=(1+k)(1+k)n≥(1+k)(1+kn)=1+kn+k+k2n=1+k(n+1)+(1+k)≥1+k(n+1)

Provingalgorithms

AF(int x) { f = 2; while ( x > 1 ) { if ( x % f == 0 ) { System.out.println(f); x = x / f; } else { f++; } }}

IteraVvealgorithm⇒ Loopinvariant

Recursivealgorithm⇒ InducVonproof

FM(int[] A, int l, int r) { if (l<r) { m=⎣(l+r)/2⎦ return max( FM(A,l,m), FM(A,m+1,r)); } else { return A[l];}

ProvingalgorithmsFM(int[] A, int l, int r) { if (l<r) { m=⎣(l+r)/2⎦ return max(FM(A,l,m),FM(A,m+1,r)); } else { return A[l]; }

•  Pre-condiVon:0≤l≤r≤A.length•  Post-condiVon:returnmaxvaluestoredinA[l:r]•  Basecase:themaxofanarrayofsize1isthevaluestored.•  InducVonHypothesis:foralllandrsuchthat|r-l|≤n,FM(A,l,r)

returnsthemaxinA[l:r]•  InducVonstep:forall|r-l|=n+1,then

FM(A,l,r)=max(FM(A,l,m),FM(A,m+1,r))

ProvingalgorithmsFM(int[] A, int l, int r) { if (l<r) { m=⎣(l+r)/2⎦ return max(FM(A,l,m),FM(A,m+1,r)); } else { return A[l]; }

•  InducVonHypothesis:foralllandrsuchthat|r-l|≤n,FM(A,l,r)returnsthemaxinA[l:r]

•  InducVonstep:Assume|r-l|=n+1.FM(A,l,r)=max(FM(A,l,m),FM(A,m+1,r))(I.H.)FM(A,l,m)&FM(A,m+1,r)returnmaxofA[l,m]&A[m+1,r]SincemaxofA[l,r]isinA[l,m]orA[m+1,r],thenFM(A,l,r)returnsmaxofA[l:r]•  TerminaVon:Ateveryrecursivecall,|r-l|decreasesstrictlyunVl

|r-l|=0,whichisthebasecase.

LoopInvariantsAF(int x) { f = 2; while ( x > 1 ) { if ( x % f == 0 ) { System.out.println(f); x = x / f; } else { f++; } }}

Invariant:x=thenumberthatremainstobefactorandwehaveremovedallfactors<f

Loopinvariant

•  Ini0aliza0on:xcontainsallfactors•  Maintenance:Atthebeginningoftheloopweremoved

allfactors<f.AmeroneiteraVonoftheloop,iffisafactorofx,thenweremoveitanddonotincreasef.Otherwise,wecansafelyincrementfandxissuchthatweremovedallfactors≤f.

•  Termina0on:Thenumberthatremainstobefactored=1andwehaveremovedallfactorsincludingthelargestfactor(I.e.:wehavefoundallthefactors)

Discussion•  DifficultyistoidenVfyagoodloopinvariant.Theproofis

usuallysimpler(iniValizaVon,maintenance,terminaVon).

•  Aloopinvariantcanbeanypredicateaslongasithelpstoprovethattheloophelpstosolvetheproblem.

•  Withrecursivealgorithm,idenVfyingthepropertywewanttoproveiseasierbecauseitisthepost-condiVon.

Invariantvs.InducVon

IteraVvealgorithm Recursivealgorithm

Loopinvariant InducVonhypothesis

IniValizaVon Basecase

Maintenance InducVonstep

TerminaVon TerminaVon

⇔

⇔

⇔

⇔

RunningVme

•  PrimiVveoperaVons–  RunningVmeisconstant,indep.ofproblemsize

•  Assigningavaluetoavariable•  Callingamethod;returningfromamethod•  ArithmeVcoperaVons,comparisons•  Indexingintoanarray•  Followingobjectreference•  CondiVonals

•  RunningVme≡NumberofprimiVveoperaVons•  Loops:SumtherunningVmeofeachiteraVon•  findMin,inserVonSort

Recurrences

•  Forrecursivealgorithms,weexpresstherunningVmeT(n)foraninputofsizenasafuncVonofT(a)forsomea<n

•  Example:–  Binarysearch:T(n)=T(n/2)+a– MergeSort:T(n)=2T(n/2)+cn

Solvingrecurrences

•  Solvingrecurrence≡giveexplicitformulaforT(n)•  SubsVtuVonmethod:

–  ReplaceoccurrencesofT()bytheirvalue–  RepeatunVlpaUernemerges

•  ProvebyinducVonthatguessiscorrect

ExampleSoluVonofT(n)=2T(n/2)+cn,T(1)=0forn≥0apowerof2?

T(n)=2(2T(n/4)+cn/2)+cn=22T(n/4)+2cn=22(2T(n/8)+cn/4)+2cn=23T(n/8)+3cn…=2kT(n/2k)+kcn

Wehaven=2k,thusk=log2(n) and we replace k in T(n). T(n)=2log(n)T(1)+log2(n)cn=cnlog(n)

ThenprovebyinducVon...

10

Proof by induction

Proposition. If T (n) satisfies the following recurrence, then T (n) = n log2 n.

Pf 2. [by induction on n]

・Base case: when n = 1, T(1) = 0.

・Inductive hypothesis: assume T(n) = n log2 n.

・Goal: show that T(2n) = 2n log2 (2n).

assuming n

is a power of 20 if n = 1 2 T (n / 2) + n otherwiseT(n) =

T(2n) = 2 T(n) + 2n

= 2 n log2 n + 2n

= 2 n (log2 (2n) – 1) + 2n

= 2 n log2 (2n). ▪

Big-OhnotaVong(n)isO(f(n))iffthereexistconstantscandn0suchthatg(n)≤cf(n)foralln≥n0

•  f(n)isΩ(g(n))iffg(n)isO(f(n))•  f(n)isΘ(g(n))ifff(n)isO(g(n))andf(n)isΩ(g(n))

ExampleShowthatFib(n)isO(2n)

Proof:WewillshowbyinducVonthatforalln≥0,Fib(n)<2n

•  Basecase:n=0,Fib(0)=0<20=1;n=1,Fib(1)=1<21=2;•  InducVonhypothesis:forallk≤ nFib(k)<2k.Fib(n+1)?•  InducVonstep:

Fib(n+1)=Fib(n)+Fib(n-1)<2n+2n-1

<2n+2n=2n+1

UsingthepreviousdefiniVon,wesetc=1andn0=0,thenFib(n)isO(2n).

Example2

•  Previously,weusedtherecursiveformulaT(n)=2T(n/2)+n,withT(1)=0,tocharacterizethenumberofoperaVonsofMergeSort.

•  WeshowedthatT(n)=nlog2(n)

•  Then,MergeSortrunsinO(nlog2(n))

Hierarchy of big-Oh classes

Big-OhnotaVon

•  If and then

•  If then

•  If and then

…

Simplification rules

Big-OhnotaVon

f1(n)∈O(g(n)) f2 (n)∈O(g(n))

f1(n)+ f2 (n)∈O(g(n))

f1(n)∈O(g(n)) k. f1(n)∈O(g(n))

f1(n)∈O(g(n)) f2 (n)∈O(h(n))

f1(n). f2 (n)∈O(g(n).h(n))

LogidenVVes

•  log(ab)=log(a)+log(b)•  log(an)=nlog(a)•  loga(n)=logb(n)/logb(a)•  alogb(n)=nlogb(a)

RunningVmeofaForloop

for (i=1; i<N; i=i*2) { … }

Whatistherunning&meofthisloop?

ValueofiamerkiteraVons:2k

Wehavei<N=>2k<N=>k<log2(N).

Thereislessthanlog2(N)iteraVons.

TherunningVmeofthisloopisO(log(n)).

#operaVons:O(log(n))O(1)=O(log(n))

Somethingrunningin

O(1)

AbstractDataTypes

•  ImplementaVon:Arraysorlinked-lists•  BasicoperaVons

–  getFirst(),get(n),getLast()–  removeFirst(),removeLast(),remove(o)–  addFirst(o),addLast(o),add(o)–  empty(),size()

•  Advantagesanddisadvantagesoverarrays•  Stack,Queues,deques,rotaVngarrays

OperaVonsondequeswithArray

•  Queue() { L = new Array[N]; head = tail = -1;

} •  isEmpty() {

if ((head == -1)&&(tail==-1)) return true; else return false;

} •  isFull() {

if ((head - tail % N) == 1) return true; else return false;

}

OperaVonsondequeswithArray

•  Enqueue(o) throw Exception { if ( isFull() ) { throw new Exception(“Full stack”) } if ( isEmpty() ) { head = tail = 0; } else { tail = ( ( tail + 1 ) % N ); } L[tail] = o;

} •  Dequeue() {

if ( isEmpty() ) { throw new Exception(“Empty stack”) } Object o = L[head]; if (((head – tail) % N) = 1) { head = tail = -1; } else { head = ( ( head + 1 ) % N ); } return o;

}