Introduction to Computer ScienceCSCI109

AndrewGoodneyFall2019

China– Tianhe-2

ReadingsSt.Amant,1-4,8

Lecture6:FirstHalfReview October6th,2019

Where are we?

1

Review

u Lasttimewegotalittleahead

u Sowe’llreviewthefirsthalfofthesemester

2

Lecture #1

3

Lecture #1

4

Computational Thinking

u “thoughtprocessesinvolvedinformulatingproblemsandtheirsolutionssothatthesolutionsarerepresentedinaformthatcanbeeffectivelycarriedoutbyaninformation-processingagent” (Cuny,Snyder,Wing)

v wayofsolvingproblems,designingsystems,andunderstandinghumanbehaviorthatdrawsonconceptsfundamentaltocomputerscienceu Toflourishintoday'sworld,computationalthinkinghastobeafundamentalpart

ofthewaypeoplethinkandunderstandtheworldv creatingandmakinguseofdifferentlevelsofabstraction,tounderstandand

solveproblemsmoreeffectivelyv thinkingalgorithmically andwiththeabilitytoapplymathematicalconcepts

suchasinduction todevelopmoreefficient,fair,andsecuresolutionsv understandingtheconsequencesofscale,notonlyforreasonsofefficiency

butalsoforeconomicandsocialreasons

5

Humansthinking(i.e.,transforminginformation)todeviseproceduresforexecutionbyinformationtransformers(humanand/ormachine)

Before Mechanical Computers

Electroniccomputerswereprecededbymechanicalcomputersandmechanicalcomputerswereprecededby…

…looms

6

Discrete Machines: State

uHowdoestheloombehaveasafunctionoftime?u Atanygiventimeasetofthreadsisraisedandtherestarelowered

uWritingdownthesequenceofraised(andlowered)threadstellsusthestepsthemachinewentthroughtoproducethecloth/tapestry/whatever

u Thepatternofraised(andlowered)threadsiscalledthestate ofthemachine

7

CS Topic: State

u StateisaverycommonCSconcept

u Herewehavethestateofaphysicalmachine

u InCSwetalkaboutthe“state”ofanobjectv Ofadatabasev Ofarobotv Ofa“state-machine”(finite,Turing,etc…)v Ofasystem(physicalorvirtual)v …

u Thenweneedawaytodescribethestatev Givesusthenotionofanencoding

8

CS Topic: Discrete Machines, State and Encoding

u Choosingastaterepresentationtakesskill.Thestateshouldbev Parsimonious:itshouldbea“small”descriptorofwhatthemachineis

doingatanygiventimev Adequate:itshouldbe“bigenough”tocaptureeverything“interesting”

aboutthemachine

u Thesearesometimescontradictory.Theyarealsoqualitativeanddependonwhatbehaviorofthemachinewewanttodescribe

uUsuallyyouneedavocabulary(encoding)todescribestate.Inthecaseofaloom,statecanbeexpressedasabinarypattern(1forraised,0forlowered)

9

Discrete Machines: Abstraction

u Theloomisadiscretemachinev Stateisbinarypattern– i.e.discretev Thenotionoftimeisdiscrete– i.e.timeismodeledasproceedinginstepsor

finitechunks

uMoreprecisely,theloomcanbeusefullymodeledasadiscretemachinev Becauseofcoursebeingaphysicaldevicethereisvariation,nothingis

exactlyprecisev Butmodelingthemachineasdiscreetisgoodenoughandworksforthis

purpose

u Thisisanexampleofanabstraction – akeyconceptinComputerScience 10

CS topic: Abstraction

u Oneofthefundamental“things”wedoinCS

u Reducingordistillingaproblemorconcepttotheessentialqualitiesv Simplesetofcharacteristicsthataremostrelevanttotheproblem

u Many(most,all)ofwhatwedoinengineeringandcomputerscienceinvolvesabstractions

u Heretheabstractionismodellingtheloomasasimplediscreetstatemachinev Makesitpossibletounderstandv Andmakesitpossibleto“program”theloom

11

Lecture #1

u Whatmakesacomputer?v Lotsofthingscanhelpuscompute(informationtransformation)v Computersneed

u Memoryu Control-flow

u State

u Abstraction

12

Lecture #2

13

Motivation

u Whatdocomputersdo?v Mathwithbinarynumbers

u Sowhatdoweneedtobuildacomputer?v Placetostorebinarynumbersv Waytodomath

14

Arithmetic/Logic

u “math”weneedtodowithnumbersinmemoryv ADDv SUBTRACTv MULTIPLYv DIVIDEv AND,OR,XOR,NOTv Etc…

u Assumewecanbuildacircuitthatcandothisu Takesnumbersrepresentedasdigital(electrical)values,producesresultsasthesame

15

OP2

Start building a circuit…

16

bus

Instructing the CPU

u Nowwecanmakeinstructions…

u Instructionsarebinarynumbersthattellthecircuitwhattodo

u Selectthe1st operand,2nd operand,destinationandfunction

u Withaseriesofsuchinstructionsthecircuitcanperformarbitrarycomputations

17

Where to get the instructions?

u InstructionMemory

18Controller

ALU

How to compute?

u Fillinstructionmemorywithdesiredprogram

u Initializedatamemory

u Runaninstruction(givenbyprogramcounter)v Thenincrementprogramcounterv Runnextinstruction,incrementprogramcounter…

u Someearlycomputerswereprettymuchjustthis

19

The Central Processing Unit (CPU)

20

u Controller+ALU=CentralProcessingUnit(CPU)u CPUhasasmallamountoftemporarymemorywithinit

v Registersv Aspecialregistercalledtheprogramcounter(PC)

u CPUperformsthefollowingcyclerepeatedly

FetchInstruction

DecodeInstruction

ExecuteInstruction

Fetch-Decode-Excecute

u Fetchv Getthenextinstructionfrommemory

u Decodev SendthepropersignalsfromthecontrollertotheALUandRegisters

u Executev LettheALUdoitsworktoproducearesult

21

The Storage Hierarchy

22

Cheaper&larger

FasterRegisters

RAM(memory)

SecondaryStorage(DiskSpace)

Trade-offs

u Anaside…

u Identifyingtrade-offsisafundamentalengineeringskill

u Understandingandbalancingtrade-offsispartofdesignprocess

u Notalwayseasytomanage!

u Conflictinginterests

u Speedvs.spaceisverycommontradeoffinCSv Soifyouwantfasterexecutionyouneedmorememory

23

Cache

u Small(butbiggerthanregisters)u Volatileu Fast(notasfastasregisters,butfasterthanRAM)uWhattokeepinthecache?

v Thingsthatprogramsarelikelytoneedinthefuturev Localityprinciple:

u Lookatwhatitemsinmemoryarebeingusedu Keepitemsfromnearbylocations(spatiallocality)u Keepitemsthatwererecentlyused(temporallocality)

24

Modern Computer Architecture diagram

25

Controller

ALU

I/O Devices

(USB, etc)

CPU

Registers &

Program Counter

Memory

DRAM

Disk

L2

Cache

L3

Cache

Boot ROM

I/O Controller

Disk controller

On die, but not

part of "CPU"

Programming a CPU

u Howtocompute?

u Developaseriesoflow-levelinstructionsv Usingtheregistersand/ormainmemoryforstoragev Usingonlylow-leveloperationsmadeavailablebytheparticularCPU

u ”Assemblylanguage”v Ormaybeevenmachinecode(probablynot,though)

26

Typical Operationsu ADDRi Rj Rk AddcontentsofregistersRi andRj andputresultinregisterRku SUBTRACTRi Rj Rk SubtractregisterRj fromregisterRi andputresultinregisterRku ANDRi Rj Rk BitwiseANDcontentsofregistersRi,Rj andputresultinregisterRku NOTRi BitwiseNOTthecontentsofregisterRiu ORRi Rj Rk BitwiseORthecontentsofregistersRi,Rj andputresultinregisterRku SETRi value SetregisterRi togivenvalueu SHIFT-LEFTRi ShiftbitsofregisterRi leftu SHIFT-RIGHTRi ShiftbitsofregisterRi rightu MOVERi Rj CopycontentsfromregisterRi toregisterRju LOADMi Ri CopycontentsofmemorylocationMi toregisterRiu WRITERi Mi CopycontentsofregisterRi tomemorylocationMiu GOTOMi JumptoinstructionstoredinmemorylocationMiu COND_GOTORi Rj Mi IfRi >Rj,jumptoinstructionstoredinmemorylocationMi

27

Lecture #2 Summary

u Computersdotwothings:v BinaryMathv Movedata

u Sowebuildastatemachine(CPU):v Controller,Registers,ALUv Fetch-Decode-ExecuteCycle

u MemoryHierarchyandCaching

u AssemblyLanguageProgramming

28

Lecture #3

29

u “Thearchitecturelevelgivesusaverydetailedviewofwhathappensonacomputer.Buttryingtounderstandeverythingacomputerdoesatthislevelwouldbe…(insertanalogyaboutperspective).Ifallwecanseeisfinedetail,itcanbehardtograspwhat’shappeningonalargerscale.”

30

Problem Solving

u Architectureputsthecomputerunderthemicroscopev Imaginesolving*all*problemsbythinkingaboutthecomputeratthe

architecturelevel

u Earlycomputerscientists*had*todothisv Luckilywedon’t.

31

Problem Solving

u Computersareusedtosolveproblemsu Abstractionforproblems

v Howtorepresentaproblem?v Howtobreakdownaproblemintosmallerparts?v Whatdoesasolutionlooklike?

u Twokeybuildingblocksv Abstractdatatypesv Algorithms

32

Abstract Data Types

uModelsofcollectionsofinformationv Chosentohelpsolveaproblem

u Typicallyatanabstractlevelv Don’tdealwithimplementationdetails:memorylayout,pointers,etc.

“…describeswhatcanbedonewithacollectionofinformation,withoutgoingdowntothelevelofcomputerstorage.”[St.Amant,pp.53]

33

Motivation for Abstract Data Structures

u Thenatureofsomedata,andthewayweneedtoaccessesitoftenrequiressomestructure,ororganizationtomakethingsefficient(orevenpossible)

u Data:largesetofnames(maybeattendancedata)

u Problems:didJelenaattendon9/9?HowmanylecturesdidMarioattend?Whichstudentsdidn’tattend8/26?

34

Sequences, Trees and Graphs

35

u Sequence:alistv Itemsarecalledelementsv Itemnumberiscalledtheindex

uGraph

u TreeEric

Emily Jane

Terry Bob

Jim

Mike Chris

Bob

Sequences aka Lists

u Sequencesareourfirstfundamentaldatastructure

u Sequencesholditemsv Items=whateverweneed.It’sabstract.

u Sequenceshavethenotionoforderv Itemscomeoneafteranother

u Sequencescanbeaccessedbyindex,orrelativev Findthe5th itemv Ormovetonextorpreviousfromcurrentitem

u The“how”(implementation)isnotimportant(now)v Arrays(C,C++),Vectors(C++),ArrayList (Java),Lists(Python)…v Thesearealldifferentimplementationsofthisabstractdatastructure

36

Sequence Tasks

u Most“questions”(problems)thataresolvedusingsequencesareessentiallyoneoftwoquestions:

u IsitemAinsequenceX?

u WhereinsequenceYisitemB?

u Bothoftheseareansweredbysearchingthesequence

37

Sequences: Searching

u Sequentialsearch:startat1,proceedtonextlocation…

u Ifnamesinthelistaresorted (sayinalphabeticalorder),thenhowtoproceed?v Startinthe‘middle’v Decideifthenameyou’relookingforisinthefirsthalforsecondv ‘Zoomin’tothecorrecthalfv Startinthe‘middle’v Decideifthenameyou’relookingforisinthefirsthalforsecondv ‘Zoomin’tothecorrecthalfv …

u Whichismoreefficient(underwhatconditions)?38

bruteforce

divide-and-conquer

Sorting

u Ifsearchingasortedsequenceismoreefficient(persearch),thisimpliesweneedawaytosortasequence!

u SortingalgorithmsarefundamentaltoCSv UsedALOTtoteachvariousCSandprogrammingconcepts

u ComputerScientistslikecomingupwithbettermoreefficientwaystosortdatav Evenhavecontests!

u We’lllookattwoalgorithmswithverydifferentdesignsv SelectionSortv QuickSort

39

Sorting: Selection Sort

40

u Sorting:puttingasetofitemsinorderu Simplestway:selectionsort

v Marchdowntheliststartingatthebeginningandfindthesmallestnumber

v Exchangethesmallestnumberwiththenumberatlocation1v Marchdowntheliststartingatthesecondlocationandfindthesmallestnumber(overallsecond-smallestnumber)

v Exchangethesmallestnumberwiththenumberatlocation2v …

Sorting: Quicksort

41

u Picka‘middle’elementinthesequence(thisiscalledthepivot)

u Putallelementssmallerthanthepivotonitsleft

u Putallelementslargerthanthepivotontheright

u Nowyouhavetwosmaller sortingproblemsbecauseyouhaveanunsortedlisttotheleftofthepivotandanunsortedlisttotherightofthepivot

u Sortthesequenceontheleft(useQuicksort!)v Picka‘middle’elementinthesequence(thisiscalledthepivot)

v Putallelementssmallerthanthepivotonitsleft

v Putallelementslargerthanthepivotontheright

v Nowyouhavetwosmaller sortingproblemsbecauseyouhaveanunsortedlisttotheleftofthepivotandanunsortedlisttotherightofthepivot

v Sortthesequenceontheleft(useQuicksort!)

v Sortthesequenceontheright(useQuicksort!)

u Sortthesequenceontheright(useQuicksort!)v Picka‘middle’elementinthesequence(thisiscalledthepivot)

v Putallelementssmallerthanthepivotonitsleft

v Putallelementslargerthanthepivotontheright

v Nowyouhavetwosmaller sortingproblemsbecauseyouhaveanunsortedlisttotheleftofthepivotandanunsortedlisttotherightofthepivot

v Sortthesequenceontheleft(useQuicksort!)

v Sortthesequenceontheright(useQuicksort!)

Quicksort

42

Lecture #3 Summary

u Solvingaproblemwithacomputerusuallyinvolves:v Astructuredwaytostore(organize)datav Analgorithmthataccessesandmodifiesthatdata

u Algorithmshavecharacteristics,likebrute-force ordivide-and-conquer thathelpusunderstandhowtheywork

u Thinkingaboutabstractdatatypesandalgorithmsfreesusfromworryingabouttheimplementationdetails

u SequencesareafundamentalADTusedtoorganizedatainanorderedlist.u Sequencescanbesearched:

v Linearsearch(brute-force)v Binarysearch(divide-and-conquer),butrequiressortedlist

u Sequencescanbesorted:v Selectionsort(brute-force)v Quick-sort(divide-and-conquer

43

Lecture #4

44

Abstract Data Types

uModelsofcollectionsofinformationu Typicallyatanabstractlevel

“…describeswhatcanbedonewithacollectionofinformation,withoutgoingdowntothelevelofcomputerstorage.”[St.Amant,pp.53]

45

Sequences, Trees and Graphs

46

u Sequence:alistv Itemsarecalledelementsv Itemnumberiscalledtheindex

uGraph

u TreeEric

Emily Jane

Terry Bob

Jim

Mike Chris

Bob

Motivation for Abstract Data Structures (Graphs, Trees)

u Thenatureofsomedata,andthewayweneedtoaccessesitoftenrequiressomestructure,ororganizationtomakethingsefficient(orevenpossible)

u Data:largesetofpeopleandtheirfamilyrelationshipusedforgeneticresearch

u Problems:twopeopleshareararegenetictrait,howcloselyaretherelated?(motivatesforatree)

47

Motivation for Abstract Data Structures (Graphs, Trees)

u Dataset:roadsandintersections.

u Problem:howtotravelfromAtoB@5pmonaFriday?Howtoavoidtrafficvs.preferfreeways?(motivatesaweightedgraph)

u Dataset:freightenterscountryatbigport(LA/LongBeach).

u Problem:Howtoroutefreightgiventrainlines/connections?v Routefastest,vs.lowestcost?

u Dataset:airportlocations

u Problem:howtorouteanddeliverapackagetoanyaddressintheUSwithminimumcost?ThinkUPS,FedEx

48

Motivation for Abstract Data Structures (Graphs, Trees)

u Dataset:networkswitchesandtheirconnectivity(networklinks)

u Problem:Choseasubsetofnetworklinksthatconnectallswitcheswithoutloops(networksdon’tlikeloops).Motivatesgraphs,andgraph->treealgorithm

49

Motivation for Abstract Data Structures (Graphs, Trees)

u Dataset:potentialsolutionstoabigproblem

u Problem:howtofindanoptimalsolutiontotheproblem,withoutsearchingeverypossibility(solutionspacetoobig).Motivatesgraphsandgraphsearchtosolveproblems.

u Otherdata/problemsthatmotivategraphs/trees:v Financialnetworksandmoneyflows,socialnetworks,renderingHTML

code,compilers,3Dgraphicsandgameengines… andmore

50

Trees

u Eachnode/vertexhasexactlyoneparentnode/vertex

u Noloopsu Directed(links/edgespointinaparticulardirection)

u Undirected(links/edgesdon’thaveadirection)

u Weighted(links/edgeshaveweights)

u Unweighted (links/edgesdon’thaveweights)

51

Eric

Emily Jane

Terry Bob

Which of these are NOT trees?

52

1

2

3

5

6

7

4 8

Graph/Tree Traversal

u Traversingagraphoratree:“moving”andexaminingthenodestoenumeratethenodesorlookforsolutions

u Example:findalllivingdescendantsofXinourgeneticdatabase.

u Fortraversingagraphwepickastartingnode,thentwomethodsareobvious:v Depthfirst

u Goasdeep(farawayfromstartingnode)aspossiblebeforebacktracking

v Breadthfirstu Examineonelayeratatime

53

Tree Traversal

u DepthfirsttraversalEric,Emily,Terry,Bob,Drew,Pam,Kim,Jane

u BreadthfirsttraversalEric,Emily,Jane,Terry,Bob,Drew,Pam,KimEric,Jane,Emily,Bob,Terry,Pam,Drew,Kim

54

Eric

Emily Jane

Terry Bob

Drew Pam

Kim

Tree Traversal

u Depthfirstvs.Breadthfirsteventuallyvisitallnodes,butdosoinadifferentorder

u Usedtoanswerdifferentquestionsv Depthfirst:goodforgametrees,evaluatingdownacertainpathv Breadthfirst:lookforshortestpathbetweentwonodes(e.g for

computernetworks)

u Roughly:v Depthfirst:find‘a’solutiontotheproblemv Breadthfirst:find‘the’solutiontotheproblem

55

Graphs: Directed and Undirected

56

Tia

Jim

Mike

Chris

Bob

Joe

Sofie

Undirected

Tia

Jim

Mike

Chris

Bob

Joe

Sofie

Directed

Graph to Tree Conversion Algorithms

u Sometimesthequestionisbestansweredbyatree,butwehaveagraph

u Needtoconvertgraphtotree(bydeletingedges)

u Usuallywanttocreatea“spanningtree”

57

Spanning Trees

u Spanningtree:Anytreethatcoversallverticesv “Cover”=“include”ingraph-speak

u Example:graphofsocialnetworkconnections.Wanttocreatea“phonetree”todisseminateinformationintheeventofanemergency

u Example:networkofswitcheswithredundantlinksandmultiplepathsbetweenswitches(thereareloopsakacyclesinthegraph).Needtochoseasetoflinksthatconnectsallswitcheswithnoloops.

58

Minimum Spanning trees

uSpanningtree:Anytreethatcoversallvertices,notascommonastheMST

uMinimum spanningtree(MST):Treeofminimaltotaledgecost

u Ifyouhaveagraphwithweightededges,aMSTisthetreewherethesumoftheweightsoftheedgesisminimum

uThereisatleastoneMST,couldbemorethanoneu IfyouhaveunweightededgesanyspanningtreeisaMST 59

uWhycomputetheminimumspanningtree?v Minimizethecostofconnectionsbetweencities(logistics/shipping)

v Minimizeofcostofwiresinalayout(printedcircuit,integratedcircuitdesign)

60

Computing the MST

uTwogreedyalgorithmstocomputetheMSTv Prim’salgorithm:Startwithanynodeandgreedilygrowthetreefromthere

v Kruskal’salgorithm:Orderedgesinascendingorderofcost.Addnextedgetothetreewithoutcreatingacycle.

u ‘Greedy’meanssolutionisrefinedateachstepusingthemostobviousnextstep,withthehopethateventualsolutionisgloballyoptimal

61

Prim’s algorithm

u Initializetheminimumspanningtreewithavertexchosenatrandom.

u Findalltheedgesthatconnectthetreetonewvertices(i.euncovered,ordisconnected),findtheminimumandaddittothetree

u Keeprepeatingstep2untilallverticesareaddedtotheMST

(adaptedfrom:https://www.programiz.com/dsa )

62

Kruskal’s algorithm

u Sortalltheedgesfromlowweighttohigh

u Taketheedgewiththelowestweight,ifaddingtheedgewouldcreateacycle,thenrejectthisedgeandselecttheedgewiththenextlowestweight

u Keepaddingedgesuntilwereachallvertices.

(adaptedfrom:https://www.programiz.com/dsa )

63

Shortest path

u Foragivensourcevertex(node)inthegraph,itfindsthepathwithlowestcost(i.e.theshortestpath)betweenthatvertexandeveryothervertex.

u SayyoursourcevertexisMikeu LowestcostpathfromMiketoJimisMike– Bob- Tia– Jim(cost3)

u LowestcostpathfromMiketoJoeisMike– Bob– Tia– Jim– Joe(cost4)

v Veryimportantfornetworkingapplications!

64

Tia

Jim

Mike

Chris

Bob

Joe

Sofie1 1

2

4

3

1

1

3

41

1

Dijkstra’s algorithm: Basic idea

u Fanoutfromtheinitialnode

u Inthebeginningthedistancestotheneighborsoftheinitialnodeareknown.Allothernodesaretentativelyinfinitedistanceaway.

u Thealgorithmimprovestheestimatestotheothernodesstepbystep.

u Asyoufanout,performtheoperationillustratedinthisexample:ifthecurrentnodeA ismarkedwithadistanceof4,andtheedgeconnectingitwithaneighborB haslength2,thenthedistancetoB (throughA)willbe4+2=6.IfBwaspreviouslymarkedwithadistancegreaterthan6thenchangeitto6.Otherwise,keepthecurrentvalue.

65

Lecture 4 Summary

u TreesandGraphsv Sometimesneedtomodelinteractions,connectionsbetweendatav Vertices,edgesv Directed/undirectedv Weighted/unweighted

u GraphTraversalv BFS,DFS

u GraphtoTreev Spanningtrees,minimumspanningtrees

u Prim’s,Kruskal’s

u Shortestpath:Dijkstra’s

66

Lecture #5

67

Recursion

u Recursion,recursionrelations,recursivedatastructures,recursivealgorithms

u Definingadatastructureoralgorithmintermsofitself

u Manyproblemsareeasiertounderstand(implement,solve)asrecursivealgorithms

68

Recursion: abstract data types

uDefiningabstractdatatypesintermsofthemselves(e.g.,treescontaintrees)

u Soalistis:Theitematthefrontofthe

list,andthentherestofthelist(whichis,anitemandthentherestofthelist…)

69

[1,3,5,7,32,6,7,121,7…]

Recursion: abstract data types

uDefiningabstractdatatypesintermsofthemselves(e.g.,treescontaintrees)

u SoatreeisEitherasinglevertex,oravertexthatistheparent

ofoneormoretrees

70

Eric

Emily Jane

Terry Bob

Drew Pam

Kim

Recursion and algorithms

u Conceptofrecursionappliestoalgorithmsaswell

u Somealgorithmsaredefinedrecursively:v Fibonaccinumbers:

u Fib(n)=0(n=0),1(n=1),fib(n-1)+fib(n-2)

u Somecanbeexpressediteratively:v Factorial=n*(n-1)*(n-2)*(n-3)…*1

u Orrecursively:v Factorial=n*factorial(n-1)

71

Recursion and algorithms

u Ifanabstractdatatypecanbethoughtofrecursively(likealist)theseofteninspirerecursivealgorithmsaswell

u Listsum:v Sumofalist=valueoffirstitem+sumoftherestofthelist

72

Recursion: algorithms

u Definingalgorithmsintermsofthemselves(e.g.,quicksort)Checkwhetherthesequencehasjustoneelement.Ifitdoes,stopCheckwhetherthesequencehastwoelements.Ifitdoes,andtheyareintherightorder,stop.Iftheyareinthewrongorder,swapthem,stop.Chooseapivotelementandrearrangethesequencetoputlower-valuedelementsononesideofthepivot,higher-valuedelementsontheothersideQuicksorttheleftsublistQuicksorttherightsublist

73

Recursion: algorithms

uHowdoyouwriteaselectionsortrecursively?uHowdoyouwriteabreadth-firstsearchofatreerecursively?Whataboutadepth-firstsearch?

74

Recursive Selection Sort

u Howtodothis?

u Needtothinkabouttheprobleminrecursiveterms:v Thinkoftheprobleminawaythatgetssmallereachtimeyouconsider

it…v Alsoneedstohaveaterminatingcondition(basecase)

u Thinkingofselectionsortinthisway…

75

Recursive selection sort

u Selectionsortfindsminimumelement,swapstofront.Thenfindsnextsmallest,swapsto2nd… andsoon

u Observation:thefrontelementiseither:v Alreadytheminimumorv Theminimumisintherestofthelist

u Observation:oncewemovetheminimumtothefrontofthelist,wecancallselectionsortontherestofthelist

76

Recursive selection sort

u Weactuallyneedtworecursivealgorithms:v find_min(list):recursivelyfindtheindexoftheminimumitemv selection_sort(list):

u Ifthelengthofthelistisone,stop,thelistissortedu callfind_min()tofindtheminimumelement,swapwiththefrontofthelist

(ifnecessary)u Callselection_sort()ontherestofthelist

v Stopwhen”restoflist”isoneitem

77

Recursive DFS, BFS

u RecursiveDFSisprettyeasy:v foreachneighboruofv:

u Ifuis‘unvisited’:calldfs(u)

u RecursiveBFS…

78

Analysis of algorithms

uHowlongdoesanalgorithmtaketorun?timecomplexity

uHowmuchmemorydoesitneed?spacecomplexity

79

Estimating running time

uHowtoestimatealgorithmrunningtime?vWriteaprogramthatimplementsthealgorithm,runit,andmeasurethetimeittakes

vAnalyzethealgorithm(independentofprogramminglanguageandtypeofcomputer)andcalculateinageneralwayhowmuchworkitdoestosolveaproblemofagivensize

uWhichisbetter?Why?80

Analysis of binary search

u n=8,thealgorithmtakes3stepsu n=32,thealgorithmtakes5stepsu Forageneraln,thealgorithmtakeslog2nsteps 81

Big O notation

u Characterizefunctionsaccordingtohowfasttheygrow

u Thegrowthrateofafunctioniscalledtheorderofthefunction.(hencetheO)

u BigOnotationusuallyonlyprovidesanupperbound onthegrowthrateofthefunction

u Asymptoticgrowth

f(x)=O(g(x))asx->∞ifandonlyifthereexistsapositivenumberM suchthatf(x)≤M*g(x)forallx>x0

82

Conventions

uO(1)denotesafunctionthatisaconstantv f(n)=3,g(n)=100000,h(n)=4.7areallsaidtobeO(1)

u Forafunctionf(n)=n2 itwouldbeperfectlycorrecttocallitO(n2)or O(n3)(orforthatmatterO(n100))

uHoweverbyconventionwecallitbythesmallestordernamely O(n2)v Why?

83

What do they have in common?

u (Binary)searchofasortedlist:O(log2n)

u Selectionsort:O(n2)

u Quicksort:O(nlogn)

u Breadthfirsttraversalofatree:O(V)

u Depthfirsttraversalofatree:O(V)

u Prim’salgorithmtofindtheMSTofagraph:O(V2)

u Kruskal’s algorithmtofindtheMSTofagraph:O(ElogE)

u Dijkstra’s algorithmtofindtheshortestpathfromanodeinagraphtoallothernodes:O(V2)

84

Subset sum problem

u Givenasetofintegersandanintegers,doesanynon-emptysubsetsumtos?

u {1,4,67,-1,42,5,17}ands=24 Nou {4,3,17,12,10,20}ands =19 Yes {4,3,12}

u IfasethasN elements,ithas2N subsets.u CheckingthesumofeachsubsettakesamaximumofNoperations

u Tocheckallthesubsetstakes2NNoperationsu Someclevernesscanreducethisbyabit(2Nbecomes 2N/2,butallknownalgorithmsareexponential

85

Travelling salesperson problem

u Givenalistofcitiesandthedistancesbetweeneachpairofcities,whatistheshortestpossibleroutethatvisitseachcityexactlyonceandreturnstotheorigincity?

u Givenagraphwhereedgesarelabeledwithdistancesbetweenvertices.Startataspecifiedvertex,visitallotherverticesexactlyonceandreturntothestartvertexinsuchawaythatsumoftheedgeweightsisminimized

u Therearen! routes(anumberontheorderofnn - muchbiggerthan2n)

u O(n!)

86

Enumerating permutations

u Listallpermutations(i.e.allpossibleorderings)ofnnumbers

uWhatistheorderofanalgorithmthatcandothis?

87

u Sowehave:v Knapsack/Subsetsum:N*2N

v Setpermutation:n!v Travelingsalesman:n!

88

Analysis of problems

u Studyofalgorithmsilluminatesthestudyofclasses ofproblems

u Ifapolynomialtimealgorithmexiststosolveaproblemthentheproblemiscalledtractable

u Ifaproblemcannotbesolvedbyapolynomialtimealgorithmthenitiscalledintractable

u Thisdividesproblemsintothree groups:v Problemswithknownpolynomialtimealgorithmsv Problemsthatareproventohavenopolynomial-timealgorithmv Problemswithnoknownpolynomialtimealgorithmbutnotyetproventobeintractable

89

Tractable and Intractable

u Tractableproblems(P)v Sortingalistv Searchinganunorderedlistv Findingaminimumspanningtree

inagraph

90

u Intractablev Listingallpermutations(all

possibleorderings)ofnnumbers

u Mightbe(in)tractablev Subsetsum:givenasetof

numbers,isthereasubsetthataddsuptoagivennumber?

v Travellingsalesperson:ncities,n!routes,findtheshortestroute

Theseproblemshavenoknownpolynomialtimesolution

Howevernoonehasbeenabletoprovethatsuchasolutiondoesnotexist

Tractability and Intractability

u ‘Propertiesofproblems’(NOT ‘propertiesofalgorithms’)

u Tractable:problemcanbesolvedbyapolynomialtimealgorithm(orsomethingmoreefficient)

u Intractable:problemcannotbesolvedbyapolynomialtimealgorithm(allsolutionsareproventobemoreinefficientthanpolynomialtime)

u Unknown:notknowniftheproblemistractableorintractable(noknownpolynomialtimesolution,noproofthatapolynomialtimesolutiondoesnotexist)

91

Subset sum problem

u Givenasetofintegersandanintegers,doesanynon-emptysubsetsumtos?

u {1,4,67,-1,42,5,17}ands=24 Nou {4,3,17,12,10,20}ands =19 Yes {4,3,12}

u IfasethasN elements,ithas2N subsets.u CheckingthesumofeachsubsettakesamaximumofNoperations

u Tocheckallthesubsetstakes2NNoperationsu Someclevernesscanreducethisbyabit(2Nbecomes 2N/2,butallknownalgorithmsareexponential)

92

P and NP

u P:setofproblemsthatcanbesolvedinpolynomialtime

u Considersubsetsumv Noknownpolynomialtimealgorithmv However,ifyougivemeasolutiontotheproblem,itiseasyformetocheckifthesolutioniscorrect– i.e.Icanwriteapolynomialtimealgorithmtocheckifagivensolutioniscorrect

uNP:setofproblemsforwhichasolutioncanbecheckedinpolynomialtime

93

Easytosolve(implieseasytocheck)

Easytocheckifsolutionisgood

Easy to Solve vs. Easy to Check

u Easytosolve:sortingv Solve:sortthelistinO(nlogn)v Check:isthelistsorted?O(n)v ClearlysortingisinP

u Hardtosolve:sub-setsumv Solve:generateallsubsets:O(2n)v Check:sum-upsubset.O(n)

u Hardtosolve:integerfactorizationv Solve:checkallnumbersbetween2andsqrt(n)O(2w)v Check:isonenumberafactorofanother?DivideandcheckO(n2)

94

P=NP?

95

uAllproblemsinP arealsoinNPuArethereanyproblemsinNP thatarenotalsoinP?

uInotherwords,is

P =NP ?uCentralopenquestioninComputerScience

P vs. NP Example

u Publickeyencryptionusestwolargeprimenumbersp,qu Ifk=p*q, thenwecansendk intheclearneed pandq todecrypt

u WhyisthisPvs.NP?v p*q clearlyPalgorithmv Findingpandq givenjustk isO(2w)wherew=sizeofthenumber

(digitsorbits)

u IfP=NPthenpublickeyencryptionwouldbe“broken”u Sidenote:ascomputershavegottenfaster,keysizegoesup,makingproblemexponentiallyharderv Keysarenow>=2048bits->22048 isapreposterouslylargenumberv Check1Bkeys/second=1.7x10600 yearstocrack

96

Midterm Style Questions

97

1. Basedontheinformationpresentedinclassandthelectureslides,whichcomponentisnotpartofamodernCPU:A. Arithmetic/logicunitB. ProgramCounterC. CachememoryD. DiskcontrollerE. Registers

2. Whichchoiceforpivotalwaysallowsoptimalruntimeofthequicksortalgorithm?

A.MaximumelementB.MinimumelementC.AverageamongallelementsD.AveragebetweenmaximumandminimumelementsE.Noneoftheabove

3. Inordertofindthek-thsmallestelementinalistofnintegerswerunasmany

iterationsofSelectionSortasnecessaryandthenwestop.Whatisthecomplexityofthisalgorithmintermsofk,n?A. O(k*log(n))B. O(k*n*log(n))C. O(n*log(n))D. O(k*n)E. Notenoughinformationisgiventodeterminethecorrectanswer

4. WhichisTRUEaboutDFS(depthfirstsearch)vs.BFS(breadthfirstsearch)?

A.BFSonaveragehassmallerruntimethanDFSB.DFSonaveragehassmallerruntimethanBFSC.BFScannotdetectloops(i.ewouldnotworkongraphs)D.DFScannotdetectloopsE.Noneoftheaboveistrue

5. Whichisthecorrectorderingofmemoryfromfastestaccesstoslowestaccess?

A. Registers,RAM,Cache,DiskB. Disk,RAM,Cache,RegistersC. Registers,Cache,RAM,DiskD. Cache,Registers,RAM,DiskE. Disk,Cache,RAM,Registers

1. Basedontheinformationpresentedinclassandthelectureslides,whichcomponentisnotpartofamodernCPU:A. Arithmetic/logicunitB. ProgramCounterC. CachememoryD. DiskcontrollerE. Registers

2. Whichchoiceforpivotalwaysallowsoptimalruntimeofthequicksortalgorithm?

A.MaximumelementB.MinimumelementC.AverageamongallelementsD.AveragebetweenmaximumandminimumelementsE.Noneoftheabove

3. Inordertofindthek-thsmallestelementinalistofnintegerswerunasmany

iterationsofSelectionSortasnecessaryandthenwestop.Whatisthecomplexityofthisalgorithmintermsofk,n?A. O(k*log(n))B. O(k*n*log(n))C. O(n*log(n))D. O(k*n)E. Notenoughinformationisgiventodeterminethecorrectanswer

4. WhichisTRUEaboutDFS(depthfirstsearch)vs.BFS(breadthfirstsearch)?

A.BFSonaveragehassmallerruntimethanDFSB.DFSonaveragehassmallerruntimethanBFSC.BFScannotdetectloops(i.ewouldnotworkongraphs)D.DFScannotdetectloopsE.Noneoftheaboveistrue

5. Whichisthecorrectorderingofmemoryfromfastestaccesstoslowestaccess?

A. Registers,RAM,Cache,DiskB. Disk,RAM,Cache,RegistersC. Registers,Cache,RAM,DiskD. Cache,Registers,RAM,DiskE. Disk,Cache,RAM,Registers

Midterm Style Questions

98

6. Considertheabovegraph.WhatisthesequenceofnodesproducedbyrunningBreadthFirstSearch(BFS)startingfromv1?ForBFSonadirectedgraphonlyfollowoutgoingedges.Ifanodehasmorethanoneoutgoingedge,followtheedgetothevertexwiththelowestindexfirst(nodeindexiskforvertexVk).

A. v1 v2 v4 v5 v7 v6 v3B. v1 v2 v4 v5 v6 v7 v3C. v1 v2 v4 v3 v6 v5 v7D. v1 v2 v4 v5 v3 v6 v7E. v1 v2 v4 v5 v3 v7 v6

7. Forthesamegraphasabove,whatisthesequenceofnodesproducedbyrunningDepthFirstSearch(DFS)startingfromv1.ForDFSonadirectedgraphonlyfollowoutgoingedges.Ifanodehasmorethanoneoutgoingedge,followtheedgetothevertexwiththelowestindexfirst(nodeindexiskforvertexVk).

A. v1 v2 v4 v3 v5 v6 v7B. v1 v2 v4 v5 v3 v7 v6C. v1 v2 v4 v3 v6 v5 v7D. v1 v2 v4 v5 v7 v6 v3E.v1 v2 v4v5v3v7v6

8. WhichoftheproblemsdescribedCANNOTbesolvedoptimallywithanMST(minimumspanningtree)?

A.Buildtheshortest-lengthbridgenetworkbetweenasetofislands.

B.Eliminateloopsinacomputernetwork.C.Givenalistofcitiesandthedistancesbetweeneachpair,findtheshortestpossibleroutethatvisitseachcityandreturnstothestartingcity.

D.Eliminatemultiplepathsbetweenanytwoverticesinagraph.E.AlloftheaboveCANbesolvedoptimallywithaMST.

6

V1 V2

2

4 10 1 3

V3 V4 V5 2 2

V6 V7

5

1

8 4

9. ImaginethatitwasprovedthatP=NP.Considerthefollowingstatements:I) Wewillbeabletoverifyanycomputationalprobleminpolynomialtime.

II) Wewillbeabletosolveanycomputationalprobleminpolynomialtime.

III) WecanbuildhardwareimplementationofanondeterministicTuringmachine

IV) Allcomputationalproblemsthatcanbeverifiedinpolynomialtimewillbesolvable

inpolynomialtime

V) WewillbeabletosolveanyNPhardprobleminpolynomialtime

A. IIandVaretrueB. OnlyIIistrueC. OnlyIVistrueD. IandIVaretrueE. AllstatementsI–Varetrue

10. Whatistheroleoftheprogramcounter?A. ItisaspecialregisterintheCPUusedtostorehigh-prioritydataB. IttracksthenumberofinstructionsexecutedbythecurrentlyexecutingprogramC. Ittracksrecentlyaccessedmemorylocationstohelpimplementtheprincipleof

temporallocality

D. ItspecifiesthememorylocationfromwheretheCPUwillfetchthenextinstructionE. IttracksthenumberofrunningprogramsaskingforaccesstotheCPU

11. WhichofthefollowingisTRUEaboutbinarysearch?A. Consideringtheinputdata,binarysearchwillALWAYShaveasmallerruntimevs.sequentialsearchonthesamedata.

B. BinarysearchcanbeappliedtoanylistC. BinarysearchhasruntimecomplexityofO(2N)foranunsortedlistD. BinarysearchcanbeimplementedrecursivelyE. Noneoftheaboveistrue

12. WhichstatementisFALSE?A. EverytreeisagraphB. EverygraphhasacycleC. AtreehasnocyclesD. Everyconnected,undirected,weightedgraphhasatleastonespanningtreeE. Thenumberofverticesintheminimumspanningtreeofagraphisthesameasthenumberofverticesinthegraphitself

Midterm Style Questions

99

13. Whenconsideredintermsofbig-O,chosetheorderingofthefollowingfunctionsthatisinorderofincreasingasymptoticgrowthrates:A. 2n,2n-1,2n+1B. 2n-1,2n+1,2nC. 2n-1,2n,2n+1D. Allthreeofthesefunctionshavethesamebig-Ogrowthrate,soorderisnot

important.14. ThereexistsadatastructurecalledaMaxHeapthatwillreturnit’smaximumelement

(andthenremoveitfromtheheap)inO(logn).YoucanconvertanunsortedlisttoaMaxHeapinO(n).Thus,youcansortanunsortedlistusingaMaxHeap(resultinginareverseordescendingorderedsort).Whatisthetimecomplexityofthissortingalgorithm?A. O(n2log(n))B. O(n2)C. O(log(n)*log(n))D. O(nlog(n))E. O(n)

15. Whatbestdescribesthedefinitionofanalgorithm?

A. Anunambiguousandordered,stepbystepdescriptionofactionstosolveaproblemwrittentobeunderstandablebyhumans

B. sequenceofinstructionwrittentorundirectlyonacomputerC. codedlanguageusedbyprogrammerstowriteinstructionsthatacomputercan

understandtodowhattheprogrammerwantsD. Acodewhosetargetalphabetcontainsonlydigitsand/orstringsofdigitsE. Amathematicalcalculationaccordingtosomewell-knownformula

16. Youareinamazeandafriendsuggeststhatyouputyourrighthandonthewalland

followthewalluntilyoufindtheexit.This“righthandrule”representsanalgorithmforsolvingthemaze.Whichalgorithmdiscussedinclassdoestheapproachcorrespondto?A.BreadthFirstSearchB.DepthFirstSearchC.Kruskal’sAlgorithmD.BinarySearch

1. Basedontheinformationpresentedinclassandthelectureslides,whichcomponentisnotpartofamodernCPU:A. Arithmetic/logicunitB. ProgramCounterC. CachememoryD. DiskcontrollerE. Registers

2. Whichchoiceforpivotalwaysallowsoptimalruntimeofthequicksortalgorithm?

A.MaximumelementB.MinimumelementC.AverageamongallelementsD.AveragebetweenmaximumandminimumelementsE.Noneoftheabove

3. Inordertofindthek-thsmallestelementinalistofnintegerswerunasmany

iterationsofSelectionSortasnecessaryandthenwestop.Whatisthecomplexityofthisalgorithmintermsofk,n?A. O(k*log(n))B. O(k*n*log(n))C. O(n*log(n))D. O(k*n)E. Notenoughinformationisgiventodeterminethecorrectanswer

4. WhichisTRUEaboutDFS(depthfirstsearch)vs.BFS(breadthfirstsearch)?

A.BFSonaveragehassmallerruntimethanDFSB.DFSonaveragehassmallerruntimethanBFSC.BFScannotdetectloops(i.ewouldnotworkongraphs)D.DFScannotdetectloopsE.Noneoftheaboveistrue

5. Whichisthecorrectorderingofmemoryfromfastestaccesstoslowestaccess?

A. Registers,RAM,Cache,DiskB. Disk,RAM,Cache,RegistersC. Registers,Cache,RAM,DiskD. Cache,Registers,RAM,DiskE. Disk,Cache,RAM,Registers

Midterm Style Questions

100

17. Thealgorithmfrom#16canfailonsomemazes.Themazescanbethoughtofasgraphs.Ifthegraphrepresentationofthemazehascertainproperties,thenthe“righthandrule”mightfail.Whichpropertycancausefailure?A.Weighted(somepassagesaresmaller,orsomepassagesforceyoutocrawl)B.Cyclic(thereareloopsinthemaze) C.Planar(themazecanbedrawnasagraphwherenoedgescross) D.Acyclic(therearenoloopsinthemaze)

18. WhichofthefollowingisTRUEabouttreesandgraphs?

A.AlltreesaregraphsB.EverygraphhasexactlyoneMSTC.AlltreesaredirectedgraphsD.AllgraphsaretreesE.NotallgraphshaveaMST

19. TheJacquardLoom(andsimilarmachines)areconsideredinformationtransformers,

butnotcomputers.Whichanswerbestdescribeswhy:A.Programmingthesemachinesdoesn’tscaleB.Programmingthesemachinesrequirespunch-cardsC.MachineslikethesedonothavememoryorcontrolflowD.Machineslikethesearetoooldtobeconsideredcomputers

20. Whenaninstructionisloadedfrommemory,itisdesirabletoloadthecontentsofafewsucceedingmemoryaddressesintothecache.Whyisthat?A.TheCPUisunabletoonlyloadoneinstructionatatimeB.Thosecontentsarelikelytobeusefulintheimmediatefutureaccordingtothe

spatiallocalityprincipleC.Thosecontentsarelikelytobeusefulintheimmediatefutureaccordingtothe

temporallocalityprincipleD.Thecontentsstoredaftertheinstructionarethevaluesusedinthecomputationof

theinstructionandthereforemustbeloadedwiththeinstruction.

21. Thesubset-sumproblemhastimecomplexityO(N*2N).WheredoesthefactorNcomefrom?A:ThatishowmanysubsetsasetofsizeNhas.B:O(N)isthetimecomplexityrequiredtocheckeachpossiblesubsetsum.C:Thatisthetimecomplexityofthealgorithmthatgeneratesthesubsets.D:Noneoftheabove.

17. Thealgorithmfrom#16canfailonsomemazes.Themazescanbethoughtofasgraphs.Ifthegraphrepresentationofthemazehascertainproperties,thenthe“righthandrule”mightfail.Whichpropertycancausefailure?A.Weighted(somepassagesaresmaller,orsomepassagesforceyoutocrawl)B.Cyclic(thereareloopsinthemaze) C.Planar(themazecanbedrawnasagraphwherenoedgescross) D.Acyclic(therearenoloopsinthemaze)

18. WhichofthefollowingisTRUEabouttreesandgraphs?

A.AlltreesaregraphsB.EverygraphhasexactlyoneMSTC.AlltreesaredirectedgraphsD.AllgraphsaretreesE.NotallgraphshaveaMST

19. TheJacquardLoom(andsimilarmachines)areconsideredinformationtransformers,

butnotcomputers.Whichanswerbestdescribeswhy:A.Programmingthesemachinesdoesn’tscaleB.Programmingthesemachinesrequirespunch-cardsC.MachineslikethesedonothavememoryorcontrolflowD.Machineslikethesearetoooldtobeconsideredcomputers

20. Whenaninstructionisloadedfrommemory,itisdesirabletoloadthecontentsofafewsucceedingmemoryaddressesintothecache.Whyisthat?A.TheCPUisunabletoonlyloadoneinstructionatatimeB.Thosecontentsarelikelytobeusefulintheimmediatefutureaccordingtothe

spatiallocalityprincipleC.Thosecontentsarelikelytobeusefulintheimmediatefutureaccordingtothe

temporallocalityprincipleD.Thecontentsstoredaftertheinstructionarethevaluesusedinthecomputationof

theinstructionandthereforemustbeloadedwiththeinstruction.

21. Thesubset-sumproblemhastimecomplexityO(N*2N).WheredoesthefactorNcomefrom?A:ThatishowmanysubsetsasetofsizeNhas.B:O(N)isthetimecomplexityrequiredtocheckeachpossiblesubsetsum.C:Thatisthetimecomplexityofthealgorithmthatgeneratesthesubsets.D:Noneoftheabove.