Design and Analysis of Algorithms

Text Book: Horowitz, S. Sahni, Fundamentals of Computer AlgorithmsReference Book: Introduction to algorithms By Thomas H. Cormen

Notion: Algorithms

• An algorithm is a sequence of unambiguous instructions for solving a computational problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.

“computer”

problem

algorithm

input output

Algorithm is thus a sequence of computational steps that transform the input into the output.

• More precisely, an algorithm is a method or process to solve a problem satisfying the following properties:– Finiteness• terminates after a finite number of steps

– Definiteness • Each step must be rigorously and unambiguously

specified.– Input• Valid inputs must be clearly specified.

– Output• can be proved to produce the correct output given a

valid can be proved to produce the correct output given a valid input.

– Effectiveness• Steps must be sufficiently simple and basic.• Can be carried out with pen and paper

• These are also called characteristics of an algorithm

Examples

• Is the following a legitimate algorithm?

i 1

While (i <= 10) do

a i + 1

Print the value of a

End of loop

Stop

• Four key terms:– Natural Language– Algorithm– Program : A program is the expression of an algorithm in a

programming language– Psuedocode-mix of algorithm and some programming

language.Key Points:

• Each step of an algorithm must be unambiguous.• The same algorithm can be represented in several different

ways. • There might exists more than one algorithm for a certain

problem.• Algorithms for the same problem can be based on very

different ideas and can solve the problem with dramatically different speeds.

Two main issues related to algorithms

• How to design algorithms

• How to analyze algorithm efficiency

Analysis of Algorithms

• How good is the algorithm? (Determined by the complexity)– time efficiency– space efficiency

• Does there exist a better algorithm?– lower bounds– optimality

Importance of Analyze Algorithm

• Need to recognize limitations of various algorithms for solving a problem

• Need to understand relationship between problem size and running time– When is a running program not good enough?

• Need to learn how to analyze an algorithm's running time without coding it

• Need to learn techniques for writing more efficient code

What do we analyze about them?

• Correctness– Does the input/output relation match algorithm

requirement?• Amount of work done (aka complexity) – Basic operations to do task

• Amount of space used– Memory used

• Simplicity, clarity– Verification and implementation.

• Optimality– Is it impossible to do better?

Models of Computation

Reference: The design and analysis of computer algorithms by Aho Ullman

Three models of computation

a) Random Access Machinesb) RASP Machines(Stored program model)c) Turing Machines

Random Access Machine (RAM)

RAM Assumptions

1) Each register holds an integer2) Program can’t modify itself3) Memory instructions involve simple arithmetic

a) Addition, subtractionb) Multiplication, division and control states (got, if-

then, etc.)

RASP Machinestored program model

• Same as RAM but allow program to change itself as it is now stored in the memory

• Same power as RAM

• Example Von Neumann architecture• Let RAM use memory registers to store modifiable

program of RASP

Turing Machine

• A Turing machine includes

– A (conceptual) tape that extends infinitely in both directions

• Holds the input to the Turing machine

• Serves as memory

• Is divided into cells

– A unit that reads one cell of the tape at a time and writes a symbol in that cell

– It is controlled by finite automaton that has finite number of states.

• Example of a finite automaton containing two states is a lightswitch. In this example, the two states are on and off. If the state of the lightswitch is on, pushing the switch is the input that will cause a transition to the off state.

• Each cell contains one symbol

– Symbols must come from a finite set of symbols called the alphabet

• Alphabet for a given Turing machine

– Contains a special symbol b (for “blank”)

– Usually contains the symbols 0 and 1

– Sometimes contains additional symbols

• Each operation involves

– Writing a symbol in the cell (replacing the symbol already there)

– Going into a new state (could be same state)

– Moving one cell left or right

• Each instruction says something like

if (you are in state i) and (you are reading symbol j) then

write symbol k onto the tape

go into state s

move in direction d