Lecture 01
Transcript of Lecture 01
Copyright Li Zimao @ 2007-2008-1 SCUEC
Design and Analysis of Algorithms
Li ZimaoCollege of Computer Science
South Central University for Nationalites
Email: [email protected] or [email protected] Address: 9- 420 Office Hours: By Appointment
Copyright Li Zimao @ 2007-2008-1 SCUEC
Textbook Cover
What does the cover include? Puzzles, Notions, and Stories of Algorithms.
Copyright Li Zimao @ 2007-2008-1 SCUEC
Text Book and Reference Books
Text Book: Anany Levitin, Introduction to the Design and Analysis of Algori
thms, Tsinghua University Press, 2003, 39RMB.
Reference Books: M.H. Alsuwaiyel, Algorithms Design Techniques and Analysis,
Publishing House of Electronics Industry, 2003, 40RMB. Thomas H. Cormen etc., Introduction to Algorithms (Second Edi
tion), Higher Education Press & The MIT Press, 68RMB. Others, search on the website with keyword “Design and Analysi
s of algorithm”, you will find more...
Copyright Li Zimao @ 2007-2008-1 SCUEC
Grading Schemes
Total 100 Class participation and Assignments: 5% Programming Projects: 10% Midterm Test: 15% Final Exam: 70%
Bonus is possible for those positive and active students
Copyright Li Zimao @ 2007-2008-1 SCUEC
Caution
Please finish your assignments and projects independently. If you really meet problems, you can seek help from me or other
students, but remember not copying, CHEATING!
Please sign attendance when needed, but remember not signing for others, CHEATING!Power off (or ring off) your mobile phone
Feedback (positive or negative) is encouraged and welcome, and IMPORTANT. (ask questions!)Thinking, before, in and after class is encouraged and IMPORTANT.
Copyright Li Zimao @ 2007-2008-1 SCUEC
Why Study this Course?
Donald E. Knuth stated “Computer Science is the study of algorithms” Cornerstone of computer science. Programs will not exist
without algorithms.
Closely related to our lives
Help to guide how others analyze and solve problems
Help to develop the ability of analyzing and solving problems via computers
Very interesting if you can concentrate on this course
Copyright Li Zimao @ 2007-2008-1 SCUEC
Course Prerequisite
Data Structure
Discrete Mathematics
C, Java or other programming languages
Advanced Mathematics
Copyright Li Zimao @ 2007-2008-1 SCUEC
Algorithm: A brief History
Muhammad ibn Musa al-Khwarizmi, one
of the most influential mathematicians in 9th century in Baghdad, wrote a textbook in Arabic about adding, multiplying, dividing numbers, and extracting square roots and computing π. www.lib.virginia.edu/science/parshall/khwariz.html
Many centuries later, decimal system was adopted in Europe, and the procedures in Al Khwarizmi’s book were named after him as “Algorithms”.
(image of Al Khwarizmi from http://jeff560.tripod.com/)
Copyright Li Zimao @ 2007-2008-1 SCUEC
Let’s start our lecture formally
PedagogyWhat I hear, I forget.What I see, I remember.What I do, I understand.
What does this mean?
Copyright Li Zimao @ 2007-2008-1 SCUEC
Expected Outcomes
Student should be able to Define algorithm formally and informally. Explain the idea of different algorithms for computing
the greatest common devisor Describe the process of algorithm design and analysis Design recursive and non-recursive algorithms for co
mputing a Fibonacci number
Copyright Li Zimao @ 2007-2008-1 SCUEC
Notion: Algorithm
You all have learned the course DATA STRUCTURE or DISCRETE MATHEMATICS, and have your own sense on algorithm. to your opinion, what is algorithm?
Copyright Li Zimao @ 2007-2008-1 SCUEC
Notion: Algorithms
“computer”
problem
algorithm
input output
An algorithm is a sequence of unambiguous instructions for solving a computational problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.
Copyright Li Zimao @ 2007-2008-1 SCUEC
Notion: Algorithm More precisely, an algorithm is a method or process to solve a problem satisfying the following properties:
Finiteness– terminates after a finite number of steps
Definiteness – Each step must be rigorously and unambiguously specified. -e.g., ”stir until lumpy”
Input– Valid inputs must be clearly specified.
Output– can be proved to produce the correct output given a valid can be proved to
produce the correct output given a valid input. Effectiveness
– Steps must be sufficiently simple and basic.-e.g., check if 2 is the largest integer n for which
there is a solution to the equation xn + yn = zn in positive integers x, y, and z
Copyright Li Zimao @ 2007-2008-1 SCUEC
Examples
Is the following a legitimate algorithm?
i 1
While (i <= 10) do
a i + 1
Print the value of a
End of loop
Stop
Copyright Li Zimao @ 2007-2008-1 SCUEC
Notion: Algorithm*
The exact definition of algorithm is given by Alonzo Church and Alan Turing in 1936. -calculus system of Church Deterministic Turning machine model Equivalent definition
For Turning machine model, see Michael Sipser, Introduction to the Theory of Computation, China Mach
ine Press. John E. Hopcroft, Rajeev Motwani, Introduction to Automata Theory, L
anguages, and Computation (Second Edition), Tsinghua University Press.
Copyright Li Zimao @ 2007-2008-1 SCUEC
Notion: Algorithm*
According to the definition of Algorithms, “almost all” problems are unsolvable. For example: Determine if a polynomial with n (n > 1) variables has
an integral solution. Determine if there is a ‘1111111’ pattern in ’s
decimal format. Halting Problem:
– Determine if a program halts on an input
Copyright Li Zimao @ 2007-2008-1 SCUEC
Examples of Algorithms – Computing the Greatest Common Divisor of Two Integers
gcd(m, n): the largest integer that divides both m and n.
First try -- Euclid’s algorithm: gcd(m, n) = gcd(n, m mod n)
Step1: If n = 0, return the value of m as the answer and stop; otherwise, proceed to Step 2.
Step2: Divide m by n and assign the value of the remainder to r. Step 3: Assign the value of n to m and the value of r to n. Go to Step 1.
Copyright Li Zimao @ 2007-2008-1 SCUEC
Methods of Specifying an Algorithm
Natural language Ambiguous
“Mike ate the sandwich on a bed.”
Pseudocode A mixture of a natural language and programming language-like
structures Precise and succinct. Pseudocode in this course
– omits declarations of variables– use indentation to show the scope of such statements as for, if, and while.– use for assignment
Copyright Li Zimao @ 2007-2008-1 SCUEC
Pseudocode of Euclid’s Algorithm
Algorithm Euclid(m, n)
//Computes gcd(m, n) by Euclid’s algorithm//Input: Two nonnegative, not-both-zero integers m and n //Output: Greatest common divisor of m and n while n ≠ 0 do
r m mod nm nn r
return mQuestions: Finiteness: how do we know that Euclid’s algorithm actually comes to a stop? Definiteness: nonambiguity Effectiveness: effectively computable.
Copyright Li Zimao @ 2007-2008-1 SCUEC
Second Try for gcd(m, n)
Consecutive Integer Algorithm Step1: Assign the value of min{m, n} to t.
Step2: Divide m by t. If the remainder of this division is 0, go to Step3;otherwise, go to Step 4.
Step3: Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step4.
Step4: Decrease the value of t by 1. Go to Step2.Questions Finiteness Definiteness Effectiveness Which algorithm is faster, the Euclid’s or this one?
Copyright Li Zimao @ 2007-2008-1 SCUEC
Third try for gcd(m, n)
Middle-school procedure Step1: Find the prime factors of m.
Step2: Find the prime factors of n.
Step3: Identify all the common factors in the two prime expansions found in Step1 and Step2. (If p is a common factor occurring Pm and Pn times in m and n, respectively, it should be repeated in min{Pm, Pn} times.)
Step4: Compute the product of all the common factors and return it as the gcd of the numbers given.
Question Is this a legitimate algorithm?
Copyright Li Zimao @ 2007-2008-1 SCUEC
Sieve of EratosthenesInput: Integer n ≥ 2
Output: List of primes less than or equal to n
for p ← 2 to n do
A[p] ← p
for p ← 2 to sqrt(n) do
if A[p] 0 //p hasn’t been previously eliminated from the list j ← p* p
while j ≤ n do
A[j] ← 0 //mark element as eliminated
j ← j + p
Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Copyright Li Zimao @ 2007-2008-1 SCUEC
What can we learn from the previous 3 examples?
Each step of an algorithm must be unambiguous.
The same algorithm can be represented in several different ways. (different pseudocodes)
There might exists more than one algorithm for a certain problem.
Algorithms for the same problem can be based on very different ideas and can solve the problem with dramatically different speeds.
Copyright Li Zimao @ 2007-2008-1 SCUEC
Fundamentals of Algorithmic Problem Solving
Understanding the problem Asking questions, do a few examples by hand, think about special
cases, etc.
Deciding on Exact vs. approximate problem solving Appropriate data structure
Design an algorithmProving correctnessAnalyzing an algorithm Time efficiency : how fast the algorithm runs Space efficiency: how much extra memory the algorithm needs.
Coding an algorithm
Copyright Li Zimao @ 2007-2008-1 SCUEC
Algorithm Design and Analysis Process
Copyright Li Zimao @ 2007-2008-1 SCUEC
Two main issues related to algorithms
How to design algorithms
How to analyze algorithm efficiency
Copyright Li Zimao @ 2007-2008-1 SCUEC
Algorithm Design Techniques/Strategies
Brute force
Divide and conquer
Decrease and conquer
Transform and conquer
Space and time tradeoffs
Greedy approach
Dynamic programming
Backtracking
Branch and bound
Copyright Li Zimao @ 2007-2008-1 SCUEC
Analysis of Algorithms
How good is the algorithm? time efficiency space efficiency
Does there exist a better algorithm? lower bounds optimality
Copyright Li Zimao @ 2007-2008-1 SCUEC
Example: Fibonacci NumberFibonacci’s original question: Suppose that you are given a newly-born pair of rabbits, o
ne male, one female. Rabbits are able to mate at the age of one month so that at
the end of its second month a female can produce another pair of rabbits.
Suppose that our rabbits never die. Suppose that the female always produces one new pair (on
e male, one female) every month.
Question: How many pairs will there be in one year? 1. In the beginning: (1 pair) 2. End of month 1: (1 pair) Rabbits are ready to mate. 3. End of month 2: (2 pairs) A new pair of rabbits are born. 4. End of month 3: (3 pairs) A new pair and two old pairs. 5. End of month 4: (5 pairs) ... 6. End of month 5: (8 pairs) ... 7. after 12 months, there will be 233 rabits
(image of Leonardo Fibonacci from http://www.math.ethz.ch/fibonacci)
Copyright Li Zimao @ 2007-2008-1 SCUEC
Recurrence Relation of Fibonacci Number fib(n):
{0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …}
Copyright Li Zimao @ 2007-2008-1 SCUEC
Algorithm: Fibonacci
Problem: What is fib(200)? What about fib(n), where n is any positive integer?
Algorithm 1 fib(n)if n = 0 then
return (0)if n = 1then
return (1)return (fib(n − 1) + fib(n − 2))
Questions that we should ask ourselves.1. Is the algorithm correct?2. What is the running time of our algorithm?3. Can we do better?
Copyright Li Zimao @ 2007-2008-1 SCUEC
Answer of Questions
Is the algorithm correct? Yes, we simply follow the definition of Fibonacci numbers
How fast is the algorithm? If we let the run time of fib(n) be T(n), then we can formulate
T(n) = T(n − 1) + T(n − 2) + 3 1.6n
T(200) 2139
The world fastest computer BlueGene/L, which can run 248 instructions per second, will take 291 seconds to compute. (291 seconds = 7.85 × 1019 years )
Can Moose’s law, which predicts that CPU get 1.6 times faster each year, solve our problem?
No, because the time needed to compute fib(n) also have the same “growth” rate
– if we can compute fib(100) in exactly a year,– then in the next year, we will still spend a year to compute fib(101)– if we want to compute fib(200) within a year, we need to wait for 100 years.
Copyright Li Zimao @ 2007-2008-1 SCUEC
Improvement
Can we do better? Yes, because many computations in the previous algorithm arere
peated.
Algorithm 2: fib(n)
comment: Initially we create an array A[0: n]
A[0] ← 0,A[1] ← 1
for i = 2 to n do
A[i] = A[i − 1] + A[i − 2]
return (A[n])
Copyright Li Zimao @ 2007-2008-1 SCUEC
Important problem types
sorting
searching
string processing
graph problems
combinatorial problems
geometric problems
numerical problems
Copyright Li Zimao @ 2007-2008-1 SCUEC
Fundamental data structures
list
array
linked list
string
stack
queue
priority queue
graph
tree
set and dictionary