S. J. Shyu Chap. 1 Introduction 1

The Design and Analysis of Algorithms

Chapter 1 Introduction

S. J. Shyu

S. J. Shyu Chap. 1 Introduction 2

Why do we need to study algorithms?

We study algorithms so that we can learn strategies to design efficient algorithms.

Besides, we study algorithms to understand the difficulty of designing good algorithms for some problems, namely NP-complete problems.

S. J. Shyu Chap. 1 Introduction 3

Consider the sorting problem.

Sorting problem: To sort a set of elements into

increasing or decreasing order.11, 7, 14, 1, 5, 9, 10

↓sort 1, 5, 7, 9, 10, 11, 14 

Insertion sort Quick sort

S. J. Shyu Chap. 1 Introduction 4

Comparison of these two algorithms implemented on two

computers:A bad algorithm implemented

on a fast computer does not perform as well as a good algorithm implemented on a slow computer. Insertion Sort on IBM SP2 Quick Sort on PC486 When n>400, PC performs better than

SP2.

S. J. Shyu Chap. 1 Introduction 5

Insertion Sort on IBM SP2 vs. Quick Sort on PC

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

0.02

100 200 300 400 500 600 700 800 900

n

CP

U t

ime

(sec

.)

InsertionSort

QuickSort

S. J. Shyu Chap. 1 Introduction 6

High Speed Computer High Speed Computation(if no idea about algorithms)

A good knowledge of algorithms makes your computer effective and efficient.

S. J. Shyu Chap. 1 Introduction 7

Analysis of algorithms Measure the goodness of algorithms

efficiency asymptotic notations: e.g. O(n2) worst case average case amortized

Measure the difficulty of problems NP-complete undecidable lower bound

Is an algorithm optimal?

S. J. Shyu Chap. 1 Introduction 8

Algorithms vs. Problems

How fast can an algorithm be? (in solving a problem)

Complexity Theory

How hard can a problem be? (for any problem solving it)

Computability Theory

S. J. Shyu Chap. 1 Introduction 9

0/1 Knapsack ProblemGiven a set of n items where each ite

m Pi has a value Vi , weight Wi and a limit M of the total weights, we want to select a subset of items such that the total weight does not exceed M and the total value is maximized.

S. J. Shyu Chap. 1 Introduction 10

0/1 Knapsack problem

M (weight limit)=14best solution: P1, P2, P3, P5 (optimal)

10 +5 +1 +3 = 19This problem is NP-complete.

P1 P2 P3 P4 P5 P6 P7 P8

Value 10 5 1 9 3 4 11 17

Weight

7 3 3 10 1 9 22 15

S. J. Shyu Chap. 1 Introduction 11

Traveling salesperson problem

Given: A set of n planar pointsFind: A closed tour which includes all points exactly once such that its total length is minimized.

This problem is NP-complete.

S. J. Shyu Chap. 1 Introduction 12

Partition problem Given: A set of positive integers S

Find: S1 and S2 such that S1S2=, S1S2=S, and

(Partition into S1 and S2 such that the sum of S1 is equal to that of S2)

e.g. S={1, 7, 10, 4, 6, 8, 13} S1={1, 10, 4, 8, 3} S2={7, 6, 13}

This problem is NP-complete.

21 SiSi

ii

S. J. Shyu Chap. 1 Introduction 13

Given: an art gallery Determine: min # of guards

and their placements such that the entire art gallery can be monitored.

NP-complete

Art gallery problem

S. J. Shyu Chap. 1 Introduction 14

Set Cover ProblemGiven: some subsets from a set S Determine: min # of subsets whose union covers SNP-complete

0 1 0 0 0 01 1 0 1 0 00 0 0 1 1 00 0 1 0 1 1

ABCD

a b c d e f

S. J. Shyu Chap. 1 Introduction 15

Knapsack problem

P5, P2, P1, P8,

W 1 + 3 + 7=11 +15 > 14 (=M)X 1 1 1 3/15V 3 + 5 + 10 + (3/15)17 = 21.4

P1 P2 P3 P4 P5 P6 P7 P8

Value 10 5 1 9 3 4 11 17

Weight

7 3 3 10 1 9 22 15

V/W 10/7 5/3 1/3 9/10 3/1 4/9 11/22 17/15P5 P2 P1 P8 P4 P7 P6 P3

Value 3 5 10 17 9 11 4 1

Weight

1 3 7 15 10 22 9 3

V/W 3 1.67 1.43 1.13 0.9 0.5 0.44 0.33

S. J. Shyu Chap. 1 Introduction 16

Minimal Spanning Trees Given a weighted graph G, a spanning tree

T is a tree where all vertices of G are vertices of T and if an edge of T connects Vi and Vj, its weight is the weight of e(Vi, Vj) in G.

A minimal spanning tree of G is a spanning tree of G whose total weight is minimized.

S. J. Shyu Chap. 1 Introduction 17

Minimum Spanning Trees

solved by greedy method# of possible spanning trees for n points: nn-2

n=10→108, n=100→10196

S. J. Shyu Chap. 1 Introduction 18

Convex hull Given a set of

planar points, find a smallest convex polygon which contains all points.

It is not obvious to find a convex hull by examining all possible solutions

divide-and-conquer

S. J. Shyu Chap. 1 Introduction 19

One-center problem Given a set of planar points,

find a smallest circle which contains all points.

Prune-and-search

S. J. Shyu Chap. 1 Introduction 20

Easy Problems vs. Difficult Problems

NP problems

NP-complete problems

P problems

Sorting, Minimal Spanning Tree,Shortest Path, Convex Hall, Knapsack,Longest Common Subsequence, …

Traveling salesperson problem, Gallery Guards, 0/1 Knapsack, Multiple Sequence Alignment, …

S. J. Shyu Chap. 1 Introduction 21

Many strategies, such as the greedy approach divide-and-conquer approach branch and bound prune and search dynamic programming and so on will be introduced in this course.

S. J. Shyu Chap. 1 Introduction 22

Strategies for P problems

Greedy Dynamic Programming Divide and Conquer Prune and Search …

S. J. Shyu Chap. 1 Introduction 23

Strategies for NP-hard problems

Optimization Branch and bound Dynamic programming Exhausted search …

Approximation Simple heuristics with a guaranteed

error ratio (e.g. Greedy, Random, …) Sophisticated heuristics (e.g. Greedy,

2-opt, k-opt, …) Meta-heuristics