Algorithms More than just solving the problem

Fluency with Information Technology

2012-02-22 Katherine Deibel, Fluency in Information Technology 1

INFO100 and CSE100

Katherine Deibel

What is an algorithm?

Algorithms are what computer scientists study

It's not just writing code!

2012-02-22 Katherine Deibel, Fluency in Information Technology 2

Katherine Deibel, Fluency in Information Technology 3

What Computer Scientists Do

We solve problems EFFICIENTLY: Define what the problem is

What inputs are given? What needs to be solved? What needs to be outputted?

Develop a solution process Is it accurate? Is it efficient? Could we improve it?

Implementation How do we organize the data? Does the choice of software/hardware change

the efficiency?

2012-02-22

Katherine Deibel, Fluency in Information Technology 4

Algorithms Everywhere

Theory All about algorithms

Graphics Even more algorithms

Architecture/Hardware Branch prediction, memory schemes, etc.

Networks Ensure transmission, data compression, etc.

Human-Computer Interaction Efficiency of human input/output

2012-02-22

What is an algorithm?

"[An] algorithm is a procedure and sequence of actions to accomplish some task. The concept of an algorithm is often illustrated by the example of a recipe, although many algorithms are much more complex; algorithms often have steps that repeat (iterate) or require decisions (such as logic or comparison). In most higher level programs, algorithms act in complex patterns, each using smaller and smaller sub-methods which are built up to the program as a whole."

Source: Computer User's online dictionary

2012-02-22 Katherine Deibel, Fluency in Information Technology 5

What is an algorithm?

Simply put:An algorithm is a precise description of the steps and methods used to solve a problem or produce a specified result

2012-02-22 Katherine Deibel, Fluency in Information Technology 6

What is an algorithm?

Simply put:An algorithm is a precise description of the steps and methods used to solve a problem or produce a specified result

2012-02-22 Katherine Deibel, Fluency in Information Technology 7

Properties of Algorithms

For an algorithm to be well specified, it must have the following

Inputs specified

Outputs specified

Definiteness

Effectiveness

Finiteness

Form of data to process

Form of results to produce

Agent always knows what to do next

Agent able to do all commands

Will stop with answer, or say ‘none’

2012-02-22 Katherine Deibel, Fluency in Information Technology 8

Properties applied to a recipe

Title Ingredients (inputs) Steps

Exceptions

When to stop Servings (outputs)

2012-02-22 Katherine Deibel, Fluency in Information Technology 9

Katherine Deibel, Fluency in Information Technology 10

Context Matters

Algorithms are abstract but how and where you execute them matters

Think about baking bread In an electric oven

Brick oven

A bread machine The recipes may need to

differ depending on the machine in use

2012-02-22

Algorithms vs Programs

The recipe metaphor breaks down here Algorithms are abstract:

Not specific to any hardware

Not specific to any programming language

Programs are instantiations of algorithms Put into a specific language

Meant to work in specified contexts

2012-02-22 Katherine Deibel, Fluency in Information Technology 11

Previous Algorithms

We have already seen several algorithms (and programs) Constructing a table of Fahrenheit to

Celsius conversions (Ch. 20)

Computing the price of espresso drink (Ch. 18)

Computing weight in gold

Fingerspelling (parsing of input)

Sorting AlgorithmsEveryday application

2012-02-22 Katherine Deibel, Fluency in Information Technology 13

Elements of Sorting

Inputs specified:A list of items in any order

Outputs specified:The items in ascending order

Definiteness: Effectiveness: Finiteness:

Clearly can be done in finite time

2012-02-22 Katherine Deibel, Fluency in Information Technology 14

Turn to your neighbor…

With your neighbor(s), discuss how you would sort a deck of cards into numerical order (As, 2s, 3s,…, Qs, Ks) How many different ways can you

think of? Is any better than the

others? How do you know?

2012-02-22 Katherine Deibel, Fluency in Information Technology 15

Random Sort

Algorithm:Check if deck is sorted

If not sorted, shuffle the deck

Repeat Will it finish? Will it be correct? Is it efficient?

2012-02-22 Katherine Deibel, Fluency in Information Technology 16

Insertion Sort

Algorithm:

Take first card and start a sorted pile

For each remaining card

Find the position where it belongs in the sorted pile

Place card in correct spot

2012-02-22 Katherine Deibel, Fluency in Information Technology 17

Insertion Sort

Deck: 4, 7, 2, 5, J, 8, …

1. Deck: 4 | 7, 2, 5, J, 8, …

2. Deck: 4, 7 | 2, 5, J, 8, …

3. Deck: 2, 4, 7 | 5, J, 8, …

4. Deck: 2, 4, 5, 7 | J, 8, …

5. Deck: 2, 4, 5, 7, J | 8, …

6. Deck: 2, 4, 5, 7, 8, J | …

2012-02-22 Katherine Deibel, Fluency in Information Technology 18

Insertion Sort

Does it end? Yes Will it be correct? Yes Is it efficient?

Fairly efficient in real world

Programming has varying performance:▪ Best: Linear O(n)

▪ Worst: Quadratic O(n2)

2012-02-22 Katherine Deibel, Fluency in Information Technology 19

Merge Sort

Algorithm:

Split deck into two piles

Merge sort first pile

Merge sort second pile

Merge the two sorted piles This is a recursive function in that it

is continually applied to itself

2012-02-22 Katherine Deibel, Fluency in Information Technology 20

Merge Sort Example

Deck: 3, 6, 7, 2, 1, 8, 4, 5,

1. 3, 6, 7, 2 | 1, 8, 4, 5 (split)

2. 3, 6 | 7, 2 | 1, 8, 4, 5 (split)

3. 3, 6 | 7, 2 | 1, 8, 4, 5 (sort left)

4. 3, 6 | 2, 7 | 1, 8, 4, 5 (sort right)

5. 2, 3, 6, 7 | 1, 8, 4, 5 (merge)

6. 2, 3, 6, 7 | 1, 8 | 4, 5 (split)

7. 2, 3, 6, 7 | 1, 8 | 4, 5 (sort left)

8. 2, 3, 6, 7 | 1, 8 | 4, 5 (sort right)

9. 2, 3, 6, 7 | 1, 4, 5, 8 (merge)

10. 1, 2, 3, 4, 5, 6, 7, 8 (merge)

2012-02-22 Katherine Deibel, Fluency in Information Technology 21

Merge Sort

Does it end? Yes Will it be correct? Yes Is it efficient?

Fairly efficient in real world

Programming has constantperformance:

▪ Best: O(n log2 n)

▪ Worst: O(n log2 n)

▪ Worse than linear, better than quadratic

2012-02-22 Katherine Deibel, Fluency in Information Technology 22

Many Sorting Algorithms

Multiple approaches for sorting Some more efficient in time in general

Some more efficient depending on nature of input

Examples of sorting performance:http://www.sorting-algorithms.com/

2012-02-22 Katherine Deibel, Fluency in Information Technology 23

EncryptionKeeping Secrets via Algorithms

2012-02-22 Katherine Deibel, Fluency in Information Technology 24

Encryption / Decryption

Encrypt: Transform data so that it is no longer understandable

Decrypt: Transform encrypted data to be understandable again

2012-02-22 Katherine Deibel, Fluency in Information Technology 25

Encryption and decryptionare algorithms

Caesar Ciper

Supposedly used by Julius Caesar Algorithm:

To encrypt:Move each letter n letters ahead

To decrypt:Move each letter n letters back

The movement wraps around the alphabet Y + 3 B

2012-02-22 Katherine Deibel, Fluency in Information Technology 26

Caesar Ciper

Example when n = 5 A F

B G

…

M R

…

Y D

Z E

2012-02-22 Katherine Deibel, Fluency in Information Technology 27

N QNPJ HFYX

I LIKE CATS

Katherine Deibel, Fluency in Information Technology 28

Caesar Cipher

Simple algorithm Easy to implement Easy to crack

Every play a cryptogram

The twelve most common letters in English (descending): ETAOIN SHRDLU

2012-02-22

RSA

Public key cryptography system invented in 1978 by Ron Rivest, Adi Shamir, and Leonard Adleman

Before we get into the details… Are any of you international students?

Is it before 1999?

2012-02-22 Katherine Deibel, Fluency in Information Technology 29

Katherine Deibel, Fluency in Information Technology 30

Encryption is a Munition

After WWII, encryption algorithms were deemed as military equipment Actually listed as munitions

Export to foreign countries tightly controlled Debates about teaching to international students

Richard White had the RSA algorithm tattooed on his arm and was theoretically unable to travel internationally

Printing and discussing of encryption algorithms is now recognized as free speech Bernstein (Pretty Good Privacy) v. United States

2012-02-22

Back to RSA

The basics of RSA:

1. Pick two large prime numbers p and q

2. Compute n = pq, m = (p-1)(q-1)

3. Choose a number e such that 1 < e < m and e and m's only common divisor is 1

4. Calculate d = e-1 mod m

5. Publish n and e; d and m are kept private

Encryption of integer i: c = ie mod n

Decryption of integer c: i = cd mod m2012-02-22 Katherine Deibel, Fluency in Information Technology 31

Katherine Deibel, Fluency in Information Technology 32

Makes sense, right?

RSA involves some fairly complex number theory

What you need to know Based on prime numbers

Factoring numbers into prime components is computationally difficult

Larger prime numbers make RSA really secure (but only 99.9999999% secure)

2012-02-22

Public key systems

RSA is an example of a public key system The encryption involves a numerical

computation with several parameters Some parameters are shared (public keys)

Some parameters are not (private keys) Dominant form of encryption today

Scales by increasing the length of parameters

2012-02-22 Katherine Deibel, Fluency in Information Technology 33

Building your own algorithmsSome tools

2012-02-22 Katherine Deibel, Fluency in Information Technology 34

Solving a Large Problem

Large problems share many properties: They are daunting—there’s so much to do!

We don’t know were to begin

Not sure we know all of the tasks that must be done to produce a solution

Not sure we know how to do all of the parts— new knowledge may be required

Not sure it is within our capability—maybe an expert is needed

2012-02-22 Katherine Deibel, Fluency in Information Technology 35

Problem Decomposition

“Divide and conquer” is a political strategy, military strategy and IT strategy

Top-level Plan for Algorithm Building

1. Describe (in any language) a series of steps that produce a solution

2. For each step, solve it or decompose further

3. For steps needing decomposition, repeat 2

4. Assemble solutions and test correctness

5. When solution fully assembled, evaluate

2012-02-22 Katherine Deibel, Fluency in Information Technology 36

1. Give Steps to a Solution

Specify (in any language) a series of steps that produce a solution For a huge problem the steps may at first be

vague, but they can be (and must be) made more precise as the whole picture emerges

The goal is an algorithm(s), so List and describe the inputs

List and describe the outputs Be guided by figuring out how to transform

the inputs into the outputs

2012-02-22 Katherine Deibel, Fluency in Information Technology 37

2 and 3. Solve or Decompose

For each step, solve it or decompose it further, i.e. apply same technique

Most “top level” steps can’t be brained out, and need further decomposition

“Top level” steps often seem huge, too

The technique allows one to concentrate on only one problem at a time

As before, focus on transforming inputs to intermediary outputs to final outputs

2012-02-22 Katherine Deibel, Fluency in Information Technology 38

Katherine Deibel, Fluency in Information Technology 39

4. Assemble and Test

Putting solutions together can be tough because of different assumptions made while solving the parts

A common practice is to combine parts along the way and to test continuously

Because of the need to test, pick a good order to solve the problems

Mistakes and backtracking are inevitable

2012-02-22

Example: Lab 8

Lab 8 showed some of the basic ideas of writing an algorithm / program Breaking down parts of the problem

Testing along the way

Continual effort to just progress forward

2012-02-22 Katherine Deibel, Fluency in Information Technology 40

Example: Chapter 22

Implements Smooth Motion page:http://preview.tinyurl.com/fit-smooth

Fairly big application 125 lines

2500 nonspace characters

6 functions Chapter details the process

of divide-and-conquer

2012-02-22 Katherine Deibel, Fluency in Information Technology 41

Summary

Algorithms describe the precise steps to solve a problem Fundamental concept in computer science

Concerned with correctness, efficiency, and finiteness

Building algorithms is about abstracting and breaking down a problem We do this all the time

The challenge of programming is telling a computer how to do the algorithm

2012-02-22 Katherine Deibel, Fluency in Information Technology 42