INTRODUCTION TO ALGORITHMICS - Tadeusz Łubazpt2.tele.pw.edu.pl/rawski/Intr_Algor_1.pdf · composed...
Transcript of INTRODUCTION TO ALGORITHMICS - Tadeusz Łubazpt2.tele.pw.edu.pl/rawski/Intr_Algor_1.pdf · composed...
INTRODUCTION TO ALGORITHMICS
Lectures’ agendaLectures’ agenda: � Introduction� Structure of algorithm� Data structures� Programming languages� Algorithmic methods� Correctness of algorithms� Complexity of algorithms
M.Rawski Introduction to Algorithmics
Literature
Textbooks
� Harel D.: Algorithmics. The Spirit of Computing.
Supplementary readings
� Wirth N.: Algorithms + Data Structures = Programs
� Aho A.J., Hopcroft J.E., Ullman J.D.: Data Structures and Algorithms
� Aho A.J., Hopcroft J.E.: The Design and Analysis of Computer Algorithms,
� Cormen T., Leiserson C., Rivest R.: Introduction to Algorithms
� Dijkstra E.: A Discipline of Programming
M.Rawski Introduction to Algorithmics
What we will be talking about?
�The topic of this lecture is algorithmics
�Basic concepts
�algorithmic problem and algorithm
�structure of algorithms and data structures
�correctness and efficiency of algorithms
?
M.Rawski Introduction to Algorithmics
Amazing machines
� Computer control aircraft, space ships and atomic power stations
� Help in performing immensely complicated mathematical
computations
� Route and control millions of telephone calls in networks spanning
continents
� Execute tasks with enormous precision
� Relieve us from many boring duties
� Deliver diverse entertainment
M.Rawski Introduction to Algorithmics
How computers are perceived
�Computer from one point of view �processing unit�monitor�keyboard
�Computer from other point of view� set of switches – bits� small number of extremely trivial
operation
bit
1
0
M.Rawski Introduction to Algorithmics
Operations on bits
� Flipping
� Zeroing
� Testing
1 1 0 1 0 0 1 0
1 1 0 0 0 0 1 0
1 1 0 0 0 1 1 0
1 1 0 1 0 0 1 0
1 1 0 0 0 0 1 0
1 1 0 0 0 0 1 0if this bit is set
flip this bit
1 1 0 0 0 0 1 0
1 1 0 0 0 1 1 0
M.Rawski Introduction to Algorithmics
How bits do this ?
Simple operations Complex applicationsSimple operations Complex applications
�Process�Algorithm
InputData
PROCESS Results
ALGORITMS
M.Rawski Introduction to Algorithmics
Some gastronomy example ...
�Baking of cake - process carried out from the ingredients, by a baker with use of oven, and, the most significantly, according to the recipe.
� ingredients input data
� cake result
� recipe algorithm
�Algorithm (software) prescribes the activities that constitute the process carried out by the hardware
M.Rawski Introduction to Algorithmics
Some gastronomy example ... (cnt)
� activities process� algorithm + processes software� utensils + oven hardware(The baker can be considered as a part of the hardware)
RECIPE(software)
RECIPE(software)
CAKECAKEOWEN, UTENSILS,
BAKER(hardware)
OWEN, UTENSILS, BAKER
(hardware)INGREDIENTSINGREDIENTS
SOFTWARESOFTWARE
RESULTSRESULTSCOMPUTERCOMPUTERINPUT DATAINPUT DATA
M.Rawski Introduction to Algorithmics
Algorithmics vs. informatics
�Algorithmics – area of human study, knowledge and expertise that concerns algorithms
Algorithmics Informatics
�Informatics- computer science
M.Rawski Introduction to Algorithmics
Some history� Euclid (400 - 300 B.C.) – Greek mathematician, author of the algorithm for
finding the greatest common divisor (GCD) of two positive integers� Mohammed al-Khwarizmi (IX Century) – Persian mathematician, author
of the rules for adding, subtracting, multiplying and dividing decimal numbers� Joseph Jacquard (1801) – Frenchman, inventor of weaving loom controlled
by cards with holes punched at various locations� Charles Babbage (1833) – English mathematician , inventor of „ the
difference engine”, conceived and planned a machine called „the analytical engine” controlled by algorithms encoded on cards with holes
� Augusta Ada King - mathematician, „programmer" of Babbage’s machine� Herman Hollerith (1890) – American engineer, inventor of the machine
that helped tabulate national census in USA� First electronic, general-purpose computers - 1940s (Alan Turing, Howard
Aiken, John Mauchly, John von Neumann)� Recognition of computer science as an independent academic discipline
occurred in the mid-1960s - Association for Computing Machinery (ACM) published recommendation for computer science programs of study
M.Rawski Introduction to Algorithmics
� On the one hand a lot of people does not know anything about
computer science, on the other hand the member of the ever-
growing class of people seam to know about it everything
� But in fact large and important part of the science of computing
are not sufficiently know by members of both groups
Before we set out on a journey into computers world...
I have computer, I am computer scientist
M.Rawski Introduction to Algorithmics
Limitations of computers
� Computers are capable of:�analysis of enormous
quantity of data,
�control sophisticated robots,
�play chess on the level of master
� Computers are incapable of: �determine the person’s age
(with error margin of few years) analyzing picture
�direct the robot to build a bird’s nest from a pile of twigs
�beat 12-year-old amateur chess player when rules are changed
M.Rawski Introduction to Algorithmics
Levels of detail
�A recipe for chocolate mousse�„Melt chocolate and 2 tablespoons water in boiler. When
melted, stir in powdered sugar; add butter bit by bit. Set aside. Beat egg yolks about 5 minutes, until…”
�Why not say „take 121 grains of powdered sugar, pour it into the melted chocolate...”, or even more precisely „move your arm toward the ingredients at an angle of 140...”
M.Rawski Introduction to Algorithmics
Levels of detail
� The hardware knows how to stir sugar into melted
chocolate
� If hardware knew how to prepare chocolate mixture the
recipe could include simple instruction „prepare
chocolate mixture”
M.Rawski Introduction to Algorithmics
Levels of detail (cnt)
� It is important to precise the set of basic instructions, algorithm is considered to be capable of prescribing
� Without doing so there is no point in trying to find algorithms� Basic instructions must be stated clearly and precisely (we cannot
accept things like „beat egg whites until foamy”)� The overall quality of algorithm depends crucially on the selection
of basic actions and their appropriateness to the matter at hand
We speak about algorithms, but few moments ago we operated
single bits (programming languages)
levels of detail of basic instructions of algorithm
basic actions adequate for solving given problem
M.Rawski Introduction to Algorithmics
Short algorithms -long processes
� list of personnel records� record = name + personal details + salary� task : sum of all salaries of all employees
� ALGORITHM(1) make „aside” a note of the number 0 ;(2) proceed through the list, adding each employee’s salary to the
noted number(3) having reached the end of the list, return the number as
output
M.Rawski Introduction to Algorithmics
Short algorithms -long processes (cnt)
€ 27,000E. Silver
••
••
••
€ 15,000D. Green
€ 32,000B. White
€ 25,000J. Brown
SalaryName 0
25,000
57,000
72,000
567,000
•
•
•
start
end
Value of noted number
•
•
•
M.Rawski Introduction to Algorithmics
Short algorithms -long processes (cnt)
� Fixed algorithm prescribes many processes � The text of the algorithm is short and fixed in length but process it
controls varies with the length of employee list� The algorithm requires only a single noted number in order to do
the job no matter how long employee list is.� There is an infinite number of inputs
� The algorithm is supposed to perform satisfactory for an infinite number of input data
� Input data must be legal (salary list, not list of bestsellers)
� Someone must specify which employee lists are legal (32,000 or maybe 32K)
M.Rawski Introduction to Algorithmics
The algorithmic problem and its solution
� Algorithmic problem � Algorithmic solution
characterization of all legal inputs
(specification ofthe allowed inputs)
characterization of desiredoutput as a function of inputs
+
any legal input
desired output
ALGORITHM
M.Rawski Introduction to Algorithmics
The algorithmic problem and its solution (cnt)
� Solution of algorithmic problem = algorithm, which for any legal data will produce desired output by executing the process that is prescribed and governed by the algorithm
� It is easy to find solution that works fine only for some legal input data:
1. return 0 as output
M.Rawski Introduction to Algorithmics
Bounds of basic actions
� Each basic actions must be carried out in a finite amount of time
� Infinitely long actions are unacceptable� Infinitely short actions are unacceptable too� Limits on the resources used in performing actions have
to be enforced too
� Obviously the assumptions about basic actions hold for real computers
M.Rawski Introduction to Algorithmics
Algorithmic problem and its solution –summary
� Algorithmic problem consists of :�a characterisation of a legal, possibly infinite, collections of potential
input sets, and�a specification of the desired outputs as function of the inputs
� It is assumed that either description of the allowed basic actions (or a hardware configuration together with its built-in basic actions) are provided in advance
� A solution to an algorithmic problem = algorithm �composed of elementary instruction prescribing actions from the
agreed-on set�producing the output as required for any legal input set
Algorithmic problems can be incredibly complex. A lot of algorithmic problems do not admit satisfactory solutions, while others do not admit any solution at all.
M.Rawski Introduction to Algorithmics
Complexity of algorithmic problems in practice
�
�Inputs – legal chess position�Desired output – best move for White
�
�2000000 papers are to be distributed to 10000 locations in 100 towns using 50 trucks and 60 drivers, in a way that the total number of kilometers driven is minimized
1111
1000100010001000
1E+061E+061E+061E+06
1E+091E+091E+091E+09
1111 2222 3333 4444 5555 6666 7777 8888 9999 10101010
2 3 4 5 6 7 8 9 102 6 24 120 720 5040 40320 362880 3628800
M.Rawski Introduction to Algorithmics
Algorithm – how to do it?
� Order a little robot (a processor) to do „this and that” to sum all salaries of employees �Robot is told to make a not of 0�Next he is told to work its way through the employee list,
finding salaries and adding them to the noted number�After going through whole list hi should present noted number
M.Rawski Introduction to Algorithmics
Control structures
� It is obvious that the order of carried actions is crucial
� The algorithm must therefore contain control instructions to „push” the processor in this or that direction, telling it what to do at each step and when to stop and say „I am done”
AlgorithmControl
instruction
PROCESOR
A→ B → C → … → X → Y → Z
Order of execution of basic instructions
M.Rawski Introduction to Algorithmics
Control structures (cnt)
� Direct sequencing - „do A followed by B ”�Each semicolon or period in the recipe hides an implicit „and
then” phrase, for example „gently fold in chocolate; [and then] reheat slightly...”
� Conditional branching - „if Q then do A otherwise do B ” or just „if Q then do A” , where Q is some condition�„reheat slightly, if necessary...”
� These to control constructs do not explain how algorithm of fixed (maybe even short) length can describe processes that can grow increasingly long, depending on particular input.
M.Rawski Introduction to Algorithmics
Control structures (cnt)
� Bounded iteration - „do A exactly N times”, where Nis a number�„add two spoons of water”
� Conditional iteration (unbounded) - ”repeat A until Q” or „while Q do A”, where Q is a condition�„beat egg whites until foamy...”
M.Rawski Introduction to Algorithmics
Sum of all salaries once more...
� Let us assume that the input includes not only the list of employees, but also its length (the number of employees)
� ALGORITHM(1) make note of 0; point to the first salary on the list;(2) do the following N – 1 times:
(2.1) add the salary pointed at to the noted number;(2.2) point to the next salary;
(3) add the salary pointed at to the noted number;(4) produce the noted number as output.
Why N – 1 times? What happens if N = 0? How to act if the input does not include N.
M.Rawski Introduction to Algorithmics
Combining control structures
� An algorithm can contain many control-flow constructs in nontrivial combinations.
� Nested iterations (loops)
(1) do exactly N times (outer loop) (1.1) while Q do A (inner loop)
� Inner loop may contain more loops, branches...� What is complexity of such nested loop?
M.Rawski Introduction to Algorithmics
Bubblesort
� Sorting is one of the most interesting topics in algorithmics
� Task�input : unordered list of elements�output : list sorted in ascending order
M.Rawski Introduction to Algorithmics
Bubblesort (cnt)
� Input – list of words: dog, book, type, day, sun
� If the jumbled list is traversed in sequence, one element at a time, and whenever two adjacent elements are found to be in the wrong order they are exchanged, then on completion of the traversal, the larger element is in its rightful place; namely at the end of the list
start → … → … → end
First traversal
start → … → end
Second traversal
M.Rawski Introduction to Algorithmics
Bubblesort (cnt)
� Elements „bubble up” to the top of the list like bubbles in the pipe filled with water
24
12
78
14
26
8
78
24
12
26
14
8
78
26
24
12
14
8
78
26
24
14
12
8
start → … → … → end
M.Rawski Introduction to Algorithmics
Bubblesort (cnt)
� Bubblesort algorithm
(1) do the following N – 1 times:(1.1) point to the first element;(1.2) do the following N – 1 times :
(1.2.1) compare the element pointed to with the next element;
(1.2.2) if the compared elements are in the wrong order, exchange them;
(1.2.3) point to the next element.
M.Rawski Introduction to Algorithmics
„Goto” statement
� „Goto” statement – „goto G”, where G marks some point in the text of the algorithm
� Controversial construct�An algorithm that contains many „goto” statements directing
control backwards and forwards in a tangled fashion quickly becomes very difficult to understand
�Introduces technical difficulties – what happens if a „goto” statement directs the processor into the midst of a loop?
� Many researchers are opposed to using this statement freely in algorithm