CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI....
Transcript of CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI....
![Page 1: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/1.jpg)
9/2/2019
Sofya Raskhodnikova
Theory of Computation
LECTURE 1Theory of Computation
• Course information
• Overview of the area
• Finite Automata
Sofya Raskhodnikova; Intro Theory of Computation
![Page 2: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/2.jpg)
Course information
1. Course staff
2. Course website(s)
3. Piazza bonus
4. Prerequisites
5. Textbook(s)
6. Syllabus
7. Homework logistics
8. Collaboration policy
9. Exams and grading
Sofya Raskhodnikova; Intro Theory of Computation 9/2/2019 L1.2
![Page 3: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/3.jpg)
Tips for the course
• Concepts in this course take some time to sink in:
be careful not to fall behind.
• Do the assigned reading on each topic before the
corresponding lecture.
• Take advantage of office hours.
• Be active in lectures/recitations and on piazza.
• Allocate lots of time for the course: comparable
to a project course, but spread more evenly.
9/2/2019 L1.3
![Page 4: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/4.jpg)
Tips for the course: HW
• Start working on HW early.
• Spread your HW time over multiple days.
• You can work in groups (up to 4 people), but
spend 1-2 hours thinking about it on your own
before your group meeting.
9/2/2019 L1.4
![Page 5: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/5.jpg)
Tips: learning problem solving
To learn problem solving, you have to do it:
• Try to think how you would solve any presented
problem before you read/hear the answer.
• Do exercises in addition to HW.
9/2/2019 L1.5
![Page 6: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/6.jpg)
Tips: how to read a math text
• Not like reading a mystery novel.
• The goal is not to get the answers, but to learn
the techniques.
• Always try to foresee what is coming next.
• Always think how you would approach a
problem before reading the solution.
• This applies to things that are not explicitly
labeled as problems.
9/2/2019 L1.6
![Page 7: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/7.jpg)
Skills we will work on
• Mathematical reasoning
• Expressing your ideas
– abstractly (suppress inessential details)
– precisely (rigorously)
• Mathematical modeling
• Algorithmic thinking
• Problem solving
• Having FUN with all of the above!!!
9/2/2019 L1.7
![Page 8: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/8.jpg)
Could they ask me questions
about CS 332 material on job interviews?
You bet.
9/2/2019 L1.8
![Page 9: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/9.jpg)
What is Theory of Computation?
• You’ve learned about computers and
programming
• Much of this knowledge is specific to particular
computing environment
9/2/2019 L1.10
![Page 10: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/10.jpg)
What is Theory of Computation?
• Theory– General ideas that apply to many systems
– Expressed simply, abstractly, precisely
• Abstraction suppresses inessential details
• Precision enables rigorous analysis
– Correctness proofs for algorithms and system designs
– Formal analysis of complexity
• Proof that there is no algorithm to solve some problem in
some setting (with certain cost)
9/2/2019 L1.11
![Page 11: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/11.jpg)
This course
• Theory basics– Models for machines
– Models for the problems machines can be used to solve
– Theorems about what kinds of machines can solve what kinds
of problems, and at what cost
– Theory needed for sequential single-processor computing
• Not covered:
– Parallel machines
– Distributed systems
– Quantum computation
– Sublinear computation
– Real-time systems
– Mobile computing
– Embedded systems
– …9/2/2019 L1.12
![Page 12: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/12.jpg)
Machine models
• Finite Automata (FAs): machines with fixed amount
of unstructured memory
– useful for modeling chips, communication protocols,
adventure games, some control systems, …
• Pushdown Automata (PDAs): FAs with unbounded
structured memory in the form of a pushdown stack
– useful for modeling parsing, compilers, some calculations
• Turing Machines (TMs): FAs with unbounded tape
– Model for general sequential computation (real computer).
– Equivalent to RAMs, various programming languages models
– Suggest general notion of computability
9/2/2019 L1.13
![Page 13: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/13.jpg)
Machine models
• Resource-bounded TMs (time and space bounded):
– “not that different” on different models: “within a polynomial
factor”
• Probabilistic TMs: extension of TMs that allows
random choices
Most of these models have nondeterministic variants:
can make nondeterministic “guesses”
9/2/2019 L1.14
![Page 14: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/14.jpg)
Problems solved by machines
1. What is a problem?
2. What does it mean for a machine to “solve”
a problem?
In this course, problem is a language.
A language is a set of strings over some “alphabet”
9/2/2019 L1.15
![Page 15: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/15.jpg)
Examples of languages
• L1= {binary representations of natural numbers divisible by 2}
• L2= {binary representations of primes}
• L3= {sequences of decimal numbers, separated by commas, that
can be divided into 2 groups with the same sum}
– (5,3,1,3) L3, (15,7,5,9,1) L3.
• L4= {C programs that loop forever on some input}
• L5= {representations of graphs containing a Hamiltonian cycle}
– {(1,2,3,4,5); (1,2),(1,3),(2,3),…}
1
2
3
45
alphabet = {0,1}
alphabet = {0,1,…,9,comma}
visits each node exactly once
vertices edges
alphabet = all symbols: digits, commas, parens
9/2/2019 L1.16
![Page 16: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/16.jpg)
Theorems about classes of languages
We will define classes of languages and prove
theorems about them:
• inclusion: Every language recognizable (i.e., solvable)
by a FA is also recognizable by a TM.
• non-inclusion: Not every language recognizable by a
TM is also recognizable by a FA.
• completeness: “Hardest” language in a class
• robustness: alternative characterizations of classes
– e.g., FA-recognizable languages by regular expressions
(UNIX)
9/2/2019 L1.17
![Page 17: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/17.jpg)
Why study theory of computation?
• a language for talking about program behavior
• feasibility (what can and cannot be done)
– halting problem, NP-completeness
• analyzing correctness and resource usage
• computationally hard problems are essential for
cryptography
• computation is fundamental to understanding the world
– cells, brains, social networks, physical systems all can be
viewed as computational devices
• IT IS FUN!!!
9/2/2019 L1.18
![Page 18: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/18.jpg)
Is it useful for developers?
Boss, I can’t find an efficient algorithm, because no such
algorithm is possible.
9/2/2019 L1.20Sofya Raskhodnikova; based on cartoon by Garey & Johnson, 1979
Boss, I can’t find an efficient algorithm. I guess I ‘m just too dumb.
![Page 19: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/19.jpg)
Parts of the course
9/2/2019
I. Automata Theory
II. Computability Theory
III.Complexity Theory
L1.21
![Page 20: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/20.jpg)
Finite automata (FA)
00,1
00
1
1
1
0111 0111
0111
01110111
0111 -- accepted
01110 -- rejected
9/2/2019
Each string is either accepted or rejected by the automaton
depending on whether it is in an accept state at the end.
L1.22
![Page 21: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/21.jpg)
Anatomy of finite automaton
9/2/2019
00,1
00
1
1
1states
states
q0
q1
q2
q3
start state (q0)
accept states
(F)
L1.23
![Page 22: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/22.jpg)
Formal Definition
9/2/2019
Q is the set of states
Σ is the alphabet
: Q Σ → Q is the transition function
q0 Q is the start state
F Q is the set of accept states
A finite automaton is a 5-tuple M = (Q, Σ, , q0, F)
L(M) = the language of machine M
= set of all (finite) strings machine M accepts
M recognizes the language L(M)L1.24
![Page 23: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/23.jpg)
Examples of FAs
9/2/2019
0,1q0
L(M) = {w | w is a string of 0s and 1s}
q0
L(M) = {ε} where ε denotes the empty string
0,1q10,1
L1.25
![Page 24: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/24.jpg)
Examples of FAs
9/2/2019
q0 q1
0 0
1
1
L(M) = {w | w has an even number of 1s}
L1.26
![Page 25: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/25.jpg)
1 0
1
q q00q0 q001
0 0 1
0,1
Build an automaton that accepts all (and only those)
strings that contain 001
9/2/2019
Examples of FAs
L1.27
![Page 26: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/26.jpg)
Formal definition of FA
L2.299/2/2019 Sofya Raskhodnikova; based on slides by Nick Hopper
Q = {q0, q1, q2, q3}
Σ = {0,1}
: Q Σ → Q transition function*q0 Q is start state
F = {q1, q2} Q accept states
M = (Q, Σ, , q0, F) where
M q2
00,1
00
1
1
1
q0
q1
q3
0 1
q0 q0 q1
q1 q2 q2
q2 q3 q2
q3 q0 q2
*
![Page 27: CS332 Elements of Theory of Computation › sofya › cs332 › lecture-notes › CS332-Lec01.pdfI. Automata Theory II. Computability Theory III.Complexity Theory L1.21. Finite automata](https://reader033.fdocuments.us/reader033/viewer/2022053012/5f0f86a07e708231d4449932/html5/thumbnails/27.jpg)
Language of FA
9/2/2019 L2.30Sofya Raskhodnikova; based on slides by Nick Hopper
L(M) = the language of machine M
= set of all strings machine M accepts
M recognizes the language L(M)