Post on 22-Aug-2020
IST 4Information and Logic
mon tue wed thr fri30 M1 1= todayT
6 M1
13 1 2M2x= hw#x out
oh oh
oh oh
= todayT
TM
20 227 M2
x= hw#x due
idt
oh oh
h h27 M2
4 311 3 4Mx= MQx out
midtermsoh
oh oh
oh = office hours oh
11 3 418 4 525
Mx= MQx out
Mx= MQx dueoh oh
ohoh
25
1 5oh
oh
oh
oh
Last lecture: Associate Memories, by Yue Li, y
A word that is associated with the following?with the following?
A word that is associated with the following?with the following?
MemoryMemoryand
ThoughtThought
MemoryInformatMemoryInformatand and
ThoughtLogicThoughtLogic
Funes the MemoriousWill be posted on the class website
Funes the MemoriousA short story by: Jorge Luis Borges
1899-19861899 1986
“With no effort, he had learned English French Portuguese andEnglish, French, Portuguese andLatin. I suspect, however, that he was not very capable of thought.”
“To think is to forget differences, generalize make abstractions Ingeneralize, make abstractions. Inthe teeming world of Funes, there were only details, almost immediate in th i ”their presence.”
MemoryMQ2• You are invited to write short essay on the topic of theM t Q ti
QDeadline Tuesday 4/28/2015 at 10pm
Magenta Question.• Recommended length is 3 pages (not more)• Submit the essay in PDF format to ta4@paradise.caltech.eduSubmit the essay in PDF format to ta4@paradise.caltech.edu
file name lastname-firstname.pdf• No collaboration. No extensionsGrading of MQ:3 points (out of 103)
50% for content quality, 50% for writing quality
Some students will be given an opportunityto give a short presentation for up to 3 additional points
Thursday 6/5/2014 2:30pm
Last year
1 W ll N G L
Thursday, 6/5/2014 2:30pm –
1. Willis Nguy - A Greater Loss
2. Sarah Brandsen - The Woman Who Cannot Forget
3. Angela Gui - The Persistence of Language
4. Jiyun Ivy Xiao - How I Trained My Memoryy y y y
5. Ankit Kumar - The Vedic Oral Tradition
6 Leon Ding - Chess Memory6. Leon Ding Chess Memory
7. Nancy Wen - Scent of a Memory
8 Grace Lee A Memory about Art 8. Grace Lee - A Memory about Art
Babylonian Clay Tablets G k P fGreek Proofs...
MemoryM moryof mathematical knowledge
A4
A4
A4
A4
A key idea inA key idea inInformation
fi it i lIs there a finite universal set of building blocks?
‘ thi ’Can construct ‘everything’
DNA ABCDEDNA ABCDE...123...
A key idea inA key idea inInformation
fi it i lIs there a finite universal set of building blocks?
‘ thi ’Can construct ‘everything’
squaressquaressquaressquaressquares
Squaring the RectangleSquaring the Rectangle
Can we find a smaller number of identical squares?
Can we find a smaller number of identical squares?
How?How?
Idea: Be greedy
How?How?
We are almost done!
N h ?Now what?F G t t N b ?From Geometry to Numbers?
Euclidean algorithm for finding the GCDGreatest Common DivisorGreatest Common Divisor
40
5
10
1515
5
1515
Euclid,300BC10
GCD = 5
,
Building BlocksSeparation
compute usin the rules of the syntaxcompute using the rules of the syntaxindependent of the semantics
l ithalgorithms
AlgorithmA procedure to build A procedure to build ‘everything’ from a
set of building blocks
b
set of building blocks
We need to remember•The building blocksg•The algorithms
12137+35823 What are the building blocks of addition?blocks of addition?
The ‘words’ of addition
0 1 2 3 4 5 6 7 8 9
0 0 1 2 3 4 5 6 7 8 9
1 1 2 3 4 5 6 7 8 9 102 2 3 4 5 6 7 8 9 10 112 2 3 4 5 6 7 8 9 10 113 3 4 5 6 7 8 9 10 11 124 4 5 6 7 8 9 10 11 12 135 5 6 7 8 9 10 11 12 13 145 5 6 7 8 9 10 11 12 13 146 6 7 8 9 10 11 12 13 14 157 7 8 9 10 11 12 13 14 15 168 8 9 10 11 12 13 14 15 16 178 8 9 10 11 12 13 14 15 16 179 9 10 11 12 13 14 15 16 17 18
There is a trade-off between There is a trade off between
th si f b ildi bl ks memoriesthe size of building blocks: memories
d th l it f th l ith and the complexity of the algorithm: composition of the memoriesp
Remembering a large memory h f with a composition of small memories
Syntax BoxesSyntax BoxesBIG f llBIG from small
Syntax Boxes (s-box)0 1 2 3 4 5 6 7 8 9
0 0 1 2 3 4 5 6 7 8 9
a b
inputs 1 1 2 3 4 5 6 7 8 9 102 2 3 4 5 6 7 8 9 10 113 3 4 5 6 7 8 9 10 11 12
S B
inputs
4 4 5 6 7 8 9 10 11 12 135 5 6 7 8 9 10 11 12 13 146 6 7 8 9 10 11 12 13 14 15
S-Box
7 7 8 9 10 11 12 13 14 15 168 8 9 10 11 12 13 14 15 16 179 9 10 11 12 13 14 15 16 17 18ooutputsp
Binary Syntax Boxes (s-box)
ba b
inputs
a b o
S-Box
outputso
p
Binary Syntax Boxes (s-box)
b0 0
inputs
a b o
0 0 0
outputs0
p
Binary Syntax Boxes (s-box)
b0 1
inputs
a b o
0 0 010 1
outputs1
p
Binary Syntax Boxes (s-box)
b1 0
inputs
a b o
0 0 010
1 011
outputs1
p
Suggest a name for this memory box?
b1 1
inputs
a b o
0 0 010
1 011
11111 1
outputs1
p
Suggest a name for this memory box?
b
o = max (a,b)Can we remember max (a,b,c) with max(a,b)?
a ba b o
0 0 010
1 01 1
1111 1 1
o
o = max (a,b)
o = max (a,b)Can we remember max (a,b,c) with max(a,b)?
a b Composition:build big s-boxes
c
build big s boxes from small s-boxes
c
o = max (a,b,c)
How to compute the following (XOR)?
Can we remember wwith the max s-box? b
o = max (a,b)
a b
with the max s box?
?a b o
0 0 0
a b w
0 0 0
101 01 1
1110 0
101 0
011
1 1 1
1 011
10
o
o = max (a,b)NO Why not?
Composition:Can we remember wwith the max s-box?
a b?
pbuild big s-boxes from small s-boxes
with the max s box?
a b
0 0 0
The output of every small s-box is bigger or equal to its inputs
w
Big s-box0 0
101 0
011
or equal to its inputs
1 011
10 The output of the big
s-box must be bigger l t it i t
w
or equal to its inputsNO Why not?
A Magic (Universal) Boxg ( )
A binary s box that can A binary s-box that can compute any binary s-box?
a ba b m
p y y
a b m
0 010
11 M10
1 011
110
M
11 0m
A Magic Boxg
Can you compute the min(x,y)
Can you compute the following with the magic box?
a ba b m x y oa b m
0 010
11 M
x y
0 010
o
0010
1 011
110
M 101 0
11
00111 0
m11 1
A Magic BoxHint 1: g
Can you compute the
Hint 1: min(x,y)
Can you compute the following with the magic box?
a ba b m x y oa b m
0 010
11 M
x y
0 010
o
0010
1 011
110
M 101 0
11
00111 0
m11 1
A Magic BoxHint 2: g
Can you compute the
Hint 2: min(x,y)
Can you compute the following with the magic box?
a ba b m x y oa b m
0 010
11 M
x y
0 010
o
0010
1 011
110
M 101 0
11
00111 0
m11 1
A Magic Boxg
x y
min(x,y)
x y
a b m
M
x y o1a b m
0 010
11
x y
0 010
o
00
1
101 0
11
110
101 0
11
001M11 0 11 1
o0
A Magic Boxg
x y
min(x,y)
x y
a b m
M
x y o0a b m
0 010
11
x y
0 010
o
00
0
101 0
11
110
101 0
11
001M11 0 11 1
o1
A Magic BoxHINT
g
Can you compute the
HINT max(x,y)
Can you compute the following with the m-box?
a ba b m x y oa b m
0 010
11 M
x y
0 010
o
0110
1 011
110
M 101 0
11
11111 0
m11 1
A Magic Boxg
x y
max(x,y)
x y
a b m x y o
MM
a b m
0 010
11
x y
0 010
o
0110
1 011
110
101 0
11
111M11 0 11 1
o
A Magic Boxg
x y
max(x,y)
x y0 0
a b m x y o
MM
a b m
0 010
11
x y
0 010
o
0110
1 011
110
101 0
11
111M
11
11 0 11 1
o0
A Magic Boxg
x y
max(x,y)
x y0 1
a b m x y o
MM
a b m
0 010
11
x y
0 010
o
0110
1 011
110
101 0
11
111M
01
11 0 11 1
o1
A Magic Boxg
x y
max(x,y)
x y1 1
a b m x y o
MM
a b m
0 010
11
x y
0 010
o
0110
1 011
110
101 0
11
111M
00
11 0 11 1
o1
m-Box: A two input binary syntax nbox that can compute any (two
input) binary syntax box?input) binary syntax box?How many differentbinary 2 input How will you prove it?
a ba b m x y o
binary 2-input s-boxes?
How will you prove it?24 = 16
a b m
0 010
11
x y
0 010
o
**10
1 011
110
M 101 0
11
***11 0
m
11
S nt x B xSyntax Boxesproof of universalityproof of universality
4 Useful Boxes
min(x,y) 1-y 1
x y o
0 0 1
x y o
0 0 1 0 010
1 0
111
0 010
1 0
101
max(x,y)
1 011
11
1 011
10
1-y
yy
a b m x y oa b m
0 010
11
x y o
0 010
10
M
101 0
11
110
101 0
11
010
o
11 0 11 0
1
y 1-y
a b m x y oa b m
0 010
11
x y o
0 010
11
M
101 0
11
110
101 0
11
111
o
11 0 11 1
y
M
a b m x y oy 1-y
a b m
0 010
11
x y o
0 010
11
M
101 0
11
110
101 0
11
111
o
111 0 11 11
4 Useful Boxes
min(x,y) 1-y 1
( ) So what?max(x,y)
Need to prove:So what?
Need to prove:any (t i t) bin nt x any (two input) binary syntax box can be computed by box can be computed by the 4 Useful Boxes
An Arbitrary Two Input Box
x y oTwo 1-input boxes!
0 010
**
1 011
**
x= 0 then
x= 1 then x= 1 then
What are the possible values of p
00
01
10
110 1 0 1
An Arbitrary Two Input Box
min(x,y) 1-y 1
( )max(x,y)
y
What are the possible values of
y
x y op
00
01
10
11
0 010
**0 1 0 1
Can we compute it with the m-box?1 0
11**
A Selector Box
x y o
0 0 *
x= 0 then
1 th 0 010
1 0
***
x= 1 then How can you compute this box?
1 011 *
selector x
o
x= 0 then A Selector Boxx= 1 then
1-x x1 x x
min(a,b) min(a,b)
max(a,b)
o
x= 0 then A Selector Boxx= 1 then
0 10 1
min(a,b) min(a,b)
0
max(a,b)
0
o
x= 0 then A Selector Boxx= 1 then
1 01 0
min(a,b) min(a,b)
0
max(a,b)
0
o
QED
How to compute the following (XOR)?
x y o
0 0 0
x= 0 then
1 th 0 010
1 0
011
x= 1 then
1 011
10??
selector x
o
How to compute the following (XOR)?
x y o
0 0 0
x= 0 then
1 th 0 010
1 0
011
x= 1 then
1 011
101-yy
selector x
o
y
x y o
0 0 0
x= 0 then
1 th 0 010
1 0
011
x= 1 then
1 011
10
1 yy
M
selector x
1-yy
o
Does the magic continue?D m g u
Given a 2-input binary box that can compute any 2-input binary box
Can it compute any 3-input binary box?
a b x zy
M
o o
3-input binary s-box
x y ozHow many differentbinary 3-input s boxes?
0 010
**
00
s-boxes?
28 = 2561 0
11**
0 0 *
0010 0
101 0
***
1111 0
11**
11
3-input binary s-box
x y ozTwo 2-input boxes!
0 010
**
00
1 011
**
0 0 *
001
z= 0 then
z= 1 then 0 010
1 0
***
111
z= 1 then
1 011
**
11
x y o
0 0 *
z
0x y x yAll boxes can be computed with M
0 010
1 0**
000
11 *0 0 *
01
101 0
**
11
11 *1
selectorselector z
z= 0 then o
z 0 then
z= 1 then
Does the magic continue?
G l b f 2 b b
D m g u
Given a magical box for any 2-input binary box
We proved that it is magical for any 3-input binary box!p g f y p y
Is it magical for any n-input binary box????YES!!!!Proof by induction on the number of inputs
a bProof by induction on the number of inputs
M
....
o o
x y x yAll boxes can be computed with M
........
(n-1) inputs (n-1) inputs
selectorselector z
z= 0 then o
z 0 then
z= 1 then
Questions about algorithmsalgorithms
and building blocks?Feasibility
Given a set of building blocks: What can/cannot be constructed?
Efficiency and complexity
Size: If feasible how many blocks are needed?
Time: How long will it take to complete the construction?
Size: If feasible, how many blocks are needed?
A word that is associated with the following?with the following?
A word that is associated with the following?with the following?
FaceFace
A word that is associated with the following?with the following?
FaceFace
You have one week!You have one week!
Diff i i ti 7 di it b 60Difference in approximation - 7 digits base 60