IST 4Information and Logic

mon tue wed thr fri30 M1 1= todayT

6 M1

13 1 2M2x= hw#x out

oh oh

oh oh

= todayT

TM

20 227 M2

x= hw#x due

idt

oh oh

h h27 M2

4 311 3 4Mx= MQx out

midtermsoh

oh oh

oh = office hours oh

11 3 418 4 525

Mx= MQx out

Mx= MQx dueoh oh

ohoh

25

1 5oh

oh

oh

oh

Last lecture: Associate Memories, by Yue Li, y

A word that is associated with the following?with the following?

A word that is associated with the following?with the following?

MemoryMemoryand

ThoughtThought

MemoryInformatMemoryInformatand and

ThoughtLogicThoughtLogic

Funes the MemoriousWill be posted on the class website

Funes the MemoriousA short story by: Jorge Luis Borges

1899-19861899 1986

“With no effort, he had learned English French Portuguese andEnglish, French, Portuguese andLatin. I suspect, however, that he was not very capable of thought.”

“To think is to forget differences, generalize make abstractions Ingeneralize, make abstractions. Inthe teeming world of Funes, there were only details, almost immediate in th i ”their presence.”

MemoryMQ2• You are invited to write short essay on the topic of theM t Q ti

QDeadline Tuesday 4/28/2015 at 10pm

Magenta Question.• Recommended length is 3 pages (not more)• Submit the essay in PDF format to ta4@paradise.caltech.eduSubmit the essay in PDF format to ta4@paradise.caltech.edu

file name lastname-firstname.pdf• No collaboration. No extensionsGrading of MQ:3 points (out of 103)

50% for content quality, 50% for writing quality

Some students will be given an opportunityto give a short presentation for up to 3 additional points

Thursday 6/5/2014 2:30pm

Last year

1 W ll N G L

Thursday, 6/5/2014 2:30pm –

1. Willis Nguy - A Greater Loss

2. Sarah Brandsen - The Woman Who Cannot Forget

3. Angela Gui - The Persistence of Language

4. Jiyun Ivy Xiao - How I Trained My Memoryy y y y

5. Ankit Kumar - The Vedic Oral Tradition

6 Leon Ding - Chess Memory6. Leon Ding Chess Memory

7. Nancy Wen - Scent of a Memory

8 Grace Lee A Memory about Art 8. Grace Lee - A Memory about Art

Babylonian Clay Tablets G k P fGreek Proofs...

MemoryM moryof mathematical knowledge

A4

A4

A4

A4

A key idea inA key idea inInformation

fi it i lIs there a finite universal set of building blocks?

‘ thi ’Can construct ‘everything’

DNA ABCDEDNA ABCDE...123...

A key idea inA key idea inInformation

fi it i lIs there a finite universal set of building blocks?

‘ thi ’Can construct ‘everything’

squaressquaressquaressquaressquares

Squaring the RectangleSquaring the Rectangle

Can we find a smaller number of identical squares?

Can we find a smaller number of identical squares?

How?How?

Idea: Be greedy

How?How?

We are almost done!

N h ?Now what?F G t t N b ?From Geometry to Numbers?

Euclidean algorithm for finding the GCDGreatest Common DivisorGreatest Common Divisor

40

5

10

1515

5

1515

Euclid,300BC10

GCD = 5

,

Building BlocksSeparation

compute usin the rules of the syntaxcompute using the rules of the syntaxindependent of the semantics

l ithalgorithms

AlgorithmA procedure to build A procedure to build ‘everything’ from a

set of building blocks

b

set of building blocks

We need to remember•The building blocksg•The algorithms

12137+35823 What are the building blocks of addition?blocks of addition?

The ‘words’ of addition

0 1 2 3 4 5 6 7 8 9

0 0 1 2 3 4 5 6 7 8 9

1 1 2 3 4 5 6 7 8 9 102 2 3 4 5 6 7 8 9 10 112 2 3 4 5 6 7 8 9 10 113 3 4 5 6 7 8 9 10 11 124 4 5 6 7 8 9 10 11 12 135 5 6 7 8 9 10 11 12 13 145 5 6 7 8 9 10 11 12 13 146 6 7 8 9 10 11 12 13 14 157 7 8 9 10 11 12 13 14 15 168 8 9 10 11 12 13 14 15 16 178 8 9 10 11 12 13 14 15 16 179 9 10 11 12 13 14 15 16 17 18

There is a trade-off between There is a trade off between

th si f b ildi bl ks memoriesthe size of building blocks: memories

d th l it f th l ith and the complexity of the algorithm: composition of the memoriesp

Remembering a large memory h f with a composition of small memories

Syntax BoxesSyntax BoxesBIG f llBIG from small

Syntax Boxes (s-box)0 1 2 3 4 5 6 7 8 9

0 0 1 2 3 4 5 6 7 8 9

a b

inputs 1 1 2 3 4 5 6 7 8 9 102 2 3 4 5 6 7 8 9 10 113 3 4 5 6 7 8 9 10 11 12

S B

inputs

4 4 5 6 7 8 9 10 11 12 135 5 6 7 8 9 10 11 12 13 146 6 7 8 9 10 11 12 13 14 15

S-Box

7 7 8 9 10 11 12 13 14 15 168 8 9 10 11 12 13 14 15 16 179 9 10 11 12 13 14 15 16 17 18ooutputsp

Binary Syntax Boxes (s-box)

ba b

inputs

a b o

S-Box

outputso

p

Binary Syntax Boxes (s-box)

b0 0

inputs

a b o

0 0 0

outputs0

p

Binary Syntax Boxes (s-box)

b0 1

inputs

a b o

0 0 010 1

outputs1

p

Binary Syntax Boxes (s-box)

b1 0

inputs

a b o

0 0 010

1 011

outputs1

p

Suggest a name for this memory box?

b1 1

inputs

a b o

0 0 010

1 011

11111 1

outputs1

p

Suggest a name for this memory box?

b

o = max (a,b)Can we remember max (a,b,c) with max(a,b)?

a ba b o

0 0 010

1 01 1

1111 1 1

o

o = max (a,b)

o = max (a,b)Can we remember max (a,b,c) with max(a,b)?

a b Composition:build big s-boxes

c

build big s boxes from small s-boxes

c

o = max (a,b,c)

How to compute the following (XOR)?

Can we remember wwith the max s-box? b

o = max (a,b)

a b

with the max s box?

?a b o

0 0 0

a b w

0 0 0

101 01 1

1110 0

101 0

011

1 1 1

1 011

10

o

o = max (a,b)NO Why not?

Composition:Can we remember wwith the max s-box?

a b?

pbuild big s-boxes from small s-boxes

with the max s box?

a b

0 0 0

The output of every small s-box is bigger or equal to its inputs

w

Big s-box0 0

101 0

011

or equal to its inputs

1 011

10 The output of the big

s-box must be bigger l t it i t

w

or equal to its inputsNO Why not?

A Magic (Universal) Boxg ( )

A binary s box that can A binary s-box that can compute any binary s-box?

a ba b m

p y y

a b m

0 010

11 M10

1 011

110

M

11 0m

A Magic Boxg

Can you compute the min(x,y)

Can you compute the following with the magic box?

a ba b m x y oa b m

0 010

11 M

x y

0 010

o

0010

1 011

110

M 101 0

11

00111 0

m11 1

A Magic BoxHint 1: g

Can you compute the

Hint 1: min(x,y)

Can you compute the following with the magic box?

a ba b m x y oa b m

0 010

11 M

x y

0 010

o

0010

1 011

110

M 101 0

11

00111 0

m11 1

A Magic BoxHint 2: g

Can you compute the

Hint 2: min(x,y)

Can you compute the following with the magic box?

a ba b m x y oa b m

0 010

11 M

x y

0 010

o

0010

1 011

110

M 101 0

11

00111 0

m11 1

A Magic Boxg

x y

min(x,y)

x y

a b m

M

x y o1a b m

0 010

11

x y

0 010

o

00

1

101 0

11

110

101 0

11

001M11 0 11 1

o0

A Magic Boxg

x y

min(x,y)

x y

a b m

M

x y o0a b m

0 010

11

x y

0 010

o

00

0

101 0

11

110

101 0

11

001M11 0 11 1

o1

A Magic BoxHINT

g

Can you compute the

HINT max(x,y)

Can you compute the following with the m-box?

a ba b m x y oa b m

0 010

11 M

x y

0 010

o

0110

1 011

110

M 101 0

11

11111 0

m11 1

A Magic Boxg

x y

max(x,y)

x y

a b m x y o

MM

a b m

0 010

11

x y

0 010

o

0110

1 011

110

101 0

11

111M11 0 11 1

o

A Magic Boxg

x y

max(x,y)

x y0 0

a b m x y o

MM

a b m

0 010

11

x y

0 010

o

0110

1 011

110

101 0

11

111M

11

11 0 11 1

o0

A Magic Boxg

x y

max(x,y)

x y0 1

a b m x y o

MM

a b m

0 010

11

x y

0 010

o

0110

1 011

110

101 0

11

111M

01

11 0 11 1

o1

A Magic Boxg

x y

max(x,y)

x y1 1

a b m x y o

MM

a b m

0 010

11

x y

0 010

o

0110

1 011

110

101 0

11

111M

00

11 0 11 1

o1

m-Box: A two input binary syntax nbox that can compute any (two

input) binary syntax box?input) binary syntax box?How many differentbinary 2 input How will you prove it?

a ba b m x y o

binary 2-input s-boxes?

How will you prove it?24 = 16

a b m

0 010

11

x y

0 010

o

**10

1 011

110

M 101 0

11

***11 0

m

11

S nt x B xSyntax Boxesproof of universalityproof of universality

4 Useful Boxes

min(x,y) 1-y 1

x y o

0 0 1

x y o

0 0 1 0 010

1 0

111

0 010

1 0

101

max(x,y)

1 011

11

1 011

10

1-y

yy

a b m x y oa b m

0 010

11

x y o

0 010

10

M

101 0

11

110

101 0

11

010

o

11 0 11 0

1

y 1-y

a b m x y oa b m

0 010

11

x y o

0 010

11

M

101 0

11

110

101 0

11

111

o

11 0 11 1

y

M

a b m x y oy 1-y

a b m

0 010

11

x y o

0 010

11

M

101 0

11

110

101 0

11

111

o

111 0 11 11

4 Useful Boxes

min(x,y) 1-y 1

( ) So what?max(x,y)

Need to prove:So what?

Need to prove:any (t i t) bin nt x any (two input) binary syntax box can be computed by box can be computed by the 4 Useful Boxes

An Arbitrary Two Input Box

x y oTwo 1-input boxes!

0 010

**

1 011

**

x= 0 then

x= 1 then x= 1 then

What are the possible values of p

00

01

10

110 1 0 1

An Arbitrary Two Input Box

min(x,y) 1-y 1

( )max(x,y)

y

What are the possible values of

y

x y op

00

01

10

11

0 010

**0 1 0 1

Can we compute it with the m-box?1 0

11**

A Selector Box

x y o

0 0 *

x= 0 then

1 th 0 010

1 0

***

x= 1 then How can you compute this box?

1 011 *

selector x

o

x= 0 then A Selector Boxx= 1 then

1-x x1 x x

min(a,b) min(a,b)

max(a,b)

o

x= 0 then A Selector Boxx= 1 then

0 10 1

min(a,b) min(a,b)

0

max(a,b)

0

o

x= 0 then A Selector Boxx= 1 then

1 01 0

min(a,b) min(a,b)

0

max(a,b)

0

o

QED

How to compute the following (XOR)?

x y o

0 0 0

x= 0 then

1 th 0 010

1 0

011

x= 1 then

1 011

10??

selector x

o

How to compute the following (XOR)?

x y o

0 0 0

x= 0 then

1 th 0 010

1 0

011

x= 1 then

1 011

101-yy

selector x

o

y

x y o

0 0 0

x= 0 then

1 th 0 010

1 0

011

x= 1 then

1 011

10

1 yy

M

selector x

1-yy

o

Does the magic continue?D m g u

Given a 2-input binary box that can compute any 2-input binary box

Can it compute any 3-input binary box?

a b x zy

M

o o

3-input binary s-box

x y ozHow many differentbinary 3-input s boxes?

0 010

**

00

s-boxes?

28 = 2561 0

11**

0 0 *

0010 0

101 0

***

1111 0

11**

11

3-input binary s-box

x y ozTwo 2-input boxes!

0 010

**

00

1 011

**

0 0 *

001

z= 0 then

z= 1 then 0 010

1 0

***

111

z= 1 then

1 011

**

11

x y o

0 0 *

z

0x y x yAll boxes can be computed with M

0 010

1 0**

000

11 *0 0 *

01

101 0

**

11

11 *1

selectorselector z

z= 0 then o

z 0 then

z= 1 then

Does the magic continue?

G l b f 2 b b

D m g u

Given a magical box for any 2-input binary box

We proved that it is magical for any 3-input binary box!p g f y p y

Is it magical for any n-input binary box????YES!!!!Proof by induction on the number of inputs

a bProof by induction on the number of inputs

M

....

o o

x y x yAll boxes can be computed with M

........

(n-1) inputs (n-1) inputs

selectorselector z

z= 0 then o

z 0 then

z= 1 then

Questions about algorithmsalgorithms

and building blocks?Feasibility

Given a set of building blocks: What can/cannot be constructed?

Efficiency and complexity

Size: If feasible how many blocks are needed?

Time: How long will it take to complete the construction?

Size: If feasible, how many blocks are needed?

A word that is associated with the following?with the following?

A word that is associated with the following?with the following?

FaceFace

A word that is associated with the following?with the following?

FaceFace

You have one week!You have one week!

Diff i i ti 7 di it b 60Difference in approximation - 7 digits base 60