IST 4 Information and Logic - California Institute of ...vigeland.caltech.edu/ist4/lectures/lect...

IST 4 Information and Logic

mon tue wed thr fri 3 M1 1 10 M1

17 1 2 M2

24 2

1 M2 8 3 15 3 4 22 4 5 29

5 5

x= hw#x out

x= hw#x due

Mx= MQx out

Mx= MQx due

midterms

oh oh

oh oh

oh oh

oh

oh

oh oh

oh

oh

oh

oh

oh

oh

oh

oh = office hours oh

T

= today T oh

oh

oh

oh

sun

Boolean Proofs: Axioms + Fun

A1-A4

L1 T1

T2

T3

L2

T4

T0

Proof Dependency

Review of Quiz #6 What is the complement of

Use the following DeMorgan Laws:

And the axioms:

What is the complement of

DeMorgan Laws:

And the axioms:

DM

DM

A4

A4 A3

A2 A3

A1

Syllogism to Algebra George Boole, 1847

George Boole 1815 –1864

George Boole Early Days

George Boole 1815-1864

Born in Lincoln, England an industrial town

His father was a shoemaker with a passion for mathematics and science

When George was 8 he surpassed his father’s knowledge in mathematics

By age 14 he was fluent in Latin, German, French, Italian and English… and algebra…

When his was 15 he had to go to work to support his family, he became a math teacher in the Wesleyan Methodist academy in Doncaster (40 miles away…)

Lost his job after two years….

Lost two more teaching jobs…

When he was 20 he opened his own school in his hometown - Lincoln

source: wikipedia



In 1841 (26) he published three papers in the newly established Cambridge Mathematical Journal

In 1844 (29) he published “On a General Method of Analysis” The paper won the first (newly established) Gold medal for Mathematics awarded by Royal Society

In 1846 (31) he applied for a professor position in the newly established Queen’s College – 3 campuses in Ireland

In 1847 (32) ‘while waiting to hear from Ireland’, he published “The Mathematical Analysis of Logic”

George Boole Early Career



In 1847 (32) ‘while waiting to hear from Ireland’, he published “The Mathematical Analysis of Logic”

In 1849 (34), his was offered a position of the first professor of mathematics at Queen’s college at Cork

He married Mary Everest (1832- 1916) in 1855 (23,40) and they had five daughters Niece of George Everest (Mt. Everest…) led the expedition to map the Himalayas

8848 m 29,029 ft George Boole Ireland

source: wikipedia In 1864, died of pneumonia (49)

The heritage continues…

Mary Ellen Margaret

Mary Everest Boole George Boole

Ethel Alicia Lucy

Geoffrey Everest Hinton

U of Toronto Google Brain Team

(deep learning)

great-great-grandson


1849-1864, taught at at Queen’s college in Cork

George Boole Geography

source: wikipedia

Cork, Ireland Source: www.flickr.com

1849-1855: lived here until got married

Boole was never a student at a university… only a professor…

Leibniz never held an academic position…


Gottfried Leibniz 1646-1716

It is all about Algorizmi 780-850AD Gottfried Leibniz

1646-1716


??? Binary

Boolean

Shannon 1916-2001

Boolean Algebra Boolean is not Binary...

Two-valued Boolean Algebra

Boolean Algebra: set of elements B={0,1}, two binary operations OR and AND

xy OR(x,y) 00 01 10 11

0 1 1 1

xy AND(x,y) 00 01 10 11

0 0 0 1

0 iff both x and y are 0 1 iff both x and y are 1

We proved it is a Boolean algebra

Four-valued Boolean Algebra

Two-valued Boolean Algebra: set of elements B={0,1}, two binary operations OR and AND

Four-valued Boolean Algebra: set of elements ?? two binary operations ?? and ??

Elements:

0-1 vectors:

(10) (00) (11) (01)

operations?

+

Elements are: (00), (11), (10), (01)

00 01 10 11

00

01

10

11

01

01

11

11

10 11

11 11

10 11

11 11

00

01

10

11

+

Elements are: (00), (11), (10), (01)

00 01 10 11

00

01

10

11

01

01

11

11

10 11

11 11

10 11

11 11

00

01

10

11

Bitwise OR

+

Elements are: (00), (11), (10), (01)

00 01 10 11

00

01

10

11

01

01

11

11

10 11

11 11

10 11

11 11

00

01

10

11

Bitwise OR xy OR(x,y) 00 01 10 11

0 1 1 1

Elements are: (00), (11), (10), (01)

00 01 10 11

00

00

00

00

00

01

00

01

00 00

00 01

10 10

10 11

00

01

10

11

Elements are: (00), (11), (10), (01)

00 01 10 11

00

00

00

00

00

01

00

01

00 00

00 01

10 10

10 11

00

01

10

11

Bitwise AND

Elements are: (00), (11), (10), (01)

00 01 10 11

00

00

00

00

00

01

00

01

00 00

00 01

10 10

10 11

00

01

10

11

Bitwise AND xy AND(x,y) 00 01 10 11

0 0 0 1

Elements:

Operations:

(10) (00) (11) (01)

Bitwise AND Bitwise OR

Is it a Boolean algebra?

0 1

Bitwise Complement

+ 00 01 10 11

00

01

10

11

01

01

11

11

10 11

11 11

10 11

11 11

00

01

10

11

00 01 10 11

00

00

00

00

00

01

00

01

00 00

00 01

10 10

10 11

00

01

10

11

+ 00 01 10 11

00

01

10

11

01

01

11

11

10 11

11 11

10 11

11 11

00

01

10

11

00 01 10 11

00

00

00

00

00

01

00

01

00 00

00 01

10 10

10 11

00

01

10

11

We can prove it!

Elements:

Operations:

(10) (00) (11) (01)


It is a Boolean algebra!

0 1

Bitwise Complement

In general: 0-1 vectors a Boolean algebra?

Elements:

Operations:

(001) (000) (111) (010)


0 1

Bitwise Complement

True for a two-valued Boolean algebra

True for any Boolean algebra with 0-1 vectors and bitwise OR and AND??

0-1 vectors Boolean algebra

(011) (100) (101) (110)

n=3

n=1

arbitrary finite n

|s|

Idea 1: Any identity that is true for 0-1 is true for 0-1 vectors with bitwise OR/AND

Idea 2: Axioms are identities True for 0-1 à true for 0-1 vectors

The 0-1 Theorem 0-1 Theorem:

An identity is true for 0-1 vectors with bitwise OR and AND if and only if it is true for two valued (0-1) with bitwise OR and AND

Proof: The easy direction

Assume an identity true for 0-1 vectors

True for 0-1

The 0-1 Theorem 0-1 Theorem:

Proof: The non-obvious direction

Assume an identity true for 0-1

Need to prove true for any 0-1 vectors

An identity is true for 0-1 vectors with bitwise OR and AND if and only if it is true for two valued (0-1) with bitwise OR and AND

Example:

Theorem 2:

Proof:

0 + 0 ⋅ 0 = 00 + 0 ⋅1 = 01 + 1 ⋅ 0 = 11 + 1 ⋅1 = 1

The identity is true for 0-1

Need to prove it for 0-1 vectors

Theorem 2: Proof (for 0-1 vectors): If an identity is not true in general; then there is an assignment of elements that violates the equality

Hence, there must be a position in the binary vector that is violated

There exists a 0-1 assignment to the identity that violates the equality, CONTRADICTION!!

By contradiction Assume true for all 0-1 assignments and not true for some other assignment

Example:

Recap: The 0-1 Theorem An identity is true for 0-1 vectors with bitwise OR/AND if and only if it is true for two valued (0-1) with OR/AND

Proof: The easy direction •  Assume an identity true for 0-1 vectors •  0-1 is a special case: True for 0-1

Q The non-obvious direction •  Assume an identity true for 0-1

•  Need to prove true for 0-1 vectors

•  Assume there exists a general ‘identity violating’ assignment •  Show that there is a 0-1 ‘identity violating’ assignment

CONTRADICTION!!

Operations: Bitwise AND Bitwise OR Bitwise

Complement

It is a Boolean algebra for any finite n

0-1 vectors of length n, there are 2n vectors Elements:

Application of the the 0-1 Theorem

It is a Boolean algebra with for n=1

By the 0-1 Theorem:

Boolean Algebra Two-value? Or not?

Prove or Disprove

At least one of the following is true:

Prove or Disprove


xy OR(x,y) 00 01 10 11

0 1 1 1

True for two-valued Boolean algebra

Is it true for 0-1 vectors Boolean algebras?

+ 00 01 10 11

00

01

10

11

01

01

11

11

10 11

11 11

10 11

11 11

00

01

10

11

Prove or Disprove


Is it true for 0-1 vectors Boolean algebras?

NO

If

Then or

Claim is NOT TRUE in general!

The 0-1 Theorem: True only for identities!!!

+ 00 01 10 11

00

01

10

11

01

01

11

11

10 11

11 11

10 11

11 11

00

01

10

11

Examples ‘other’ Boolean Algebras

•  0-1 vectors

•  Algebra of subsets

•  Arithmetic Boolean algebras

next

next

Boolean algebra subsets of a set |s|

Algebra of Subsets S is the set of all points

Elements: all possible subsets of a set S How many elements? 2|s| 8

Algebra of Subsets S is the set of all points

Elements: all possible subsets of a set S

+ is union of sets is intersection of sets

A boring example:

Elements:

S =

Operations: union and intersection

Complement:

Elements:

Corresponding 0-1 vectors:

(10) (00) (11) (01)

S =

Algebra of subsets is isomorphic to the algebra of 0-1 vectors, it is a Boolean algebra!

Is the algebra of subsets a Boolean Algebra? YES

syntax the same – different semantics

01 11 00 10

00

10

01

11

Elements are: (00), (11), (10), (01)

01 11 00 10

00

10

01

11

Elements are: (00), (11), (10), (01)

00 10 01 11

10

11

01

10 11 11

11

11 11 11

11 01

Union Bitwise OR

Elements are: (00), (11), (10), (01)

01 11 00 10

00

10

01

11

00 00 00 00

00

00

00

10 00 10

00

10 01 11

01 01

Intersection Bitwise AND

(110) (011) (111)

(110) (011) (010)

Boolean algebra Boolean integers |s|

Euclid, 300BC

405 297 27

884 68 612

Interesting triples??

Euclid, 300BC

405 = 3x3x3x3x5

297 = 3x3x3x11 27 = 3x3x3

884 = 2x2x13x17 68 = 2 x 2 x 17

612 = 2x2x3x3x17

Greatest Common Divisor

Application: Simplifying fractions

Euclid, 300BC

405 = 3x3x3x3x5

297 = 3x3x3x11 4455 = 3x3x3x3x5x11

884 = 2x2x13x17 7956 = 2x2x3x3x13x17

612 = 2x2x3x3x17

Least Common Multiple

Application: Adding fractions

Euclid, 300BC

297x405 = 120,285

gcd(297,405) = 27 lcm(297,405) = 4455

Least Common Multiple

Greatest Common Divisor

27x4455 = 120,285

gcd(a,b) x lcm(a,b) = a x b

405 = 3x3x3x3x5

297 = 3x3x3x11

405 = 3x3x3x3x5

297 = 3x3x3x11

A fun fact:

Arithmetic Boolean Algebra

lcm = lowest common multiple gcd = greatest common divisor

The set of elements: {1,2,3,6}

The operations: lcm and gcd

1 is Boolean 0 6 is Boolean 1


What is the complement?

1 2

3 6







Elements: Set of all the divisors of an integer n

Is it a Boolean algebra?

Is it a Boolean Algebra?

6 = 3x2

3 = 3

2 = 2

1 = 1

11

00

10

01

LCM GCD

11 01

Bitwise OR Bitwise AND

YES

For which n does it work?

The complement?




1 2

4 8




For which n does it work?

Elements: Set of all the divisors of an integer n

Prime factors appear at most once in n

Square-free integers, Boolean integers

Bunitskiy Algebra 1899

Elements:

The set of divisors of a Boolean integer

{1,2,3,5,6,10,15,30}


The 0 and 1 elements: 1 and 30

Boolean Integers 2 x 3 x 5 = 30 2 x 3 x 7 = 42

Every prime in the prime factorization is a power of one (square-free integer)

Euclid, 300BC George Boole 1815 –1864

+ 00 01 10 11

00

01

10

11

01

01

11

11

10 11

11 11

10 11

11 11

00

01

10

11

00 01 10 11

00

00

00

00

00

01

00

01

00 00

00 01

10 10

10 11

00

01

10

11

1 2 3 6

1

1 2 3 6

1 2 3 6

2 2 6 6

3 6 3 6

6 6 6 6 1 2 3 6

1 1 3 3

1 2 1 2

1 1 1 1 1

2

3

6

2

3

6

lcm gcd

Bunitskiy algebra is isomorphic to the algebra of 0-1 vectors, it is a Boolean algebra!

syntax the same – different semantics

Boolean algebra an amazing theorem

Examples of Boolean Algebras

•  0-1 (two valued) Boolean algebra OR / AND •  0-1 vectors bitwise OR / bitwise AND •  Algebra of subsets union / intersection •  Arithmetic Boolean algebras lcm / gcd

Size 2k They are isomorphic!

Representation Theorem (Stone 1936): Every finite Boolean algebra is isomorphic to a Boolean algebra of 0-1 vectors

Algebra 1 Algebra 2

elements elements

operations operations

Representation Theorem (Stone 1936): Every finite Boolean algebra is isomorphic to a Boolean algebra with elements being bit vectors of finite length with bitwise operations OR and AND

Two Boolean algebras with m elements are isomorphic

Provides intuition beyond the axioms: We can ‘naturally’ reason about results in Boolean algebra

Every Boolean algebra has 2 elements k

Marshall Stone Marshall Stone

1903-1989

Proved in 1936 90AB = years After Boole The Boolean Syntax invented in 1847 has a unique representative semantic!!!

Harlan Fiske Stone 12th Chief Justice of the US 1941-1946

Marshall entered Harvard in 1919 intending to continue his studies at Harvard law school; fell in love with Mathematics, and the rest is history…

Marshall had a passion for travel. He began traveling when he was young and was on a trip to India when he died....

One week!

2 symbol adder c

s

d1 d2

c

parity

majority

1 2

3

4

Prove 1 Prove 2

multiply

Need to provide a complete proof No duality arguments

Sum of products (no need to expand to DNF)

See the solution to Quiz #6

IST 4 Information and Logic - California Institute of ...vigeland.caltech.edu/ist4/lectures/lect...

Documents

Transcript of IST 4 Information and Logic - California Institute of ...vigeland.caltech.edu/ist4/lectures/lect...