Discrete Mathematics - All chapters

Fall 2002 CMSC 203 - Discrete Structures 1

Let’s get started with...Let’s get started with...

LogicLogic!!


LogicLogic• Crucial for mathematical reasoningCrucial for mathematical reasoning• Used for designing electronic circuitryUsed for designing electronic circuitry

• Logic is a system based on Logic is a system based on propositionspropositions..• A proposition is a statement that is either A proposition is a statement that is either

truetrue or or falsefalse (not both). (not both).• We say that the We say that the truth valuetruth value of a of a

proposition is either true (T) or false (F).proposition is either true (T) or false (F).

• Corresponds to Corresponds to 11 and and 00 in digital circuits in digital circuits


The Statement/Proposition The Statement/Proposition GameGame

““Elephants are bigger than mice.”Elephants are bigger than mice.”

Is this a statement?Is this a statement? yesyes

Is this a proposition?Is this a proposition? yesyes

What is the truth What is the truth value value

of the proposition?of the proposition?truetrue



““520 < 111”520 < 111”




of the proposition?of the proposition?falsfals

ee



““y > 5”y > 5”


Is this a proposition?Is this a proposition? nono

Its truth value depends on the value of Its truth value depends on the value of y, but this value is not specified.y, but this value is not specified.

We call this type of statement a We call this type of statement a propositional functionpropositional function or or open open sentencesentence..



““Today is January 1 and 99 < 5.”Today is January 1 and 99 < 5.”




of the proposition?of the proposition?falsfals

ee



““Please do not fall asleep.”Please do not fall asleep.”

Is this a statement?Is this a statement? nono

Is this a proposition?Is this a proposition? nono

Only statements can be propositions.Only statements can be propositions.

It’s a request.It’s a request.



““If elephants were red,If elephants were red,

they could hide in cherry trees.”they could hide in cherry trees.”




of the proposition?of the proposition?probably probably

falsefalse



““x < y if and only if y > x.”x < y if and only if y > x.”




of the proposition?of the proposition?truetrue

… … because its truth value because its truth value does not depend on does not depend on specific values of x and specific values of x and y.y.


Combining PropositionsCombining Propositions

As we have seen in the previous examples, As we have seen in the previous examples, one or more propositions can be combined one or more propositions can be combined to form a single to form a single compound propositioncompound proposition..

We formalize this by denoting propositions We formalize this by denoting propositions with letters such as with letters such as p, q, r, s,p, q, r, s, and and introducing several introducing several logical operatorslogical operators. .


Logical Operators Logical Operators (Connectives)(Connectives)

We will examine the following logical We will examine the following logical operators:operators:

• Negation Negation (NOT)(NOT)• Conjunction Conjunction (AND)(AND)• Disjunction Disjunction (OR)(OR)• Exclusive or Exclusive or (XOR)(XOR)• Implication Implication (if – then)(if – then)• Biconditional Biconditional (if and only if)(if and only if)

Truth tables can be used to show how these Truth tables can be used to show how these operators can combine propositions to operators can combine propositions to compound propositions.compound propositions.


Negation (NOT)Negation (NOT)

Unary Operator, Symbol: Unary Operator, Symbol:

PP PP

true (T)true (T) false (F)false (F)

false (F)false (F) true (T)true (T)


Conjunction (AND)Conjunction (AND)

Binary Operator, Symbol: Binary Operator, Symbol:

PP QQ PPQQ

TT TT TT

TT FF FF

FF TT FF

FF FF FF


Disjunction (OR)Disjunction (OR)


PP QQ PPQQ

TT TT TT

TT FF TT

FF TT TT

FF FF FF


Exclusive Or (XOR)Exclusive Or (XOR)


PP QQ PPQQ

TT TT FF

TT FF TT

FF TT TT

FF FF FF


Implication (if - then)Implication (if - then)


PP QQ PPQQ

TT TT TT

TT FF FF

FF TT TT

FF FF TT


Biconditional (if and only Biconditional (if and only if)if)


PP QQ PPQQ

TT TT TT

TT FF FF

FF TT FF

FF FF TT


Statements and OperatorsStatements and OperatorsStatements and operators can be combined in Statements and operators can be combined in

any way to form new statements.any way to form new statements.

PP QQ PP QQ ((P)P)((Q)Q)

TT TT FF FF FF

TT FF FF TT TT

FF TT TT FF TT

FF FF TT TT TT


Statements and Statements and OperationsOperations

Statements and operators can be combined in Statements and operators can be combined in any way to form new statements.any way to form new statements.

PP QQ PPQQ (P(PQ)Q) ((P)P)((Q)Q)

TT TT TT FF FF

TT FF FF TT TT

FF TT FF TT TT

FF FF FF TT TT


Equivalent StatementsEquivalent Statements

PP QQ (P(PQ)Q) ((P)P)((Q)Q) (P(PQ)Q)((P)P)((Q)Q)

TT TT FF FF TT

TT FF TT TT TT

FF TT TT TT TT

FF FF TT TT TT

The statements The statements (P(PQ) and (Q) and (P) P) ( (Q) are Q) are logically logically

equivalentequivalent, since , since (P(PQ) Q) ((P) P) ( (Q) is always true.Q) is always true.


Tautologies and Tautologies and ContradictionsContradictions

A tautology is a statement that is always A tautology is a statement that is always true.true.

Examples: Examples: • RR((R)R) (P(PQ)Q)((P)P)((Q)Q)

If SIf ST is a tautology, we write ST is a tautology, we write ST.T.




A contradiction is a statement that is alwaysA contradiction is a statement that is always

false.false.

Examples: Examples: • RR((R)R) (((P(PQ)Q)((P)P)((Q))Q))

The negation of any tautology is a contra-The negation of any tautology is a contra-

diction, and the negation of any contradiction diction, and the negation of any contradiction is is

a tautology.a tautology.


ExercisesExercisesWe already know the following tautology: We already know the following tautology:

(P(PQ) Q) ((P)P)((Q)Q)

Nice home exercise: Nice home exercise:

Show that Show that (P(PQ) Q) ((P)P)((Q).Q).

These two tautologies are known as De These two tautologies are known as De Morgan’s laws.Morgan’s laws.

Table 5 in Section 1.2Table 5 in Section 1.2 shows many useful laws. shows many useful laws.

Exercises 1 and 7 in Section 1.2Exercises 1 and 7 in Section 1.2 may help you may help you get used to propositions and operators.get used to propositions and operators.


Let’s Talk About LogicLet’s Talk About Logic

• Logic is a system based on Logic is a system based on propositionspropositions..

• A proposition is a statement that is either A proposition is a statement that is either truetrue or or falsefalse (not both). (not both).

• We say that the We say that the truth valuetruth value of a of a proposition is either true (T) or false (F).proposition is either true (T) or false (F).

• Corresponds to Corresponds to 11 and and 00 in digital circuits in digital circuits


Logical Operators Logical Operators (Connectives)(Connectives)

• Negation Negation (NOT)(NOT)• Conjunction Conjunction (AND)(AND)• Disjunction Disjunction (OR)(OR)• Exclusive or Exclusive or (XOR)(XOR)• Implication Implication (if – then)(if – then)• Biconditional Biconditional (if and only if)(if and only if)

Truth tables can be used to show how these Truth tables can be used to show how these operators can combine propositions to operators can combine propositions to compound propositions.compound propositions.



A tautology is a statement that is always A tautology is a statement that is always true.true.

Examples: Examples: • RR((R)R) (P(PQ)Q)((P)P)((Q)Q)





A contradiction is a statement that is alwaysA contradiction is a statement that is alwaysfalse.false.

Examples: Examples: • RR((R)R)• (((P(PQ)Q)((P)P)((Q))Q))

The negation of any tautology is a The negation of any tautology is a contradiction, and the negation of any contradiction, and the negation of any contradiction is a tautology.contradiction is a tautology.


Propositional FunctionsPropositional Functions

Propositional function (open sentence):Propositional function (open sentence):

statement involving one or more variables,statement involving one or more variables,

e.g.: x-3 > 5.e.g.: x-3 > 5.

Let us call this propositional function P(x), Let us call this propositional function P(x), where P is the where P is the predicatepredicate and x is the and x is the variablevariable..

What is the truth value of P(2) ?What is the truth value of P(2) ? falsefalse

What is the truth value of P(8) ?What is the truth value of P(8) ?

What is the truth value of P(9) ?What is the truth value of P(9) ?

falsefalse

truetrue


Propositional FunctionsPropositional Functions

Let us consider the propositional function Let us consider the propositional function Q(x, y, z) defined as: Q(x, y, z) defined as:

x + y = z.x + y = z.

Here, Q is the Here, Q is the predicatepredicate and x, y, and z are and x, y, and z are the the variablesvariables..

What is the truth value of Q(2, 3, 5) What is the truth value of Q(2, 3, 5) ??

truetrue

What is the truth value of Q(0, 1, What is the truth value of Q(0, 1, 2) ?2) ?What is the truth value of Q(9, -9, 0) ?What is the truth value of Q(9, -9, 0) ?

falsefalse

truetrue


Universal QuantificationUniversal Quantification

Let P(x) be a propositional function.Let P(x) be a propositional function.

Universally quantified sentenceUniversally quantified sentence::

For all x in the universe of discourse P(x) is For all x in the universe of discourse P(x) is true.true.

Using the universal quantifier Using the universal quantifier ::

x P(x) x P(x) “for all x P(x)” or “for every x P(x)”“for all x P(x)” or “for every x P(x)”

(Note: (Note: x P(x) is either true or false, so it is a x P(x) is either true or false, so it is a proposition, not a propositional function.)proposition, not a propositional function.)


Universal QuantificationUniversal Quantification

Example: Example:

S(x): x is a UMBC student.S(x): x is a UMBC student.

G(x): x is a genius.G(x): x is a genius.

What does What does x (S(x) x (S(x) G(x)) G(x)) mean ? mean ?

““If x is a UMBC student, then x is a genius.”If x is a UMBC student, then x is a genius.”

oror

““All UMBC students are geniuses.”All UMBC students are geniuses.”


Existential QuantificationExistential Quantification

Existentially quantified sentenceExistentially quantified sentence::There exists an x in the universe of discourse There exists an x in the universe of discourse for which P(x) is true.for which P(x) is true.

Using the existential quantifier Using the existential quantifier ::x P(x) x P(x) “There is an x such that P(x).”“There is an x such that P(x).”

“ “There is at least one x such that There is at least one x such that P(x).”P(x).”

(Note: (Note: x P(x) is either true or false, so it is a x P(x) is either true or false, so it is a proposition, but no propositional function.)proposition, but no propositional function.)


Existential QuantificationExistential Quantification

Example: Example: P(x): x is a UMBC professor.P(x): x is a UMBC professor.G(x): x is a genius.G(x): x is a genius.

What does What does x (P(x) x (P(x) G(x)) G(x)) mean ? mean ?

““There is an x such that x is a UMBC There is an x such that x is a UMBC professor and x is a genius.”professor and x is a genius.”oror““At least one UMBC professor is a genius.”At least one UMBC professor is a genius.”


QuantificationQuantification

Another example:Another example:

Let the universe of discourse be the real numbers.Let the universe of discourse be the real numbers.

What does What does xxy (x + y = 320)y (x + y = 320) mean ? mean ?

““For every x there exists a y so that x + y = 320.”For every x there exists a y so that x + y = 320.”

Is it true?Is it true?

Is it true for the natural Is it true for the natural numbers?numbers?

yesyes

nono


Disproof by CounterexampleDisproof by Counterexample

A counterexample to A counterexample to x P(x) is an object c x P(x) is an object c so that P(c) is false. so that P(c) is false.

Statements such as Statements such as x (P(x) x (P(x) Q(x)) can be Q(x)) can be disproved by simply providing a disproved by simply providing a counterexample.counterexample.

Statement: “All birds can fly.”Statement: “All birds can fly.”Disproved by counterexample: Penguin.Disproved by counterexample: Penguin.


NegationNegation

((x P(x)) is logically equivalent to x P(x)) is logically equivalent to x (x (P(x)).P(x)).

((x P(x)) is logically equivalent to x P(x)) is logically equivalent to x (x (P(x)).P(x)).

See Table 3 in Section 1.3.See Table 3 in Section 1.3.

I recommend exercises 5 and 9 in Section 1.3.I recommend exercises 5 and 9 in Section 1.3.


… … and now for something and now for something completely different…completely different…

Set TheorySet TheoryActually, you will see that logic Actually, you will see that logic and set theory are very closely and set theory are very closely

related.related.


Set TheorySet Theory

• Set: Collection of objects (“elements”)Set: Collection of objects (“elements”)

• aaAA “a is an element of A” “a is an element of A” “a is a member of A” “a is a member of A”

• aaAA “a is not an element of A” “a is not an element of A”

• A = {aA = {a11, a, a22, …, a, …, ann} } “A contains…”“A contains…”

• Order of elements is meaninglessOrder of elements is meaningless

• It does not matter how often the same It does not matter how often the same element is listed.element is listed.


Set EqualitySet Equality

Sets A and B are equal if and only if they Sets A and B are equal if and only if they contain exactly the same elements.contain exactly the same elements.

Examples:Examples:

• A = {9, 2, 7, -3}, B = {7, 9, -3, A = {9, 2, 7, -3}, B = {7, 9, -3, 2} :2} :

A = BA = B

• A = {dog, cat, horse}, A = {dog, cat, horse}, B = {cat, horse, squirrel, dog} B = {cat, horse, squirrel, dog} ::

A A B B

• A = {dog, cat, horse}, A = {dog, cat, horse}, B = {cat, horse, dog, dog} : B = {cat, horse, dog, dog} : A = BA = B


Examples for SetsExamples for Sets

““Standard” Sets:Standard” Sets:

• Natural numbers Natural numbers NN = {0, 1, 2, 3, …} = {0, 1, 2, 3, …}

• Integers Integers ZZ = {…, -2, -1, 0, 1, 2, …} = {…, -2, -1, 0, 1, 2, …}

• Positive Integers Positive Integers ZZ++ = {1, 2, 3, 4, …} = {1, 2, 3, 4, …}

• Real Numbers Real Numbers RR = {47.3, -12, = {47.3, -12, , …}, …}

• Rational Numbers Rational Numbers QQ = {1.5, 2.6, -3.8, 15, = {1.5, 2.6, -3.8, 15, …}…}(correct definition will follow)(correct definition will follow)



• A = A = “empty set/null set”“empty set/null set”

• A = {z}A = {z} Note: zNote: zA, but z A, but z {z}{z}

• A = {{b, c}, {c, x, d}}A = {{b, c}, {c, x, d}}

• A = {{x, y}} A = {{x, y}} Note: {x, y} Note: {x, y} A, but {x, y} A, but {x, y} {{x, y}} {{x, y}}

• A = {x | P(x)}A = {x | P(x)}“set of all x such that P(x)”“set of all x such that P(x)”

• A = {x | xA = {x | xNN x > 7} = {8, 9, 10, …}x > 7} = {8, 9, 10, …}“set builder notation”“set builder notation”



We are now able to define the set of rational We are now able to define the set of rational numbers Q:numbers Q:

QQ = {a/b | a = {a/b | aZZ bbZZ++} }

or or

QQ = {a/b | a = {a/b | aZZ bbZ Z bb0} 0}

And how about the set of real numbers R?And how about the set of real numbers R?

RR = {r | r is a real number} = {r | r is a real number}

That is the best we can do.That is the best we can do.


SubsetsSubsetsA A B B “A is a subset of B” “A is a subset of B”A A B if and only if every element of A is also B if and only if every element of A is also an element of B. an element of B.We can completely formalize this:We can completely formalize this:A A B B x (xx (xA A x xB)B)

Examples:Examples:

A = {3, 9}, B = {5, 9, 1, 3}, A A = {3, 9}, B = {5, 9, 1, 3}, A B ? B ?

truetrue

A = {3, 3, 3, 9}, B = {5, 9, 1, 3}, A A = {3, 3, 3, 9}, B = {5, 9, 1, 3}, A B ? B ?

falsefalse

truetrue

A = {1, 2, 3}, B = {2, 3, 4}, A A = {1, 2, 3}, B = {2, 3, 4}, A B ? B ?


SubsetsSubsetsUseful rules:Useful rules:• A = B A = B (A (A B) B) (B (B A) A) • (A (A B) B) (B (B C) C) A A C C (see Venn Diagram)(see Venn Diagram)

UU

AABB

CC


SubsetsSubsets

Useful rules:Useful rules: A for any set A A for any set A • AA A for any set A A for any set A

Proper subsets:Proper subsets:

A A B B “A is a proper subset of B”“A is a proper subset of B”

A A B B x (xx (xA A x xB) B) x (xx (xB B x xA)A)

oror

A A B B x (xx (xA A x xB) B) x (xx (xB B x xA) A)


Cardinality of SetsCardinality of Sets

If a set S contains n distinct elements, nIf a set S contains n distinct elements, nNN,,we call S a we call S a finite setfinite set with with cardinality ncardinality n..

Examples:Examples:

A = {Mercedes, BMW, Porsche}, |A| = 3A = {Mercedes, BMW, Porsche}, |A| = 3

B = {1, {2, 3}, {4, 5}, 6}B = {1, {2, 3}, {4, 5}, 6} |B| = |B| = 44C = C = |C| = 0|C| = 0

D = { xD = { xN N | x | x 7000 } 7000 } |D| = 7001|D| = 7001

E = { xE = { xN N | x | x 7000 } 7000 } E is infinite!E is infinite!


The Power SetThe Power Set

P(A) P(A) “power set of A”“power set of A”

P(A) = {B | B P(A) = {B | B A} A} (contains all subsets of A)(contains all subsets of A)

Examples:Examples:

A = {x, y, z}A = {x, y, z}

P(A)P(A) = {= {, {x}, {y}, {z}, {x, y}, {x, z}, {y, z}, , {x}, {y}, {z}, {x, y}, {x, z}, {y, z}, {x, y, z}}{x, y, z}}

A = A = P(A) = {P(A) = {}}

Note: |A| = 0, |P(A)| = 1Note: |A| = 0, |P(A)| = 1


The Power SetThe Power SetCardinality of power sets:Cardinality of power sets:

| P(A) | = 2| P(A) | = 2|A||A|

• Imagine each element in A has an “Imagine each element in A has an “onon//offoff” switch” switch• Each possible switch configuration in A Each possible switch configuration in A

corresponds to one element in 2corresponds to one element in 2AA

zzzzzzzzzzzzzzzzzzyyyyyyyyyyyyyyyyyyxxxxxxxxxxxxxxxxxx8877665544332211AA

• For 3 elements in A, there are For 3 elements in A, there are 22222 = 8 elements in P(A)2 = 8 elements in P(A)


Cartesian ProductCartesian ProductThe The ordered n-tupleordered n-tuple (a (a11, a, a22, a, a33, …, a, …, ann) is an ) is an

ordered collectionordered collection of objects. of objects.

Two ordered n-tuples (aTwo ordered n-tuples (a11, a, a22, a, a33, …, a, …, ann) and ) and

(b(b11, b, b22, b, b33, …, b, …, bnn) are equal if and only if they ) are equal if and only if they

contain exactly the same elements contain exactly the same elements in the same in the same orderorder, i.e. a, i.e. aii = b = bii for 1 for 1 i i n. n.

The The Cartesian productCartesian product of two sets is defined as: of two sets is defined as:

AAB = {(a, b) | aB = {(a, b) | aA A b bB}B}

Example:Example: A = {x, y}, B = {a, b, c} A = {x, y}, B = {a, b, c}AAB = {(x, a), (x, b), (x, c), (y, a), (y, b), (y, c)}B = {(x, a), (x, b), (x, c), (y, a), (y, b), (y, c)}


Cartesian ProductCartesian Product

The The Cartesian productCartesian product of two sets is defined of two sets is defined as: Aas: AB = {(a, b) | aB = {(a, b) | aA A b bB}B}

Example:Example:

A = {good, bad}, B = {student, prof}A = {good, bad}, B = {student, prof}

AAB = {B = {(good, student),(good, student), (good, prof),(good, prof), (bad, student),(bad, student), (bad, prof)(bad, prof)}}

(student, good),(student, good), (prof, good),(prof, good), (student, bad),(student, bad), (prof, bad)(prof, bad)}} BBA = A = {{


Cartesian ProductCartesian Product

Note that:Note that:

• AA = = • A = A = • For non-empty sets A and B: AFor non-empty sets A and B: AB B A AB B B BAA

• |A|AB| = |A|B| = |A||B||B|

The Cartesian product of The Cartesian product of two or more setstwo or more sets is is defined as:defined as:

AA11AA22……AAnn = {(a = {(a11, a, a22, …, a, …, ann) | a) | aiiA for 1 A for 1 i i

n}n}


Set OperationsSet Operations

Union: AUnion: AB = {x | xB = {x | xA A x xB}B}

Example:Example: A = {a, b}, B = {b, c, d} A = {a, b}, B = {b, c, d}

AAB = {a, b, c, d} B = {a, b, c, d}

Intersection: AIntersection: AB = {x | xB = {x | xA A x xB}B}

Example:Example: A = {a, b}, B = {b, c, d} A = {a, b}, B = {b, c, d}

AAB = {b}B = {b}



Two sets are called Two sets are called disjointdisjoint if their intersection if their intersection is empty, that is, they share no elements:is empty, that is, they share no elements:

AAB = B =

The The differencedifference between two sets A and B between two sets A and B contains exactly those elements of A that are contains exactly those elements of A that are not in B:not in B:

A-B = {x | xA-B = {x | xA A x xB}B}Example: Example: A = {a, b}, B = {b, c, d}, A-B = {a}A = {a, b}, B = {b, c, d}, A-B = {a}



The The complementcomplement of a set A contains exactly of a set A contains exactly those elements under consideration that are those elements under consideration that are not in A: not in A: AAcc = U-A = U-A

Example: Example: U = U = NN, B = {250, 251, 252, …}, B = {250, 251, 252, …} BBcc = {0, 1, 2, …, 248, 249} = {0, 1, 2, …, 248, 249}


Set OperationsSet OperationsTable 1 in Section 1.5 shows many useful Table 1 in Section 1.5 shows many useful

equations.equations. How can we prove AHow can we prove A(B(BC) = (AC) = (AB)B)(A(AC)?C)?

Method I:Method I: xxAA(B(BC)C) xxA A x x(B(BC)C) xxA A (x (xB B x xC)C) (x(xA A x xB) B) (x (xA A x xC)C)

(distributive law for logical expressions)(distributive law for logical expressions) xx(A(AB) B) x x(A(AC)C) xx(A(AB)B)(A(AC)C)


Set OperationsSet OperationsMethod II: Method II: Membership tableMembership table

1 means “x is an element of this set”1 means “x is an element of this set”0 means “x is not an element of this set” 0 means “x is not an element of this set”

11111111111 1 11 1 1

11111111001 1 01 1 0

11111111001 0 11 0 1

11111111001 0 01 0 0

11111111110 1 10 1 1

00001100000 1 00 1 0

00110000000 0 10 0 1

00000000000 0 00 0 0

(A(AB) B) (A(AC)C)AACCAABBAA(B(BC)C)BBCCA B CA B C



Every logical expression can be transformed Every logical expression can be transformed into an equivalent expression in set theory and into an equivalent expression in set theory and vice versa.vice versa.

You could work on Exercises 9 and 19 in You could work on Exercises 9 and 19 in Section 1.5 to get some practice. Section 1.5 to get some practice.


… … and the following and the following mathematical appetizer is mathematical appetizer is

about…about…

FunctionsFunctions


FunctionsFunctions

A A functionfunction f from a set A to a set B is an f from a set A to a set B is an assignmentassignment of exactly one element of B to each of exactly one element of B to each element of A.element of A.

We writeWe write

f(a) = bf(a) = b

if b is the unique element of B assigned by the if b is the unique element of B assigned by the function f to the element a of A.function f to the element a of A.

If f is a function from A to B, we writeIf f is a function from A to B, we write

f: Af: ABB

(note: Here, “(note: Here, ““ has nothing to do with if… “ has nothing to do with if… then)then)


FunctionsFunctions

If f:AIf f:AB, we say that A is the B, we say that A is the domaindomain of f and B of f and B is the is the codomaincodomain of f. of f.

If f(a) = b, we say that b is the If f(a) = b, we say that b is the imageimage of a and of a and a is the a is the pre-imagepre-image of b. of b.

The The rangerange of f:A of f:AB is the set of all images of B is the set of all images of elements of A.elements of A.

We say that f:AWe say that f:AB B mapsmaps A to B. A to B.


FunctionsFunctions

Let us take a look at the function f:PLet us take a look at the function f:PC withC withP = {Linda, Max, Kathy, Peter}P = {Linda, Max, Kathy, Peter}C = {Boston, New York, Hong Kong, Moscow}C = {Boston, New York, Hong Kong, Moscow}

f(Linda) = Moscowf(Linda) = Moscowf(Max) = Bostonf(Max) = Bostonf(Kathy) = Hong Kongf(Kathy) = Hong Kongf(Peter) = New Yorkf(Peter) = New York

Here, the range of f is C.Here, the range of f is C.


FunctionsFunctions

Let us re-specify f as follows:Let us re-specify f as follows:

f(Linda) = Moscowf(Linda) = Moscow

f(Max) = Bostonf(Max) = Boston

f(Kathy) = Hong Kongf(Kathy) = Hong Kong

f(Peter) = Bostonf(Peter) = Boston

Is f still a function?Is f still a function? yesyes

{Moscow, Boston, Hong {Moscow, Boston, Hong Kong}Kong}

What is its range?What is its range?


FunctionsFunctions

Other ways to represent f:Other ways to represent f:

BostonBostonPeterPeter

Hong Hong KongKongKathyKathy

BostonBostonMaxMax

MoscowMoscowLindaLinda

f(x)f(x)xx LindaLinda

MaxMax

KathyKathy

PeterPeter

BostonBoston

New New YorkYorkHong Hong KongKong

MoscowMoscow


FunctionsFunctions

If the domain of our function f is large, it is If the domain of our function f is large, it is convenient to specify f with a convenient to specify f with a formulaformula, e.g.:, e.g.:

f:f:RRRR f(x) = 2xf(x) = 2x

This leads to:This leads to:f(1) = 2f(1) = 2f(3) = 6f(3) = 6f(-3) = -6f(-3) = -6……


FunctionsFunctions

Let fLet f11 and f and f22 be functions from A to be functions from A to RR..

Then the Then the sumsum and the and the productproduct of f of f11 and f and f22 are are also functions from A to also functions from A to RR defined by: defined by:

(f(f11 + f + f22)(x) = f)(x) = f11(x) + f(x) + f22(x)(x)

(f(f11ff22)(x) = f)(x) = f11(x) f(x) f22(x)(x)

Example:Example:

ff11(x) = 3x, f(x) = 3x, f22(x) = x + 5(x) = x + 5

(f(f11 + f + f22)(x) = f)(x) = f11(x) + f(x) + f22(x) = 3x + x + 5 = 4x + 5(x) = 3x + x + 5 = 4x + 5

(f(f11ff22)(x) = f)(x) = f11(x) f(x) f22(x) = 3x (x + 5) = 3x(x) = 3x (x + 5) = 3x22 + 15x + 15x


FunctionsFunctions

We already know that the We already know that the rangerange of a function of a function f:Af:AB is the set of all images of elements aB is the set of all images of elements aA.A.

If we only regard a If we only regard a subsetsubset S SA, the set of all A, the set of all images of elements simages of elements sS is called the S is called the imageimage of of S.S.

We denote the image of S by f(S):We denote the image of S by f(S):

f(S) = {f(s) | sf(S) = {f(s) | sS}S}


FunctionsFunctions

Let us look at the following well-known function:Let us look at the following well-known function:





What is the image of S = {Linda, Max} ?What is the image of S = {Linda, Max} ?

f(S) = {Moscow, Boston}f(S) = {Moscow, Boston}

What is the image of S = {Max, Peter} ?What is the image of S = {Max, Peter} ?

f(S) = {Boston}f(S) = {Boston}


Properties of FunctionsProperties of Functions

A function f:AA function f:AB is said to be B is said to be one-to-oneone-to-one (or (or injectiveinjective), if and only if), if and only if

x, yx, yA (f(x) = f(y) A (f(x) = f(y) x = y) x = y)

In other words:In other words: f is one-to-one if and only if it f is one-to-one if and only if it does not map two distinct elements of A onto does not map two distinct elements of A onto the same element of B.the same element of B.



And again…And again…





Is f one-to-one?Is f one-to-one?

No, Max and Peter are No, Max and Peter are mapped onto the mapped onto the same element of the same element of the image.image.

g(Linda) = Moscowg(Linda) = Moscow

g(Max) = Bostong(Max) = Boston

g(Kathy) = Hong g(Kathy) = Hong KongKong

g(Peter) = New Yorkg(Peter) = New York

Is g one-to-one?Is g one-to-one?

Yes, each element is Yes, each element is assigned a unique assigned a unique element of the element of the image.image.



How can we prove that a function f is one-to-How can we prove that a function f is one-to-one?one?

Whenever you want to prove something, first Whenever you want to prove something, first take a look at the relevant definition(s):take a look at the relevant definition(s):

x, yx, yA (f(x) = f(y) A (f(x) = f(y) x = y) x = y)

Example:Example:

f:f:RRRR

f(x) = xf(x) = x22

Disproof by counterexample:

f(3) = f(-3), but 3 f(3) = f(-3), but 3 -3, so f is not one-to-one. -3, so f is not one-to-one.



… … and yet another example:and yet another example:

f:f:RRRR

f(x) = 3xf(x) = 3x

One-to-one: One-to-one: x, yx, yA (f(x) = f(y) A (f(x) = f(y) x = y) x = y)

To show:To show: f(x) f(x) f(y) whenever x f(y) whenever x y y

x x y y 3x 3x 3y 3y f(x) f(x) f(y), f(y),

so if x so if x y, then f(x) y, then f(x) f(y), that is, f is one-to- f(y), that is, f is one-to-one.one.



A function f:AA function f:AB with A,B B with A,B R is called R is called strictly strictly increasingincreasing, if , if

x,yx,yA (x < y A (x < y f(x) < f(y)), f(x) < f(y)),

and and strictly decreasingstrictly decreasing, if, if

x,yx,yA (x < y A (x < y f(x) > f(y)). f(x) > f(y)).

Obviously, a function that is either strictly Obviously, a function that is either strictly increasing or strictly decreasing is increasing or strictly decreasing is one-to-oneone-to-one..



A function f:AA function f:AB is called B is called ontoonto, or , or surjectivesurjective, if , if and only if for every element band only if for every element bB there is an B there is an element aelement aA with f(a) = b.A with f(a) = b.

In other words, f is onto if and only if its In other words, f is onto if and only if its rangerange is is its its entire codomainentire codomain..

A function f: AA function f: AB is a B is a one-to-one one-to-one correspondencecorrespondence, or a , or a bijectionbijection, if and only if it is , if and only if it is both one-to-one and onto.both one-to-one and onto.

Obviously, if f is a bijection and A and B are Obviously, if f is a bijection and A and B are finite sets, then |A| = |B|.finite sets, then |A| = |B|.



Examples:Examples:

In the following examples, we use the arrow In the following examples, we use the arrow representation to illustrate functions f:Arepresentation to illustrate functions f:AB. B.

In each example, the complete sets A and B are In each example, the complete sets A and B are shown.shown.



Is f injective?Is f injective?

No.No.

Is f surjective?Is f surjective?

No.No.

Is f bijective?Is f bijective?

No.No.

LindaLinda

MaxMax

KathyKathy

PeterPeter

BostonBoston


MoscowMoscow




No.No.


Yes.Yes.


No.No.

LindaLinda

MaxMax

KathyKathy

PeterPeter

BostonBoston


MoscowMoscow

PaulPaul




Yes.Yes.


No.No.


No.No.

LindaLinda

MaxMax

KathyKathy

PeterPeter

BostonBoston


MoscowMoscow

LLüübeckbeck




No! f is not evenNo! f is not evena function!a function!

LindaLinda

MaxMax

KathyKathy

PeterPeter

BostonBoston


MoscowMoscow

LLüübeckbeck




Yes.Yes.


Yes.Yes.


Yes.Yes.

LindaLinda

MaxMax

KathyKathy

PeterPeter

BostonBoston


MoscowMoscow

LLüübeckbeckHelenaHelena


InversionInversion

An interesting property of bijections is that An interesting property of bijections is that they have an they have an inverse functioninverse function..

The The inverse functioninverse function of the bijection of the bijection f:Af:AB is the function fB is the function f-1-1:B:BA with A with

ff-1-1(b) = a whenever f(a) = b. (b) = a whenever f(a) = b.


InversionInversion

Example:Example:



f(Kathy) = Hong f(Kathy) = Hong KongKong

f(Peter) = Lf(Peter) = Lüübeckbeck

f(Helena) = New f(Helena) = New YorkYork

Clearly, f is Clearly, f is bijective.bijective.

The inverse function The inverse function ff-1-1 is given by: is given by:

ff-1-1(Moscow) = Linda(Moscow) = Linda

ff-1-1(Boston) = Max(Boston) = Max

ff-1-1(Hong Kong) = (Hong Kong) = KathyKathy

ff-1-1(L(Lüübeck) = Peterbeck) = Peter

ff-1-1(New York) = (New York) = HelenaHelena

Inversion is only Inversion is only possible for possible for bijectionsbijections(= invertible (= invertible functions)functions)


InversionInversion

LindaLinda

MaxMax

KathyKathy

PeterPeter

BostonBoston


MoscowMoscow

LLüübeckbeckHelenaHelena

ff

ff-1-1

ff-1-1:C:CP is no P is no function, because function, because it is not defined it is not defined for all elements of for all elements of C and assigns two C and assigns two images to the pre-images to the pre-image New York.image New York.


CompositionCompositionThe The compositioncomposition of two functions g:A of two functions g:AB B and f:Band f:BC, denoted by fC, denoted by fg, is defined by g, is defined by

(f(fg)(a) = f(g(a))g)(a) = f(g(a))

This means that This means that • firstfirst, function g is applied to element , function g is applied to element aaA,A, mapping it onto an element of B, mapping it onto an element of B,• thenthen, function f is applied to this element , function f is applied to this element of of B, mapping it onto an element of C. B, mapping it onto an element of C.• ThereforeTherefore, the composite function maps , the composite function maps from A to C. from A to C.


CompositionComposition

Example:Example:

f(x) = 7x – 4, g(x) = 3x,f(x) = 7x – 4, g(x) = 3x,

f:f:RRRR, g:, g:RRRR

(f(fg)(5) = f(g(5)) = f(15) = 105 – 4 = 101g)(5) = f(g(5)) = f(15) = 105 – 4 = 101

(f(fg)(x) = f(g(x)) = f(3x) = 21x - 4g)(x) = f(g(x)) = f(3x) = 21x - 4


CompositionComposition

Composition of a function and its inverse:Composition of a function and its inverse:

(f(f-1-1f)(x) = ff)(x) = f-1-1(f(x)) = x(f(x)) = x

The composition of a function and its The composition of a function and its inverse is the inverse is the identity functionidentity function i(x) = x. i(x) = x.


GraphsGraphs

TheThe graphgraph of a functionof a function f:Af:AB is the set of B is the set of ordered pairs {(a, b) | aordered pairs {(a, b) | aA and f(a) = b}.A and f(a) = b}.

The graph is a subset of AThe graph is a subset of AB that can be B that can be used to visualize f in a two-dimensional used to visualize f in a two-dimensional coordinate system.coordinate system.


Floor and Ceiling FunctionsFloor and Ceiling Functions

The The floorfloor and and ceilingceiling functions map the real functions map the real numbers onto the integers (numbers onto the integers (RRZZ).).

The The floorfloor function assigns to r function assigns to rRR the largest the largest zzZZ with z with z r, denoted by r, denoted by rr..

Examples:Examples: 2.32.3 = 2, = 2, 22 = 2, = 2, 0.50.5 = 0, = 0, -3.5-3.5 = - = -44

The The ceilingceiling function assigns to r function assigns to rRR the the smallest zsmallest zZZ with z with z r, denoted by r, denoted by rr..

Examples:Examples: 2.32.3 = 3, = 3, 22 = 2, = 2, 0.50.5 = 1, = 1, -3.5-3.5 = - = -3 3


ExercisesExercises

I recommend Exercises 1 and 15 in Section I recommend Exercises 1 and 15 in Section 1.6.1.6.

It may also be useful to study the graph It may also be useful to study the graph displays in that section. displays in that section.

Another question: What do all graph displays Another question: What do all graph displays for any function f:for any function f:RRR R have in common?have in common?


… … and now for…and now for…

SequencesSequences


SequencesSequences

SequencesSequences represent represent ordered listsordered lists of elements. of elements.

A A sequencesequence is defined as a function from a subset is defined as a function from a subset of of NN to a set S. We use the notation a to a set S. We use the notation ann to denote to denote the image of the integer n. We call athe image of the integer n. We call ann a term of the a term of the sequence.sequence.

Example:Example:

subset of subset of NN: 1 2 3 4 5 …: 1 2 3 4 5 …

S: 2 4 6 8 10 …S: 2 4 6 8 10 …


SequencesSequences

We use the notation {aWe use the notation {ann} to describe a } to describe a sequence.sequence.

Important: Do not confuse this with the {} Important: Do not confuse this with the {} used in set notation.used in set notation.

It is convenient to describe a sequence with a It is convenient to describe a sequence with a formulaformula..

For example, the sequence on the previous For example, the sequence on the previous slide can be specified as {aslide can be specified as {ann}, where a}, where ann = 2n. = 2n.


The Formula GameThe Formula Game

1, 3, 5, 7, 9, …1, 3, 5, 7, 9, … aann = 2n - 1 = 2n - 1

-1, 1, -1, 1, -1, -1, 1, -1, 1, -1, ……

aann = (-1) = (-1)nn

2, 5, 10, 17, 26, 2, 5, 10, 17, 26, ……

aann = n = n22 + 1 + 1

0.25, 0.5, 0.75, 1, 1.25 0.25, 0.5, 0.75, 1, 1.25 ……

aann = 0.25n = 0.25n

3, 9, 27, 81, 243, …3, 9, 27, 81, 243, … aann = 3 = 3nn

What are the formulas that describe the What are the formulas that describe the following sequences afollowing sequences a11, a, a22, a, a33, … ?, … ?


StringsStrings

Finite sequences are also called Finite sequences are also called stringsstrings, , denoted by adenoted by a11aa22aa33…a…ann..

The The lengthlength of a string S is the number of of a string S is the number of terms that it consists of.terms that it consists of.

The The empty stringempty string contains no terms at all. It contains no terms at all. It has length zero.has length zero.


SummationsSummations

It represents the sum aIt represents the sum amm + a + am+1m+1 + a + am+2m+2 + … + + … + aann..

The variable j is called the The variable j is called the index of index of summationsummation, running from its , running from its lower limitlower limit m m to its to its upper limitupper limit n. We could as well have n. We could as well have used any other letter to denote this index.used any other letter to denote this index.

n

mjjaWhat does stand for?What does stand for?



It is 1 + 2 + 3 + 4 + 5 + 6 = 21.It is 1 + 2 + 3 + 4 + 5 + 6 = 21.

We write it as .We write it as .

1000

1

2

j

j

What is the value What is the value of ?of ?

6

1j

j

It is so much work to calculate this…It is so much work to calculate this…

What is the value What is the value of ?of ?

100

1j

j

How can we express the sum of the first How can we express the sum of the first 1000 terms of the sequence {a1000 terms of the sequence {ann} with a} with ann=n=n22 for for n = 1, 2, 3, … ?n = 1, 2, 3, … ?



It is said that Friedrich Gauss came up with the It is said that Friedrich Gauss came up with the following formula:following formula:

n

j

nnj

1 2

)1(

When you have such a formula, the result of When you have such a formula, the result of any summation can be calculated much more any summation can be calculated much more easily, for example:easily, for example:

50502

10100

2

)1100(100100

1

j

j


Arithemetic SeriesArithemetic Series

How does:How does:

n

j

nnj

1 2

)1(

Observe that:Observe that:

1 + 2 + 3 +…+ n/2 + (n/2 + 1) +…+ (n - 2) + 1 + 2 + 3 +…+ n/2 + (n/2 + 1) +…+ (n - 2) + (n - 1) + n(n - 1) + n

??????

= [1 + n] + [2 + (n - 1)] + [3 + (n - 2)] +…+ [n/2 + = [1 + n] + [2 + (n - 1)] + [3 + (n - 2)] +…+ [n/2 + (n/2 + 1)](n/2 + 1)] = (n + 1) + (n + 1) + (n + 1) + … + (n + 1) = (n + 1) + (n + 1) + (n + 1) + … + (n + 1) (with (with n/2 terms)n/2 terms) = n(n + 1)/2.= n(n + 1)/2.


Geometric SeriesGeometric Series

How does:How does:

n

j

nj

a

aa

0

)1(

)1(

1

Observe that:Observe that:

S = 1 + a + aS = 1 + a + a22 + a + a33 + … + a + … + ann

??????

aS = a + aaS = a + a22 + a + a33 + … + a + … + ann + a + a(n+1)(n+1)

so, (aS - S) = (a - 1)S = aso, (aS - S) = (a - 1)S = a(n+1)(n+1) - 1 - 1

Therefore, 1 + a + aTherefore, 1 + a + a22 + … + a + … + ann = (a = (a(n+1)(n+1) - 1) / (a - 1) / (a - 1).- 1).For example: 1 + 2 + 4 + 8 +… + 1024 = For example: 1 + 2 + 4 + 8 +… + 1024 = 2047.2047.


Useful SeriesUseful Series

1.1.

2.2.

3.3.

4.4.

n

j

nj

a

aa

0

)1(

)1(

1

n

j

nnj

1 2

)1(

n

j

nnnj

1

2

6

)12)(1(

n

j

nnj

1

223

4

)1(


Double SummationsDouble SummationsCorresponding to nested loops in C or Java, Corresponding to nested loops in C or Java, there is also double (or triple etc.) summation:there is also double (or triple etc.) summation:

Example:Example:

5

1

2

1i j

ij

5

1

)2(i

ii

5

1

3i

i

451512963


Double SummationsDouble Summations

Table 2 in Section 1.7 contains some very Table 2 in Section 1.7 contains some very useful formulas for calculating sums.useful formulas for calculating sums.

Exercises 15 and 17 make a nice homework.Exercises 15 and 17 make a nice homework.


Enough Mathematical Enough Mathematical Appetizers! Appetizers!

Let us look at something more interesting:Let us look at something more interesting:

AlgorithmsAlgorithms


Algorithms Algorithms

What is an algorithm?What is an algorithm?

An algorithm is a finite set of precise An algorithm is a finite set of precise instructions for performing a computation or instructions for performing a computation or for solving a problem.for solving a problem.

This is a rather vague definition. You will get to This is a rather vague definition. You will get to know a more precise and mathematically know a more precise and mathematically useful definition when you attend CS420. useful definition when you attend CS420.

But this one is good enough for now…But this one is good enough for now…


Algorithms Algorithms

Properties of algorithms:Properties of algorithms:

• InputInput from a specified set, from a specified set,• OutputOutput from a specified set (solution), from a specified set (solution),• DefinitenessDefiniteness of every step in the computation, of every step in the computation,• CorrectnessCorrectness of output for every possible input, of output for every possible input,• FinitenessFiniteness of the number of calculation steps, of the number of calculation steps,• EffectivenessEffectiveness of each calculation step and of each calculation step and• GeneralityGenerality for a class of problems. for a class of problems.


Algorithm ExamplesAlgorithm Examples

We will use a pseudocode to specify algorithms, We will use a pseudocode to specify algorithms, which slightly reminds us of Basic and Pascal.which slightly reminds us of Basic and Pascal.

Example:Example: an algorithm that finds the maximum an algorithm that finds the maximum element in a finite sequenceelement in a finite sequence

procedureprocedure max(a max(a11, a, a22, …, a, …, ann: integers): integers)max := amax := a11

forfor i := 2 i := 2 toto n nifif max < a max < aii thenthen max := a max := aii

{max is the largest element}{max is the largest element}



Another example:Another example: a linear search algorithm, a linear search algorithm, that is, an algorithm that linearly searches a that is, an algorithm that linearly searches a sequence for a particular element.sequence for a particular element.

procedureprocedure linear_search(x: integer; a linear_search(x: integer; a11, a, a22, …, , …, aann: : integers) integers)i := 1i := 1while while (i (i n and x n and x a aii))

i := i + 1i := i + 1ifif i i n n thenthen location := i location := ielseelse location := 0 location := 0{location is the subscript of the term that {location is the subscript of the term that equals x, or is zero if x is not found}equals x, or is zero if x is not found}



If the terms in a sequence are ordered, a If the terms in a sequence are ordered, a binary search algorithm is more efficient than binary search algorithm is more efficient than linear search.linear search.

The binary search algorithm iteratively The binary search algorithm iteratively restricts the relevant search interval until it restricts the relevant search interval until it closes in on the position of the element to be closes in on the position of the element to be located.located.



a c d f g h j l m o p r s u v x za c d f g h j l m o p r s u v x z

binary search for the letter ‘j’binary search for the letter ‘j’

center center elementelement

search search intervalinterval



a c d f g h j l m a c d f g h j l m o p r s u v x zo p r s u v x z






a c d f ga c d f g h j l m h j l m o p r s u v x zo p r s u v x z






a c d f ga c d f g h j h j l ml m o p r s u v x zo p r s u v x z






a c d f ga c d f g h h j j l ml m o p r s u v x zo p r s u v x z




found !found !


Algorithm ExamplesAlgorithm Examplesprocedureprocedure binary_search(x: integer; a binary_search(x: integer; a11, a, a22, …, , …, aann: : integers) integers)i := 1 i := 1 {i is left endpoint of search interval}{i is left endpoint of search interval}j := n j := n {j is right endpoint of search interval}{j is right endpoint of search interval} while while (i < j)(i < j)beginbegin

m := m := (i + j)/2(i + j)/2ifif x > a x > amm thenthen i := m + 1 i := m + 1elseelse j := m j := m

endendifif x = a x = aii thenthen location := i location := ielseelse location := 0 location := 0{location is the subscript of the term that {location is the subscript of the term that equals x, or is zero if x is not found}equals x, or is zero if x is not found}


ComplexityComplexity

In general, we are not so much interested in In general, we are not so much interested in the time and space complexity for small the time and space complexity for small inputs.inputs.

For example, while the difference in time For example, while the difference in time complexity between linear and binary search complexity between linear and binary search is meaningless for a sequence with n = 10, it is meaningless for a sequence with n = 10, it is gigantic for n = 2is gigantic for n = 23030..



For example, let us assume two algorithms A For example, let us assume two algorithms A and B that solve the same class of problems.and B that solve the same class of problems.

The time complexity of A is 5,000n, the one The time complexity of A is 5,000n, the one for B is for B is 1.11.1nn for an input with n elements. for an input with n elements.

For n = 10, A requires 50,000 steps, but B For n = 10, A requires 50,000 steps, but B only 3, so B seems to be superior to A.only 3, so B seems to be superior to A.

For n = 1000, however, A requires 5,000,000 For n = 1000, however, A requires 5,000,000 steps, while B requires 2.5steps, while B requires 2.510104141 steps. steps.



This means that algorithm B cannot be used This means that algorithm B cannot be used for large inputs, while algorithm A is still for large inputs, while algorithm A is still feasible.feasible.

So what is important is the So what is important is the growthgrowth of the of the complexity functions.complexity functions.

The growth of time and space complexity with The growth of time and space complexity with increasing input size n is a suitable measure increasing input size n is a suitable measure for the comparison of algorithms. for the comparison of algorithms.



Comparison:Comparison: time complexity of algorithms A and B time complexity of algorithms A and B

Algorithm AAlgorithm A Algorithm BAlgorithm BInput SizeInput Size

nn

1010

100100

1,0001,000

1,000,0001,000,000

5,000n5,000n

50,00050,000

500,000500,000

5,000,0005,000,000

55101099

1.11.1nn33

2.52.510104141

13,78113,781

4.84.810104139241392



This means that algorithm B cannot be used This means that algorithm B cannot be used for large inputs, while running algorithm A is for large inputs, while running algorithm A is still feasible.still feasible.

So what is important is the So what is important is the growthgrowth of the of the complexity functions.complexity functions.

The growth of time and space complexity with The growth of time and space complexity with increasing input size n is a suitable measure increasing input size n is a suitable measure for the comparison of algorithms. for the comparison of algorithms.


The Growth of FunctionsThe Growth of Functions

The growth of functions is usually described The growth of functions is usually described using the using the big-O notationbig-O notation..

Definition:Definition: Let f and g be functions from the Let f and g be functions from the integers or the real numbers to the real integers or the real numbers to the real numbers.numbers.We say that f(x) is O(g(x)) if there are We say that f(x) is O(g(x)) if there are constants C and k such thatconstants C and k such that

|f(x)| |f(x)| C|g(x)| C|g(x)|

whenever x > k.whenever x > k.



When we analyze the growth of When we analyze the growth of complexity complexity functionsfunctions, f(x) and g(x) are always positive. , f(x) and g(x) are always positive.

Therefore, we can simplify the big-O Therefore, we can simplify the big-O requirement torequirement to

f(x) f(x) C Cg(x) whenever x > k.g(x) whenever x > k.

If we want to show that f(x) is O(g(x)), we only If we want to show that f(x) is O(g(x)), we only need to find need to find oneone pair (C, k) (which is never pair (C, k) (which is never unique).unique).


The Growth of FunctionsThe Growth of FunctionsThe idea behind the big-O notation is to The idea behind the big-O notation is to establish an establish an upper boundaryupper boundary for the growth for the growth of a function f(x) for large x.of a function f(x) for large x.

This boundary is specified by a function g(x) This boundary is specified by a function g(x) that is usually much that is usually much simplersimpler than f(x). than f(x).

We accept the constant C in the requirementWe accept the constant C in the requirement

f(x) f(x) C Cg(x) whenever x > k,g(x) whenever x > k,

because because C does not grow with x.C does not grow with x.

We are only interested in large x, so it is OK ifWe are only interested in large x, so it is OK iff(x) > Cf(x) > Cg(x) for x g(x) for x k. k.



Example:Example:

Show that f(x) = xShow that f(x) = x22 + 2x + 1 is O(x + 2x + 1 is O(x22).).

For x > 1 we have:For x > 1 we have:

xx22 + 2x + 1 + 2x + 1 x x22 + 2x + 2x22 + x + x22

xx22 + 2x + 1 + 2x + 1 4x 4x22

Therefore, for C = 4 and k = 1:Therefore, for C = 4 and k = 1:

f(x) f(x) Cx Cx22 whenever x > k. whenever x > k.

f(x) is O(xf(x) is O(x22).).



Question: If f(x) is O(xQuestion: If f(x) is O(x22), is it also O(x), is it also O(x33)?)?

Yes.Yes. x x33 grows faster than x grows faster than x22, so x, so x33 grows also grows also faster than f(x).faster than f(x).

Therefore, we always have to find the Therefore, we always have to find the smallestsmallest simple function g(x) for which f(x) is simple function g(x) for which f(x) is O(g(x)). O(g(x)).



““Popular” functions g(n) arePopular” functions g(n) aren log n, 1, 2n log n, 1, 2nn, n, n22, n!, n, n, n!, n, n33, log n, log n

Listed from slowest to fastest growth:Listed from slowest to fastest growth:• 11• log nlog n• nn• n log nn log n• nn22

• nn33

• 22nn

• n!n!



A problem that can be solved with polynomial A problem that can be solved with polynomial worst-case complexity is called worst-case complexity is called tractabletractable..

Problems of higher complexity are called Problems of higher complexity are called intractable.intractable.

Problems that no algorithm can solve are Problems that no algorithm can solve are called called unsolvableunsolvable..

You will find out more about this in CS420.You will find out more about this in CS420.


Useful Rules for Big-OUseful Rules for Big-O

For any For any polynomialpolynomial f(x) = a f(x) = annxxnn + a + an-1n-1xxn-1n-1 + … + + … + aa00, where a, where a00, a, a11, …, a, …, ann are real numbers, are real numbers,f(x) is O(xf(x) is O(xnn).).

If fIf f11(x) is O(g(x) is O(g11(x)) and f(x)) and f22(x) is O(g(x) is O(g22(x)), then (x)), then (f(f11 + f + f22)(x) is O(max(g)(x) is O(max(g11(x), g(x), g22(x)))(x)))

If fIf f11(x) is O(g(x)) and f(x) is O(g(x)) and f22(x) is O(g(x)), then(x) is O(g(x)), then(f(f11 + f + f22)(x) is O(g(x)).)(x) is O(g(x)).

If fIf f11(x) is O(g(x) is O(g11(x)) and f(x)) and f22(x) is O(g(x) is O(g22(x)), then (x)), then

(f(f11ff22)(x) is O(g)(x) is O(g11(x) g(x) g22(x)).(x)).


Complexity ExamplesComplexity Examples

What does the following algorithm compute?What does the following algorithm compute?

procedureprocedure who_knows(a who_knows(a11, a, a22, …, a, …, ann: integers): integers)m := 0m := 0forfor i := 1 to n-1 i := 1 to n-1

forfor j := i + 1 to n j := i + 1 to nifif |a |aii – a – ajj| > m | > m thenthen m := |a m := |aii – a – ajj||

{m is the maximum difference between any two {m is the maximum difference between any two numbers in the input sequence}numbers in the input sequence}Comparisons: n-1 + n-2 + n-3 + … + 1Comparisons: n-1 + n-2 + n-3 + … + 1 = (n – 1)n/2 = 0.5n= (n – 1)n/2 = 0.5n22 – 0.5n – 0.5n

Time complexity is O(nTime complexity is O(n22).).


Complexity ExamplesComplexity Examples

Another algorithm solving the same problem:Another algorithm solving the same problem:

procedureprocedure max_diff(a max_diff(a11, a, a22, …, a, …, ann: integers): integers)min := a1min := a1max := a1max := a1forfor i := 2 to n i := 2 to n

ifif a aii < min < min thenthen min := a min := aii

elseelse if a if aii > max > max thenthen max := a max := aii

m := max - minm := max - min

Comparisons: 2n - 2Comparisons: 2n - 2

Time complexity is O(n).Time complexity is O(n).


Let us get into…Let us get into…

Number TheoryNumber Theory


Introduction to Number TheoryIntroduction to Number Theory

Number theory is about Number theory is about integersintegers and their and their properties.properties.

We will start with the basic principles ofWe will start with the basic principles of

• divisibility,divisibility,• greatest common divisors,greatest common divisors,• least common multiples, andleast common multiples, and• modular arithmeticmodular arithmetic

and look at some relevant algorithms. and look at some relevant algorithms.


DivisionDivision

If a and b are integers with a If a and b are integers with a 0, we say that 0, we say that a a dividesdivides b if there is an integer c so that b = b if there is an integer c so that b = ac.ac.

When a divides b we say that a is a When a divides b we say that a is a factorfactor of b of b and that b is a and that b is a multiplemultiple of a. of a.

The notation The notation a | ba | b means that a divides b. means that a divides b.

We write We write a X ba X b when a does not divide b when a does not divide b(see book for correct symbol).(see book for correct symbol).


Divisibility TheoremsDivisibility Theorems

For integers a, b, and c it is true thatFor integers a, b, and c it is true that

• if a | b and a | c, then a | (b + c)if a | b and a | c, then a | (b + c) Example:Example: 3 | 6 and 3 | 9, so 3 | 15. 3 | 6 and 3 | 9, so 3 | 15.

• if a | b, then a | bc for all integers cif a | b, then a | bc for all integers c Example:Example: 5 | 10, so 5 | 20, 5 | 30, 5 | 40, … 5 | 10, so 5 | 20, 5 | 30, 5 | 40, …

• if a | b and b | c, then a | cif a | b and b | c, then a | c Example:Example: 4 | 8 and 8 | 24, so 4 | 24. 4 | 8 and 8 | 24, so 4 | 24.


PrimesPrimes

A positive integer p greater than 1 is called A positive integer p greater than 1 is called prime if the only positive factors of p are 1 and prime if the only positive factors of p are 1 and p.p.

A positive integer that is greater than 1 and is A positive integer that is greater than 1 and is not prime is called composite.not prime is called composite.

The fundamental theorem of arithmetic:The fundamental theorem of arithmetic:

Every positive integer can be written Every positive integer can be written uniquelyuniquely as the as the product of primesproduct of primes, where the prime , where the prime factors are written in order of increasing size.factors are written in order of increasing size.


PrimesPrimes

Examples:Examples:

3·53·5

48 =48 =

17 =17 =

100 100 ==512 512 ==515 515 ==28 =28 =

15 =15 =

2·2·2·2·3 = 22·2·2·2·3 = 244·3·3

1717

2·2·5·5 = 22·2·5·5 = 222·5·522

2·2·2·2·2·2·2·2·2 = 22·2·2·2·2·2·2·2·2 = 299

5·1035·103

2·2·72·2·7


PrimesPrimes

If n is a composite integer, then n has a prime If n is a composite integer, then n has a prime divisor less than or equal .divisor less than or equal .

This is easy to see: if n is a composite integer, This is easy to see: if n is a composite integer, it must have two prime divisors pit must have two prime divisors p11 and p and p22 such such that pthat p11pp22 = n. = n.

pp11 and p and p22 cannot both be greater than cannot both be greater than , because then p, because then p11pp22 > n. > n.

n

n


The Division AlgorithmThe Division Algorithm

Let Let aa be an integer and be an integer and dd a positive integer. a positive integer.Then there are unique integers Then there are unique integers qq and and rr, with , with 0 0 r < d r < d, such that , such that a = dq + ra = dq + r..

In the above equation, In the above equation, • dd is called the divisor, is called the divisor, • aa is called the dividend, is called the dividend, • qq is called the quotient, and is called the quotient, and • rr is called the remainder. is called the remainder.



Example:Example:

When we divide 17 by 5, we haveWhen we divide 17 by 5, we have

17 = 517 = 53 + 2.3 + 2.

• 17 is the dividend,17 is the dividend,• 5 is the divisor,5 is the divisor,• 3 is called the quotient, and3 is called the quotient, and• 2 is called the remainder.2 is called the remainder.




What happens when we divide -11 by 3 ?What happens when we divide -11 by 3 ?

Note that the remainder cannot be negative.Note that the remainder cannot be negative.

-11 = 3-11 = 3(-4) + 1.(-4) + 1.

• -11 is the dividend,-11 is the dividend,• 3 is the divisor,3 is the divisor,• -4 is called the quotient, and-4 is called the quotient, and• 1 is called the remainder.1 is called the remainder.


Greatest Common DivisorsGreatest Common DivisorsLet a and b be integers, not both zero.Let a and b be integers, not both zero.The largest integer d such that d | a and d | b is The largest integer d such that d | a and d | b is called the called the greatest common divisorgreatest common divisor of a and b. of a and b.The greatest common divisor of a and b is denoted The greatest common divisor of a and b is denoted by gcd(a, b).by gcd(a, b).

Example 1:Example 1: What is gcd(48, 72) ? What is gcd(48, 72) ?The positive common divisors of 48 and 72 are The positive common divisors of 48 and 72 are 1, 2, 3, 4, 6, 8, 12, 16, and 24, so gcd(48, 72) = 24. 1, 2, 3, 4, 6, 8, 12, 16, and 24, so gcd(48, 72) = 24.

Example 2:Example 2: What is gcd(19, 72) ? What is gcd(19, 72) ?The only positive common divisor of 19 and 72 isThe only positive common divisor of 19 and 72 is1, so gcd(19, 72) = 1. 1, so gcd(19, 72) = 1.


Greatest Common DivisorsGreatest Common Divisors

Using prime factorizations:Using prime factorizations:

a = pa = p11aa1 1 p p22

aa2 2 … p… pnnaan n , b = p, b = p11

bb1 1 p p22bb2 2 … p… pnn

bbn n ,,

where pwhere p11 < p < p22 < … < p < … < pnn and a and aii, b, bii NN for 1 for 1 i i n n

gcd(a, b) = pgcd(a, b) = p11min(amin(a11, b, b1 1 )) p p22

min(amin(a22, b, b2 2 )) … p… pnnmin(amin(ann, b, bn n ))

Example:Example:

a = 60 a = 60 = =

2222 3 311 5 511

b = 54 b = 54 = =

2211 3 333 5 500

gcd(a, b) gcd(a, b) = =

2211 3 311 5 50 0 = 6 = 6


Relatively Prime IntegersRelatively Prime IntegersDefinition:Definition:

Two integers a and b are Two integers a and b are relatively primerelatively prime if if gcd(a, b) = 1.gcd(a, b) = 1.

Examples:Examples:

Are 15 and 28 relatively prime?Are 15 and 28 relatively prime?Yes, gcd(15, 28) = 1.Yes, gcd(15, 28) = 1.Are 55 and 28 relatively prime?Are 55 and 28 relatively prime?Yes, gcd(55, 28) = 1.Yes, gcd(55, 28) = 1.Are 35 and 28 relatively prime?Are 35 and 28 relatively prime?No, gcd(35, 28) = 7.No, gcd(35, 28) = 7.


Relatively Prime IntegersRelatively Prime Integers

Definition:Definition:

The integers aThe integers a11, a, a22, …, a, …, ann are are pairwise pairwise

relatively primerelatively prime if gcd(a if gcd(aii, a, ajj) = 1 whenever 1 ) = 1 whenever 1

i < j i < j n. n.

Examples:Examples:

Are 15, 17, and 27 pairwise relatively prime?Are 15, 17, and 27 pairwise relatively prime?No, because gcd(15, 27) = 3.No, because gcd(15, 27) = 3.

Are 15, 17, and 28 pairwise relatively prime?Are 15, 17, and 28 pairwise relatively prime?Yes, because gcd(15, 17) = 1, gcd(15, 28) = 1 Yes, because gcd(15, 17) = 1, gcd(15, 28) = 1 and gcd(17, 28) = 1.and gcd(17, 28) = 1.


Least Common MultiplesLeast Common MultiplesDefinition:Definition:

The The least common multipleleast common multiple of the positive of the positive integers a and b is the smallest positive integers a and b is the smallest positive integer that is divisible by both a and b.integer that is divisible by both a and b.

We denote the least common multiple of a and We denote the least common multiple of a and b by lcm(a, b).b by lcm(a, b).

Examples:Examples:

lcm(3, 7) lcm(3, 7) ==

2121

lcm(4, 6) lcm(4, 6) ==

1212

lcm(5, 10) lcm(5, 10) ==

1010


Least Common MultiplesLeast Common Multiples

Using prime factorizations:Using prime factorizations:

a = pa = p11aa1 1 p p22

aa2 2 … p… pnnaan n , b = p, b = p11

bb1 1 p p22bb2 2 … p… pnn

bbn n ,,

where pwhere p11 < p < p22 < … < p < … < pnn and a and aii, b, bii NN for 1 for 1 i i n n

lcm(a, b) = plcm(a, b) = p11max(amax(a11, b, b1 1 )) p p22

max(amax(a22, b, b2 2 )) … p… pnnmax(amax(ann, b, bn n ))

Example:Example:

a = 60 a = 60 = =

2222 3 311 5 511

b = 54 b = 54 = =

2211 3 333 5 500

lcm(a, b) lcm(a, b) = =

2222 3 333 5 51 1 = 4 = 427275 = 5405 = 540


GCD and LCMGCD and LCM

a = 60 a = 60 = =

2222 3 311 5 511

b = 54 b = 54 = =

2211 3 333 5 500

lcm(a, b) lcm(a, b) = =

2222 3 333 5 51 1 = 540 = 540

gcd(a, b) gcd(a, b) = =

2211 3 311 5 50 0 = 6 = 6

Theorem: aTheorem: ab b ==

gcd(a,b)gcd(a,b)lcm(a,lcm(a,b)b)


Modular ArithmeticModular Arithmetic

Let a be an integer and m be a positive integer.Let a be an integer and m be a positive integer.We denote by We denote by a mod ma mod m the remainder when a the remainder when a is divided by m.is divided by m.

Examples:Examples:

9 mod 4 9 mod 4 ==

11

9 mod 3 9 mod 3 ==

00

9 mod 10 9 mod 10 ==

99

-13 mod 4 -13 mod 4 ==

33


CongruencesCongruences

Let a and b be integers and m be a positive Let a and b be integers and m be a positive integer. We say that integer. We say that a is congruent to b a is congruent to b modulo mmodulo m if if m divides a – b.m divides a – b.

We use the notation We use the notation a a b (mod m) b (mod m) to indicate to indicate that a is congruent to b modulo m.that a is congruent to b modulo m.

In other words:In other words:a a b (mod m) if and only if b (mod m) if and only if a mod m = b mod a mod m = b mod mm. .


CongruencesCongruencesExamples:Examples:Is it true that 46 Is it true that 46 68 (mod 11) ? 68 (mod 11) ?Yes, because 11 | (46 – 68).Yes, because 11 | (46 – 68).Is it true that 46 Is it true that 46 68 (mod 22)? 68 (mod 22)?Yes, because 22 | (46 – 68).Yes, because 22 | (46 – 68).For which integers z is it true that z For which integers z is it true that z 12 (mod 12 (mod 10)?10)?It is true for any zIt is true for any z{…,-28, -18, -8, 2, 12, 22, 32, {…,-28, -18, -8, 2, 12, 22, 32, …}…}

Theorem:Theorem: Let m be a positive integer. The Let m be a positive integer. The integers a and b are congruent modulo m if and integers a and b are congruent modulo m if and only if there is an integer k such that a = b + km.only if there is an integer k such that a = b + km.


CongruencesCongruences

Theorem:Theorem: Let m be a positive integer. Let m be a positive integer. If a If a b (mod m) and c b (mod m) and c d (mod m), then d (mod m), then a + c a + c b + d (mod m) and ac b + d (mod m) and ac bd (mod m). bd (mod m).Proof:Proof: We know that a We know that a b (mod m) and c b (mod m) and c d (mod m) d (mod m) implies that there are integers s and t with implies that there are integers s and t with b = a + sm and d = c + tm. b = a + sm and d = c + tm. Therefore,Therefore,b + d = (a + sm) + (c + tm) = (a + c) + m(s + t) b + d = (a + sm) + (c + tm) = (a + c) + m(s + t) andandbd = (a + sm)(c + tm) = ac + m(at + cs + stm).bd = (a + sm)(c + tm) = ac + m(at + cs + stm).Hence, a + c Hence, a + c b + d (mod m) and ac b + d (mod m) and ac bd (mod bd (mod m).m).


The Euclidean Algorithm The Euclidean Algorithm

The The Euclidean AlgorithmEuclidean Algorithm finds the finds the greatest greatest common divisorcommon divisor of two integers a and b. of two integers a and b.

For example, if we want to find gcd(287, 91), For example, if we want to find gcd(287, 91), we we dividedivide 287 by 91: 287 by 91:

287 = 91287 = 913 + 143 + 14

We know that for integers a, b and c,We know that for integers a, b and c,if a | b and a | c, then a | (b + c).if a | b and a | c, then a | (b + c).

Therefore, any divisor of 287 and 91 must also Therefore, any divisor of 287 and 91 must also be a divisor of 287 - 91be a divisor of 287 - 913 = 14.3 = 14.

Consequently, gcd(287, 91) = gcd(14, 91).Consequently, gcd(287, 91) = gcd(14, 91).



In the next step, we divide 91 by 14:In the next step, we divide 91 by 14:

91 = 1491 = 146 + 76 + 7

This means that gcd(14, 91) = gcd(14, 7).This means that gcd(14, 91) = gcd(14, 7).

So we divide 14 by 7:So we divide 14 by 7:

14 = 714 = 72 + 02 + 0

We find that 7 | 14, and thus gcd(14, 7) = 7.We find that 7 | 14, and thus gcd(14, 7) = 7.

Therefore, gcd(287, 91) = 7.Therefore, gcd(287, 91) = 7.



In In pseudocodepseudocode, the algorithm can be , the algorithm can be implemented as follows: implemented as follows:

procedureprocedure gcd(a, b: positive integers) gcd(a, b: positive integers)x := ax := ay := by := bwhilewhile y y 0 0beginbegin

r := x r := x modmod y yx := yx := yy := ry := r

endend {x is gcd(a, b)}{x is gcd(a, b)}


Representations of IntegersRepresentations of Integers

Let b be a positive integer greater than 1.Let b be a positive integer greater than 1.Then if n is a positive integer, it can be Then if n is a positive integer, it can be expressed expressed uniquelyuniquely in the form: in the form:

n = an = akkbbkk + a + ak-1k-1bbk-1k-1 + … + a + … + a11b + ab + a00,,

where k is a nonnegative integer,where k is a nonnegative integer,aa00, a, a11, …, a, …, akk are nonnegative integers less than b, are nonnegative integers less than b,and aand akk 0. 0.

Example for b=10:Example for b=10:

859 = 8859 = 8101022 + 5 + 5101011 + 9 + 9101000



Example for b=2 (binary expansion):Example for b=2 (binary expansion):

(10110)(10110)22 = 1 = 12244 + 1 + 12222 + 1 + 12211 = (22) = (22)1010

Example for b=16 (hexadecimal Example for b=16 (hexadecimal expansion):expansion):

(we use letters A to F to indicate numbers 10 to (we use letters A to F to indicate numbers 10 to 15)15)

(3A0F)(3A0F)1616 = 3 = 3161633 + 10 + 10161622 + 15 + 15161600 = (14863) = (14863)1010


Representations of IntegersRepresentations of IntegersHow can we construct the base b expansion of an How can we construct the base b expansion of an integer n?integer n?

First, divide n by b to obtain a quotient qFirst, divide n by b to obtain a quotient q00 and and remainder aremainder a00, that is,, that is,

n = bqn = bq00 + a + a00, where 0 , where 0 a a00 < b. < b.

The remainder aThe remainder a00 is the rightmost digit in the is the rightmost digit in the base b expansion of n.base b expansion of n.

Next, divide qNext, divide q00 by b to obtain: by b to obtain:

qq00 = bq = bq11 + a + a11, where 0 , where 0 a a11 < b. < b.

aa11 is the second digit from the right in the base b is the second digit from the right in the base b expansion of n. Continue this process until you expansion of n. Continue this process until you obtain a quotient equal to zero.obtain a quotient equal to zero.



Example:Example: What is the base 8 expansion of (12345)What is the base 8 expansion of (12345)10 10 ??

First, divide 12345 by 8:First, divide 12345 by 8:12345 = 812345 = 81543 + 11543 + 1

1543 = 81543 = 8192 + 7192 + 7192 = 8192 = 824 + 024 + 024 = 824 = 83 + 03 + 03 = 83 = 80 + 30 + 3

The result is: (12345)The result is: (12345)1010 = (30071) = (30071)88..



procedure procedure base_b_expansion(n, b: positive base_b_expansion(n, b: positive integers)integers)q := nq := nk := 0k := 0whilewhile q q 0 0beginbegin

aakk := q mod b := q mod bq := q := q/bq/bk := k + 1k := k + 1

endend {the base b expansion of n is (a{the base b expansion of n is (ak-1k-1 … a … a11aa00))b b }}


Addition of IntegersAddition of Integers

Let a = (aLet a = (an-1n-1aan-2n-2…a…a11aa00))22, b = (b, b = (bn-1n-1bbn-2n-2…b…b11bb00))2.2.

How can we add these two binary numbers?How can we add these two binary numbers?First, add their rightmost bits:First, add their rightmost bits:

aa00 + b + b00 = c = c002 + s2 + s00,,

where swhere s00 is the is the rightmost bitrightmost bit in the binary in the binary expansion of a + b, and cexpansion of a + b, and c00 is the is the carrycarry..

Then, add the next pair of bits and the carry:Then, add the next pair of bits and the carry:

aa11 + b + b1 1 + c+ c00 = c = c112 + s2 + s11,,

where swhere s11 is the is the next bitnext bit in the binary in the binary expansion of a + b, and cexpansion of a + b, and c11 is the carry. is the carry.



Continue this process until you obtain cContinue this process until you obtain cn-1n-1..

The leading bit of the sum is sThe leading bit of the sum is snn = c = cn-1n-1..

The result is:The result is:

a + b = (sa + b = (snnssn-1n-1…s…s11ss00))22



Example:Example:Add a = (1110)Add a = (1110)22 and b = (1011) and b = (1011)22..

aa00 + b + b00 = 0 + 1 = 0 = 0 + 1 = 02 + 1, so that c2 + 1, so that c00 = 0 and s = 0 and s00 = 1. = 1.

aa11 + b + b1 1 + c+ c00 = 1 + 1 + 0 = 1 = 1 + 1 + 0 = 12 + 0, so c2 + 0, so c11 = 1 and s = 1 and s11 = 0. = 0.



ss44 = c = c33 = 1. = 1.

Therefore, s = a + b = (11001)Therefore, s = a + b = (11001)22..



How do we (humans) add two integers?How do we (humans) add two integers?

Example: Example: 75837583 + + 49324932

5511552211

111111 carrycarry

Binary expansions: Binary expansions: (1011)(1011)22

+ + (1010)(1010)22

1100

carrycarry11

1100

11

11(( ))22



Let a = (aLet a = (an-1n-1aan-2n-2…a…a11aa00))22, b = (b, b = (bn-1n-1bbn-2n-2…b…b11bb00))2.2.

How can we How can we algorithmically algorithmically add these two add these two binary numbers?binary numbers?First, add their rightmost bits:First, add their rightmost bits:

aa00 + b + b00 = c = c002 + s2 + s00,,

where swhere s00 is the is the rightmost bitrightmost bit in the binary in the binary expansion of a + b, and cexpansion of a + b, and c00 is the is the carrycarry..

Then, add the next pair of bits and the carry:Then, add the next pair of bits and the carry:

aa11 + b + b1 1 + c+ c00 = c = c112 + s2 + s11,,

where swhere s11 is the is the next bitnext bit in the binary expansion in the binary expansion of a + b, and cof a + b, and c11 is the carry. is the carry.



Continue this process until you obtain cContinue this process until you obtain cn-1n-1..

The leading bit of the sum is sThe leading bit of the sum is snn = c = cn-1n-1..

The result is:The result is:

a + b = (sa + b = (snnssn-1n-1…s…s11ss00))22



Example:Example:Add a = (1110)Add a = (1110)22 and b = (1011) and b = (1011)22..

aa00 + b + b00 = 0 + 1 = 0 = 0 + 1 = 02 + 1, so that c2 + 1, so that c00 = 0 and s = 0 and s00 = 1. = 1.




ss44 = c = c33 = 1. = 1.

Therefore, s = a + b = (11001)Therefore, s = a + b = (11001)22..



procedure procedure add(a, b: positive integers)add(a, b: positive integers)c := 0c := 0for j := 0 to n-1for j := 0 to n-1beginbegin

d := d := (a(ajj + b + bjj + c)/2 + c)/2ssjj := a := ajj + b + bjj + c – 2d + c – 2dc := dc := d

endendssnn := c := c{the binary expansion of the sum is (s{the binary expansion of the sum is (snnssn-1n-1……ss11ss00))22}}


Let’s proceed to…Let’s proceed to…

Mathematical Mathematical ReasoningReasoning


Mathematical ReasoningMathematical Reasoning

We need We need mathematical reasoningmathematical reasoning to to

• determine whether a mathematical determine whether a mathematical argument is argument is correct or incorrect and correct or incorrect and• construct mathematical arguments.construct mathematical arguments.

Mathematical reasoning is not only important Mathematical reasoning is not only important for conducting for conducting proofsproofs and and program program verificationverification, but also for , but also for artificial artificial intelligenceintelligence systems (drawing inferences). systems (drawing inferences).


TerminologyTerminology

An An axiomaxiom is a basic assumption about is a basic assumption about mathematical structured that needs no proof.mathematical structured that needs no proof.

We can use a We can use a proofproof to demonstrate that a to demonstrate that a particular statement is true. A proof consists of a particular statement is true. A proof consists of a sequence of statements that form an argument.sequence of statements that form an argument.

The steps that connect the statements in such a The steps that connect the statements in such a sequence are the sequence are the rules of inferencerules of inference..

Cases of incorrect reasoning are called Cases of incorrect reasoning are called fallaciesfallacies..

A A theoremtheorem is a statement that can be shown to is a statement that can be shown to be true. be true.


TerminologyTerminology

A A lemmalemma is a simple theorem used as an is a simple theorem used as an intermediate result in the proof of another intermediate result in the proof of another theorem.theorem.

A A corollarycorollary is a proposition that follows is a proposition that follows directly from a theorem that has been proved.directly from a theorem that has been proved.

A A conjectureconjecture is a statement whose truth is a statement whose truth value is unknown. Once it is proven, it value is unknown. Once it is proven, it becomes a theorem.becomes a theorem.


Rules of InferenceRules of Inference

Rules of inferenceRules of inference provide the justification of provide the justification of the steps used in a proof.the steps used in a proof.

One important rule is called One important rule is called modus ponensmodus ponens or the or the law of detachmentlaw of detachment. It is based on the . It is based on the tautology tautology (p(p(p(pq)) q)) q. We write it in the following way: q. We write it in the following way:

ppp p q q________ qq

The two The two hypotheseshypotheses p and p p and p q are q are

written in a column, and the written in a column, and the conclusionconclusionbelow a bar, where below a bar, where means means “therefore”.“therefore”.



The general form of a rule of inference is:The general form of a rule of inference is:

pp11

pp22 .. .. .. ppnn________ qq

The rule states that if pThe rule states that if p11 andand p p22 andand … … andand p pnn are all true, then q is true as are all true, then q is true as well.well.

These rules of inference can be used These rules of inference can be used in any mathematical argument and do in any mathematical argument and do not not require any proof.require any proof.



pp__________ ppqq AdditionAddition

ppqq__________ pp SimplificatioSimplificatio

nn

pp qq__________ ppqq

ConjunctionConjunction

qq ppq q __________ pp

Modus Modus tollenstollens

ppqq qqr r __________ ppr r

Hypothetical Hypothetical syllogismsyllogism

ppqq pp__________ q q

Disjunctive Disjunctive syllogismsyllogism


ArgumentsArguments

Just like a rule of inference, an Just like a rule of inference, an argument argument consists of one or more hypotheses and a consists of one or more hypotheses and a conclusion. conclusion.

We say that an argument isWe say that an argument is valid valid, if whenever , if whenever all its hypotheses are true, its conclusion is all its hypotheses are true, its conclusion is also true.also true.

However, if any hypothesis is false, even a However, if any hypothesis is false, even a valid argument can lead to an incorrect valid argument can lead to an incorrect conclusion. conclusion.


ArgumentsArgumentsExample:Example:

““If 101 is divisible by 3, then 101If 101 is divisible by 3, then 10122 is divisible is divisible by 9. 101 is divisible by 3. Consequently, 101by 9. 101 is divisible by 3. Consequently, 10122 is divisible by 9.”is divisible by 9.”

Although the argument is Although the argument is validvalid, its conclusion , its conclusion is is incorrectincorrect, because one of the hypotheses is , because one of the hypotheses is false (“101 is divisible by 3.”).false (“101 is divisible by 3.”).

If in the above argument we replace 101 with If in the above argument we replace 101 with 102, we could correctly conclude that 102102, we could correctly conclude that 10222 is is divisible by 9.divisible by 9.


ArgumentsArgumentsWhich rule of inference was used in the last Which rule of inference was used in the last argument?argument?

p: “101 is divisible by 3.”p: “101 is divisible by 3.”

q: “101q: “10122 is divisible by 9.” is divisible by 9.”

pp ppq q __________ qq

Modus Modus ponensponens

Unfortunately, one of the hypotheses (p) is Unfortunately, one of the hypotheses (p) is false.false.Therefore, the conclusion q is incorrect.Therefore, the conclusion q is incorrect.


ArgumentsArguments


““If it rains today, then we will not have a If it rains today, then we will not have a barbeque today. If we do not have a barbeque barbeque today. If we do not have a barbeque today, then we will have a barbeque today, then we will have a barbeque tomorrow.tomorrow.Therefore, if it rains today, then we will have a Therefore, if it rains today, then we will have a barbeque tomorrow.”barbeque tomorrow.”

This is a This is a validvalid argument: If its hypotheses are argument: If its hypotheses are true, then its conclusion is also true.true, then its conclusion is also true.


ArgumentsArguments

Let us formalize the previous argument:Let us formalize the previous argument:

p: “It is raining today.”p: “It is raining today.”

q: “We will not have a barbecue today.”q: “We will not have a barbecue today.”

r: “We will have a barbecue tomorrow.”r: “We will have a barbecue tomorrow.”

So the argument is of the following form:So the argument is of the following form:

ppqq qqr r __________ ppr r

Hypothetical Hypothetical syllogismsyllogism


ArgumentsArguments


Gary is either intelligent or a good actor.Gary is either intelligent or a good actor.If Gary is intelligent, then he can count If Gary is intelligent, then he can count from 1 to 10.from 1 to 10.Gary can only count from 1 to 2.Gary can only count from 1 to 2.Therefore, Gary is a good actor.Therefore, Gary is a good actor.

i: “Gary is intelligent.”i: “Gary is intelligent.”a: “Gary is a good actor.”a: “Gary is a good actor.”c: “Gary can count from 1 to 10.”c: “Gary can count from 1 to 10.”


ArgumentsArguments

i: “Gary is intelligent.”i: “Gary is intelligent.”a: “Gary is a good actor.”a: “Gary is a good actor.”c: “Gary can count from 1 to 10.”c: “Gary can count from 1 to 10.”

Step 1:Step 1: cc HypothesisHypothesisStep 2:Step 2: i i c c HypothesisHypothesisStep 3:Step 3: i i Modus tollens Steps 1 & 2Modus tollens Steps 1 & 2Step 4:Step 4: a a i i HypothesisHypothesisStep 5:Step 5: a a Disjunctive SyllogismDisjunctive Syllogism

Steps 3 & 4Steps 3 & 4

Conclusion: Conclusion: aa (“Gary is a good actor.”) (“Gary is a good actor.”)


ArgumentsArguments

Yet another example:Yet another example:

If you listen to me, you will pass CS 320.If you listen to me, you will pass CS 320.You passed CS 320.You passed CS 320.Therefore, you have listened to me.Therefore, you have listened to me.

Is this argument valid?Is this argument valid?

NoNo, it assumes ((p, it assumes ((pq)q) q) q) p. p.

This statement is not a tautology. It is This statement is not a tautology. It is falsefalse if p if p is false and q is true.is false and q is true.


Rules of Inference for Quantified Rules of Inference for Quantified StatementsStatements

x P(x)x P(x)____________________ P(c) if P(c) if ccUU

Universal Universal instantiationinstantiation

P(c) for an arbitrary cP(c) for an arbitrary cUU______________________________________ x P(x)x P(x)

Universal Universal generalizatiogeneralizationn

x P(x)x P(x)____________________________________________ P(c) for some element cP(c) for some element cUU

Existential Existential instantiationinstantiation

P(c) for some element cP(c) for some element cUU________________________________________ x P(x) x P(x)

Existential Existential generalizatiogeneralizationn



Example:Example:

Every UMB student is a genius. Every UMB student is a genius. George is a UMB student.George is a UMB student.Therefore, George is a genius.Therefore, George is a genius.

U(x): “x is a UMB student.”U(x): “x is a UMB student.”G(x): “x is a genius.”G(x): “x is a genius.”



The following steps are used in the argument:The following steps are used in the argument:

Step 1:Step 1: x (U(x) x (U(x) G(x)) G(x)) HypothesisHypothesisStep 2:Step 2: U(George) U(George) G(George) G(George) Univ. Univ. instantiation instantiation using Step 1using Step 1

x P(x)x P(x)____________________ P(c) if P(c) if ccUU

Universal Universal instantiationinstantiation

Step 3:Step 3: U(George) U(George) HypothesisHypothesisStep 4:Step 4: G(George) G(George) Modus ponensModus ponens

using Steps 2 & 3using Steps 2 & 3


Proving TheoremsProving Theorems

Direct proof:Direct proof:

An implication pAn implication pq can be proved by showing q can be proved by showing that if p is true, then q is also true.that if p is true, then q is also true.

Example:Example: Give a direct proof of the theorem Give a direct proof of the theorem “If n is odd, then n“If n is odd, then n22 is odd.” is odd.”

Idea:Idea: Assume that the hypothesis of this Assume that the hypothesis of this implication is true (n is odd). Then use rules of implication is true (n is odd). Then use rules of inference and known theorems to show that q inference and known theorems to show that q must also be true (nmust also be true (n22 is odd). is odd).



n is odd.n is odd.

Then n = 2k + 1, where k is an integer.Then n = 2k + 1, where k is an integer.

Consequently, nConsequently, n22 = (2k + 1) = (2k + 1)22.. = 4k= 4k22 + 4k + 1 + 4k + 1 = 2(2k= 2(2k22 + 2k) + 1 + 2k) + 1

Since nSince n22 can be written in this form, it is odd. can be written in this form, it is odd.



Indirect proof:Indirect proof:

An implication pAn implication pq is equivalent to its q is equivalent to its contra-contra-positivepositive q q p. Therefore, we can prove pp. Therefore, we can prove pq q by showing that whenever q is false, then p is by showing that whenever q is false, then p is also false.also false.

Example:Example: Give an indirect proof of the theorem Give an indirect proof of the theorem

“If 3n + 2 is odd, then n is odd.”“If 3n + 2 is odd, then n is odd.”

Idea:Idea: Assume that the conclusion of this Assume that the conclusion of this implication is false (n is even). Then use rules of implication is false (n is even). Then use rules of inference and known theorems to show that p inference and known theorems to show that p must also be false (3n + 2 is even).must also be false (3n + 2 is even).


Proving TheoremsProving Theoremsn is even.n is even.

Then n = 2k, where k is an integer.Then n = 2k, where k is an integer.

It follows that 3n + 2 = 3(2k) + 2 It follows that 3n + 2 = 3(2k) + 2 = 6k + 2= 6k + 2= 2(3k + 1)= 2(3k + 1)

Therefore, 3n + 2 is even.Therefore, 3n + 2 is even.

We have shown that the contrapositive of the We have shown that the contrapositive of the implication is true, so the implication itself is implication is true, so the implication itself is also true also true (If 2n + 3 is odd, then n is odd).(If 2n + 3 is odd, then n is odd).


Follow me for a walk through...Follow me for a walk through...

MathematicalMathematical

InductionInduction


InductionInduction

The The principle of mathematical inductionprinciple of mathematical induction is a useful tool for proving that a certain is a useful tool for proving that a certain predicate is true for predicate is true for all natural numbersall natural numbers..

It cannot be used to discover theorems, but It cannot be used to discover theorems, but only to prove them.only to prove them.


InductionInductionIf we have a propositional function P(n), and If we have a propositional function P(n), and we want to prove that P(n) is true for any we want to prove that P(n) is true for any natural number n, we do the following:natural number n, we do the following:

• Show that P(0) is true.Show that P(0) is true. (basis step)(basis step)• Show that if P(n) then P(n + 1) for any Show that if P(n) then P(n + 1) for any nnN.N. (inductive step)(inductive step)• Then P(n) must be true for any nThen P(n) must be true for any nN. N. (conclusion)(conclusion)


InductionInductionExample:Example:

Show that n < 2Show that n < 2nn for all positive integers n. for all positive integers n.

Let P(n) be the proposition “n < 2Let P(n) be the proposition “n < 2nn.”.”

1. Show that P(1) is true.1. Show that P(1) is true.(basis step)(basis step)

P(1) is true, because 1 < 2P(1) is true, because 1 < 211 = 2. = 2.


InductionInduction2. Show that if P(n) is true, then P(n + 1) is 2. Show that if P(n) is true, then P(n + 1) is

true.true.(inductive step)(inductive step)

Assume that n < 2Assume that n < 2nn is true. is true.We need to show that P(n + 1) is true, i.e.We need to show that P(n + 1) is true, i.e.n + 1 < 2n + 1 < 2n+1n+1

We start from n < 2We start from n < 2nn::n + 1 < 2n + 1 < 2nn + 1 + 1 2 2nn + 2 + 2nn = 2 = 2n+1n+1

Therefore, if n < 2Therefore, if n < 2nn then n + 1 < 2 then n + 1 < 2n+1n+1


InductionInduction

• Then P(n) must be true for any positive Then P(n) must be true for any positive integer.integer.(conclusion)(conclusion)

n < 2n < 2nn is true for any positive integer. is true for any positive integer.

End of proof.End of proof.


InductionInduction

Another Example (“Gauss”):Another Example (“Gauss”):

1 + 2 + … + n = n (n + 1)/21 + 2 + … + n = n (n + 1)/2

• Show that P(0) is true.Show that P(0) is true.(basis step)(basis step)

For n = 0 we get 0 = 0. For n = 0 we get 0 = 0. True.True.


InductionInduction• Show that if P(n) then P(n + 1) for any Show that if P(n) then P(n + 1) for any

nnNN. . (inductive step)(inductive step)

1 + 2 + … + n = n (n + 1)/21 + 2 + … + n = n (n + 1)/2

1 + 2 + … + n 1 + 2 + … + n + (n + 1)+ (n + 1) = n (n + 1)/2 = n (n + 1)/2 + (n + (n + 1)+ 1)

= (2n + 2 + n (n + 1))/2= (2n + 2 + n (n + 1))/2

= (2n + 2 + n= (2n + 2 + n22 + n)/2 + n)/2

= (2 + 3n + n= (2 + 3n + n2 2 )/2)/2

= (n + 1) (n + 2)/2= (n + 1) (n + 2)/2

= = (n + 1)(n + 1) ( ((n + 1)(n + 1) + 1)/2 + 1)/2


InductionInduction

• Then P(n) must be true for any nThen P(n) must be true for any nN. N. (conclusion)(conclusion)

1 + 2 + … + n = n (n + 1)/2 is true for all 1 + 2 + … + n = n (n + 1)/2 is true for all nnN.N.



InductionInduction

There is another proof technique that is very There is another proof technique that is very similar to the principle of mathematical similar to the principle of mathematical induction.induction.

It is called It is called the second principle of the second principle of mathematical inductionmathematical induction..

It can be used to prove that a propositional It can be used to prove that a propositional function P(n) is true for any natural number n. function P(n) is true for any natural number n.


InductionInduction

The second principle of mathematical The second principle of mathematical induction:induction:

• Show that P(0) is true.Show that P(0) is true.(basis step)(basis step)

• Show that if P(0) and P(1) and … and P(n),Show that if P(0) and P(1) and … and P(n),then P(n + 1) for any nthen P(n + 1) for any nN.N.(inductive step)(inductive step)



InductionInduction

Example: Show that every integer greater Example: Show that every integer greater than 1 can be written as the product of than 1 can be written as the product of primes.primes.

• Show that P(2) is true.Show that P(2) is true. (basis step)(basis step)

2 is the product of one prime: itself.2 is the product of one prime: itself.


InductionInduction• Show that if P(2) and P(3) and … and P(n),Show that if P(2) and P(3) and … and P(n),

then P(n + 1) for any nthen P(n + 1) for any nN. N. (inductive step)(inductive step)

Two possible cases:Two possible cases:• If (n + 1) is If (n + 1) is primeprime, then obviously P(n + 1) is , then obviously P(n + 1) is

true.true.• If (n + 1) is If (n + 1) is compositecomposite, it can be written as , it can be written as

the product of two integers a and b such thatthe product of two integers a and b such that2 2 a a b < n + 1. b < n + 1.

By the By the induction hypothesisinduction hypothesis, both a and b , both a and b can be written as the product of primes.can be written as the product of primes.

Therefore, n + 1 = aTherefore, n + 1 = ab can be written as the b can be written as the product of primes.product of primes.


InductionInduction



We have shown that We have shown that every integer every integer greater than 1greater than 1 can be written as the can be written as the product of primes.product of primes.


If I told you once, it must If I told you once, it must be...be...

RecursionRecursion


Recursive DefinitionsRecursive Definitions

RecursionRecursion is a principle closely related to is a principle closely related to mathematical induction.mathematical induction.

In a In a recursive definitionrecursive definition, an object is , an object is defined in terms of itself.defined in terms of itself.

We can recursively define We can recursively define sequencessequences, , functionsfunctions and and setssets..


Recursively Defined Recursively Defined SequencesSequences

Example:Example:

The sequence {aThe sequence {ann} of powers of 2 is given by} of powers of 2 is given byaann = 2 = 2nn for n = 0, 1, 2, … . for n = 0, 1, 2, … .

The same sequence can also be defined The same sequence can also be defined recursivelyrecursively::

aa00 = 1 = 1

aan+1n+1 = 2a = 2an n for n = 0, 1, 2, …for n = 0, 1, 2, …

Obviously, induction and recursion are similar Obviously, induction and recursion are similar principles.principles.


Recursively Defined Recursively Defined FunctionsFunctions

We can use the following method to define a We can use the following method to define a function with the function with the natural numbersnatural numbers as its as its domain:domain:

• Specify the value of the function at zero.Specify the value of the function at zero.• Give a rule for finding its value at any integer Give a rule for finding its value at any integer from its values at smaller integers. from its values at smaller integers.

Such a definition is called Such a definition is called recursiverecursive or or inductive definitioninductive definition. .



Example:Example:

f(0) = 3f(0) = 3

f(n + 1) = 2f(n) + 3f(n + 1) = 2f(n) + 3

f(0) = 3f(0) = 3

f(1) = 2f(0) + 3 = 2f(1) = 2f(0) + 3 = 23 + 3 = 93 + 3 = 9

f(2) = 2f(1) + 3 = 2f(2) = 2f(1) + 3 = 29 + 3 = 219 + 3 = 21

f(3) = 2f(2) + 3 = 2f(3) = 2f(2) + 3 = 221 + 3 = 4521 + 3 = 45

f(4) = 2f(3) + 3 = 2f(4) = 2f(3) + 3 = 245 + 3 = 9345 + 3 = 93



How can we recursively define the How can we recursively define the factorial function f(n) = n! ? factorial function f(n) = n! ?

f(0) = 1f(0) = 1f(n + 1) = (n + 1)f(n) f(n + 1) = (n + 1)f(n)

f(0) = 1f(0) = 1f(1) = 1f(0) = 1f(1) = 1f(0) = 11 = 11 = 1f(2) = 2f(1) = 2f(2) = 2f(1) = 21 = 21 = 2f(3) = 3f(2) = 3f(3) = 3f(2) = 32 = 62 = 6f(4) = 4f(3) = 4f(4) = 4f(3) = 46 = 246 = 24



A famous example: The Fibonacci A famous example: The Fibonacci numbers numbers

f(0) = 0, f(1) = 1f(0) = 0, f(1) = 1f(n) = f(n – 1) + f(n - 2) f(n) = f(n – 1) + f(n - 2)

f(0) = 0f(0) = 0f(1) = 1f(1) = 1f(2) = f(1) + f(0) = 1 + 0 = 1f(2) = f(1) + f(0) = 1 + 0 = 1f(3) = f(2) + f(1) = 1 + 1 = 2f(3) = f(2) + f(1) = 1 + 1 = 2f(4) = f(3) + f(2) = 2 + 1 = 3f(4) = f(3) + f(2) = 2 + 1 = 3f(5) = f(4) + f(3) = 3 + 2 = 5f(5) = f(4) + f(3) = 3 + 2 = 5f(6) = f(5) + f(4) = 5 + 3 = 8f(6) = f(5) + f(4) = 5 + 3 = 8


Recursively Defined SetsRecursively Defined Sets

If we want to recursively define a set, we If we want to recursively define a set, we need to provide two things:need to provide two things:

• an an initial setinitial set of elements, of elements,• rulesrules for the construction of for the construction of additionaladditional elements from elements in the set. elements from elements in the set.

Example:Example: Let S be recursively defined by: Let S be recursively defined by:

3 3 S S

(x + y) (x + y) S if (x S if (x S) and (y S) and (y S) S)

S is the set of positive integers divisible by 3.S is the set of positive integers divisible by 3.



Proof:Proof:

Let A be the set of all positive integers divisible Let A be the set of all positive integers divisible by 3.by 3.

To show that A = S, we must show that To show that A = S, we must show that A A S and S S and S A. A.

Part I:Part I: To prove that A To prove that A S, we must show that S, we must show that every positive integer divisible by 3 is in S.every positive integer divisible by 3 is in S.

We will use mathematical induction to show We will use mathematical induction to show this. this.



Let P(n) be the statement “3n belongs to S”.Let P(n) be the statement “3n belongs to S”.

Basis step:Basis step: P(1) is true, because 3 is in S. P(1) is true, because 3 is in S.

Inductive step:Inductive step: To show: To show:If P(n) is true, then P(n + 1) is true.If P(n) is true, then P(n + 1) is true.

Assume 3n is in S. Since 3n is in S and 3 is in Assume 3n is in S. Since 3n is in S and 3 is in S, it follows from the recursive definition of S S, it follows from the recursive definition of S thatthat3n + 3 = 3(n + 1) is also in S.3n + 3 = 3(n + 1) is also in S.

Conclusion of Part I:Conclusion of Part I: A A S. S.


Recursively Defined SetsRecursively Defined SetsPart II:Part II: To show: S To show: S A. A.

Basis step:Basis step: To show: To show: All initial elements of S are in A. 3 is in A. All initial elements of S are in A. 3 is in A. True.True.

Inductive step:Inductive step: To show: To show:(x + y) is in A whenever x and y are in S.(x + y) is in A whenever x and y are in S.

If x and y are both in A, it follows that 3 | x If x and y are both in A, it follows that 3 | x and 3 | y. From Theorem I, Section 2.3, it and 3 | y. From Theorem I, Section 2.3, it follows that 3 | (x + y).follows that 3 | (x + y).

Conclusion of Part II:Conclusion of Part II: S S A. A.Overall conclusion:Overall conclusion: A = S. A = S.




The well-formed formulae of variables, The well-formed formulae of variables, numerals and operators from {+, -, *, /, ^} numerals and operators from {+, -, *, /, ^} are defined by:are defined by:

x is a well-formed formula if x is a numeral or x is a well-formed formula if x is a numeral or variable.variable.

(f + g), (f – g), (f * g), (f / g), (f ^ g) are well-(f + g), (f – g), (f * g), (f / g), (f ^ g) are well-formed formulae if f and g are.formed formulae if f and g are.



With this definition, we can construct With this definition, we can construct formulae such as:formulae such as:

(x – y)(x – y)

((z / 3) – y)((z / 3) – y)

((z / 3) – (6 + 5))((z / 3) – (6 + 5))

((z / (2 * 4)) – (6 + 5))((z / (2 * 4)) – (6 + 5))


Recursive AlgorithmsRecursive Algorithms

An algorithm is called An algorithm is called recursiverecursive if it solves a if it solves a problem by reducing it to an instance of the problem by reducing it to an instance of the same problem with smaller input.same problem with smaller input.

Example I:Example I: Recursive Euclidean Algorithm Recursive Euclidean Algorithm

procedureprocedure gcd(a, b: nonnegative integers with gcd(a, b: nonnegative integers with a < b)a < b)

ifif a = 0 a = 0 then then gcd(a, b) := bgcd(a, b) := b

elseelse gcd(a, b) := gcd(b gcd(a, b) := gcd(b modmod a, a) a, a)



Example II:Example II: Recursive Fibonacci Algorithm Recursive Fibonacci Algorithm

procedureprocedure fibo(n: nonnegative integer) fibo(n: nonnegative integer)

ifif n = 0 n = 0 then then fibo(0) := 0fibo(0) := 0

else ifelse if n = 1 n = 1 then then fibo(1) := 1fibo(1) := 1

elseelse fibo(n) := fibo(n – 1) + fibo(n – 2) fibo(n) := fibo(n – 1) + fibo(n – 2)



Recursive Fibonacci Evaluation:Recursive Fibonacci Evaluation:

f(4)f(4)

f(3)f(3)

f(2)f(2)

f(1)f(1) f(0)f(0)

f(1)f(1)

f(2)f(2)

f(1)f(1) f(0)f(0)


Recursive AlgorithmsRecursive Algorithmsprocedureprocedure iterative_fibo(n: nonnegative integer) iterative_fibo(n: nonnegative integer)ifif n = 0 n = 0 then then y := 0y := 0elseelsebeginbegin

x := 0x := 0y := 1y := 1for for i := 1 i := 1 toto n-1 n-1beginbegin

z := x + yz := x + yx : = yx : = yy := zy := z

endendendend {y is the n-th Fibonacci number}{y is the n-th Fibonacci number}



For every recursive algorithm, there is an For every recursive algorithm, there is an equivalent iterative algorithm.equivalent iterative algorithm.

Recursive algorithms are often shorter, more Recursive algorithms are often shorter, more elegant, and easier to understand than their elegant, and easier to understand than their iterative counterparts.iterative counterparts.

However, iterative algorithms are usually However, iterative algorithms are usually more efficient in their use of space and time. more efficient in their use of space and time.


One, two, three, we’re…One, two, three, we’re…

CountingCounting


Basic Counting PrinciplesBasic Counting Principles

Counting problemsCounting problems are of the following kind: are of the following kind:

““How manyHow many different 8-letter passwords are different 8-letter passwords are there?”there?”

““How manyHow many possible ways are there to pick 11 possible ways are there to pick 11 soccer players out of a 20-player team?”soccer players out of a 20-player team?”

Most importantly, counting is the basis for Most importantly, counting is the basis for computing computing probabilities of discrete eventsprobabilities of discrete events..

(“What is the probability of winning the lottery?”) (“What is the probability of winning the lottery?”)


Basic Counting PrinciplesBasic Counting PrinciplesThe sum rule:The sum rule:If a task can be done in nIf a task can be done in n11 ways and a second ways and a second task in ntask in n22 ways, and if these two tasks cannot ways, and if these two tasks cannot be done at the same time, then there are nbe done at the same time, then there are n11 + + nn22 ways to do either task. ways to do either task.

Example:Example: The department will award a free computer to The department will award a free computer to either a CS student or a CS professor. either a CS student or a CS professor. How many different choices are there, if there How many different choices are there, if there are 530 students and 15 professors?are 530 students and 15 professors?

There are 530 + 15 = 545 choices.There are 530 + 15 = 545 choices.



Generalized sum rule:Generalized sum rule:

If we have tasks TIf we have tasks T11, T, T22, …, T, …, Tmm that can be that can be done in ndone in n11, n, n22, …, n, …, nmm ways, respectively, and ways, respectively, and no two of these tasks can be done at the same no two of these tasks can be done at the same time, then there are ntime, then there are n11 + n + n22 + … + n + … + nmm ways to ways to do one of these tasks.do one of these tasks.



The product rule:The product rule:

Suppose that a procedure can be broken down Suppose that a procedure can be broken down into two successive tasks. If there are ninto two successive tasks. If there are n11 ways ways to do the first task and nto do the first task and n22 ways to do the ways to do the second task after the first task has been done, second task after the first task has been done, then there are nthen there are n11nn22 ways to do the procedure. ways to do the procedure.


Basic Counting PrinciplesBasic Counting PrinciplesExample:Example:How many different license plates are there How many different license plates are there that containing exactly three English letters ?that containing exactly three English letters ?

Solution:Solution:

There are 26 possibilities to pick the first There are 26 possibilities to pick the first letter, then 26 possibilities for the second one, letter, then 26 possibilities for the second one, and 26 for the last one. and 26 for the last one.

So there are 26So there are 26262626 = 17576 different 26 = 17576 different license plates.license plates.



Generalized product rule:Generalized product rule:

If we have a procedure consisting of If we have a procedure consisting of sequential tasks Tsequential tasks T11, T, T22, …, T, …, Tmm that can be done that can be done in nin n11, n, n22, …, n, …, nmm ways, respectively, then there ways, respectively, then there are nare n11 n n22 … … n nmm ways to carry out the ways to carry out the procedure.procedure.


Basic Counting PrinciplesBasic Counting PrinciplesThe sum and product rules can also be phrased The sum and product rules can also be phrased in terms of in terms of set theoryset theory..

Sum rule:Sum rule: Let A Let A11, A, A22, …, A, …, Amm be disjoint sets. be disjoint sets. Then the number of ways to choose any element Then the number of ways to choose any element from one of these sets is |Afrom one of these sets is |A11 A A22 … … A Amm | = | =|A|A11| + |A| + |A22| + … + |A| + … + |Amm|.|.

Product rule:Product rule: Let A Let A11, A, A22, …, A, …, Amm be finite sets. be finite sets. Then the number of ways to choose one element Then the number of ways to choose one element from each set in the order Afrom each set in the order A11, A, A22, …, A, …, Am m is is |A|A11 A A22 … … A Amm | = |A | = |A11| | |A |A22| | … … |A |Amm|.|.


Inclusion-ExclusionInclusion-ExclusionHow many bit strings of length 8 either start with How many bit strings of length 8 either start with a 1 or end with 00?a 1 or end with 00?

Task 1:Task 1: Construct a string of length 8 that starts Construct a string of length 8 that starts with a 1.with a 1.

There is one way to pick the first bit (1), There is one way to pick the first bit (1), two ways to pick the second bit (0 or 1),two ways to pick the second bit (0 or 1),two ways to pick the third bit (0 or 1),two ways to pick the third bit (0 or 1),......two ways to pick the eighth bit (0 or 1).two ways to pick the eighth bit (0 or 1).

Product rule:Product rule: Task 1 can be done in 1 Task 1 can be done in 12277 = 128 = 128 ways.ways.


Inclusion-ExclusionInclusion-ExclusionTask 2:Task 2: Construct a string of length 8 that ends Construct a string of length 8 that ends with 00.with 00.

There are two ways to pick the first bit (0 or 1), There are two ways to pick the first bit (0 or 1), two ways to pick the second bit (0 or 1),two ways to pick the second bit (0 or 1),......two ways to pick the sixth bit (0 or 1),two ways to pick the sixth bit (0 or 1),one way to pick the seventh bit (0), andone way to pick the seventh bit (0), andone way to pick the eighth bit (0).one way to pick the eighth bit (0).

Product rule:Product rule: Task 2 can be done in 2 Task 2 can be done in 266 = 64 = 64 ways.ways.


Inclusion-ExclusionInclusion-Exclusion

Since there are 128 ways to do Task 1 and 64 ways Since there are 128 ways to do Task 1 and 64 ways to do Task 2, does this mean that there are 192 bit to do Task 2, does this mean that there are 192 bit strings either starting with 1 or ending with 00 ?strings either starting with 1 or ending with 00 ?

No, because here Task 1 and Task 2 can be done No, because here Task 1 and Task 2 can be done at at the same timethe same time..

When we carry out Task 1 and create strings When we carry out Task 1 and create strings starting with 1, some of these strings end with 00.starting with 1, some of these strings end with 00.

Therefore, we sometimes do Tasks 1 and 2 at the Therefore, we sometimes do Tasks 1 and 2 at the same time, so same time, so the sum rule does not applythe sum rule does not apply..


Inclusion-ExclusionInclusion-Exclusion

If we want to use the sum rule in such a case, If we want to use the sum rule in such a case, we have to subtract the cases when Tasks 1 we have to subtract the cases when Tasks 1 and 2 are done at the same time.and 2 are done at the same time.

How many cases are there, that is, how many How many cases are there, that is, how many strings start with 1 strings start with 1 andand end with 00? end with 00?

There is one way to pick the first bit (1), There is one way to pick the first bit (1), two ways for the second, …, sixth bit (0 or 1),two ways for the second, …, sixth bit (0 or 1),one way for the seventh, eighth bit (0).one way for the seventh, eighth bit (0).

Product rule:Product rule: In 2 In 255 = 32 cases, Tasks 1 and 2 = 32 cases, Tasks 1 and 2 are carried out at the same time.are carried out at the same time.


Inclusion-ExclusionInclusion-ExclusionSince there are 128 ways to complete Task 1 and Since there are 128 ways to complete Task 1 and 64 ways to complete Task 2, and in 32 of these 64 ways to complete Task 2, and in 32 of these cases Tasks 1 and 2 are completed at the same cases Tasks 1 and 2 are completed at the same time, there aretime, there are

128 + 64 – 32 = 160 ways to do either task.128 + 64 – 32 = 160 ways to do either task.

In set theory, this corresponds to sets AIn set theory, this corresponds to sets A11 and A and A22 that are that are notnot disjoint. Then we have: disjoint. Then we have:

|A|A11 A A22| = |A| = |A11| + |A| + |A22| - |A| - |A11 A A22||

This is called the This is called the principle of inclusion-principle of inclusion-exclusion.exclusion.


Tree DiagramsTree DiagramsHow many bit strings of length four do not have How many bit strings of length four do not have two consecutive 1s?two consecutive 1s?

Task 1Task 1 Task 2Task 2 Task 3Task 3 Task 4Task 4(1(1stst bit) bit) (2(2ndnd bit) bit) (3(3rdrd bit) bit) (4(4thth bit) bit)

00

0000

00

1111

0011 00 00

11

11 0000 00

1111

00There are 8 strings.There are 8 strings.


The Pigeonhole PrincipleThe Pigeonhole PrincipleThe pigeonhole principle:The pigeonhole principle: If (k + 1) or more If (k + 1) or more objects are placed into k boxes, then there is at objects are placed into k boxes, then there is at least one box containing two or more of the least one box containing two or more of the objects.objects.

Example 1:Example 1: If there are 11 players in a soccer If there are 11 players in a soccer team that wins 12-0, there must be at least one team that wins 12-0, there must be at least one player in the team who scored at least twice.player in the team who scored at least twice.

Example 2:Example 2: If you have 6 classes from Monday If you have 6 classes from Monday to Friday, there must be at least one day on to Friday, there must be at least one day on which you have at least two classes.which you have at least two classes.


The Pigeonhole PrincipleThe Pigeonhole Principle

The generalized pigeonhole principle:The generalized pigeonhole principle: If N If N objects are placed into k boxes, then there is at objects are placed into k boxes, then there is at least one box containing at least least one box containing at least N/kN/k of the of the objects.objects.

Example 1:Example 1: In our 60-student class, at least 12 In our 60-student class, at least 12 students will get the same letter grade (A, B, C, students will get the same letter grade (A, B, C, D, or F).D, or F).


The Pigeonhole PrincipleThe Pigeonhole Principle

Example 2:Example 2: Assume you have a drawer Assume you have a drawer containing a random distribution of a dozen containing a random distribution of a dozen brown socks and a dozen black socks. It is dark, brown socks and a dozen black socks. It is dark, so how many socks do you have to pick to be so how many socks do you have to pick to be sure that among them there is a matching pair?sure that among them there is a matching pair?

There are two types of socks, so if you pick at There are two types of socks, so if you pick at least 3 socks, there must be either at least two least 3 socks, there must be either at least two brown socks or at least two black socks.brown socks or at least two black socks.

Generalized pigeonhole principle: Generalized pigeonhole principle: 3/23/2 = 2. = 2.


Permutations and Permutations and CombinationsCombinations

How many ways are there to pick a set of 3 How many ways are there to pick a set of 3 people from a group of 6?people from a group of 6?

There are 6 choices for the first person, 5 for There are 6 choices for the first person, 5 for the second one, and 4 for the third one, so the second one, and 4 for the third one, so there arethere are66554 = 120 ways to do this.4 = 120 ways to do this.

This is not the correct result!This is not the correct result!

For example, picking person C, then person A, For example, picking person C, then person A, and then person E leads to the and then person E leads to the same groupsame group as as first picking E, then C, and then A.first picking E, then C, and then A.However, these cases are counted However, these cases are counted separatelyseparately in the above equation.in the above equation.



So how can we compute how many different So how can we compute how many different subsets of people can be picked (that is, we subsets of people can be picked (that is, we want to disregard the order of picking) ?want to disregard the order of picking) ?

To find out about this, we need to look at To find out about this, we need to look at permutations.permutations.

A A permutationpermutation of a set of distinct objects is an of a set of distinct objects is an ordered arrangement of these objects.ordered arrangement of these objects.

An ordered arrangement of r elements of a set An ordered arrangement of r elements of a set is called an is called an r-permutationr-permutation..



Example:Example: Let S = {1, 2, 3}. Let S = {1, 2, 3}.The arrangement 3, 1, 2 is a permutation of S.The arrangement 3, 1, 2 is a permutation of S.The arrangement 3, 2 is a 2-permutation of S.The arrangement 3, 2 is a 2-permutation of S.

The number of r-permutations of a set with n The number of r-permutations of a set with n distinct elements is denoted by distinct elements is denoted by P(n, r).P(n, r).

We can calculate P(n, r) with the product rule:We can calculate P(n, r) with the product rule:

P(n, r) = nP(n, r) = n(n – 1)(n – 1)(n – 2) (n – 2) ……(n – r + 1).(n – r + 1).

(n choices for the first element, (n – 1) for the (n choices for the first element, (n – 1) for the second one, (n – 2) for the third one…)second one, (n – 2) for the third one…)



Example:Example:

P(8, 3) = 8P(8, 3) = 8776 = 3366 = 336 = (8= (87766554433221)/(51)/(54433221)1)

General formula:General formula:

P(n, r) = n!/(n – r)!P(n, r) = n!/(n – r)!

Knowing this, we can return to our initial Knowing this, we can return to our initial question:question:How many ways are there to pick a set of 3 How many ways are there to pick a set of 3 people from a group of 6 (disregarding the people from a group of 6 (disregarding the order of picking)?order of picking)?



An An r-combinationr-combination of elements of a set is an of elements of a set is an unordered selection of r elements from the set.unordered selection of r elements from the set.Thus, an r-combination is simply a subset of the Thus, an r-combination is simply a subset of the set with r elements.set with r elements.

Example:Example: Let S = {1, 2, 3, 4}. Let S = {1, 2, 3, 4}.Then {1, 3, 4} is a 3-combination from S.Then {1, 3, 4} is a 3-combination from S.

The number of r-combinations of a set with n The number of r-combinations of a set with n distinct elements is denoted by C(n, r).distinct elements is denoted by C(n, r).

Example:Example: C(4, 2) = 6, since, for example, the C(4, 2) = 6, since, for example, the 2-combinations of a set {1, 2, 3, 4} are {1, 2}, 2-combinations of a set {1, 2, 3, 4} are {1, 2}, {1, 3}, {1, 4}, {2, 3}, {2, 4}, {3, 4}.{1, 3}, {1, 4}, {2, 3}, {2, 4}, {3, 4}.



How can we calculate C(n, r)?How can we calculate C(n, r)?

Consider that we can obtain the r-permutation Consider that we can obtain the r-permutation of a set in the following way:of a set in the following way:

First,First, we form all the r-combinations of the set we form all the r-combinations of the set(there are C(n, r) such r-combinations).(there are C(n, r) such r-combinations).

Then,Then, we generate all possible orderings in we generate all possible orderings in each of these r-combinations (there are P(r, r) each of these r-combinations (there are P(r, r) such orderings in each case).such orderings in each case).

Therefore, we have:Therefore, we have:

P(n, r) = C(n, r)P(n, r) = C(n, r)P(r, r)P(r, r)



C(n, r) = P(n, r)/P(r, r)C(n, r) = P(n, r)/P(r, r) = n!/(n – r)!/(r!/(r – r)!)= n!/(n – r)!/(r!/(r – r)!) = n!/(r!(n – r)!)= n!/(r!(n – r)!)

Now we can answer our initial question:Now we can answer our initial question:

How many ways are there to pick a set of 3 How many ways are there to pick a set of 3 people from a group of 6 (disregarding the people from a group of 6 (disregarding the order of picking)?order of picking)?

C(6, 3) = 6!/(3!C(6, 3) = 6!/(3!3!) = 720/(63!) = 720/(66) = 720/36 = 206) = 720/36 = 20

There are 20 different ways, that is, 20 different There are 20 different ways, that is, 20 different groups to be picked.groups to be picked.



Corollary:Corollary: Let n and r be nonnegative integers with r Let n and r be nonnegative integers with r n. n.Then C(n, r) = C(n, n – r).Then C(n, r) = C(n, n – r).

Note that Note that “picking a group of r people from “picking a group of r people from a group of n people”a group of n people” is the same as is the same as “splitting a group of n people into a group “splitting a group of n people into a group of r people and another group of (n – r) of r people and another group of (n – r) people”.people”.

Please also look at proof on page 252.Please also look at proof on page 252.



Example:A soccer club has 8 female and 7 male members. For today’s match, the coach wants to have 6 female and 5 male players on the grass. How many possible configurations are there?

C(8, 6) C(7, 5) = 8!/(6! C(7, 5) = 8!/(6!2!) 2!) 7!/(5! 7!/(5!2!)2!) = 28= 2821 21 = 588= 588


CombinationsCombinationsWe also saw the following:We also saw the following:

),(!)!(

!

)]!([)!(

!),( rnC

rrn

n

rnnrn

nrnnC

This symmetry is intuitively plausible. For This symmetry is intuitively plausible. For example, let us consider a set containing six example, let us consider a set containing six elements (n = 6).elements (n = 6).

Picking twoPicking two elements and elements and leaving fourleaving four is is essentially the same as essentially the same as picking fourpicking four elements elements and and leaving twoleaving two..

In either case, our number of choices is the In either case, our number of choices is the number of possibilities to number of possibilities to dividedivide the set into the set into one set containing two elements and another one set containing two elements and another set containing four elements.set containing four elements.


CombinationsCombinations

Pascal’s Identity:Pascal’s Identity:

Let n and k be positive integers with n Let n and k be positive integers with n k. k.Then C(n + 1, k) = C(n, k – 1) + C(n, k).Then C(n + 1, k) = C(n, k – 1) + C(n, k).

How can this be explained?How can this be explained?

What is it good for?What is it good for?


CombinationsCombinationsImagine a set S containing n elements and a set Imagine a set S containing n elements and a set T containing (n + 1) elements, namely all T containing (n + 1) elements, namely all elements in S plus a new element elements in S plus a new element aa..

Calculating C(n + 1, k) is equivalent to answering Calculating C(n + 1, k) is equivalent to answering the question: How many subsets of T containing k the question: How many subsets of T containing k items are there?items are there?

Case I:Case I: The subset contains (k – 1) elements of S The subset contains (k – 1) elements of S plus the element plus the element aa: C(n, k – 1) choices.: C(n, k – 1) choices.

Case II:Case II: The subset contains k elements of S and The subset contains k elements of S and does not contain does not contain aa: C(n, k) choices.: C(n, k) choices.

Sum Rule:Sum Rule: C(n + 1, k) = C(n, k – 1) + C(n, k). C(n + 1, k) = C(n, k – 1) + C(n, k).


Pascal’s TrianglePascal’s TriangleIn Pascal’s triangle, each number is the sum of In Pascal’s triangle, each number is the sum of the numbers to its upper left and upper right:the numbers to its upper left and upper right:

11

11 11

11 22 11

11 33 33 11

11 44 66 44 11

…… …… …… …… …… ……


Pascal’s TrianglePascal’s TriangleSince we have C(n + 1, k) = C(n, k – 1) + C(n, k) andSince we have C(n + 1, k) = C(n, k – 1) + C(n, k) andC(0, 0) = 1, we can use Pascal’s triangle to simplify C(0, 0) = 1, we can use Pascal’s triangle to simplify the computation of C(n, k):the computation of C(n, k):

C(0, 0) = C(0, 0) = 11

C(1, 0) = C(1, 0) = 11C(1, 1) = C(1, 1) = 11

C(2, 0) = C(2, 0) = 11C(2, 1) = C(2, 1) = 22C(2, 2) = C(2, 2) = 11

C(3, 0) = C(3, 0) = 11C(3, 1) = C(3, 1) = 33C(3, 2) = C(3, 2) = 33C(3, 3) = C(3, 3) = 11

C(4, 0) = C(4, 0) = 11C(4, 1) = C(4, 1) = 44C(4, 2) = C(4, 2) = 66C(4, 3) = C(4, 3) = 44C(4, 4) = C(4, 4) = 11

kk

nn


Binomial CoefficientsBinomial CoefficientsExpressions of the form C(n, k) are also called Expressions of the form C(n, k) are also called binomial coefficientsbinomial coefficients..How come?How come?A A binomial expressionbinomial expression is the sum of two is the sum of two terms, such as (a + b).terms, such as (a + b).Now consider (a + b)Now consider (a + b)2 2 = (a + b)(a + b).= (a + b)(a + b).When expanding such expressions, we have When expanding such expressions, we have to form all possible products of a term in the to form all possible products of a term in the first factor and a term in the second factor:first factor and a term in the second factor:(a + b)(a + b)22 = a·a + a·b + b·a + b·b = a·a + a·b + b·a + b·bThen we can sum identical terms:Then we can sum identical terms:(a + b)(a + b)22 = a = a22 + 2ab + b + 2ab + b22


Binomial CoefficientsBinomial CoefficientsFor (a + b)For (a + b)3 3 = (a + b)(a + b)(a + b) we have= (a + b)(a + b)(a + b) we have

(a + b)(a + b)3 3 = = aaa + aab + aba + abb + baa + bab + aaa + aab + aba + abb + baa + bab + bba + bbbbba + bbb

(a + b)(a + b)3 3 = a= a33 + 3a + 3a22b + 3abb + 3ab22 + b + b33

There is only one term a3, because there is only one possibility to form it: Choose a from all three factors: C(3, 3) = 1.There is three times the term a2b, because there are three possibilities to choose a from two out of the three factors: C(3, 2) = 3.Similarly, there is three times the term ab2 (C(3, 1) = 3) and once the term b3 (C(3, 0) = 1).


Binomial CoefficientsBinomial Coefficients

This leads us to the following formula:This leads us to the following formula:

jn

j

jnn bajnCba

0

),()(

With the help of Pascal’s triangle, this formula can With the help of Pascal’s triangle, this formula can considerably simplify the process of expanding considerably simplify the process of expanding powers of binomial expressions.powers of binomial expressions.

For example, the fifth row of Pascal’s triangleFor example, the fifth row of Pascal’s triangle(1 – 4 – 6 – 4 – 1) helps us to compute (a + b)(1 – 4 – 6 – 4 – 1) helps us to compute (a + b)44::

(a + b)(a + b)44 = a = a44 + 4a + 4a33b + 6ab + 6a22bb22 + 4ab + 4ab33 + b + b44

(Binomial (Binomial Theorem)Theorem)


Now it’s Time for…Now it’s Time for…

RecurrenRecurrencece

RelationsRelations


Recurrence RelationsRecurrence Relations

A A recurrence relationrecurrence relation for the sequence {a for the sequence {ann} } is an equation that expresses ais an equation that expresses ann is terms of is terms of one or more of the previous terms of the one or more of the previous terms of the sequence, namely, asequence, namely, a00, a, a11, …, a, …, an-1n-1, for all , for all integers n with integers n with n n n n00, where n, where n00 is a nonnegative integer. is a nonnegative integer.

A sequence is called a A sequence is called a solutionsolution of a of a recurrence relation if it terms satisfy the recurrence relation if it terms satisfy the recurrence relation.recurrence relation.



In other words, a recurrence relation is like a In other words, a recurrence relation is like a recursively defined sequence, but recursively defined sequence, but without without specifying any initial values (initial specifying any initial values (initial conditions)conditions)..

Therefore, the same recurrence relation can Therefore, the same recurrence relation can have (and usually has) have (and usually has) multiple solutionsmultiple solutions..

If If bothboth the initial conditions and the the initial conditions and the recurrence relation are specified, then the recurrence relation are specified, then the sequence is sequence is uniquely uniquely determined.determined.



Example:Example: Consider the recurrence relation Consider the recurrence relation aann = 2a = 2an-1n-1 – a – an-2n-2 for n = 2, 3, 4, … for n = 2, 3, 4, …

Is the sequence {aIs the sequence {ann} with a} with ann=3n a solution of =3n a solution of this recurrence relation?this recurrence relation?

For n For n 2 we see that 2 we see that 2a2an-1n-1 – a – an-2n-2 = 2(3(n – 1)) – 3(n – 2) = 3n = a = 2(3(n – 1)) – 3(n – 2) = 3n = ann..

Therefore, {aTherefore, {ann} with a} with ann=3n is a solution of the =3n is a solution of the recurrence relation.recurrence relation.



Is the sequence {aIs the sequence {ann} with a} with ann=5 a solution of =5 a solution of the same recurrence relation?the same recurrence relation?

For n For n 2 we see that 2 we see that 2a2an-1n-1 – a – an-2n-2 = 2 = 25 - 5 = 5 = a5 - 5 = 5 = ann..

Therefore, {aTherefore, {ann} with a} with ann=5 is also a solution of =5 is also a solution of the recurrence relation.the recurrence relation.


Modeling with Recurrence Modeling with Recurrence RelationsRelations

Example:Example:

Someone deposits $10,000 in a savings Someone deposits $10,000 in a savings account at a bank yielding 5% per year with account at a bank yielding 5% per year with interest compounded annually. How much interest compounded annually. How much money will be in the account after 30 years?money will be in the account after 30 years?

Solution:Solution:

Let PLet Pnn denote the amount in the account after denote the amount in the account after n years.n years.

How can we determine PHow can we determine Pnn on the basis of P on the basis of Pn-1n-1??



We can derive the following We can derive the following recurrence relationrecurrence relation::

PPnn = P = Pn-1n-1 + 0.05P + 0.05Pn-1n-1 = 1.05P = 1.05Pn-1n-1..

The initial condition is PThe initial condition is P00 = 10,000. = 10,000.Then we have:Then we have:

PP11 = 1.05P = 1.05P00

PP22 = 1.05P = 1.05P11 = (1.05) = (1.05)22PP00

PP33 = 1.05P = 1.05P22 = (1.05) = (1.05)33PP00

……

PPnn = 1.05P = 1.05Pn-1n-1 = (1.05) = (1.05)nnPP00

We now have a We now have a formulaformula to calculate P to calculate Pnn for any for any natural number n and can avoid the iteration.natural number n and can avoid the iteration.



Let us use this formula to find PLet us use this formula to find P3030 under the under the

initial condition Pinitial condition P00 = 10,000: = 10,000:

PP3030 = (1.05) = (1.05)303010,000 = 43,219.4210,000 = 43,219.42

After 30 years, the account contains After 30 years, the account contains $43,219.42.$43,219.42.




Let aLet ann denote the number of bit strings of denote the number of bit strings of length n that do not have two consecutive 0s length n that do not have two consecutive 0s (“valid strings”). Find a recurrence relation (“valid strings”). Find a recurrence relation and give initial conditions for the sequence and give initial conditions for the sequence {a{ann}.}.

Solution:Solution:

Idea: The number of valid strings equals the Idea: The number of valid strings equals the number of valid strings ending with a 0 plus number of valid strings ending with a 0 plus the number of valid strings ending with a 1.the number of valid strings ending with a 1.



Let us assume that n Let us assume that n 3, so that the string 3, so that the string contains at least 3 bits.contains at least 3 bits.

Let us further assume that we know the Let us further assume that we know the number anumber an-1n-1 of valid strings of length (n – 1). of valid strings of length (n – 1).

Then how many valid strings of length n are Then how many valid strings of length n are there, if the string ends with a 1?there, if the string ends with a 1?

There are aThere are an-1n-1 such strings, namely the set of such strings, namely the set of valid strings of length (n – 1) with a 1 valid strings of length (n – 1) with a 1 appended to them.appended to them.

Note:Note: Whenever we append a 1 to a valid Whenever we append a 1 to a valid string, that string remains valid.string, that string remains valid.



Now we need to know: How many valid strings of Now we need to know: How many valid strings of length n are there, if the string ends with a length n are there, if the string ends with a 00??

Valid strings of length n ending with a 0 Valid strings of length n ending with a 0 must must have a 1 as their (n – 1)st bithave a 1 as their (n – 1)st bit (otherwise they (otherwise they would end with 00 and would not be valid).would end with 00 and would not be valid).

And what is the number of valid strings of length And what is the number of valid strings of length (n – 1) that end with a 1?(n – 1) that end with a 1?

We already know that there are aWe already know that there are an-1n-1 strings of strings of length n that end with a 1.length n that end with a 1.

Therefore, there are aTherefore, there are an-2n-2 strings of length (n – 1) strings of length (n – 1) that end with a 1.that end with a 1.



So there are aSo there are an-2n-2 valid strings of length n that valid strings of length n that end with a 0 (all valid strings of length (n – 2) end with a 0 (all valid strings of length (n – 2) with 10 appended to them).with 10 appended to them).

As we said before, the number of valid strings As we said before, the number of valid strings is the number of valid strings ending with a 0 is the number of valid strings ending with a 0 plus the number of valid strings ending with a plus the number of valid strings ending with a 1.1.

That gives us the following That gives us the following recurrence recurrence relationrelation::

aann = a = an-1n-1 + a + an-2n-2



What are the What are the initial conditionsinitial conditions??

aa11 = 2 (0 and 1) = 2 (0 and 1)

aa22 = 3 (01, 10, and 11) = 3 (01, 10, and 11)

aa33 = a = a22 + a + a11 = 3 + 2 = 5 = 3 + 2 = 5

aa44 = a = a33 + a + a22 = 5 + 3 = 8 = 5 + 3 = 8

aa55 = a = a44 + a + a33 = 8 + 5 = 13 = 8 + 5 = 13

……

This sequence satisfies the same recurrence This sequence satisfies the same recurrence relation as the relation as the Fibonacci sequenceFibonacci sequence..

Since aSince a11 = f = f33 and a and a22 = f = f44, we have a, we have ann = f = fn+2n+2..


Solving Recurrence RelationsSolving Recurrence Relations

In general, we would prefer to have an In general, we would prefer to have an explicit formulaexplicit formula to compute the value of a to compute the value of ann rather than conducting n iterations.rather than conducting n iterations.

For one class of recurrence relations, we can For one class of recurrence relations, we can obtain such formulas in a systematic way.obtain such formulas in a systematic way.

Those are the recurrence relations that Those are the recurrence relations that express the terms of a sequence as express the terms of a sequence as linear linear combinationscombinations of previous terms. of previous terms.



Definition:Definition: A linear homogeneous recurrence A linear homogeneous recurrence relation of degree k with constant coefficients relation of degree k with constant coefficients is a recurrence relation of the form:is a recurrence relation of the form:

aann = c = c11aan-1n-1 + c + c22aan-2n-2 + … + c + … + ckkaan-kn-k,,

Where cWhere c11, c, c22, …, c, …, ckk are real numbers, and c are real numbers, and ckk 0. 0.

A sequence satisfying such a recurrence A sequence satisfying such a recurrence relation is uniquely determined by the relation is uniquely determined by the recurrence relation and the k initial conditionsrecurrence relation and the k initial conditions

aa00 = C = C00, a, a11 = C = C11, a, a22 = C = C22, …, a, …, ak-1k-1 = C = Ck-1k-1..



Examples:Examples:

The recurrence relation PThe recurrence relation Pnn = (1.05)P = (1.05)Pn-1n-1

is a linear homogeneous recurrence relation of is a linear homogeneous recurrence relation of degree onedegree one..

The recurrence relation fThe recurrence relation fnn = f = fn-1n-1 + f + fn-2n-2

is a linear homogeneous recurrence relation of is a linear homogeneous recurrence relation of degree twodegree two..

The recurrence relation aThe recurrence relation ann = a = an-5n-5

is a linear homogeneous recurrence relation of is a linear homogeneous recurrence relation of degree fivedegree five..


Solving Recurrence RelationsSolving Recurrence RelationsBasically, when solving such recurrence Basically, when solving such recurrence relations, we try to find solutions of the form relations, we try to find solutions of the form aann = r = rnn, where r is a constant., where r is a constant.

aann = r = rnn is a solution of the recurrence relation is a solution of the recurrence relationaann = c = c11aan-1n-1 + c + c22aan-2n-2 + … + c + … + ckkaan-kn-k if and only if if and only if

rrnn = c = c11rrn-1 n-1 + c+ c22rrn-2 n-2 + … + c+ … + ckkrrn-kn-k..

Divide this equation by rDivide this equation by rn-kn-k and subtract the and subtract the right-hand side from the left:right-hand side from the left:

rrkk - c - c11rrk-1 k-1 - c- c22rrk-2 k-2 - … - c- … - ck-1k-1r - cr - ckk = 0 = 0

This is called the This is called the characteristic equationcharacteristic equation of of the recurrence relation.the recurrence relation.


Solving Recurrence RelationsSolving Recurrence RelationsThe solutions of this equation are called the The solutions of this equation are called the characteristic rootscharacteristic roots of the recurrence relation. of the recurrence relation.

Let us consider linear homogeneous recurrence Let us consider linear homogeneous recurrence relations of relations of degree twodegree two..

Theorem:Theorem: Let c Let c11 and c and c22 be real numbers. Suppose be real numbers. Suppose that rthat r22 – c – c11r – cr – c22 = 0 has two distinct roots r = 0 has two distinct roots r11 and r and r22..

Then the sequence {aThen the sequence {ann} is a solution of the } is a solution of the recurrence relation arecurrence relation ann = c = c11aan-1n-1 + c + c22aan-2n-2 if and only if if and only if aann = = 11rr11

nn + + 22rr22nn for n = 0, 1, 2, …, where for n = 0, 1, 2, …, where 11 and and

22 are constants. are constants.

See pp. 321 and 322 for the proof.See pp. 321 and 322 for the proof.



Example:Example: What is the solution of the recurrence What is the solution of the recurrence relation arelation ann = a = an-1n-1 + 2a + 2an-2n-2 with a with a00 = 2 and a = 2 and a11 = 7 ? = 7 ?

Solution:Solution: The characteristic equation of the The characteristic equation of the recurrence relation is rrecurrence relation is r22 – r – 2 = 0. – r – 2 = 0.

Its roots are r = 2 and r = -1.Its roots are r = 2 and r = -1.

Hence, the sequence {aHence, the sequence {ann} is a solution to the } is a solution to the recurrence relation if and only if:recurrence relation if and only if:

aann = = 1122n n + + 22(-1)(-1)nn for some constants for some constants 1 1 and and 22..



Given the equation aGiven the equation ann = = 1122n n + + 22(-1)(-1)nn and the and the initial conditions ainitial conditions a00 = 2 and a = 2 and a11 = 7, it follows that = 7, it follows that

aa00 = 2 = = 2 = 1 1 + + 22

aa11 = 7 = = 7 = 112 + 2 + 2 2 (-1)(-1)

Solving these two equations gives usSolving these two equations gives us11 = 3 and = 3 and 22 = -1. = -1.

Therefore, the solution to the recurrence relation Therefore, the solution to the recurrence relation and initial conditions is the sequence {aand initial conditions is the sequence {ann} with} with

aann = 3 = 322nn – (-1) – (-1)nn. .


Solving Recurrence RelationsSolving Recurrence Relationsaann = r = rnn is a solution of the linear homogeneous is a solution of the linear homogeneous recurrence relationrecurrence relationaann = c = c11aan-1n-1 + c + c22aan-2n-2 + … + c + … + ckkaan-kn-k

if and only ifif and only if

rrnn = c = c11rrn-1 n-1 + c+ c22rrn-2 n-2 + … + c+ … + ckkrrn-kn-k..

Divide this equation by rDivide this equation by rn-kn-k and subtract the and subtract the right-hand side from the left:right-hand side from the left:

rrkk - c - c11rrk-1 k-1 - c- c22rrk-2 k-2 - … - c- … - ck-1k-1r - cr - ckk = 0 = 0

This is called the This is called the characteristic equationcharacteristic equation of of the recurrence relation.the recurrence relation.


Solving Recurrence RelationsSolving Recurrence RelationsThe solutions of this equation are called the The solutions of this equation are called the characteristic rootscharacteristic roots of the recurrence relation. of the recurrence relation.

Let us consider linear homogeneous recurrence Let us consider linear homogeneous recurrence relations of relations of degree twodegree two..

Theorem:Theorem: Let c Let c11 and c and c22 be real numbers. Suppose be real numbers. Suppose that rthat r22 – c – c11r – cr – c22 = 0 has two distinct roots r = 0 has two distinct roots r11 and r and r22..

Then the sequence {aThen the sequence {ann} is a solution of the } is a solution of the recurrence relation arecurrence relation ann = c = c11aan-1n-1 + c + c22aan-2n-2 if and only if if and only if aann = = 11rr11

nn + + 22rr22nn for n = 0, 1, 2, …, where for n = 0, 1, 2, …, where 11 and and

22 are constants. are constants.

See pp. 321 and 322 for the proof.See pp. 321 and 322 for the proof.



Example:Example: Give an explicit formula for the Give an explicit formula for the Fibonacci numbers.Fibonacci numbers.

Solution:Solution: The Fibonacci numbers satisfy the The Fibonacci numbers satisfy the recurrence relation frecurrence relation fnn = f = fn-1n-1 + f + fn-2n-2 with initial with initial conditions fconditions f00 = 0 and f = 0 and f11 = 1. = 1.

The characteristic equation is rThe characteristic equation is r22 – r – 1 = 0. – r – 1 = 0.

Its roots areIts roots are

2

51,

2

5121

rr


Solving Recurrence RelationsSolving Recurrence RelationsTherefore, the Fibonacci numbers are given byTherefore, the Fibonacci numbers are given by

nn

nf

2

51

2

5121

for some constants for some constants 11 and and 22..

We can determine values for these constants We can determine values for these constants so that the sequence meets the conditions fso that the sequence meets the conditions f00

= 0 and f= 0 and f11 = 1: = 1: 0210 f

12

51

2

51211

f



The unique solution to this system of two The unique solution to this system of two equations and two variables isequations and two variables is

5

1,

5

121

So finally we obtained an explicit formula for So finally we obtained an explicit formula for the Fibonacci numbers:the Fibonacci numbers:

nn

nf

2

51

5

1

2

51

5

1



But what happens if the characteristic equation But what happens if the characteristic equation has only one root?has only one root?

How can we then match our equation with the How can we then match our equation with the initial conditions ainitial conditions a00 and a and a1 1 ??

Theorem:Theorem: Let c Let c11 and c and c22 be real numbers with be real numbers with cc22 0. Suppose that r 0. Suppose that r22 – c – c11r – cr – c22 = 0 has only one = 0 has only one root rroot r00. . A sequence {aA sequence {ann} is a solution of the recurrence } is a solution of the recurrence relation arelation ann = c = c11aan-1n-1 + c + c22aan-2n-2 if and only if if and only if aann = = 11rr00

nn + + 22nrnr00nn, for n = 0, 1, 2, …, where , for n = 0, 1, 2, …, where 11

and and 22 are constants. are constants.


Solving Recurrence RelationsSolving Recurrence RelationsExample:Example: What is the solution of the recurrence What is the solution of the recurrence relation arelation ann = 6a = 6an-1n-1 – 9a – 9an-2n-2 with a with a00 = 1 and a = 1 and a11 = 6? = 6?

Solution:Solution: The only root of r The only root of r22 – 6r + 9 = 0 is r – 6r + 9 = 0 is r00 = 3. = 3.Hence, the solution to the recurrence relation isHence, the solution to the recurrence relation is

aann = = 1133nn + + 22n3n3nn for some constants for some constants 11 and and 22..

To match the initial condition, we needTo match the initial condition, we need

aa00 = 1 = = 1 = 11

aa11 = 6 = = 6 = 113 + 3 + 2233

Solving these equations yields Solving these equations yields 11 = 1 and = 1 and 22 = 1. = 1.

Consequently, the overall solution is given byConsequently, the overall solution is given by

aann = 3 = 3nn + n3 + n3nn..


You Never Escape Your…You Never Escape Your…

RelationsRelations


RelationsRelationsIf we want to describe a relationship between If we want to describe a relationship between elements of two sets A and B, we can use elements of two sets A and B, we can use ordered pairsordered pairs with their first element taken with their first element taken from A and their second element taken from B. from A and their second element taken from B.

Since this is a relation between Since this is a relation between two setstwo sets, it is , it is called a called a binary relationbinary relation..

Definition:Definition: Let A and B be sets. A binary Let A and B be sets. A binary relation from A to B is a subset of Arelation from A to B is a subset of AB.B.

In other words, for a binary relation R we have In other words, for a binary relation R we have R R A AB. We use the notation aRb to denote B. We use the notation aRb to denote that (a, b)that (a, b)R and aR and aRb to denote that (a, b)b to denote that (a, b)R.R.


RelationsRelationsWhen (a, b) belongs to R, a is said to be When (a, b) belongs to R, a is said to be relatedrelated to b by R. to b by R.Example:Example: Let P be a set of people, C be a set Let P be a set of people, C be a set of cars, and D be the relation describing which of cars, and D be the relation describing which person drives which car(s).person drives which car(s).P = {Carl, Suzanne, Peter, Carla}, P = {Carl, Suzanne, Peter, Carla}, C = {Mercedes, BMW, tricycle}C = {Mercedes, BMW, tricycle}D = {(Carl, Mercedes), (Suzanne, Mercedes),D = {(Carl, Mercedes), (Suzanne, Mercedes), (Suzanne, BMW), (Peter, tricycle)} (Suzanne, BMW), (Peter, tricycle)}This means that Carl drives a Mercedes, This means that Carl drives a Mercedes, Suzanne drives a Mercedes and a BMW, Peter Suzanne drives a Mercedes and a BMW, Peter drives a tricycle, and Carla does not drive any drives a tricycle, and Carla does not drive any of these vehicles.of these vehicles.


Functions as RelationsFunctions as Relations

You might remember that a You might remember that a functionfunction f from a f from a set A to a set B assigns a unique element of B set A to a set B assigns a unique element of B to each element of A.to each element of A.

The The graphgraph of f is the set of ordered pairs (a, b) of f is the set of ordered pairs (a, b) such that b = f(a).such that b = f(a).

Since the graph of f is a subset of ASince the graph of f is a subset of AB, it is a B, it is a relationrelation from A to B. from A to B.

Moreover, for each element Moreover, for each element aa of A, there is of A, there is exactly one ordered pair in the graph that has exactly one ordered pair in the graph that has aa as its first element. as its first element.


Functions as RelationsFunctions as Relations

Conversely, if R is a relation from A to B such Conversely, if R is a relation from A to B such that every element in A is the first element of that every element in A is the first element of exactly one ordered pair of R, then a function exactly one ordered pair of R, then a function can be defined with R as its graph.can be defined with R as its graph.

This is done by assigning to an element aThis is done by assigning to an element aA A the unique element bthe unique element bB such that (a, b)B such that (a, b)R.R.


Relations on a SetRelations on a Set

Definition:Definition: A relation on the set A is a relation A relation on the set A is a relation from A to A.from A to A.

In other words, a relation on the set A is a In other words, a relation on the set A is a subset of Asubset of AA.A.

Example:Example: Let A = {1, 2, 3, 4}. Which ordered Let A = {1, 2, 3, 4}. Which ordered pairs are in the relation R = {(a, b) | a < b} ?pairs are in the relation R = {(a, b) | a < b} ?


Relations on a SetRelations on a Set

Solution:Solution: R = { R = {(1, 2),(1, 2),(1, 3),(1, 3),(1, 4),(1, 4),(2, 3),(2, 3),(2, 4),(2, 4),(3, 4)}(3, 4)}

44

33

22

11

44332211RR11 11

22

33

44

22

33

44

XX XX XX

XX XX

XX


Relations on a SetRelations on a SetHow many different relations can we How many different relations can we define on a set A with n elements?define on a set A with n elements?

A relation on a set A is a subset of AA relation on a set A is a subset of AA.A.How many elements are in AHow many elements are in AA ?A ?

There are nThere are n22 elements in A elements in AA, so how many A, so how many subsets (= relations on A) does Asubsets (= relations on A) does AA have?A have?

The number of subsets that we can form out of The number of subsets that we can form out of a set with m elements is 2a set with m elements is 2mm. Therefore, 2. Therefore, 2nn22 subsets can be formed out of Asubsets can be formed out of AA.A.

Answer:Answer: We can define 2 We can define 2nn22 different relations different relations on A.on A.


Properties of RelationsProperties of RelationsWe will now look at some useful ways to We will now look at some useful ways to classify relations.classify relations.

Definition:Definition: A relation R on a set A is called A relation R on a set A is called reflexivereflexive if (a, a) if (a, a)R for every element aR for every element aA.A.

Are the following relations on {1, 2, 3, 4} Are the following relations on {1, 2, 3, 4} reflexive?reflexive?R = {(1, 1), (1, 2), (2, 3), (3, 3), (4, 4)}R = {(1, 1), (1, 2), (2, 3), (3, 3), (4, 4)} No.No.R = {(1, 1), (2, 2), (2, 3), (3, 3), (4, 4)}R = {(1, 1), (2, 2), (2, 3), (3, 3), (4, 4)} Yes.Yes.

R = {(1, 1), (2, 2), (3, 3)}R = {(1, 1), (2, 2), (3, 3)} No.No.

Definition:Definition: A relation on a set A is called A relation on a set A is called irreflexiveirreflexive if (a, a) if (a, a)R for every element aR for every element aA.A.


Properties of RelationsProperties of Relations

Definitions:Definitions:

A relation R on a set A is called A relation R on a set A is called symmetricsymmetric if if (b, a)(b, a)R whenever (a, b)R whenever (a, b)R for all a, bR for all a, bA. A.

A relation R on a set A is called A relation R on a set A is called antisymmetricantisymmetric if if a = b whenever (a, b)a = b whenever (a, b)R and (b, a)R and (b, a)R.R.

A relation R on a set A is called A relation R on a set A is called asymmetricasymmetric if if (a, b)(a, b)R implies that (b, a)R implies that (b, a)R for all a, bR for all a, bA. A.


Properties of RelationsProperties of RelationsAre the following relations on {1, 2, 3, 4} Are the following relations on {1, 2, 3, 4} symmetric, antisymmetric, or asymmetric?symmetric, antisymmetric, or asymmetric?

R = {(1, 1), (1, 2), (2, 1), (3, 3), (4, 4)}R = {(1, 1), (1, 2), (2, 1), (3, 3), (4, 4)} symmetricsymmetric

R = {(1, 1)}R = {(1, 1)} sym. and sym. and antisym.antisym.

R = {(1, 3), (3, 2), (2, 1)}R = {(1, 3), (3, 2), (2, 1)} antisym. antisym. and and asym.asym.

R = {(4, 4), (3, 3), (1, 4)}R = {(4, 4), (3, 3), (1, 4)} antisym.antisym.


Properties of RelationsProperties of Relations

Definition:Definition: A relation R on a set A is called A relation R on a set A is called transitivetransitive if whenever (a, b) if whenever (a, b)R and (b, c)R and (b, c)R, R, then (a, c)then (a, c)R for a, b, cR for a, b, cA. A.

Are the following relations on {1, 2, 3, 4} Are the following relations on {1, 2, 3, 4} transitive?transitive?

R = {(1, 1), (1, 2), (2, 2), (2, 1), (3, 3)}R = {(1, 1), (1, 2), (2, 2), (2, 1), (3, 3)} Yes.Yes.

R = {(1, 3), (3, 2), (2, 1)}R = {(1, 3), (3, 2), (2, 1)} No.No.

R = {(2, 4), (4, 3), (2, 3), (4, 1)}R = {(2, 4), (4, 3), (2, 3), (4, 1)} No.No.


Counting RelationsCounting RelationsExample:Example: How many different reflexive relations How many different reflexive relations can be defined on a set A containing n elements?can be defined on a set A containing n elements?

Solution:Solution: Relations on R are subsets of A Relations on R are subsets of AA, A, which contains nwhich contains n22 elements. elements.Therefore, different relations on A can be Therefore, different relations on A can be generated by choosing different subsets out of generated by choosing different subsets out of these nthese n22 elements, so there are 2 elements, so there are 2nn22 relations. relations.A A reflexivereflexive relation, however, relation, however, mustmust contain the contain the n elements (a, a) for every an elements (a, a) for every aA.A.Consequently, we can only choose among nConsequently, we can only choose among n22 – n – n = = n(n – 1) elements to generate reflexive relations, n(n – 1) elements to generate reflexive relations, so there are 2so there are 2n(n – 1)n(n – 1) of them. of them.


Combining RelationsCombining Relations

Relations are sets, and therefore, we can apply Relations are sets, and therefore, we can apply the usual the usual set operationsset operations to them. to them.

If we have two relations RIf we have two relations R11 and R and R22, and both of , and both of them are from a set A to a set B, then we can them are from a set A to a set B, then we can combine them to Rcombine them to R11 R R22, R, R11 R R22, or R, or R11 – R – R22..

In each case, the result will be In each case, the result will be another another relation from A to Brelation from A to B..


Combining RelationsCombining Relations… … and there is another important way to combine and there is another important way to combine relations.relations.

Definition:Definition: Let R be a relation from a set A to a Let R be a relation from a set A to a set B and S a relation from B to a set C. The set B and S a relation from B to a set C. The compositecomposite of R and S is the relation consisting of of R and S is the relation consisting of ordered pairs (a, c), where aordered pairs (a, c), where aA, cA, cC, and for which C, and for which there exists an element bthere exists an element bB such that (a, b)B such that (a, b)R and R and

(b, c)(b, c)S. We denote the composite of R and S byS. We denote the composite of R and S bySSRR..

In other words, if relation R contains a pair (a, b) In other words, if relation R contains a pair (a, b) and relation S contains a pair (b, c), then Sand relation S contains a pair (b, c), then SR R contains a pair (a, c).contains a pair (a, c).



Example:Example: Let D and S be relations on A = {1, 2, Let D and S be relations on A = {1, 2, 3, 4}.3, 4}.

D = {(a, b) | b = 5 - a} “b equals (5 – a)”D = {(a, b) | b = 5 - a} “b equals (5 – a)”

S = {(a, b) | a < b} “a is smaller than b”S = {(a, b) | a < b} “a is smaller than b”

D = {(1, 4), (2, 3), (3, 2), (4, 1)}D = {(1, 4), (2, 3), (3, 2), (4, 1)}

S = {(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (3, 4)}S = {(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (3, 4)}

SSD = {D = {(2, 4),(2, 4),(3, 3),(3, 3),(3, 4),(3, 4),(4, 2),(4, 2), (4, 3),(4, 3),

D maps an element a to the element (5 – a), D maps an element a to the element (5 – a), and afterwards S maps (5 – a) to all elements and afterwards S maps (5 – a) to all elements larger than (5 – a), resulting in larger than (5 – a), resulting in SSD = {(a,b) | b D = {(a,b) | b > 5 – a}> 5 – a} or or SSD = {(a,b) | a + b > 5}.D = {(a,b) | a + b > 5}.

(4, 4)}(4, 4)}



We already know that We already know that functionsfunctions are just are just special casesspecial cases of of relationsrelations (namely those that (namely those that map each element in the domain onto exactly map each element in the domain onto exactly one element in the codomain).one element in the codomain).

If we formally convert two functions into If we formally convert two functions into relations, that is, write them down as sets of relations, that is, write them down as sets of ordered pairs, the composite of these relations ordered pairs, the composite of these relations will be exactly the same as the composite of will be exactly the same as the composite of the functions (as defined earlier).the functions (as defined earlier).



Definition:Definition: Let R be a relation on the set A. Let R be a relation on the set A. The powers RThe powers Rnn, n = 1, 2, 3, …, are defined , n = 1, 2, 3, …, are defined inductively byinductively by

RR11 = R = R

RRn+1n+1 = R = RnnRR

In other words:In other words:

RRnn = R = RRR … … R (n times the letter R)R (n times the letter R)


Combining RelationsCombining RelationsTheorem:Theorem: The relation R on a set A is transitive The relation R on a set A is transitive if and only if Rif and only if Rnn R for all positive integers n. R for all positive integers n. Remember the definition of transitivity:Remember the definition of transitivity:Definition:Definition: A relation R on a set A is called A relation R on a set A is called transitive if whenever (a, b)transitive if whenever (a, b)R and (b, c)R and (b, c)R, R, then (a, c)then (a, c)R for a, b, cR for a, b, cA. A. The composite of R with itself contains exactly The composite of R with itself contains exactly these pairs (a, c). these pairs (a, c). Therefore, for a transitive relation R, RTherefore, for a transitive relation R, RR does R does not contain any pairs that are not in R, so Rnot contain any pairs that are not in R, so RR R R.R.Since RSince RR does not introduce any pairs that are R does not introduce any pairs that are not already in R, it must also be true that (Rnot already in R, it must also be true that (RR)R)R R R, and so on, so that R R, and so on, so that Rnn R. R.


n-ary Relationsn-ary Relations

In order to study an interesting application of In order to study an interesting application of relations, namely relations, namely databasesdatabases, we first need to , we first need to generalize the concept of binary relations to generalize the concept of binary relations to n-n-ary relationsary relations..

Definition:Definition: Let A Let A11, A, A22, …, A, …, Ann be sets. An be sets. An n-ary n-ary relationrelation on these sets is a subset of A on these sets is a subset of A11AA22……AAnn..

The sets AThe sets A11, A, A22, …, A, …, Ann are called the are called the domainsdomains of the relation, and n is called its of the relation, and n is called its degreedegree..


n-ary Relationsn-ary Relations

Example:Example:

Let R = {(a, b, c) | a = 2b Let R = {(a, b, c) | a = 2b b = 2c with a, b, c b = 2c with a, b, cNN}}

What is the degree of R?What is the degree of R?

The degree of R is 3, so its elements are triples.The degree of R is 3, so its elements are triples.

What are its domains?What are its domains?

Its domains are all equal to the set of integers.Its domains are all equal to the set of integers.

Is (2, 4, 8) in R?Is (2, 4, 8) in R?

No.No.

Is (4, 2, 1) in R?Is (4, 2, 1) in R?

Yes.Yes.


Databases and RelationsDatabases and Relations

Let us take a look at a type of database Let us take a look at a type of database representation that is based on relations, representation that is based on relations, namely thenamely the relational data model. relational data model.

A database consists of n-tuples called A database consists of n-tuples called recordsrecords, , which are made up of which are made up of fieldsfields..

These fields are the These fields are the entriesentries of the n-tuples. of the n-tuples.

The relational data model represents a The relational data model represents a database as an n-ary relation, that is, a set of database as an n-ary relation, that is, a set of records.records.


Databases and RelationsDatabases and RelationsExample:Example: Consider a database of students, Consider a database of students, whose records are represented as 4-tuples with whose records are represented as 4-tuples with the fields the fields Student NameStudent Name, , ID NumberID Number, , MajorMajor, , and and GPAGPA::

R = {(Ackermann, 231455, CS, 3.88),R = {(Ackermann, 231455, CS, 3.88), (Adams, 888323, Physics, 3.45), (Adams, 888323, Physics, 3.45), (Chou, 102147, CS, 3.79), (Chou, 102147, CS, 3.79), (Goodfriend, 453876, Math, 3.45), (Goodfriend, 453876, Math, 3.45), (Rao, 678543, Math, 3.90), (Rao, 678543, Math, 3.90), (Stevens, 786576, Psych, 2.99)} (Stevens, 786576, Psych, 2.99)}

Relations that represent databases are also called Relations that represent databases are also called tablestables, since they are often displayed as tables., since they are often displayed as tables.


Databases and RelationsDatabases and RelationsA domain of an n-ary relation is called a A domain of an n-ary relation is called a primary keyprimary key if the n-tuples are uniquely if the n-tuples are uniquely determined by their values from this domain.determined by their values from this domain.This means that no two records have the same This means that no two records have the same value from the same primary key.value from the same primary key.

In our example, which of the fields In our example, which of the fields Student Student NameName, , ID NumberID Number, , MajorMajor, and , and GPAGPA are are primary keys?primary keys?

Student NameStudent Name and and ID NumberID Number are primary are primary keys, because no two students have identical keys, because no two students have identical values in these fields.values in these fields.

In a real student database, only In a real student database, only ID NumberID Number would be a primary key.would be a primary key.



In a database, a primary key should remain one In a database, a primary key should remain one even if new records are added.even if new records are added.

Therefore, we should use a primary key of the Therefore, we should use a primary key of the intension intension of the database, containing all the n-of the database, containing all the n-tuples that can ever be included in our tuples that can ever be included in our database.database.

Combinations of domainsCombinations of domains can also uniquely can also uniquely identify n-tuples in an n-ary relation.identify n-tuples in an n-ary relation.

When the values of a When the values of a set of domainsset of domains determine an n-tuple in a relation, the determine an n-tuple in a relation, the Cartesian productCartesian product of these domains is called of these domains is called a a composite keycomposite key..


Databases and RelationsDatabases and RelationsWe can apply a variety of We can apply a variety of operationsoperations on n-ary on n-ary relations to form new relations.relations to form new relations.

Definition:Definition: The The projectionprojection P Pii11, i, i22, …, i, …, imm maps the n- maps the n-

tuple (atuple (a11, a, a22, …, a, …, ann) to the m-tuple (a) to the m-tuple (aii11, a, aii22

, …, a, …, aiimm), ),

where m where m n. n.

In other words, a projection PIn other words, a projection Pii11, i, i22, …, i, …, imm keeps the m keeps the m

components acomponents aii11, a, aii22

, …, a, …, aiim m of an n-tuple and of an n-tuple and

deletes its (n – m) other components.deletes its (n – m) other components.

Example:Example: What is the result when we apply the What is the result when we apply the projection Pprojection P2,42,4 to the student record (Stevens, to the student record (Stevens, 786576, Psych, 2.99) ?786576, Psych, 2.99) ?Solution:Solution: It is the pair (786576, 2.99). It is the pair (786576, 2.99).



In some cases, applying a projection to an In some cases, applying a projection to an entire table may not only result in fewer entire table may not only result in fewer columns, but also in columns, but also in fewer rowsfewer rows..

Why is that?Why is that?

Some records may only have differed in those Some records may only have differed in those fields that were deleted, so they become fields that were deleted, so they become identicalidentical, and there is no need to list identical , and there is no need to list identical records more than once.records more than once.



We can use the We can use the joinjoin operation to combine two operation to combine two tables into one if they share some identical tables into one if they share some identical fields.fields.

Definition:Definition: Let R be a relation of degree m and Let R be a relation of degree m and S a relation of degree n. The S a relation of degree n. The joinjoin J Jpp(R, S), where (R, S), where p p m and p m and p n, is a relation of degree m + n – n, is a relation of degree m + n – p that consists of all (m + n – p)-tuples p that consists of all (m + n – p)-tuples (a(a11, a, a22, …, a, …, am-pm-p, c, c11, c, c22, …, c, …, cpp, b, b11, b, b22, …, b, …, bn-pn-p),),where the m-tuple (awhere the m-tuple (a11, a, a22, …, a, …, am-pm-p, c, c11, c, c22, …, c, …, cpp) ) belongs to R and the n-tuple (cbelongs to R and the n-tuple (c11, c, c22, …, c, …, cpp, b, b11, , bb22, …, b, …, bn-pn-p) belongs to S.) belongs to S.



In other words, to generate Jp(R, S), we have to In other words, to generate Jp(R, S), we have to find all the elements in R whose p last find all the elements in R whose p last components match the p first components of components match the p first components of an element in S.an element in S.

The new relation contains exactly these The new relation contains exactly these matches, which are combined to tuples that matches, which are combined to tuples that contain each matching field only once.contain each matching field only once.



Example:Example: What is J What is J11(Y, R), where Y contains the (Y, R), where Y contains the fields fields Student NameStudent Name and and Year of BirthYear of Birth,,

Y = {(1978, Ackermann),Y = {(1978, Ackermann), (1972, Adams), (1972, Adams), (1917, Chou), (1917, Chou), (1984, Goodfriend), (1984, Goodfriend), (1982, Rao), (1982, Rao), (1970, Stevens)}, (1970, Stevens)},

and R contains the student records as defined and R contains the student records as defined before ?before ?



Solution:Solution: The resulting relation is: The resulting relation is:

{(1978, Ackermann, 231455, CS, 3.88),{(1978, Ackermann, 231455, CS, 3.88), (1972, Adams, 888323, Physics, 3.45), (1972, Adams, 888323, Physics, 3.45), (1917, Chou, 102147, CS, 3.79), (1917, Chou, 102147, CS, 3.79), (1984, Goodfriend, 453876, Math, 3.45), (1984, Goodfriend, 453876, Math, 3.45), (1982, Rao, 678543, Math, 3.90), (1982, Rao, 678543, Math, 3.90), (1970, Stevens, 786576, Psych, 2.99)} (1970, Stevens, 786576, Psych, 2.99)}

Since Y has two fields and R has four, the Since Y has two fields and R has four, the relation Jrelation J11(Y, R) has 2 + 4 – 1 = 5 fields.(Y, R) has 2 + 4 – 1 = 5 fields.


Representing RelationsRepresenting RelationsWe already know different ways of representing We already know different ways of representing relations. We will now take a closer look at two relations. We will now take a closer look at two ways of representation: ways of representation: Zero-one matricesZero-one matrices and and directed graphsdirected graphs..

If R is a relation from A = {aIf R is a relation from A = {a11, a, a22, …, a, …, amm} to B = } to B = {b{b11, b, b22, …, b, …, bnn}, then R can be represented by }, then R can be represented by the zero-one matrix Mthe zero-one matrix MRR = [m = [mijij] with] with

mmijij = 1, if (a = 1, if (aii, b, bjj))R, andR, and

mmijij = 0, if (a = 0, if (aii, b, bjj))R.R.

Note that for creating this matrix we first need Note that for creating this matrix we first need to list the elements in A and B in a to list the elements in A and B in a particular, particular, but arbitrary orderbut arbitrary order..


Representing RelationsRepresenting Relations

Example:Example: How can we represent the relation How can we represent the relation R = {(2, 1), (3, 1), (3, 2)} as a zero-one matrix?R = {(2, 1), (3, 1), (3, 2)} as a zero-one matrix?

Solution:Solution: The matrix M The matrix MRR is given by is given by

11

01

00

RM


Representing RelationsRepresenting RelationsWhat do we know about the matrices representing What do we know about the matrices representing a a relation on a setrelation on a set (a relation from A to A) ? (a relation from A to A) ?They are They are square square matrices.matrices.What do we know about matrices representing What do we know about matrices representing reflexivereflexive relations? relations?All the elements on the All the elements on the diagonaldiagonal of such matrices of such matrices MMrefref must be must be 1s1s..

1

.

.

.

1

1

refM


Representing RelationsRepresenting RelationsWhat do we know about the matrices What do we know about the matrices representing representing symmetric relationssymmetric relations??

These matrices are symmetric, that is, MThese matrices are symmetric, that is, MRR = (M = (MRR))tt..

1101

1001

0010

1101

RM

symmetric matrix,symmetric matrix,symmetric relation.symmetric relation.

0011

0011

0011

0011

RM

non-symmetric matrix,non-symmetric matrix,non-symmetric relation.non-symmetric relation.


Representing RelationsRepresenting Relations

The Boolean operations The Boolean operations joinjoin and and meetmeet (you (you remember?)remember?) can be used to determine the can be used to determine the matrices representing the matrices representing the unionunion and the and the intersectionintersection of two relations, respectively. of two relations, respectively.

To obtain the To obtain the joinjoin of two zero-one matrices, we of two zero-one matrices, we apply the Boolean “or” function to all apply the Boolean “or” function to all corresponding elements in the matrices.corresponding elements in the matrices.

To obtain the To obtain the meetmeet of two zero-one matrices, of two zero-one matrices, we apply the Boolean “and” function to all we apply the Boolean “and” function to all corresponding elements in the matrices.corresponding elements in the matrices.


Representing RelationsRepresenting RelationsExample:Example: Let the relations R and S be Let the relations R and S be represented by the matricesrepresented by the matrices

011

111

101

SRSR MMM

001

110

101

SM

What are the matrices representing RWhat are the matrices representing RS and RS and RS?S?

Solution:Solution: These matrices are given by These matrices are given by

000

000

101

SRSR MMM

010

001

101

RM


Representing Relations Using Representing Relations Using MatricesMatrices

Example:Example: How can we represent the relation How can we represent the relation R = {(2, 1), (3, 1), (3, 2)} as a zero-one matrix?R = {(2, 1), (3, 1), (3, 2)} as a zero-one matrix?

Solution:Solution: The matrix M The matrix MRR is given by is given by

11

01

00

RM



Example:Example: Let the relations R and S be Let the relations R and S be represented by the matricesrepresented by the matrices

011

111

101

SRSR MMM

001

110

101

SM

What are the matrices representing RWhat are the matrices representing RS and RS and RS?S?

Solution:Solution: These matrices are given by These matrices are given by

000

000

101

SRSR MMM

010

001

101

RM



Do you remember the Do you remember the Boolean product Boolean product of of two zero-one matrices?two zero-one matrices?

Let A = [aLet A = [aijij] be an m] be an mk zero-one matrix and k zero-one matrix and B = [bB = [bijij] be a k] be a kn zero-one matrix.n zero-one matrix.

Then the Then the Boolean productBoolean product of A and B, of A and B, denoted by Adenoted by AB, is the mB, is the mn matrix with (i, j)th n matrix with (i, j)th entry [centry [cijij], where], where

ccijij = (a = (ai1i1 b b1j1j) ) (a (ai2i2 b b2i2i) ) … … (a (aikik b bkjkj). ).

ccijij = 1 if and only if at least one of the terms = 1 if and only if at least one of the terms(a(ainin b bnjnj) = 1 for some n; otherwise c) = 1 for some n; otherwise cijij = 0. = 0.



Let us now assume that the zero-one matrices Let us now assume that the zero-one matrices MMA A = [a= [aijij], M], MBB = [b = [bijij] and M] and MCC = [c = [cijij] represent ] represent relations A, B, and C, respectively.relations A, B, and C, respectively.

Remember:Remember: For M For MCC = M = MAAMMBB we have: we have:

ccijij = 1 if and only if at least one of the terms = 1 if and only if at least one of the terms(a(ainin b bnjnj) = 1 for some n; otherwise c) = 1 for some n; otherwise cijij = 0. = 0.

In terms of the In terms of the relationsrelations, this means that C , this means that C contains a pair (xcontains a pair (xii, z, zjj) if and only if there is an ) if and only if there is an element yelement ynn such that (x such that (xii, y, ynn) is in relation A and ) is in relation A and (y(ynn, z, zjj) is in relation B.) is in relation B.

Therefore, C = BTherefore, C = BA (A (compositecomposite of A and B). of A and B).



This gives us the following rule:This gives us the following rule:

MMBBAA = M = MAAMMBB

In other words, the matrix representing the In other words, the matrix representing the compositecomposite of relations A and B is the of relations A and B is the Boolean Boolean productproduct of the matrices representing A and B. of the matrices representing A and B.

Analogously, we can find matrices representing Analogously, we can find matrices representing the the powers of relationspowers of relations::

MMRRnn = M = MRR[n][n] (n-th (n-th Boolean powerBoolean power).).



Example:Example: Find the matrix representing R Find the matrix representing R22, , where the matrix representing R is given bywhere the matrix representing R is given by

001

110

010

RM

Solution:Solution: The matrix for R The matrix for R22 is given by is given by

010

111

110]2[

2 RRMM


Representing Relations Using Representing Relations Using DigraphsDigraphs

Definition:Definition: A A directed graphdirected graph, or , or digraphdigraph, , consists of a set V of consists of a set V of verticesvertices (or (or nodesnodes) ) together with a set E of ordered pairs of together with a set E of ordered pairs of elements of V called elements of V called edgesedges (or (or arcsarcs).).

The vertex a is called the The vertex a is called the initial vertexinitial vertex of the of the edge (a, b), and the vertex b is called the edge (a, b), and the vertex b is called the terminal vertexterminal vertex of this edge. of this edge.

We can use arrows to display graphs.We can use arrows to display graphs.



Example:Example: Display the digraph with V = {a, b, c, d}, Display the digraph with V = {a, b, c, d}, E = {(a, b), (a, d), (b, b), (b, d), (c, a), (c, b), (d, b)}.E = {(a, b), (a, d), (b, b), (b, d), (c, a), (c, b), (d, b)}.

aabb

ccdd

An edge of the form (b, b) is called aAn edge of the form (b, b) is called a loop. loop.



Obviously, we can represent any relation R on a Obviously, we can represent any relation R on a set A by the digraph with A as its vertices and set A by the digraph with A as its vertices and all pairs (a, b)all pairs (a, b)R as its edges.R as its edges.

Vice versa, any digraph with vertices V and Vice versa, any digraph with vertices V and edges E can be represented by a relation on V edges E can be represented by a relation on V containing all the pairs in E.containing all the pairs in E.

This This one-to-one correspondenceone-to-one correspondence between between relations and digraphs means that any relations and digraphs means that any statement about relations also applies to statement about relations also applies to digraphs, and vice versa.digraphs, and vice versa.


Equivalence Relations Equivalence Relations

Equivalence relationsEquivalence relations are used to relate are used to relate objects that are similar in some way.objects that are similar in some way.

Definition:Definition: A relation on a set A is called an A relation on a set A is called an equivalence relation if it is reflexive, equivalence relation if it is reflexive, symmetric, and transitive.symmetric, and transitive.

Two elements that are related by an Two elements that are related by an equivalence relation R are called equivalence relation R are called equivalentequivalent..


Equivalence Relations Equivalence Relations

Since R is Since R is symmetricsymmetric, a is equivalent to b , a is equivalent to b whenever b is equivalent to a.whenever b is equivalent to a.

Since R is Since R is reflexivereflexive, every element is , every element is equivalent to itself.equivalent to itself.

Since R is Since R is transitivetransitive, if a and b are equivalent , if a and b are equivalent and b and c are equivalent, then a and c are and b and c are equivalent, then a and c are equivalent.equivalent.

Obviously, these three properties are necessary Obviously, these three properties are necessary for a reasonable definition of equivalence.for a reasonable definition of equivalence.


Equivalence Relations Equivalence Relations Example:Example: Suppose that R is the relation on the set Suppose that R is the relation on the set of strings that consist of English letters such that of strings that consist of English letters such that aRb if and only if l(a) = l(b), where l(x) is the length aRb if and only if l(a) = l(b), where l(x) is the length of the string x. Is R an equivalence relation?of the string x. Is R an equivalence relation?

Solution:Solution: • R is reflexive, because l(a) = l(a) and therefore R is reflexive, because l(a) = l(a) and therefore aRa for any string a. aRa for any string a.• R is symmetric, because if l(a) = l(b) then l(b) = R is symmetric, because if l(a) = l(b) then l(b) = l(a), so if aRb then bRa. l(a), so if aRb then bRa.• R is transitive, because if l(a) = l(b) and l(b) = l(c), R is transitive, because if l(a) = l(b) and l(b) = l(c), then l(a) = l(c), so aRb and bRc implies aRc. then l(a) = l(c), so aRb and bRc implies aRc.

R is an equivalence relation.R is an equivalence relation.


Equivalence Classes Equivalence Classes Definition: Definition: Let R be an equivalence relation on Let R be an equivalence relation on a set A. The set of all elements that are related a set A. The set of all elements that are related to an element a of A is called the to an element a of A is called the equivalence equivalence classclass of a. of a.

The equivalence class of a with respect to R is The equivalence class of a with respect to R is denoted by denoted by [a][a]RR..

When only one relation is under consideration, When only one relation is under consideration, we will delete the subscript R and write we will delete the subscript R and write [a][a] for for this equivalence class.this equivalence class.

If bIf b[a][a]RR, b is called a , b is called a representativerepresentative of this of this equivalence class.equivalence class.


Equivalence Classes Equivalence Classes

Example: Example: In the previous example (strings of In the previous example (strings of identical length), what is the equivalence class identical length), what is the equivalence class of the word mouse, denoted by [mouse] ?of the word mouse, denoted by [mouse] ?

Solution:Solution: [mouse] is the set of all English [mouse] is the set of all English words containing five letters.words containing five letters.

For example, ‘horse’ would be a representative For example, ‘horse’ would be a representative of this equivalence class.of this equivalence class.


Equivalence Classes Equivalence Classes Theorem: Theorem: Let R be an equivalence relation on a Let R be an equivalence relation on a set A. The following statements are equivalent:set A. The following statements are equivalent:• aRbaRb• [a] = [b][a] = [b]• [a] [a] [b] [b]

Definition:Definition: A A partition partition of a set S is a collection of of a set S is a collection of disjoint nonempty subsets of S that have S as their disjoint nonempty subsets of S that have S as their union. In other words, the collection of subsets Aunion. In other words, the collection of subsets Aii, , iiI, forms a partition of S if and only if I, forms a partition of S if and only if (i) A(i) Aii for i for iII

• AAii A Ajj = = , if i , if i j j

• iiII A Aii = S = S


Equivalence Classes Equivalence Classes Examples: Examples: Let S be the set {u, m, b, r, o, c, k, s}.Let S be the set {u, m, b, r, o, c, k, s}.Do the following collections of sets partition S ?Do the following collections of sets partition S ?

{{m, o, c, k}, {r, u, b, s}}{{m, o, c, k}, {r, u, b, s}} yes.yes.

{{c, o, m, b}, {u, s}, {r}}{{c, o, m, b}, {u, s}, {r}} no (k is missing).no (k is missing).

{{b, r, o, c, k}, {m, u, s, t}}{{b, r, o, c, k}, {m, u, s, t}} no (t is not in S).no (t is not in S).

{{u, m, b, r, o, c, k, s}}{{u, m, b, r, o, c, k, s}} yes.yes.

{{b, o, o, k}, {r, u, m}, {c, s}}{{b, o, o, k}, {r, u, m}, {c, s}} yes ({b,o,o,k} = {b,o,k}).yes ({b,o,o,k} = {b,o,k}).

{{u, m, b}, {r, o, c, k, s}, {{u, m, b}, {r, o, c, k, s}, }} no (no ( not allowed). not allowed).



Theorem: Theorem: Let R be an equivalence relation on Let R be an equivalence relation on a a set S. Then the set S. Then the equivalence classesequivalence classes of R form of R form a a partitionpartition of S. Conversely, given a partition of S. Conversely, given a partition {A{Aii | i | iI} of the set S, there is an equivalence I} of the set S, there is an equivalence relation R that has the sets Arelation R that has the sets Aii, i, iI, as its I, as its equivalence classes.equivalence classes.


Equivalence Classes Equivalence Classes Example: Example: Let us assume that Frank, Suzanne Let us assume that Frank, Suzanne and George live in Boston, Stephanie and Max and George live in Boston, Stephanie and Max live in Lübeck, and Jennifer lives in Sydney. live in Lübeck, and Jennifer lives in Sydney.

Let R be the Let R be the equivalence relationequivalence relation {(a, b) | a {(a, b) | a and b live in the same city} on the set P = and b live in the same city} on the set P = {Frank, Suzanne, George, Stephanie, Max, {Frank, Suzanne, George, Stephanie, Max, Jennifer}.Jennifer}.

Then R = {(Frank, Frank), (Frank, Suzanne),Then R = {(Frank, Frank), (Frank, Suzanne),(Frank, George), (Suzanne, Frank), (Suzanne, (Frank, George), (Suzanne, Frank), (Suzanne, Suzanne), (Suzanne, George), (George, Frank),Suzanne), (Suzanne, George), (George, Frank),(George, Suzanne), (George, George), (Stephanie,(George, Suzanne), (George, George), (Stephanie,Stephanie), (Stephanie, Max), (Max, Stephanie),Stephanie), (Stephanie, Max), (Max, Stephanie),(Max, Max), (Jennifer, Jennifer)}.(Max, Max), (Jennifer, Jennifer)}.



Then the Then the equivalence classesequivalence classes of R are: of R are:

{{Frank, Suzanne, George}, {Stephanie, Max}, {{Frank, Suzanne, George}, {Stephanie, Max}, {Jennifer}}.{Jennifer}}.

This is a This is a partition partition of P.of P.

The equivalence classes of any equivalence The equivalence classes of any equivalence relation R defined on a set S constitute a relation R defined on a set S constitute a partition of S, because every element in S is partition of S, because every element in S is assigned to assigned to exactly oneexactly one of the equivalence of the equivalence classes.classes.



Another example: Another example: Let R be the relation Let R be the relation {(a, b) | a {(a, b) | a b (mod 3)} on the set of integers. b (mod 3)} on the set of integers.

Is R an equivalence relation?Is R an equivalence relation?

Yes, R is reflexive, symmetric, and transitive.Yes, R is reflexive, symmetric, and transitive.

What are the equivalence classes of R ?What are the equivalence classes of R ?

{{…, -6, -3, 0, 3, 6, …},{{…, -6, -3, 0, 3, 6, …}, {…, -5, -2, 1, 4, 7, …}, {…, -5, -2, 1, 4, 7, …}, {…, -4, -1, 2, 5, 8, …}} {…, -4, -1, 2, 5, 8, …}}


Yes, No, Maybe...Yes, No, Maybe...

BooleanBooleanAlgebraAlgebra


Boolean AlgebraBoolean Algebra

Boolean algebra provides the operations and Boolean algebra provides the operations and the rules for working with the set the rules for working with the set {0, 1}.{0, 1}.

These are the rules that underlie These are the rules that underlie electronic electronic circuitscircuits, and the methods we will discuss are , and the methods we will discuss are fundamental to fundamental to VLSI designVLSI design..

We are going to focus on three operations:We are going to focus on three operations:

• Boolean complementation,Boolean complementation,

• Boolean sum, andBoolean sum, and

• Boolean productBoolean product


Boolean OperationsBoolean Operations

The The complementcomplement is denoted by a bar (on the is denoted by a bar (on the slides, we will use a minus sign). It is defined byslides, we will use a minus sign). It is defined by

-0 = 1 and -1 = 0.-0 = 1 and -1 = 0.

The The Boolean sumBoolean sum, denoted by + or by OR, has , denoted by + or by OR, has the following values:the following values:

1 + 1 = 1, 1 + 0 = 1, 0 + 1 = 1, 0 + 0 = 01 + 1 = 1, 1 + 0 = 1, 0 + 1 = 1, 0 + 0 = 0

The The Boolean productBoolean product, denoted by , denoted by or by or by AND, has the following values:AND, has the following values:

1 1 1 = 1, 1 1 = 1, 1 0 = 0, 0 0 = 0, 0 1 = 0, 0 1 = 0, 0 0 = 0 0 = 0


Boolean Functions and ExpressionsBoolean Functions and Expressions

Definition:Definition: Let B = {0, 1}. The variable x is Let B = {0, 1}. The variable x is called a called a Boolean variableBoolean variable if it assumes values if it assumes values only from B.only from B.

A function from BA function from Bnn, the set {(x, the set {(x11, x, x22, …, x, …, xnn) |x) |xiiB, B,

1 1 i i n}, to B is called a n}, to B is called a Boolean function Boolean function of degree nof degree n..

Boolean functions can be represented using Boolean functions can be represented using expressions made up from the variables and expressions made up from the variables and Boolean operations.Boolean operations.



The The Boolean expressionsBoolean expressions in the variables x in the variables x11, x, x22, , …, x…, xnn are defined recursively as follows: are defined recursively as follows:

• 0, 1, x0, 1, x11, x, x22, …, x, …, xnn are Boolean expressions. are Boolean expressions.

• If EIf E11 and E and E22 are Boolean expressions, then (-E are Boolean expressions, then (-E11), ),

(E (E11EE22), and (E), and (E11 + E + E22) are Boolean expressions.) are Boolean expressions.

Each Boolean expression represents a Boolean Each Boolean expression represents a Boolean function. The values of this function are obtained function. The values of this function are obtained by substituting 0 and 1 for the variables in the by substituting 0 and 1 for the variables in the expression.expression.



For example, we can create Boolean expression For example, we can create Boolean expression in the variables x, y, and z using the “building in the variables x, y, and z using the “building blocks”blocks”0, 1, x, y, and z, and the construction rules:0, 1, x, y, and z, and the construction rules:

Since x and y are Boolean expressions, so is xy.Since x and y are Boolean expressions, so is xy.

Since z is a Boolean expression, so is (-z).Since z is a Boolean expression, so is (-z).

Since xy and (-z) are expressions, so is xy + (-Since xy and (-z) are expressions, so is xy + (-z).z).

… … and so on…and so on…



Example:Example: Give a Boolean expression for the Give a Boolean expression for the Boolean function F(x, y) as defined by the Boolean function F(x, y) as defined by the following table:following table:

111100

000011

001111

000000F(x, y)F(x, y)yyxx

Possible solution:Possible solution: F(x, y) = (-x) F(x, y) = (-x)yy


Boolean Functions and ExpressionsBoolean Functions and ExpressionsAnother Example:Another Example: Possible solution I:Possible solution I:

F(x, y, z) = -(xz + y)F(x, y, z) = -(xz + y)

00001111

F(x, y, z)F(x, y, z)

11001100zz

0000

1100

1100

0000yyxx

00000011

11001100

1111

1111

00110011

Possible solution II:Possible solution II:

F(x, y, z) = (-(xz))(-y)F(x, y, z) = (-(xz))(-y)



There is a simple method for deriving a Boolean There is a simple method for deriving a Boolean expression for a function that is defined by a expression for a function that is defined by a table. This method is based on table. This method is based on mintermsminterms..

Definition:Definition: A A literal literal is a Boolean variable or is a Boolean variable or its complement. A its complement. A mintermminterm of the Boolean of the Boolean variables xvariables x11, x, x22, …, x, …, xnn is a Boolean product is a Boolean product yy11yy22…y…ynn, where y, where yii = x = xii or y or yii = -x = -xii..

Hence, a minterm is a product of n literals, with Hence, a minterm is a product of n literals, with one literal for each variable.one literal for each variable.


Boolean Functions and ExpressionsBoolean Functions and ExpressionsConsider F(x,y,z) again:Consider F(x,y,z) again: F(x, y, z) = 1 if and F(x, y, z) = 1 if and

only if:only if:

x = y = z = 0 orx = y = z = 0 or

x = y = 0, z = 1 orx = y = 0, z = 1 or

x = 1, y = z = 0x = 1, y = z = 0

Therefore,Therefore,

F(x, y, z) =F(x, y, z) =(-x)(-y)(-z) +(-x)(-y)(-z) +(-x)(-y)z +(-x)(-y)z +x(-y)(-z)x(-y)(-z)

00001111

F(x, y, z)F(x, y, z)

11001100zz

0000

1100

1100

0000yyxx

00000011

11001100

1111

1111

00110011



Definition:Definition: The Boolean functions F and G of n The Boolean functions F and G of n variables are variables are equalequal if and only if F(b if and only if F(b11, b, b22, …, b, …, bnn) ) = G(b= G(b11, b, b22, …, b, …, bnn) whenever b) whenever b11, b, b22, …, b, …, bnn belong belong to B.to B.

Two different Boolean expressions that Two different Boolean expressions that represent the same function are called represent the same function are called equivalentequivalent..

For example, the Boolean expressions xy, xy + For example, the Boolean expressions xy, xy + 0, and xy0, and xy1 are equivalent.1 are equivalent.



The The complement complement of the Boolean function F is the of the Boolean function F is the function –F, where –F(bfunction –F, where –F(b11, b, b22, …, b, …, bnn) = ) = -(F(b-(F(b11, b, b22, …, b, …, bnn)).)).

Let F and G be Boolean functions of degree n. The Let F and G be Boolean functions of degree n. The Boolean sum F+GBoolean sum F+G and and Boolean product FGBoolean product FG are then defined byare then defined by

(F + G)(b(F + G)(b11, b, b22, …, b, …, bnn) = F(b) = F(b11, b, b22, …, b, …, bnn) + G(b) + G(b11, b, b22, , …, b…, bnn))

(FG)(b(FG)(b11, b, b22, …, b, …, bnn) = F(b) = F(b11, b, b22, …, b, …, bnn) G(b) G(b11, b, b22, …, b, …, bnn))



Question:Question: How many different Boolean functions of How many different Boolean functions of degree 1 are there?degree 1 are there?

Solution:Solution: There are four of them, F There are four of them, F11, F, F22, F, F33, and F, and F44::

00

11

FF33

11

00

FF22

110011

110000

FF44FF11xx



Question:Question: How many different Boolean How many different Boolean functions of degree 2 are there?functions of degree 2 are there?

Solution:Solution: There are 16 of them, F There are 16 of them, F11, F, F22, …, F, …, F1616::

11000000FF22

00000000FF11

001100

110011

001111

000000FF33yyxx

11111100FF88

00111100FF77

00

00

00

11FF99

00001100FF55

11110000FF44

11

00

11

00FF66

00110011FF1111

11000011FF1010

00

11

11

11FF1212

11001111

FF1414

00001111

FF1313

11

11

00

11

FF1515

11111111FF1616



Question:Question: How many different Boolean How many different Boolean functions of degree n are there?functions of degree n are there?

Solution:Solution:

There are 2There are 2nn different n-tuples of 0s and 1s. different n-tuples of 0s and 1s.

A Boolean function is an assignment of 0 or 1 A Boolean function is an assignment of 0 or 1 to each of these 2to each of these 2nn different n-tuples. different n-tuples.

Therefore, there are Therefore, there are 2222nn different Boolean different Boolean

functions.functions.


DualityDuality

There are useful identities of Boolean There are useful identities of Boolean expressions that can help us to transform an expressions that can help us to transform an expression A into an equivalent expression B expression A into an equivalent expression B (see Table 5 on page 597 in the textbook).(see Table 5 on page 597 in the textbook).

We can derive additional identities with the We can derive additional identities with the help of the help of the dualdual of a Boolean expression. of a Boolean expression.

The dual of a Boolean expression is obtained The dual of a Boolean expression is obtained by interchanging Boolean sums and Boolean by interchanging Boolean sums and Boolean products and interchanging 0s and 1s.products and interchanging 0s and 1s.


DualityDualityExamples:Examples:

The dual of The dual of x(y + z)x(y + z) is is x + yz.x + yz.

The dual of The dual of -x-x1 + (-y + z) 1 + (-y + z) isis (-x + 0)((-y)z).(-x + 0)((-y)z).

The The dual of a Boolean function Fdual of a Boolean function F represented represented by a Boolean expression is the function by a Boolean expression is the function represented by the dual of this expression.represented by the dual of this expression.

This dual function, denoted by FThis dual function, denoted by Fdd, , does not does not dependdepend on the particular Boolean expression on the particular Boolean expression used to represent F.used to represent F.


DualityDuality

Therefore, an identity between functions Therefore, an identity between functions represented by Boolean expressions represented by Boolean expressions remains remains validvalid when the duals of both sides of the when the duals of both sides of the identity are taken.identity are taken.

We can use this fact, called the We can use this fact, called the duality duality principleprinciple, to derive new identities., to derive new identities.

For example, consider the absorption law For example, consider the absorption law x(x + y) = xx(x + y) = x..

By taking the duals of both sides of this By taking the duals of both sides of this identity, we obtain the equation identity, we obtain the equation x + xy = xx + xy = x, , which is also an identity (and also called an which is also an identity (and also called an absorption law).absorption law).


Definition of a Boolean AlgebraDefinition of a Boolean Algebra

All the properties of Boolean functions and All the properties of Boolean functions and expressions that we have discovered also expressions that we have discovered also apply to apply to other mathematical structuresother mathematical structures such as propositions and sets and the such as propositions and sets and the operations defined on them.operations defined on them.

If we can show that a particular structure is a If we can show that a particular structure is a Boolean algebra, then we know that all results Boolean algebra, then we know that all results established about Boolean algebras apply to established about Boolean algebras apply to this structure.this structure.

For this purpose, we need an For this purpose, we need an abstract abstract definitiondefinition of a Boolean algebra. of a Boolean algebra.


Definition of a Boolean AlgebraDefinition of a Boolean AlgebraDefinition:Definition: A Boolean algebra is a set B with two A Boolean algebra is a set B with two binary operations binary operations and and , elements 0 and 1, and , elements 0 and 1, and a unary operation – such that the following a unary operation – such that the following properties hold for all x, y, and z in B:properties hold for all x, y, and z in B:

x x 0 = x and x 0 = x and x 1 = x 1 = x (identity laws)(identity laws)

x x (-x) = 1 and x (-x) = 1 and x (-x) = 0 (-x) = 0 (domination laws)(domination laws)

(x (x y) y) z = x z = x (y (y z) and z) and (x (x y) y) z = x z = x (y (y z) and z) and (associative laws)(associative laws)

x x y = y y = y x and x x and x y = y y = y x x (commutative (commutative laws)laws)

x x (y (y z) = (x z) = (x y) y) (x (x z) and z) andx x (y (y z) = (x z) = (x y) y) (x (x z) z) (distributive laws)(distributive laws)


Logic GatesLogic GatesElectronic circuits consist of so-called gates.Electronic circuits consist of so-called gates.There are three basic types of gates:There are three basic types of gates:

xx

yy

x+x+yy OR gateOR gate

AND gateAND gatexx

yy

xyxy

xx -x-xinverterinverter


Logic GatesLogic GatesExample:Example: How can we build a circuit that How can we build a circuit that computes the function xy + (-x)y ?computes the function xy + (-x)y ?

xy + (-xy + (-x)yx)y

xx

yy

xyxy

xx -x-x

yy

(-x)y(-x)y


TheTheEndEnd

Discrete Mathematics - All chapters

News & Politics

Transcript of Discrete Mathematics - All chapters