Jeff Clune

Assistant ProfessorEvolving Artificial Intelligence Laboratory

Artificial Intelligence

Make Friends

• 3 minutes with a new person

• Funny story involving fire, climbing trees, or poor-ordering

Great diversity!

Thanks Paul

Guest Lecturer

• Thurs

Logical Deduction AI Approach

• Based on formal logic

• Example • Toasters are smaller than car trunks • Things that are smaller than something can fit inside that

something. • ergo: Toasters can fit inside car trunks

Logical Deduction AI Approach

• Based on formal logic

• Example • Toasters are smaller than cows • Things that are smaller than something can fit inside that

something. • ergo: Toasters can fit inside cows • Maybe true...but not quite right

Logical Deduction AI Approach

• Called Expert Systems

• Problems • Too hard to hand enter all facts

- oranges are smaller than watermelons - toasters don’t usually go inside of cows, etc. etc. etc. ... - Cyc

- Started 1984 - Attempts to gather all commonsense facts - Currently has over one-million hand-entered facts! (e.g. “plants die eventually”) - Can do some things: with “trees are plants”, and “plants die eventually”,“do trees die?”

- It answers “yes”

Logical Deduction AI Approach

• Called Expert Systems

• Problems • Too hard to hand enter all facts • informal knowledge is hard to codify, especially when fuzzy

- e.g. the fact that toasters don’t go inside cows

• exponential computational power required as fact database increases - even a few hundred rules becomes untenable, unless

- you give the system tips about which reasoning steps to try first - (same problem in all AI fields....to be fair)

Logical Deduction AI Approach

• Called Expert Systems

• We’re not going to cover these systems • If interested, read Section III of the textbook

History of AI

• There is a great history of AI in RN1 • Including the AI winter and our current AI spring (summer?!) • We won’t cover it in lecture, but you should read it

Modern Era

• Learning from big data sets • often, more/better data is more important than specific choice of

algorithm • information is learned from data sets, not hand-entered

• Dealing with uncertainty • Statistics/probability (often Bayesian)

Agents

from: http://www.cs.cornell.edu/courses/cs4700/2011fa/lectures/02_ProblemSolvingAsSearch.pdf

Agents

• Translate inputs (percepts) to outputs (actions)

• We want them to perform well • Need a performance measure

• What is a good performance measure for a vacuum cleaning robot?

Agents

• Performance measures: notoriously difficult to design • e.g. vacuum-robot:

- measure: amount of dirt vacuumed up - result: vacuums, dumps dirt on ground, repeat! - better measure: percent of floor clean summed over all time steps

- possibly with penalties for energy used, noise, dust, etc.

• General rule: reward what you want the agent to do, not how you think it should accomplish that goal • problem: no stepping-stones....

Agents

• Maximize expected payoff • from built-in knowledge (instincts, hard-wired knowledge...) • from learned knowledge (percept sequence) • including gathering info (active learning)

• “outcomes don’t justify decisions” • not dying doesn’t mean it was smart to play Russian Roulette

The Gambler

Every gambler knows that the secret to survivin'

Is knowin' what to throw away and knowing what to keep

'Cause every hand's a winner and every hand's a loser

The Gambler

Every gambler knows that the secret to survivin'

Is knowin' what to throw away and knowing what to keep

'Cause every hand's a winner and every hand's a loser

And the best that you can hope for is to die in your sleep

to maximize expected reap!

Environments/Problems/Domains

• Fully observable vs. partially observable • Fully observable: agents sensors can see all relevant info

- no need to keep internal state - examples of fully observable problems?

• Partially observable: - what are different reasons why world may not be observable? - examples of partially observable problems?

- Is chess fully observable? - What about the castle rule? “Can only occur if neither piece is moved…”

- What if I mark each piece as moved? - What if I list each move that has occurred?

- Would that make every environment fully observable?

Environments/Problems/Domains

• Single agent vs. multi-agent • examples of single? • multi?

Environments/Problems/Domains

• Single agent vs. multi-agent • examples of single? • multi?

- competitive - cooperative

Environments/Problems/Domains

• Deterministic vs. stochastic

Environments/Problems/Domains

• Episodic vs. sequential • Episodic: sense & act, repeat, but episode(t) is independent of

episode (t-1) - examples?

Environments/Problems/Domains

• Episodic vs. sequential • Episodic: sense & act, repeat, but episode(t) is independent of

episode (t-1) - examples?

- classification tasks

Environments/Problems/Domains

• Episodic vs. sequential • Episodic: sense & act, repeat, but episode(t) is independent of

episode (t-1) - examples?

- classification tasks

• Sequential - current action can influence all future actions - which is harder?

Environments/Problems/Domains

• Static vs. Dynamic • Dynamic: environment can change while agent is deliberating

- If you choose not to decide (or think too long), you still have made a choice!

• Examples of both?

Environments/Problems/Domains

• Discrete vs. Continuous • states, time, percepts, actions

- separate time steps - integers vs. floats

Environments/Problems/Domains

• Known vs. Unknown • Known: agent knows the laws of physics ahead of time

- or the rules of the game...how the system works

Environments/Problems/Domains

• Hardest • Partially observable • multiagent • stochastic • sequential • dynamic • continuous • unknown

Environments/Problems/Domains

• Hardest • Partially observable or fully? • single or multi agent? • deterministic or stochastic? • sequential or episodic? • dynamic or static? • discrete or continuous? • known or unknown?

What is homework one?

Possible Agents: Lookup Table

• If percepts are:

• Then

• aka “State Machine”

o

Possible Agents: Lookup Table

• State space: all of the possible agent situations

• Chess ~ 10150

• Any robot with camera: ~10250,000,000,000

• video at 27 mb/second - 30 frames per second, 640x480 pixels, 24 bits of color info

• Number of atoms in observable universe: ~1050

Possible agents: Lookup Table

• Such large lookup tables are not going to work • can’t store them, learn them, etc.

• Key challenge for AI: • make small programs perform as well as optimal/good, vast,

lookup tables

Possible Agents: Simple Reflex Agent

• Other end of the spectrum: • open-loop controller

- steadfastly refuses to alter its plan despite what’s happening in reality - (i.e. ignores/has no percepts)

• Simple reflex agent • responds to current precepts • very limited

- why? can you think of examples where you need more than the current percepts?

Possible Agents: Simple Reflex Agent

• Simple reflex agent • responds to current precepts • very limited

- only works optimally if environment is fully observable

Possible Agents: Simple Reflex Agent

• Best way to handle partial observability? • Remember important things that you can’t perceive now • I.e. build a model of the world

- internal states that represent external states

Possible Agents: Model-based Reflex Agent

• Best way to handle partial observability? • Remember important things that you can’t perceive now • I.e. build a model of the world

- internal states that represent external states

• To build a model • must guess how the world changes when you can’t see it • must guess how the agent’s actions change the world

Possible Agents: Model-based Reflex Agent

Goal-based Agents

• Have goal; can figure out autonomously how to accomplish it

• The more abstract the goal can be specified, the better • Imagine hand coding every decision a robot makes to deliver a

pizza vs. saying “deliver this pizza to 300 Water St.”

Utility-based Agents

• Have many goals • deliver pizza • fast • don’t crash • etc.

• Best way to combine these different factors into one score?

Utility-based Agents

• Have many goals • deliver pizza • fast • don’t crash • etc.

• Given uncertainty (stochasticity): • Maximize expected utility (aka expected value)

Pen and Paper

• Start to bring them please • Lots of in-class problems

Utility-based Agents

• Have many goals • deliver pizza • fast • don’t crash • etc.

• Given uncertainty (stochasticity): • Maximize expected utility (aka expected value)

Learning Agents

• Instead of programming each decision, they learn • there is usually an implicit or explicit utility function

• To key parts of an agent • Select actions to perform • Learn from what happened

- requires knowing what’s good/bad - either a reward signal - or a internal “critic”

Exploration vs. Exploitation

• Two-armed bandit problem • Let’s play

- Arm 1: payoff = ??? - Arm 2: payoff = ??? - Goal: a policy that maximized expected

value over N pulls - Problem version 1: payoffs don’t change

- your policy? - Problem version 2: payoffs change

- your policy?

Exploration vs. Exploitation

• Exploitation: Doing the best you can given your current knowledge

• Exploration: Trying things that are less-optimal (according to your current model) in order to improve the model • examples from real life?

Abstraction

• Including only what’s relevant • e.g. in chess, the color of the pieces is not important

Probability

• When to add vs. multiply? • Probability of rolling:

- P(1 ^ 2 ^ 3 ^4 ^ 5) in order for 5 rolls of one die - P(1 v 2 v 3 v 4 v 5) in 1 roll of one die

On your own

Probability

• When to add vs. multiply?

• Intuition: • X AND Y = less likely

- stranger is tall AND fat AND bald AND funny

• X OR Y = more likely - stranger is tall OR fat OR bald OR funny

• Probability of rolling: - 1 ^ 2 ^ 3 ^4 ^ 5? - 1 v 2 v 3 v 4 v 5?

Probability

• When to add vs. multiply?

• Intuition: • X AND Y = less likely

- stranger is tall AND fat AND bald AND funny

• X OR Y = more likely - stranger is tall OR fat OR bald OR funny

• Probability of rolling: - 1 ^ 2 ^ 3 ^4 ^ 5? - 1 v 2 v 3 v 4 v 5?