Using Discrete Event Simulation to Model Multi-Robot Multi ...
Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static...
Transcript of Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static...
![Page 1: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/1.jpg)
Rational Agents (Chapter 2)
![Page 2: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/2.jpg)
Agents • An agent is anything that can be viewed as perceiving its
environment through sensors and acting upon that environment through actuators
![Page 3: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/3.jpg)
Example: Vacuum-Agent • Percepts:
Location and status, e.g., [A,Dirty]
• Actions: Left, Right, Suck, NoOp
function Vacuum-Agent([location,status]) returns an action • if status = Dirty then return Suck • else if location = A then return Right • else if location = B then return Left
![Page 4: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/4.jpg)
Rational agents • For each possible percept sequence, a rational
agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and the agent’s built-in knowledge
• Performance measure (utility function): An objective criterion for success of an agent's behavior
• Expected utility:
• Can a rational agent make mistakes?
EU(action) = P(outcome | action)U(outcome)outcomes∑
![Page 5: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/5.jpg)
Back to Vacuum-Agent • Percepts:
Location and status, e.g., [A,Dirty]
• Actions: Left, Right, Suck, NoOp
function Vacuum-Agent([location,status]) returns an action • if status = Dirty then return Suck • else if location = A then return Right • else if location = B then return Left
• Is this agent rational? – Depends on performance measure, environment properties
![Page 6: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/6.jpg)
Specifying the task environment • PEAS: Performance measure, Environment, Actuators,
Sensors • P: a function the agent is maximizing (or minimizing)
– Assumed given – In practice, needs to be computed somewhere
• E: a formal representation for world states – For concreteness, a tuple (var1=val1, var2=val2, … ,varn=valn)
• A: actions that change the state according to a transition model – Given a state and action, what is the successor state
(or distribution over successor states)?
• S: observations that allow the agent to infer the world state – Often come in very different form than the state itself – E.g., in tracking, observations may be pixels and state
variables 3D coordinates
![Page 7: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/7.jpg)
PEAS Example: Autonomous taxi • Performance measure
– Safe, fast, legal, comfortable trip, maximize profits
• Environment – Roads, other traffic, pedestrians, customers
• Actuators – Steering wheel, accelerator, brake, signal,
horn • Sensors
– Cameras, LIDAR, speedometer, GPS, odometer, engine sensors, keyboard
![Page 8: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/8.jpg)
Another PEAS example: Spam filter • Performance measure
– Minimizing false positives, false negatives • Environment
– A user’s email account, email server • Actuators
– Mark as spam, delete, etc. • Sensors
– Incoming messages, other information about user’s account
![Page 9: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/9.jpg)
Environment types • Fully observable vs. partially observable • Deterministic vs. stochastic • Episodic vs. sequential • Static vs. dynamic • Discrete vs. continuous • Single agent vs. multi-agent • Known vs. unknown
![Page 10: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/10.jpg)
Fully observable vs. partially observable • Do the agent's sensors give it access to the complete
state of the environment? – For any given world state, are the values of all the variables
known to the agent?
vs.
Source: L. Zettlemoyer
![Page 11: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/11.jpg)
Deterministic vs. stochastic • Is the next state of the environment completely
determined by the current state and the agent’s action? – Is the transition model deterministic (unique successor state
given current state and action) or stochastic (distribution over successor states given current state and action)?
– Strategic: the environment is deterministic except for the actions of other agents
vs.
![Page 12: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/12.jpg)
Episodic vs. sequential • Is the agent’s experience divided into unconnected
single decisions/actions, or is it a coherent sequence of observations and actions in which the world evolves according to the transition model?
vs.
![Page 13: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/13.jpg)
Static vs. dynamic • Is the world changing while the agent is thinking?
• Semidynamic: the environment does not change with the passage of time, but the agent's performance score does
vs.
![Page 14: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/14.jpg)
Discrete vs. continuous • Does the environment provide a fixed number of distinct
percepts, actions, and environment states? – Are the values of the state variables discrete or continuous? – Time can also evolve in a discrete or continuous fashion
vs.
![Page 15: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/15.jpg)
Single-agent vs. multiagent • Is an agent operating by itself in the environment?
vs.
![Page 16: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/16.jpg)
Known vs. unknown • Are the rules of the environment (transition model and
rewards associated with states) known to the agent? – Strictly speaking, not a property of the environment, but of the
agent’s state of knowledge
vs.
![Page 17: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/17.jpg)
Examples of different environments
Observable
Deterministic
Episodic
Static
Discrete
Single agent
Fully Partially Partially
Strategic Stochastic Stochastic
Sequential Sequential Sequential
Semidynamic Dynamic Static
Discrete Discrete Continuous
Multi Multi Multi
Fully
Deterministic
Episodic
Static
Discrete
Single
Chess with a clock
Scrabble Autonomous driving
Word jumble solver
![Page 18: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/18.jpg)
Preview of the course • Deterministic environments: search, constraint
satisfaction, classical planning – Can be sequential or episodic
• Multi-agent, strategic environments: minimax search, games – Can also be stochastic, partially observable
• Stochastic environments – Episodic: Bayesian networks, pattern classifiers – Sequential, known: Markov decision processes – Sequential, unknown: reinforcement learning
![Page 19: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/19.jpg)
Review: PEAS
![Page 20: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/20.jpg)
Review: PEAS • P: Performance measure
– Function the agent is maximizing (or minimizing) • E: Environment
– A formal representation for world states – For concreteness, a tuple (var1=val1, var2=val2, … ,varn=valn)
• A: Actions – Transition model: Given a state and action, what is the
successor state (or distribution over successor states)?
• S: Sensors – Observations that allow the agent to infer the world state – Often come in very different form than the state itself
![Page 21: Rational Agents (Chapter 2)slazebni.cs.illinois.edu/fall17/lec03_agents.pdf · Semidynamic Static Dynamic Discrete Discrete Continuous Multi Multi Multi Fully Deterministic Episodic](https://reader031.fdocuments.us/reader031/viewer/2022021903/5b9ea1cf09d3f2d0208c083f/html5/thumbnails/21.jpg)
Review: Environment types • Fully observable vs. partially observable • Deterministic vs. stochastic (vs. strategic) • Episodic vs. sequential • Static vs. dynamic (vs. semidynamic) • Discrete vs. continuous • Single agent vs. multi-agent • Known vs. unknown