Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score...

31

Transcript of Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score...

Page 1: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

�� �� �� � �� �� � � � � � � �� � � �

� �� �� � �� � �� �

�� �� � � � � � � �� � �� � � �� �� �

� � � �! �� �"# # # # # # # # # # # # # # # # #

$� �� � % � �! % �� &' � �� ( �) � �* +� � � �! � �, �� � � � �� � + & � � �� - �� , � �� � � �� + � � � . � � � +� � � �* / � * � � + � � * � � �� & �

0 � �� * �

1234 56 274 3 8 2 6 9: 63; 7 74 <; 6 =; >

Page 2: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

�� ��� � �

� � � � �� � �� �� � �

� � � � � � � � � � � � �

� � �� � �� � � � � �� � � �� � ��

��� > ! "# $# $%& &

Page 3: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � �� � � � � �

� � � � � � �� �� � � � ��� �� � � � � � � � �� � � � � � Int� � �� � �

� � � � �� �� �� � � � � � � �� �� �� � � � �

� � � ��� � � � � � � � � � Int� �

� � � � � � � � � �� � �� ���

� � � � � � � � � � � �� � � � � � � � � �� � � � � � ��

� � � � � � �� � �� � � � � Int�

��� > ! "# $# $%& �

Page 4: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� �� � �� � � � �� � � � � � �� � � �� � �

practical reasoning

utilitarianism

perfect rationality

bounded rationalitycalculative rationality

metalevel rationality

bounded optimality

ecological rationality

heuristics and biases

��� > ! "# $# $%&

Page 5: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � � � � � � �� � � �� � � � �

O

A

NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

132

NEXTOBJECT

SCORE

138

NEXTOBJECT

SCORE

144

NEXTOBJECT

SCORE

151

� � � � � �� � �� � �� � � � � � �� � � � � E

� � � � � � � � f �� � � �

� � �� � � � � � � �� � � �� � � � � � � � � � �

� � � � � � �� � V f � E � � � � � � � f� E

��� > ! "# $# $%& !

Page 6: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

Int1� � �� � � � � � �� � � �� � �

� � � f � ��� � � � �� � �� � �� � � � �

f � � � � �� � �� f V f �� �

� � � � � � � � � � � � �� � � �� �� � ��

� � � ��� � � � � � � � � � � �� � �� � �� � � �� �

� �� �� �� � �� � � � � �

�� �� � �� � � � � � � �� � �

� � � � � �� � �� � � � � � � � � � �� � �

� � � � � � �� � � � � � � � � � � � �� �� � �� �� �

��� > ! "# $# $%& �

Page 7: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

�� � � � � � � �� � �� � �

NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE132

NEXTOBJECT

SCORE

138

NEXTOBJECT

SCORE

144

NEXTOBJECT

SCORE

151

� � � � � � �� � � M� � � � �� � �� � p

� � � � � � � � � � � � � � � � f � Agent p � M �

��� > ! "# $# $%&

Page 8: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

Int2� � � �� � �� � � � �� � � �� � �

p� � � � � �� � � �� �� � � � � � Agent p � M � � f � ���

� � � M� �� � � �� � � �

� � � � � p � � � � � � � � � � � � � � � � � � � � �

� � � ��� � � � � � � � � � � � � �� � � �� � � � � �� �

� � �� � � �� � � � �� � ��� � �� � � �

� �� � �� � � � �

� � � � � �� � � � � �� � � � � � � � � � � � �

� � �� � � �� �� � � �� � � �� �� � �� �� � �

��� > ! "# $# $%& �

Page 9: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

Int3� � � � � � � � � � �� � � � � � �

Agent p � M �� � � � � � � �� � �� � � � � � � � � � � � �

� � � � � � � � � � � �� � � � � �� � � � � � � � �� � � �� � �

� � � ��� � � � � � � � � � � � � � � �� � �� � � �� �

� �� �� �� � �� � � � � �

�� �� � �� � � � � � � �� � �

� � � � � �� � �� � � � � � � � � � � � �

��� > ! "# $# $%& #

Page 10: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � �� � � � � � � � � � �� �� � �

� � � � �� �� � � � � � � �

� � � � � � � � � � � � � � �

� � � � � � � � � � �� � � �� � �� � �� � �

� � � � �� � �� �� � � � � �

� �� � � � �� � �� � �

� � � � � � � � � � � � � � � � � � � �� � � � � � � � �

� � � � � � � � � � � � �

� � � � � �� � � � � � � �

��� > ! "# $# $%& >%

Page 11: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � �� � � � � � � � �� � �

� �� � �� � � � �� � � � � �� � � � � � � �� � � �

quality

time

benefit

value

cost

� � � � � � � � � � � � � ��� � � � � �� � � �

� � � � � � � � � �� � �� � � � � � � �� �

��� > ! "# $# $%& > >

Page 12: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

�� � � � �� � � � �� � � � � �� � �

� � � �� � � � �� � � � � � � �� � � � � � � � � � � � � � � � � 0

5000

10000

15000

20000

25000

30000

35000

40000

0 5 10 15 20 25 30 35 40 45 50

Perf

orm

ance

(nu

mbe

r of

row

s cl

eare

d)

Number of search nodes expanded

Performance of Best-First expansion policy (with GameSpeed = 10)

mean

��� > ! "# $# $%& >&

Page 13: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

�� � ��� �� � � � � � � � � � �� �� � �

� � ��� � � � � � �� �� �� �� � �� � � � � � � � �

? ? ? ?

� � �� �� � � �� �� � � �� �� � �

� � � �� � � � � � �� � � � � �� � ��

� �� � � � � �� � � � � � � �

� � ��� � � � �� � � � �� � � � � � � � �� � �� ��

� � �� � � � � � � � �� � � � � � � �� � � � �

��� > ! "# $# $%& > �

Page 14: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � � �� � � � � � �

� � � � � � � � � � �� � � �� � �� � � � �� � � � � �� � � � � � �� � � � �

ALGORITHMS

��� > ! "# $# $%& >

Page 15: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

Int4� � � � � � � � � � �� � � � � �

Agent p � � � � M �� � � � � � � � � �� � � � �

p � ��� � �� � �� pV Agent p � M � �� �

� � � � � � � � � �� � �� � �� � � M �

� � � �� � � � � � � � � � � � � � � � � �� � ��

� �� �� �� � �� � � � � �

� �� � � � � � �� � �

� � � � � �� � � � � �� � �� � � � �

� � � � � � � �� � �� � � � � �� � � � �� � �� �� � � � �

� � � � � � �� � � � � � �� � � � � � � � � � � � � �� � �� � �� �� � � � � �

��� > ! "# $# $%& >!

Page 16: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � �� � � � � � � � � � � � �� � � � � � � � � �� � �

� � � � � � � �� � �� �� � � � � � � � � � � � � � �� � � � � � �

� � �� �� � � � � �� �� � �� � � � � � � � �

� � � � � �� � � � � �� � � � � � � � � � � � � � � �

� � � � �� � � � � � � � � � � � �� � �� �� �� � �� � � � �� � �

� � � � � � � � �� � �� � �� � � � � � �� � � � � � �

� � � � � � � � �� � �� � �� � � � � � �� � � � � � � �

� � � �� � � � � � � � � � � � � � �� �� �� � �

� � �� � � � � � � � � �� � �� �

��� > ! "# $# $%& > �

Page 17: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

�� � � � � � � �� � � �

� �� � � � � NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

126

� � � �� � � � � � �� � � � � � � � � � � � � V

� V� � � � � � � � � � � � � � � � � � � �

� � � � � �� ��

� � � � � � � � � � � �� � � � � � � � � �

� � � � �� �� � �� � � � � � � � � � � � � � � �� � � � �

� � � �� � � � � � � � � � �� � �� � � � � �

� �� � � �� � � � �� � � � � � � � � � ��� � � � � � �� �� �

� � � � � � � ��� � � � � � �� � � � � � �� � �

��� > ! "# $# $%& >

Page 18: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

�� � � � � � � �� � � � �

� �� � � � �

� � � �� � � � � � �� � � � � � � � � � � � � �

� V� � � � � � � �� � � � � � � � � � � �� � � � � ��

� � �� � � � � � � �� � � � � � � � � � � � � � � �� � �� � � � � �

� � � � � �� ��

� � � � � � � �� � � � � � � � � � �

� � � � �� �� � �� � � � � � � � �

� � �� � � �� � � � � � � �� � � � �� � �

��� > ! "# $# $%& > �

Page 19: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � � � � � �� � �

� � � � � � � � � � �� � � � � � � � � � � � � � � �� Q �� � � � �

NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

126

QQ

Q� � � � � � �� � � � � � � � � � � �� � � � � ��

� � � � � �� ��

� � �� � � �� � � � � � �� � � � � � �� � � �

�� � � � � � � � � � �

� � � � �� ��

� �� � � � � � � � � �� � � � �� � � � � �� � � �� �� � � � �

� � � � � � � � � � � � � �� � � � � � � � � � � �� Q �� � � � �

��� > ! "# $# $%& >#

Page 20: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � � � � � �� �� � � �� � � � ��

� �� � �� � � �� �� � � � � � � ��

� � � � � � � � � �� �� � � � �

NextPiece:

NextPiece:

Control ProblemObject Level

Object LevelController

Meta−LevelControl Problem

� � � �� � �� � � �� � � � � � � � � �� � � � � � � � �� � � � �

� � � � �� � � � �� �� � � �� � � �� � � �� � � � � �� � �

� � � �� � �� �� � � � �� � � � � � � � � � � �

��� > ! "# $# $%& & %

Page 21: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � � � � � � �� � � �� � � � � �� � � � � �� � �

5

3 11

10

6

22 1 1 9

−4 −2 12 12

6

10 1 0 2 1 9 3 3

4.52 0.04 7.71

� � � � � � f� �� �� �� �� � �� � � � � � � � � � � � � � � � � � �

� �� � � � � �� � � �� � � � � � � �

�� � � � � �� �� � � � � � �� � �� � � �

� � � � � f� � � � � � �� �� �� � �� � � � � � � � �

� � � � � � � � �� �� � � � � � � � � �� �� � � �� � � �

��� > ! "# $# $%& & >

Page 22: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � � � � � � � � �� � � � �� � � �

computations

actions

rewards 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1

� � � � � �� �� � � �� � � � � � � ��� �� � � � �� �� �� � � � �� � � � �

� � �� � � � � � � �� �� � � � �� �� �� � � � � � �� �� � �

� � � � � � � � � � �� �� � � � �� � �� � � �� � � � � � �� � �� �

� � � � � � � � � � � � � � � � �� �� � � � � � � � � � � �� � � � � �� � � � ��

� ���� > ! "# $# $%& & &

Page 23: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � �� � � � � � � �� �

reject

mail sortcamera

Time

Probability

E� � � �� � � � � � � � � � � � � � �M� � � � � � �� � �� � � �� � �� �� � �

� � � � � � � p � ��� � � � � � � � � �� �� �� � �

� ��� �� � � � � �� �� � � � ��� ��� � � � � � �

��� > ! "# $# $%& & �

Page 24: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � �� � �� �� � � � � � � � � � � �� � �� � �

� � � � � � � � � � �� � �� �� � � � � � �� � �

p� � � � � � � � � � � � � � � � � � � �� � � � � � � �� �

� k V Agent p � kM � �� � � V Agent p � � � � M � �� �

� � � � � � � � � �� � � � M � k � � � � � � � � � �� p � �� � � � � � � �

�� � �� � � � � � � � �� � � �� � � � � �

� � �� �� � � � � � �� � � � � � �� �� � �

��� > ! "# $# $%& &

Page 25: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � �� � �� � � � � � � � �� � � �

� � � � � � � �� � �� � � � � � � �� �� � � � �� � � � �� � � � � � � � �� � �

� � pi � � � � � �� � � � � � � � �� � t � 2 i�

� � �� �� � � � � � � �� � � � � � �� � � �� � �� � pU

p0 ppp 321

pU� � � � � � �� � � � � �� � �� �� � �� �

� � � � � � � � �� � � � � � � � �� �� � � � � �

��� > ! "# $# $%& & !

Page 26: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

�� � � �� � � � � � � � � �� �� � � � � � � � � �� �

� � � � �� � � �� � � �� � � �� � �� �� � �

time

diagnosis quality

time

therapy qualitytherapy diagnose x � �

� � � �� � � � �� � � � � � � � � � � � � � �� �� � � � � � � � � � � �� �� �� � �

� � � � � �� � � � � �� � � � � � � � � �� � � �� � � � � � � � � � �� � �

� � � � � � � � � �� � � � � �

��� > ! "# $# $%& & �

Page 27: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � �� �� � � � � �� � �� � � � � � � � �� � � �

� � � � � � �� � � � �� �� � � � �� � � � � � � � �� � � � � � � � � � � �

� � � � � � � � �� � � � � � � � � �� � � �� � � � �� � � � � � � �

�� �� � � � �� � � � � � NEXTOBJECT

SCORE

126

NEXTOBJECT

SCORE

126

� � � � � �� � � � � � � �� � � � � � � � � � � �� � � � � � � � �

� �� � � �� � �� � � �� � � � �� �� � � � � � �

� � � � � �� � � �� � � � � � � � �� � � � � � �� � �

��� > ! "# $# $%& &

Page 28: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � � �� � �� � � �� �� � �� � � � � �� � � � � � � �

� � � � � � � �� �� � � � � � �� � �� � � �� �� �� � � �� �� �� � � �

� � � � � � � � � � �� � � � � � � � � �

� � � � � � �� � � � �� � � � � � �� � � �

� � � � � � � � � � � � �� � �� � � � � � � � � �

� � �� � � � � �� � �� � � � � � � � � � � � � � � � � � � � � � �

� � � �

� � � �� � � � � �� �� � � � � �� � � �� �� �� � ��

��� > ! "# $# $%& & �

Page 29: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � � � � � �� � � �� � � �� �� �� � � �

memory

speed

��� > ! "# $# $%& & #

Page 30: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� �� � � � � � � � � � � � �� � �� � �

� � � �� � �� �� � � � � � � �� � � � � � � �� � �

WOW !!

� � � � � � �� � � � �� � �� � �

� � � � � � � � � � � � �� � �� � �

� � � � � � � � � � � � � �

� � � �� �� � �� � �� �� � �� � � � � � �� � � � � � � ��� � � � ��

� � �� � �� � � � � � � � � � � ��

�� � � � � �� � � �� �

� � � � �� � � � � �

� � � � �� � � �

� � � � � � � � � � �� � � � � � � � �� � � �� �

U.C. BERKELEY

� � �� � � �� �� � � � � � � � �� �� � � �� � � � �

��� > ! "# $# $%& � %

Page 31: Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score 126 next object score 126 next object score 126 next object score 126 next object

� � � � �� �� � � �

�� � � � � � � � �� �� � � �

�� � � � � � � �� � �

� �� �� � � � � �� � � � � � � � � � � � � �

� � � � � � � � �� � �� � �

�� �� �� � � �� � � �� � �� �� � � � �

� � � � � � �� � � � � �� � � �� � � � �

1234 56 274 3 8 2 6 9: 63; 7 74 <; 6 =; � >