Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score...
Transcript of Int - people.eecs.berkeley.edurussell/talks/2002/russell-bounded-optimality.pdfo a next object score...
�� �� �� � �� �� � � � � � � �� � � �
� �� �� � �� � �� �
�� �� � � � � � � �� � �� � � �� �� �
� � � �! �� �"# # # # # # # # # # # # # # # # #
$� �� � % � �! % �� &' � �� ( �) � �* +� � � �! � �, �� � � � �� � + & � � �� - �� , � �� � � �� + � � � . � � � +� � � �* / � * � � + � � * � � �� & �
0 � �� * �
1234 56 274 3 8 2 6 9: 63; 7 74 <; 6 =; >
�� ��� � �
� � � � �� � �� �� � �
� � � � � � � � � � � � �
� � �� � �� � � � � �� � � �� � ��
��� > ! "# $# $%& &
� � � � � �� � � � � �
� � � � � � �� �� � � � ��� �� � � � � � � � �� � � � � � Int� � �� � �
� � � � �� �� �� � � � � � � �� �� �� � � � �
� � � ��� � � � � � � � � � Int� �
� � � � � � � � � �� � �� ���
� � � � � � � � � � � �� � � � � � � � � �� � � � � � ��
� � � � � � �� � �� � � � � Int�
��� > ! "# $# $%& �
� �� � �� � � � �� � � � � � �� � � �� � �
practical reasoning
utilitarianism
perfect rationality
bounded rationalitycalculative rationality
metalevel rationality
bounded optimality
ecological rationality
heuristics and biases
��� > ! "# $# $%&
� � � � � � � � � � �� � � �� � � � �
O
A
NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
132
NEXTOBJECT
SCORE
138
NEXTOBJECT
SCORE
144
NEXTOBJECT
SCORE
151
� � � � � �� � �� � �� � � � � � �� � � � � E
� � � � � � � � f �� � � �
� � �� � � � � � � �� � � �� � � � � � � � � � �
� � � � � � �� � V f � E � � � � � � � f� E
��� > ! "# $# $%& !
Int1� � �� � � � � � �� � � �� � �
� � � f � ��� � � � �� � �� � �� � � � �
f � � � � �� � �� f V f �� �
� � � � � � � � � � � � �� � � �� �� � ��
� � � ��� � � � � � � � � � � �� � �� � �� � � �� �
� �� �� �� � �� � � � � �
�� �� � �� � � � � � � �� � �
� � � � � �� � �� � � � � � � � � � �� � �
� � � � � � �� � � � � � � � � � � � �� �� � �� �� �
��� > ! "# $# $%& �
�� � � � � � � �� � �� � �
NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE132
NEXTOBJECT
SCORE
138
NEXTOBJECT
SCORE
144
NEXTOBJECT
SCORE
151
� � � � � � �� � � M� � � � �� � �� � p
� � � � � � � � � � � � � � � � f � Agent p � M �
��� > ! "# $# $%&
Int2� � � �� � �� � � � �� � � �� � �
p� � � � � �� � � �� �� � � � � � Agent p � M � � f � ���
� � � M� �� � � �� � � �
� � � � � p � � � � � � � � � � � � � � � � � � � � �
� � � ��� � � � � � � � � � � � � �� � � �� � � � � �� �
� � �� � � �� � � � �� � ��� � �� � � �
� �� � �� � � � �
� � � � � �� � � � � �� � � � � � � � � � � � �
� � �� � � �� �� � � �� � � �� �� � �� �� � �
��� > ! "# $# $%& �
Int3� � � � � � � � � � �� � � � � � �
Agent p � M �� � � � � � � �� � �� � � � � � � � � � � � �
� � � � � � � � � � � �� � � � � �� � � � � � � � �� � � �� � �
� � � ��� � � � � � � � � � � � � � � �� � �� � � �� �
� �� �� �� � �� � � � � �
�� �� � �� � � � � � � �� � �
� � � � � �� � �� � � � � � � � � � � � �
��� > ! "# $# $%& #
� � �� � � � � � � � � � �� �� � �
� � � � �� �� � � � � � � �
� � � � � � � � � � � � � � �
� � � � � � � � � � �� � � �� � �� � �� � �
� � � � �� � �� �� � � � � �
� �� � � � �� � �� � �
� � � � � � � � � � � � � � � � � � � �� � � � � � � � �
� � � � � � � � � � � � �
� � � � � �� � � � � � � �
��� > ! "# $# $%& >%
� � � �� � � � � � � � �� � �
� �� � �� � � � �� � � � � �� � � � � � � �� � � �
quality
time
benefit
value
cost
� � � � � � � � � � � � � ��� � � � � �� � � �
� � � � � � � � � �� � �� � � � � � � �� �
��� > ! "# $# $%& > >
�� � � � �� � � � �� � � � � �� � �
� � � �� � � � �� � � � � � � �� � � � � � � � � � � � � � � � � 0
5000
10000
15000
20000
25000
30000
35000
40000
0 5 10 15 20 25 30 35 40 45 50
Perf
orm
ance
(nu
mbe
r of
row
s cl
eare
d)
Number of search nodes expanded
Performance of Best-First expansion policy (with GameSpeed = 10)
mean
��� > ! "# $# $%& >&
�� � ��� �� � � � � � � � � � �� �� � �
� � ��� � � � � � �� �� �� �� � �� � � � � � � � �
? ? ? ?
� � �� �� � � �� �� � � �� �� � �
� � � �� � � � � � �� � � � � �� � ��
� �� � � � � �� � � � � � � �
� � ��� � � � �� � � � �� � � � � � � � �� � �� ��
� � �� � � � � � � � �� � � � � � � �� � � � �
��� > ! "# $# $%& > �
� � � � � � �� � � � � � �
� � � � � � � � � � �� � � �� � �� � � � �� � � � � �� � � � � � �� � � � �
ALGORITHMS
��� > ! "# $# $%& >
Int4� � � � � � � � � � �� � � � � �
Agent p � � � � M �� � � � � � � � � �� � � � �
p � ��� � �� � �� pV Agent p � M � �� �
� � � � � � � � � �� � �� � �� � � M �
� � � �� � � � � � � � � � � � � � � � � �� � ��
� �� �� �� � �� � � � � �
� �� � � � � � �� � �
� � � � � �� � � � � �� � �� � � � �
� � � � � � � �� � �� � � � � �� � � � �� � �� �� � � � �
� � � � � � �� � � � � � �� � � � � � � � � � � � � �� � �� � �� �� � � � � �
��� > ! "# $# $%& >!
� � � � � �� � � � � � � � � � � � �� � � � � � � � � �� � �
� � � � � � � �� � �� �� � � � � � � � � � � � � � �� � � � � � �
� � �� �� � � � � �� �� � �� � � � � � � � �
� � � � � �� � � � � �� � � � � � � � � � � � � � � �
� � � � �� � � � � � � � � � � � �� � �� �� �� � �� � � � �� � �
� � � � � � � � �� � �� � �� � � � � � �� � � � � � �
� � � � � � � � �� � �� � �� � � � � � �� � � � � � � �
� � � �� � � � � � � � � � � � � � �� �� �� � �
� � �� � � � � � � � � �� � �� �
��� > ! "# $# $%& > �
�� � � � � � � �� � � �
� �� � � � � NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
126
� � � �� � � � � � �� � � � � � � � � � � � � V
� V� � � � � � � � � � � � � � � � � � � �
� � � � � �� ��
� � � � � � � � � � � �� � � � � � � � � �
� � � � �� �� � �� � � � � � � � � � � � � � � �� � � � �
� � � �� � � � � � � � � � �� � �� � � � � �
� �� � � �� � � � �� � � � � � � � � � ��� � � � � � �� �� �
� � � � � � � ��� � � � � � �� � � � � � �� � �
��� > ! "# $# $%& >
�� � � � � � � �� � � � �
� �� � � � �
� � � �� � � � � � �� � � � � � � � � � � � � �
� V� � � � � � � �� � � � � � � � � � � �� � � � � ��
� � �� � � � � � � �� � � � � � � � � � � � � � � �� � �� � � � � �
� � � � � �� ��
� � � � � � � �� � � � � � � � � � �
� � � � �� �� � �� � � � � � � � �
� � �� � � �� � � � � � � �� � � � �� � �
��� > ! "# $# $%& > �
� � � � � � � � � �� � �
� � � � � � � � � � �� � � � � � � � � � � � � � � �� Q �� � � � �
NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
126
Q� � � � � � �� � � � � � � � � � � �� � � � � ��
� � � � � �� ��
� � �� � � �� � � � � � �� � � � � � �� � � �
�� � � � � � � � � � �
� � � � �� ��
� �� � � � � � � � � �� � � � �� � � � � �� � � �� �� � � � �
� � � � � � � � � � � � � �� � � � � � � � � � � �� Q �� � � � �
��� > ! "# $# $%& >#
� � � � � � � � � �� �� � � �� � � � ��
� �� � �� � � �� �� � � � � � � ��
� � � � � � � � � �� �� � � � �
NextPiece:
NextPiece:
Control ProblemObject Level
Object LevelController
Meta−LevelControl Problem
� � � �� � �� � � �� � � � � � � � � �� � � � � � � � �� � � � �
� � � � �� � � � �� �� � � �� � � �� � � �� � � � � �� � �
� � � �� � �� �� � � � �� � � � � � � � � � � �
��� > ! "# $# $%& & %
� � � � � � � � � � �� � � �� � � � � �� � � � � �� � �
5
3 11
10
6
22 1 1 9
−4 −2 12 12
6
10 1 0 2 1 9 3 3
4.52 0.04 7.71
� � � � � � f� �� �� �� �� � �� � � � � � � � � � � � � � � � � � �
� �� � � � � �� � � �� � � � � � � �
�� � � � � �� �� � � � � � �� � �� � � �
� � � � � f� � � � � � �� �� �� � �� � � � � � � � �
� � � � � � � � �� �� � � � � � � � � �� �� � � �� � � �
��� > ! "# $# $%& & >
� � � � � � � � � � � � �� � � � �� � � �
computations
actions
rewards 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
� � � � � �� �� � � �� � � � � � � ��� �� � � � �� �� �� � � � �� � � � �
� � �� � � � � � � �� �� � � � �� �� �� � � � � � �� �� � �
� � � � � � � � � � �� �� � � � �� � �� � � �� � � � � � �� � �� �
� � � � � � � � � � � � � � � � �� �� � � � � � � � � � � �� � � � � �� � � � ��
� ���� > ! "# $# $%& & &
� � � � � �� � � � � � � �� �
reject
mail sortcamera
Time
Probability
E� � � �� � � � � � � � � � � � � � �M� � � � � � �� � �� � � �� � �� �� � �
� � � � � � � p � ��� � � � � � � � � �� �� �� � �
� ��� �� � � � � �� �� � � � ��� ��� � � � � � �
��� > ! "# $# $%& & �
� � �� � �� �� � � � � � � � � � � �� � �� � �
� � � � � � � � � � �� � �� �� � � � � � �� � �
p� � � � � � � � � � � � � � � � � � � �� � � � � � � �� �
� k V Agent p � kM � �� � � V Agent p � � � � M � �� �
� � � � � � � � � �� � � � M � k � � � � � � � � � �� p � �� � � � � � � �
�� � �� � � � � � � � �� � � �� � � � � �
� � �� �� � � � � � �� � � � � � �� �� � �
��� > ! "# $# $%& &
� � � � � �� � �� � � � � � � � �� � � �
� � � � � � � �� � �� � � � � � � �� �� � � � �� � � � �� � � � � � � � �� � �
� � pi � � � � � �� � � � � � � � �� � t � 2 i�
� � �� �� � � � � � � �� � � � � � �� � � �� � �� � pU
p0 ppp 321
pU� � � � � � �� � � � � �� � �� �� � �� �
� � � � � � � � �� � � � � � � � �� �� � � � � �
��� > ! "# $# $%& & !
�� � � �� � � � � � � � � �� �� � � � � � � � � �� �
� � � � �� � � �� � � �� � � �� � �� �� � �
time
diagnosis quality
time
therapy qualitytherapy diagnose x � �
� � � �� � � � �� � � � � � � � � � � � � � �� �� � � � � � � � � � � �� �� �� � �
� � � � � �� � � � � �� � � � � � � � � �� � � �� � � � � � � � � � �� � �
� � � � � � � � � �� � � � � �
��� > ! "# $# $%& & �
� � � � � �� �� � � � � �� � �� � � � � � � � �� � � �
� � � � � � �� � � � �� �� � � � �� � � � � � � � �� � � � � � � � � � � �
� � � � � � � � �� � � � � � � � � �� � � �� � � � �� � � � � � � �
�� �� � � � �� � � � � � NEXTOBJECT
SCORE
126
NEXTOBJECT
SCORE
126
� � � � � �� � � � � � � �� � � � � � � � � � � �� � � � � � � � �
� �� � � �� � �� � � �� � � � �� �� � � � � � �
� � � � � �� � � �� � � � � � � � �� � � � � � �� � �
��� > ! "# $# $%& &
� � � � � � �� � �� � � �� �� � �� � � � � �� � � � � � � �
� � � � � � � �� �� � � � � � �� � �� � � �� �� �� � � �� �� �� � � �
� � � � � � � � � � �� � � � � � � � � �
� � � � � � �� � � � �� � � � � � �� � � �
� � � � � � � � � � � � �� � �� � � � � � � � � �
� � �� � � � � �� � �� � � � � � � � � � � � � � � � � � � � � � �
� � � �
� � � �� � � � � �� �� � � � � �� � � �� �� �� � ��
��� > ! "# $# $%& & �
� � � � � � � � �� � � �� � � �� �� �� � � �
memory
speed
��� > ! "# $# $%& & #
� �� � � � � � � � � � � � �� � �� � �
� � � �� � �� �� � � � � � � �� � � � � � � �� � �
WOW !!
� � � � � � �� � � � �� � �� � �
� � � � � � � � � � � � �� � �� � �
� � � � � � � � � � � � � �
� � � �� �� � �� � �� �� � �� � � � � � �� � � � � � � ��� � � � ��
� � �� � �� � � � � � � � � � � ��
�� � � � � �� � � �� �
� � � � �� � � � � �
� � � � �� � � �
� � � � � � � � � � �� � � � � � � � �� � � �� �
U.C. BERKELEY
� � �� � � �� �� � � � � � � � �� �� � � �� � � � �
��� > ! "# $# $%& � %
� � � � �� �� � � �
�� � � � � � � � �� �� � � �
�� � � � � � � �� � �
� �� �� � � � � �� � � � � � � � � � � � � �
� � � � � � � � �� � �� � �
�� �� �� � � �� � � �� � �� �� � � � �
� � � � � � �� � � � � �� � � �� � � � �
1234 56 274 3 8 2 6 9: 63; 7 74 <; 6 =; � >