Post on 19-Dec-2015
Modeling Qualitative Preferences
Using the CP-net Model
To make good decision, we must be able to assess and compare different
alternatives.
The ability to make decisions is a corner-stone of many AI applications:
• Decision-support expert systems
• Autonomous agents
• Configuration software
• …
Assessing Alternatives
We compare alternatives based on their:
• Likelihood
• Desirability
Our Focus:
Assessing/specifying outcome desirability
Specifying Preferences
UncertaintyInvolved?
Utility functiondesirable
Enough torank potential
outcomes
yes
no
Need to weigh the contributions of different outcome
Need to recognize the best feasible alternative
Utility functions
• Qantify outcome desirability– Capture the difference in desirability
between outcomes
• Necessary when:– Uncertainty is involved, or
• Drawbacks:– Complicated preference elicitation.– Generally hard optimization.
Serious practical concern
Serious comput. concern
When a utility function cannot be and/or need not be obtained, one should resort to other, more qualitative forms of preference representation.
What We Want from a Qualitative Preference Model
• Simple elicitation process based on intuitive and natural statements about preferences– No need for an expert decision analyst– Makes automatic online elicitation feasible
• As expressive as possible, subject to above
• Supports an efficient optimization process
Overview
• CP-net model for qualitative preferences.• Preference Optimization.• Complexity analysis of outcome comparisons.• Consistency testing.• Various Enhancements.
• Potential applications.• Future research directions and open problems.
Overview
• CP-net model for qualitative preferences.• Preference Optimization.• Complexity analysis of outcome comparisons.• Consistency testing.• Various Enhancements.
• Potential applications.• Future research directions and open problems.
Ceteris Paribus (cp) Statemenents
Ceteris Paribus (Lat.) – all else being equal
“ I prefer to have wine with my meal, all else being equal”
That is: given two identical meals, one with wine and one without, I prefer the former.
Conditional CP Statements
“ I prefer red wine to white wine with my meal, ceteris paribus, given that meat is served”
That is: given two identical meals in which meat is served, I prefer red wine to white wine.
Tells us nothing about two identical meals in which meat is NOT served.
Outcomes and Preferences1. Domain variables .
2. Outcome space .
3. Preferences over the outcome space.
1{ , , }nV
1( ) ( )nDom Dom
Example on Product (Computer System) Configuration:
• Domain variables are the system’s properties:
Processor Speed, Processor Manufacturer, Screen Size, etc.
• Dom(Screen Size) = {15in, 17in, 19in, 21in}
• Possible preferential statements of a customer:
I prefer 1000 MHz on 800 MHz
I prefer 19in screen on 17in screen if video card is Sony’s
Preferential Independence
If my preferences over the values of a variable v does not depends on the values of some other variables, then v is preferentially independent of all other variables.
I prefer 1000 MHz to 800 MHz (all else being equal)
If my preferences over the values of a variable v depends on the values of some other variables v1, …,vk , then v is conditionally preferentially independent of all other variables V-{v1, …,vk }, given an assignment on v1, …,vk .
I prefer 19in screen to 17in screen if video card is Sony’s (all else being equal)
Preferential Independence
A subset of variables is preferentially independent of its complement if and only if, for all assignments holds
X Y V X
1 2 1 2, , ,x x y y
1 1 2 1 1 2 2 2x y x y iff x y x y
Let be a partition of into three disjoint non-empty sets. and are conditionally preferentially independent given if and only if, for all holds
, ,X Y Z VX Y
z1 2 1 2, , ,x x y y
1 1 2 1 1 2 2 2x y z x y z iff x y z x y z
CP-nets (Boutilier, Brafman, Hoos, Poole, UAI ‘99)
An intuitive, qualitative, graphical model of An intuitive, qualitative, graphical model of preferences, that captures statements of preferences, that captures statements of conditional preferential independence.conditional preferential independence.
• DAG in which each node represents a domain variable.
• The immediate parents P(v) of a variable v in the network are those variables that affect user’s preference over the values of v.
P(screen size) = { video card manuf. }P(operating system) = { processor speed, screen
size }
A
C
D E
F
B b b
( ) ( ) :
( ) ( ) :
a b a b c c
a b a b c c
:
:
c e e
c e e
a a
:
:
c d d
c d d
:
:
d f f
d f f
Example of a CP-net
Any CP-net defines a partial order over the outcome space.
A
C
B b b
( ) ( ) :
( ) ( ) :
a b a b c c
a b a b c c
a aabc
abc abc abc
abc abc abc
abc
Consistency
worst
best
Overview
• CP-net model for qualitative preferences.• Preference Optimization.• Complexity analysis of outcome comparisons.• Consistency testing.• Various Enhancements.
• Potential applications.• Future research directions and open problems.
Preferential Optimization
Finding the preferentially optimal outcome is straightforward!
A
C
D E
F
B b b
( ) ( ) :
( ) ( ) :
a b a b c c
a b a b c c
:
:
c e e
c e e
a a
:
:
c d d
c d d
:
:
d f f
d f f
A
C
D E
F
Ba b
c
d e
f
Adding Constraints1. Domain variables .
2. Outcome space .
3. Preferences over the outcome space.
4. Constraints on the domain variables.
1{ , , }nV
1( ) ( )nDom Dom
Constraints in Product (Computer System) Configuration:
17in screens are currently unavailable.
15in screens of NEC are incompatible with the Sony’s graphical card.
Windows XP requires at least 800MHz processor speed.
Constraint-based Preferential Optimization (Boutilier et al. 97’)
Branch & Bound algorithm for determining the set of feasible, preferentially non-dominated outcomes was
suggested.• Starting with an empty set of solutions, the algorithm
continuously extends it by adding new non-dominated solutions.
• The current set of the solutions serves a lower bound for the forthcoming candidates.
>>
Constraint-based Preferential Optimization
E A B F
GC
DH
X Y Z
Initialize the set of local results R = .
Choose any variable v with no parents in G.
A Let v1 >… > vk be the preference ordering of D(v) given the assignment on P(v) in K.
K(x,y) : a1 > … > ak
E ai B F
GC
DH
X Y Z for i = 1 to k do v = vi
K(x,y) : a1 > …> ai > …> ak
Strengthen the set of constraints C by v = vi to obtain Ci
if Ci is inconsistent or exist j<i s.t. Cj Ci continue with next iteration
Reduce G to G’ by removing the variables assigned by K’.
E B F
GC
DH
Let G’1,…, G’m be the strongly connected components of G’
for j= 1 to m do
Sj = SearchSearch (G’j, KK’, Ci)
elseLet K’ be the partial
assignment induced by v = vi and Ci .
ai
for i = 1 to k do v = vi
Strengthen the set of constraints C by v = vi to obtain Ci
if Ci is inconsistent or exist j<i s.t. Cj Ci continue with next iteration
Reduce G to G’ by removing the variables assigned by K’.
E B F
GC
DH
Let G’1,…, G’m be the strongly connected components of G’
for j= 1 to m do
Sj = SearchSearch (G’j, KK’, Ci)
elseLet K’ be the partial assignment
induced by v = vi and Ci .
E ai B F
GC
DH
X Y Z
ai
Add o to R
return R
if Sj for all j m
if for each o’ R holds Ko’ Ko
foreach o K’ S1 … Sm do
Constraint-based Preferential Optimization
So what is the price of comparison (dominance testingdominance testing)
between two outcomes?
Overview
• CP-net model for qualitative preferences.• Preference Optimization.• Complexity analysis of outcome comparisons.• Consistency testing.• Various Enhancements.
• Potential applications.• Future research directions and open problems.
abc
abc abc abc
abc abc abc
abc
Flipping Sequence - Example
abc
abcabc
abc
check ( )abc abc
Dominance TestingIn (Boutilier et al.) dominance testing was treated as a search for a flipping sequence from the (purported) less preferred outcome to the (purported) more preferred outcome through a sequence of more preferred outcomes:
0 1 1m m
Where, for , 0 1i m • outcome differs from the outcome in
the value of exactly one variablei 1i
j
• , given the values of in (and )
1[ ] [ ]i ij j ( )jP i
1i
Dominance Testing for CP-nets with Binary Variables.
Backtrack free algorithm in BBHP ‘99
RemarksRemarks
Linear?Tree
ComplexityComplexityCP-net grapCP-net graphh
Complexity of Dominance Testing
First result (Boutilier et al.):
Dominance testing for binary, tree CP-nets is backtrack free.
Example:
A
C
D E
B:
:
a c c
a c c
a a
:
:
c d d
c d d
:
:
d f f
d f f
:
:
a b b
a b b
abcde abcde
a
b c
d e
abcde
cabcde
Example:
A
C
D E
B:
:
a c c
a c c
a a
:
:
c d d
c d d
:
:
d f f
d f f
:
:
a b b
a b b
abcde abcde
a
b c
d e
abcde
abcdeb
abcde
Example:
A
C
D E
B:
:
a c c
a c c
a a
:
:
c d d
c d d
:
:
d f f
d f f
:
:
a b b
a b b
abcde abcde
a
b c
d e
abcde
abcdeb
abcded
abcde
Example:
A
C
D E
B:
:
a c c
a c c
a a
:
:
c d d
c d d
:
:
d f f
d f f
:
:
a b b
a b b
abcde abcde
a
b c
d e
abcde
abcdeb
abcded
abcde
a
abcde
Example:
A
C
D E
B:
:
a c c
a c c
a a
:
:
c d d
c d d
:
:
d f f
d f f
:
:
a b b
a b b
abcde abcde
a
b c
d e
abcde
abcdeb
abcded
abcde
a
abcde
c
abcde
The algorithm for trees is not good for polytrees
A
C
B b b
( ) ( ) :
( ) ( ) :
a b a b c c
a b a b c c
a a
abc abc
c
baabc
a
abc
abc
abc
Dominance Testing for CP-nets with Binary Variables.
Lower bound
RemarksRemarks
Tree
ComplexityComplexityCP-net grapCP-net graphh
2( )O n
Dominance Testing for CP-nets with Binary Variables.
Lower bound
RemarksRemarks
Tree
Polytree
ComplexityComplexityCP-net grapCP-net graphh
2( )O n2 2 3(2 )k kO n
k - maximal indegree
Start of Analysis
Denote by the maximal number of times that a variable may be required to flip its value on a irreducible flipping sequence from to .
Lemma 1Lemma 1: Given a dominance testing problem where is a singly connected, binary CP-net, for each variable we have thatolds:
MaxReq( )ii
, ,N
i
MaxReq( )i n
N
Framework for Polytrees
• Using the upper bound established by Lemma 1, we provide a polynomial time procedure that determines the maximal number of feasible, possibly required value flips for a given variable.
• We provide a polynomial time algorithm that determines whether or not exist a flipping sequence from to . This algorithm is based on top-down execution of the previously defined procedure on the variables of the CP-net.
MaxReq
FeasibleMaxReq( ) min(MaxCan( ),MaxReq( ))i i i
Dominance Testing for CP-nets with Binary Variables.
k - maximal indegree
Lower bound
RemarksRemarks
Tree
Singly-connected DAG
Polytree
ComplexityComplexityCP-net grapCP-net graphh
2( )O n2 2 3(2 )k kO n
NP-complete Reduction from 3SAT
Dominance Testing for CP-nets with Binary Variables.
Minimal flipping sequences are polynomially bounded
Reduction from 3SAT
- maximal indegree
Lower bound
RemarksRemarks
Tree
NP-complete -connected DAG
NP-completeSingly-connected DAG
Polytree
ComplexityComplexityCP-net grapCP-net graphh
2( )O n2 2 3(2 )kO n
Dominance Testing for CP-nets with Binary Variables.
EXPTIME or in NP?
Minimal flipping sequences are polynomially bounded
Reduction from 3SAT
- maximal indegree
Lower bound
RemarksRemarks
Tree
?DAG
NP-complete -connected DAG
NP-completeSingly-connected DAG
Polytree
ComplexityComplexityCP-net grapCP-net graphh
2( )O n2 2 3(2 )kO n
Recall that ...
• Starting with an empty set of solutions, the B&B algorithm continuously extends it by adding new non-dominated solutions.
• The current set of the solutions serves a lower bound for the forthcoming candidates.
• New candidate is compared to all solutions generated until now.
The Big Picture• Generally, COP is much harder than CSP.
• If any non-dominated solution is enough, then the CP-net based COP is not harder than the corresponding CSP.
• If some non-dominated solutions are required, and the CP-net is reasonably restricted, then the CP-net based COP is still not harder than the corresponding CSP.
• In general, if some (even just 2) non-dominated solutions are required, then the CP-net based COP may be much harder.
Overview
• CP-net model for qualitative preferences.• Preference Optimization.• Complexity analysis of outcome comparisons.• Consistency testing.• Various Enhancements.
• Potential applications.• Future research directions and open problems.
Any CP-net defines a partial order over the outcomes.
A
C
B b b
( ) ( ) :
( ) ( ) :
a b a b c c
a b a b c c
a aabc
abc abc abc
abc abc abc
abc
Recall that …
worst
best
acyclic
Cyclic CP-nets are not necessarily consistent …
Consistency of Cyclic CP-nets
A
B
:
:
a b b
a b b
:
:
b a a
b a a
ab ab
ab ab
:
:
b a a
b a a
ab ab
ab ab
Localizing the Analysis
Strongly Connected Components
Context variables of a SCC
Contexts of a SCC =
assignments to its context variables
Localizing the Analysis
Theorem: A cyclic CP-net is consistent if and only if for every strongly connected component and for every context of , we have that is consistent given .
Conclusion:We can start by consistency analysis of
only strongly connected CP-nets.
Basic SCCs – Simple Cycles, Binary Variables
a
c d
b
Simple cycle
Surprisingly …
Theorem: A consistent specification of a binary, simple cycle CP-net over more than 2 variables is impossible!
A
B:
:
a b b
a b b
:
:
b a a
b a a
ab ab
ab ab
Putting Things Together
Theorem: A consistent specification of a binary, simple cycle CP-net over more than 2 variables is impossible!
Surprising observation:There are many cyclic binary CP-nets with
simple cycles SCCs that are consistent!
Theorem: A cyclic binary CP-net is consistent if and only if for every its strongly connected component and for every context of , we have that is consistent given .
Disappointing?
Specific contexts break the cycles!
Overview
• CP-net model for qualitative preferences.• Preference Optimization.• Complexity analysis of outcome comparisons.• Consistency testing.• Various Enhancements.
– Importance tradeoffs– Adding utilities
• Potential applications.• Future research directions and open problems.
Adding Variable Importance into CP-networks
Relative Importance
If it is more important to me that the value of X be high than the value of Y be high, then X is more important than Y.
Processor type is more important to me than operating system (all else being equal).
Operating system is more important than processor type (all else being equal), if the PC is used primarily for
graphical applications.If, given zDom(Z), it is more important to me that the value of X be high than the value of Y be high, then X is conditionally more important than Y.
X Y
X Yz
A
C
B
b b
( ) ( ) :
( ) ( ) :
a b a b c c
a b a b c c
a a
abc
abc
abc
abc
abc
abc
abc
abc
worst
best
A
B
C D
E
:
:
b c c
b c c
e e
a a
:
:
b d d
b d d
:
:
a b b
a b b
:
:
:
be C D
be D C
be D C
nodes variablescp-arcs (directed)
i-arcs (directed)
ci-arcs (undirected)cp-tables
ci-tables
( , ) { , }C D B ES
The same “forward sweep” algorithm works as is for TCP-nets!
Outcome Optimization in TCP-nets
1. The relative importance relations do not play a role in this case.
2. The network is traversed according to a topological ordering induced by the CP-net part of the given TCP-net.
– Any acyclic CP-net induce a partial order over the variables, according to which the variables should be processed.
– Any conditionally acyclic TCP-net induce a hierarchical set of partial orders over the variables.
The central difference between CP-nets and TCP-nets with respect to
constraint-based outcome optimization
Extension to conditionally acyclic TCP-nets
• If any non-dominated solution is enough, then the CP-net based COP is not harder than the corresponding CSP.
• “As is”, the Branch & Bound algorithm for CP-nets, executed on a conditionally acyclic TCP-net, loses its anytime behavior.
• We extended this algorithm to process TCP-net based constraint outcome optimization in an anytime fashion.
Introducing Utilities into CP-networks
Utility Functions
• Utility functions assign a real number to each outcome: u(o1)=0.4, u(o2)=4.3, …
• Actual u-values are significant:– U-values can be used to rank outcomes:
u(o1) < u(o2) implies that o1 is preferred to o2
– But much more! we can assess any uncertain combination of outcomes: the utility of an action yielding with probability 0.4 and with probability 0.6 is 0.4*u(o1) + 0.6*u(o1)
Dominance queries are easy
Optimiziation may be difficult
Generalized Additive Independ.
• The (possibly overlapping) sets X1,...,Xk of variables
are generalized additive independent (GAI) if:
– for any distributions P1, P2, with identical marginals over
the Xi, the expected utility w.r.t. U is the same for P1 and P2
• X1,...,Xk are GAI iff U can be written as
U(v) = i fi(xi )
for some component utility factors fi
UCP-nets – CP nets + Utilities• Utilities permit fine value tradeoffs and effective dominance tests• CP-nets allows effective outcome optimization
• UCP-nets use the graphical structure of a CP-net, but quantify conditional utilities
• U(x) is sum of utility factors; U(abcd) = fA(a)+fB(b)+fC(abc)+fD(cd) =
5 + 4 + .2 + .9 = 10.1
• Utility computation is linear
A B
C
D
a > a5 2
b > b4 3
a b .6 .1a b .2 .8a b .3 .8a b .9 .3
c .9 .8c .2 .3
c c
d d
The CPI-Restriction• So, far, directionality is not exploited
– This is just a GAI decomposition with redundant factors (no need for factors fA or fB; can be incorporated into fC)
• We also require:
DAG G is a valid CP-network for the preference order induced by U
• UCP-net is a specific form of GAI decomposition– Not all GAI-decompositions correspond to a UCP-net
topology
– Not all utility functions can be captured by UCP-nets: the CPI-conditions may not be satisfied
Testing CP-Ind. Condition• CP-Ind conditions implies that not all quantifications lead to
legitimate UCP-nets• A local test exists to verify CPI conditions
• X dominates its children if for all x1,x2 s.t. fX(x1,u) fX(x2,u),u, z, y :
fX(x1,u) - fX(x2,u) i fYi(yix1uizi) - fYi(yix2uizi)
• G is a UCP-net iff each variable dominates its children
• Domination is a local test
• Simpler sufficient conditions exist
X
Y1 Y2
U U2U1
Z1 Z2
Tradeoff Weight Representation
• Let’s normalize utilities to the [0,1] range
• For each X with parents U and factor fX,
– normalize each row of conditional “utility function” fX(—,u) to obtain local value function vu(X)
– for each u Dom(U), specify mulitplicative tradeoff
weight u and additive tradeoff weight u
– utility factor fX(x,u) = u vu(x) + u
• Every UCP-net has a normalized equivalent
Overview
• CP-net model for qualitative preferences.• Preference Optimization.• Complexity analysis of outcome comparisons.• Consistency testing.• Various Enhancements.
• Potential applications.• Future research directions and open problems.
Preference-based Preference-based ConfigurationConfiguration
of Web Page Contentof Web Page Content
Our Goal
Model for presenting the content of a Model for presenting the content of a web-page that:web-page that:
• Reflects the preferences of the web-page designer.
• Adjusts dynamically to the viewer’s current interests.
State of the Art – Categorization
Web page
Component: ArticleArticlePresentation alternatives:
1. Full content2. Partial content3. Heading only4. Link only5. Invisible6. …
Today:Today: Any web page is a collection of components, each one with its static, static, predefined appearance.predefined appearance.
Preference-based Configuration
• Multi-valued components of a web-page create a configuration spaceconfiguration space .
• Preference rankingPreference ranking is a total preorder over : means that configuration is subjectively
equally or more preferred to .
C
C
1 2c c 1c2c
The preference order represents the The preference order represents the subjective preferences of the web page subjective preferences of the web page
designer, not of its viewer.designer, not of its viewer.
However, the actual choices of the viewer affect the choice of configuration.
Why qualitative decision making?
• Utility assessments unlikely to be intuitive in our setting.
• Ordering different options for each component is likely to be a relatively easy task.
Looking for a Model - Objectives
• Explicit ranking of all options – infeasible.
• Desiderata:– Intuitive.– Relatively efficient reasoning.– Capture conditional preference dependencies.
A
B
C D
v ha a
:
:h
v
h
h
v
v
a
a b
b b
b
:
:v v h
h h v
b d d
b d d
:
:v h v
h v h
b c c
b c c
Each component of a web page can be Each component of a web page can be represented by a variable in the CP-represented by a variable in the CP-
network.network.
Authoring ToolHTML
DHTML
Preference specification
Web Browser
1. Interaction1. Interaction
3. Presentation3. Presentation
2. Preference-2. Preference-based based
reconfiguration of reconfiguration of the web-page.the web-page.
Plain HTML (DIV tags)
Parsed HTML
CP-network definition
Preference definition
(1 - present, 0 - hide)
Authoring ToolHTML
DHTML
Preference specification
Web Browser
1. Interaction1. Interaction
3. Presentation3. Presentation
2. Preference-2. Preference-based based
reconfiguration of reconfiguration of the web-page.the web-page.
- CP-network of the web page's components.
- Queue of the recent events (component + value).
Let be the recent event: an interaction of the viewer with
component (setting on/i
i
e
C
Reconfiguration
G
E
off). Then
1. Switch the value of and add into .
If is bigger than some threshold - remove the oldest event from .
2. Project on (specify values of recently observed components wrt. ).
i iC e E
E E
E G E
3. Traverse in a topological order and set each unspecified component
to its most preferred value wrt. the values of its predecessors in .
G
G.
Example• Elections - Presenting is unconditionally preferred to hiding.
• Traffic Accident – Hiding is unconditionally preferred to presenting.
• New Airbag – Presenting is preferred only if Traffic Accident is presented, and Elections is hidden.
• NBA – Presenting is preferred only if Traffic Accident is not presented.
• NY Times comm. – Presenting is preferred only if both Elections and Traffic Accident are presented.
• Volvo comm. – Presenting is preferred only if either New Airbag or Traffic Accident is presented.
• Nike comm. – Presenting is preferred only if NBA is presented.
TrafficAccident
Elections
New Airbag NBA
Volvo Nike
NY Times
Corresponding CP-network
TrafficAccident
Elections
New Airbag NBA
Volvo Nike
NY Times
Initially optimal presentation
TrafficAccident
Elections
New Airbag NBA
Volvo Nike
NY Times
Viewer opened “Traffic Accident”
TrafficAccident
Elections
New Airbag NBA
Volvo Nike
NY Times
Viewer closed “Elections”
TrafficAccident
Elections
New Airbag NBA
Volvo Nike
NY Times
TrafficAccident
Elections
New Airbag NBA
Volvo Nike
NY Times
What if the event queue is bigger?
TrafficAccident
Elections
New Airbag NBA
Volvo Nike
NY Times
TrafficAccident
Elections
New Airbag NBA
Volvo Nike
NY Times
Integrating Global Constraints
What if the constraints and the preferences are not defined over the same set of variables?
No obvious way to utilize preferences for search space pruning …
• Example - User Interface Configuration– Preferences are defined over the content
alternatives for the different UI components.– Geometric layout constraints.
Integrating Layout Constraints
• Layout constraints are represented as an integer linear program.
• Reconfiguration is adopted to look for a feasible, Pareto-optimal presentation.
Generally, this task is computationally hard.
Highly Constrained Problems
Reasonable approach:
• Forget about the CP-net, and generate all the solutions for the CSP.
• Using the CP-net, filter all the dominated solutions.
Generally, filtering may be painful ...
Given an outcome a, it is easy to determine the complete set of outcomes {a1 , …, am } that are (preferentially) minimally worse than a .
Preferentially optimal outcome
Underconstrained Problems
Potential Applications
• Multi-agent meeting scheduler– CP-nets support efficient computation of Pareto-
optimal solutions
• Filtering the results of search– Context can be use to disambiguate words, focus on the
more important terms
• Scheduling problems– Various constraints are more important than others
Overview
• CP-net model for qualitative preferences.• Preference Optimization.• Complexity analysis of outcome comparisons.• Consistency testing.• Various Enhancements.
• Potential applications.• Future research directions and open problems.
• Possible connections with non-monotonic reasoning.
• Model acquisition from a restricted natural language text.– Possibility for an intuitive closed loop system …
• Analyzing domains in which qualitative modeling of preferences seem to be required.– Product configuration (!)– Computational Biology (?)
Additional directions