Temporal Probabilistic Agents J. Dix, S. Kraus, VS Subrahmanian Clausthal (DE)/ Bar-Ilan(IL)/...

Temporal Probabilistic Agents

J. Dix, S. Kraus, VS

Subrahmanian

Clausthal (DE)/ Bar-Ilan(IL)/ Maryland

(USA)

1

2Prof. Dr. Jürgen Dix – DESIR seminaire – 16.03.2007

Motivation

Agents must be able to leverage existing

code rather than reinvent everything

from scratch.

Agents must be able to reason about

State

Time

Uncertainty in the state


Applications

Built 4 applications that motivated the need for this theory. 2 examples: US Navy: Finding where and when an enemy submarine will be in

the future, and determining which of many friendly assets will try to attack it. Joint with BBN, Lockheed, NRL and many other partners.

- Terrain DB- Sensor DB- Predictive Models- Route Planner

US Army: Finding where and when an enemy tank will be in the future, and determining which of many coalition assets will try to attack it, where, and when. Joint with BAE.

- Terrain Mobility Map- Sensor DB- Optimization code- Route/flight Planner


Overview

Part I: A Bird’s Eye View on IMPACT

Part II: Adding time

Part III: Adding uncertainty

Part IV: Adding time and uncertainty


I.1 Motivations

Agents are for everyone! Agentize arbitrary Legacy Code.

Knowledge is distributed and heterogenous. Code-Call machinery

Agents act wrt a clearly articulated semantics. Agent Program


Code Call: d:f(arg1,…,argk).

Code Call Atom: in(X,d:f(arg1,…,argk)).

Execute function f defined in agent d on the specified arguments. Returns a set of objects.

I.2 Data Access

Succeeds if X is in the set of objects returned

–in(X, stock:portfolio( ‘dix’)). Find all stocks in dix’ portfolio.–in(X, pred:dest( ‘ibm’)). Find all pairs (T,C) s.t. the price of ibm will change by C% within the next T units of time.

–stock:portfolio( ‘person’).–pred:dest(‘stock’)


I.3 Data Access: Code Call Conditions

A conjunction of code call atoms and comparison atoms.

in(X, pred:dest( S )) & X.percent > 30 &

X.time < 2 weeks & in(S,stock:portfolio( dix)).

The above says: find X,S such that: S is a stock in the portfolio of dix, X is a prediction that the stock S is going up

by at least 30% within the next two weeks.


I.4 What is an Agent?

Agents are built on top of legacy or specialized software modules

or systems.

They access data using API functions.

Software moduleImplementation

Interface/API



msgbox

Agent state (distributed)

Implementation

Interface/API

There is a message box with suitable functionality. Code manipulates data structures, the contents of which

(including msgbox) describe the agent’s state.



Actions

msgbox


Implementation

Interface/API

Agents have available actions to change the state. Actions are declaratively described through add/delete

lists containing code call conditions. They are implemented through software code.



Agent ProgramActions

msgbox


Implementation

Interface/API

Agents have an “agent program” encoding the agent’s operating principles

The program consists of rules that are declarative and can be easily written or modified.



Integrity ConstraintsAction Constraints


msgbox


Implementation

Interface/API

Integrity Constraints (to preserve properties of the state) Action constraints (to ensure actions do not interfere)




msgbox


Implementation

Interface/API

Based on semantics and current state, actions are executed.

These actions change its own state and other agents

states as well.

ActionsIntegrity ConstraintsAction Constraints

Semantics


I.5 Agent Program (1)Set of rules of the form

Op a(arg1,…,argn)<--- <code call condition> &

Op1 a1(<args>) & … & Opan(<args>)

Op is a “deontic modality” and is either P - permitted

O - obligatory

Do - do

F – forbidden

If code call condition is true and the deontic

modalities in the rule body are true, then

Op a(arg1,…,argn) is true.


I.5 Agent Program (2)

Most important part of

the agent.

Agent program rules must

be carefully crafted to avoid

inconsistencies.

IMPACT provides facilities

to create such rules.

A class of programs is

defined that can be

efficiently implemented:

Regular Agents.

EXAMPLE:

Dobuy(S) in(X, pred:dest( S ))

& X.percent > 30

& X.time < 2 weeks

Fbuy(S) in(S,

stock:portfolio(dix)).

&

Dodiversify_portfolio(dix)


II Adding Time

Basic framework does not allow to express time.

If a prediction package expects a stock to rise K%

after T units of time and K>25, then buy the stock

some units between (tnow + T –5) and (tnow + T –2).

Do buy(S): [(tnow + T –5),(tnow + T –2)] <--- in( ...): [tnow,tnow]

Semantics: Time is 1, 2, 3, ...

We introduce temporal annotations [t1,t2]. Semantics

needs to be defined for all timepoints.

Semantics has to reflect the obligations and

committments (into the future) made in the program.

(see Dix/Kraus/Subr.: Temporal Agent Programs, AIJ 2001)


III TP Adding Probabilities Basic framework does not allow to express

uncertainty.

Requires changes in the code call mechanism:

Code calls return random variables. Status atoms are true only with a certain probability.

This is modelled by probabilistic annotations.

Evaluating code call conditions is difficult: What is

the probability of a conjunction of events?

(see Dix/Nanni/Subr.: Probabilistic Agents, TOCL 2000)


Most commercial prediction packages

compute (based on observations of the

market) probability intervals linked with

time intervals:

If a stock is expected to go over $50 per

share (with 80% probability) in the next

two weeks, then we should buy it. If the

probability is less than 40%, we should

not.

IV Adding both Time and Probabilities


Define annotations carrying both temporal

and probabilistic information:

[strategy,tc,l,l‘,pdf]

strategy: determines probability of conjunctive events

tc: temporal constraint tnow+5<t< tnow

l,l‘ : probabilistic items valued in [0,1], e.g X,

(X+1)/2 where X is a RV with values in [0,1]

pdf: probability distribution over solutions of tc

Dobuy(ibm):[strat, tnow +5<t< tnow , 0.5,0.7,uniform]

IV. 1 TP Agents: Annotations


IV.2 TP Agent Program

Set of rules of the form

Op a(arg1,…,argn) [ ] <--- <code call condition>

[ ] & Op1 a1(<args>) [ ] & …&

Opan(<args>)[ ] where [ ] are temporal probabilistic annotations.

Formal definitions of what it means that a <code call condition> [ ] is true in a state.

SEMANTICS: Fixpoint operator associating a meaning to TP agent programs.


IV. 2 TP Agents (2)

Dobuy(X):[s, tnow +5<t< tnow, 1, 1,u]

in(X,stock:portfolio( dix) ): [s, tnow, 1, 1,u] &

in(yes, stock:over(50,X ) ): [s, tnow<t< tnow+14, 0.8,

1,u]

We defined formal semantics and algorithms

for computing feasible status sets of TP

programs. (This is quite involved due to action histories,

deontic/state consistency, machinery of fixpoint operators.)

Complexity: Existence of a feasible SS is NP-

complete.

Implementation within IMPACT is on its way.


Conclusions

Developed an agent programming language

that allows us to build agents that

Leverage commercial software without reinventing it;

Take actions in accordance with an agent program

and integrity constraints in the presence of time and

uncertainty about the state of the world

Developed a declarative semantics and the concept of

a feasible TPSI.

Developed fixpoint computation algorithm to find

feasible TPSis when they exist.

Temporal Probabilistic Agents J. Dix, S. Kraus, VS Subrahmanian Clausthal (DE)/ Bar-Ilan(IL)/...

Documents

Transcript of Temporal Probabilistic Agents J. Dix, S. Kraus, VS Subrahmanian Clausthal (DE)/ Bar-Ilan(IL)/...