Predictive and Contextual Feature Separation for Bayesian Metanetworks

Vagan TerziyanVagan Terziyan

vagan@cc.jyu.fi

Industrial Ontologies Group, University of Jyväskylä, Finland

KES-2007, Vietri sul Mare , Italy

12 September 2007

Contextual level

Predictive level

Session IS03: Context-Aware Adaptable Systems and TheirApplications (17:10, room D)

2

Contents

Bayesian Metanetworks Metanetworks for

managing conditional dependencies

Metanetworks for managing feature relevance

Feature Separation for Bayesian Metanetworks

Conclusions

Vagan Terziyan

Industrial Ontologies Group

Department of Mathematical Information Technologies

University of Jyvaskyla (Finland)

http://www.cs.jyu.fi/ai/vagan

This presentation: http://www.cs.jyu.fi/ai/KES-2007.ppt

3

X

Y

P(X)

P(Y)

P(Y|X)

Fixed conditional probability table:

Conditional dependence between variables X and Y

P(Y) = X (P(X) · P(Y|X))

P(Y|X) y1 y2 …ym

x1 p(x1| y1) p(x1| y2) p(x1| ym)

x2 p(x2| y1) p(x2| y2) p(x2| ym)

… xn p(xn| y1) p(xn| y2) … p(xn| ym)

Random variable Y {y1, y2, …, ym}

Random variable X {x1, x2, …, xn}

4

Bayesian Metanetwork

Definition. The Bayesian Metanetwork is a set of Bayesian networks, which are put on each other in such a way that the elements (nodes or conditional dependencies) of every previous probabilistic network depend on the local probability distributions associated with the nodes of the next level network.

5

Two-level Bayesian C-Metanetwork for Managing Conditional Dependencies

Contextual level

Predictive level

11

Two-level Bayesian C-Metanetwork for managing conditional dependencies

Contextual level

Predictive level A

B

X

Y

P(B|A) P(Y|X)

12

Two-level Bayesian R-Metanetwork for Modelling Relevant Features’ Selection

Contextual level

Predictive level

13

Feature relevance modelling

We consider relevance as a probability of importance of the variable to the inference of target attribute in the given context. In such definition relevance inherits all properties of a probability.

X

Y

Probability

P(X)

P(Y)-?

P(Y|X)

Relevance

Ψ(X)

Y

P0(Y) Probability to have this model is:

P((X)=”no”)= 1-X

X

Y

P(X)

P(Y|X)

Probability to have this model is:

P((X)=”yes”)= X

P1(Y)

14

General Case of Managing Relevance

X1

Y

Probability

P(X1)

P(Y)-?

P(Y|X1,X2,…,XN)

Relevance

Ψ(X1)

XN

Probability

P(XN) Relevance

Ψ(XN) X2

Probability

P(X2) Relevance

Ψ(X2)

…

1 2 )"")(()"")((

1

])1()(),...2,1|([...1

)(X X XN noXqq

XqyesXrr

XrN

s

XrPnxrXNXXYPnxs

YP

Probability

P(XN)

15

Example of Relevance Bayesian Metanetwork

X

Y

P(X)

P(Y)-?

P(Y|X) P(Ψ(X)|Ψ(A))

A

P(A) Ψ(A) Ψ(X)

)]}.1()()|(

)([)|({1

)(

XAAX

X

A

PP

XPnxXYPnx

YP

Conditional relevance !!!

16

Example of Relevance Bayesian Metanetwork

X

Y

P(X)

P(Y)

P(Y|X)

Ψ(X|A)

A

B

P(B)

P(B|A)

P(A) Ψ(A) Ψ (X)

Contextual level

Predictive level

Y B

A X

Ψ(B)

Ψ(A)

Ψ(Y)

Ψ(X)

17

Separation of contextual and predictive attributes is based on:

Part_of context Role-based context Interface-based context

18

The nature of part_of context

Machine

Environment

Sensors

XX x1 x2 x3 x4 x5 x6 x7

predictive attributes contextual attributes

air pressure

dust

humidity

temperature

emission

19

Context Description Framework (CDF) Basic Data ModelContext Description Framework (CDF) Basic Data Model

Khriyenko O., Terziyan V., A Framework for Context-Sensitive Metadata Description, In: International Journal of Metadata, Semantics and Ontologies, Inderscience Publishers, ISSN 1744-2621, 2006, Vol. 1, No. 2, pp. 154-164.

20

Part-of Context in CDF

Resource k

Resource ipart_of

Value_rProperty_n

Value_mProperty_q

Value_sProperty_p

Resource_k Property_n Value_r

Resource_i Property_q Value_m

Resource_i Property_p Value_s

true_in_context

Context_h

RDF statement

RDF container

Predictive feature Contextual features

23

Multiple Context Inheritance …Multiple Context Inheritance …

John

Golf_Clubpart_of

48 y.has_age

Parislocated_in

36members_amount

Resource Predictive features

Contextual Features (inherited from both parents)

John age

environmentenvironment__11_location environmentenvironment__11_members amount

environmentenvironment__22_location environment_2environment_2_belongs_to

Symphonic_Orchestra

Bagnoletlocated_in

Statebelongs_to

part_of

24

Role-based contextRole-based context

The example of the proactive object (human resource), which is member of several organization and which is playing different roles in each of them. The context of this object should include the description of these roles (duties, commitments, responsibilities, etc).

25

Interface-based context

The example of the domain object (aircraft) is shown in different interfaces: (a) Google Maps; (b) pilots’ control panel; (c) manufacturing design e-manual. Each interface is considered as a context, which affect on which parameters of the aircraft are to be shown

a b

c

26

Summary We are considering a context as a set of contextual

attributes, which are not directly effect probability distribution of the target attributes, but they effect on a “relevance” of the predictive attributes towards target attributes.

Bayesian Metanetwork allows modelling such context-sensitive feature relevance. The model assumes that the relevance of predictive attributes in a Bayesian network might be a random attribute itself and it provides a tool to reason based not only on probabilities of predictive attributes but also on their relevancies.

For Bayesian Metanetwork there is a need to distinguish predictive and contextual attributes and in this paper the separation of attributes is described based on three notions of a context: part_of context, role-based context and interface-based context.

27

Read more about Bayesian Metanetworks in:

Terziyan V., A Bayesian Metanetwork, In: International Journal on Artificial Intelligence Tools, Vol. 14, No. 3, 2005, World Scientific, pp. 371-384.

http://www.cs.jyu.fi/ai/papers/KI-2003.pdf

Terziyan V., Vitko O., Bayesian Metanetwork for Modelling User Preferences in Mobile Environment, In: German Conference on Artificial Intelligence (KI-2003), LNAI, Vol. 2821, 2003, pp.370-384.

http://www.cs.jyu.fi/ai/papers/IJAIT-2005.pdf

Terziyan V., Vitko O., Learning Bayesian Metanetworks from Data with Multilevel Uncertainty, In: M. Bramer and V. Devedzic (eds.), Proceedings of the First International Conference on Artificial Intelligence and Innovations, Toulouse, France, August 22-27, 2004, Kluwer Academic Publishers, pp. 187-196 .

http://www.cs.jyu.fi/ai/papers/AIAI-2004.ps