From Events to Situations: An

Event-web perspectiveVivek Singh

Advisor: Professor Ramesh Jain

University of California, Irvine

Event-web

• Connecting real users, events and places rather than just documents.

• Events and objects as basic organization and linking mechanism

▫ Multimodal

▫ Closer to real world

• Users gain insights and experiences

• IBM Smarter planet

Event-web (imminent signs)

• Image and video sites for sharing experiential data related to events

• Tweets about events of interest

• Multimodal news broadcast of events

• Detection of events in surveillance videos

Motivation: From events to situations…

• Given a plethora of event data. How can we:

▫ Disambiguate relevant and irrelevant events?

▫ Combine events into meaningful representations ?

▫ Allow inference and cascading effects

▫ Support different interpretations based on application domain

▫ Support Control & decision making

Situation based control: Motivations

1. Inherent support for event-based (temporal) reasoning

2. The ability of the controller to reason based on symbols (rather than just signals)

3. Explicit inclusion of domain semantics (to support multiple applications)

Applications

• Energy efficient buildings:

▫ When to switch off air-conditioner?

• Telepresence:

▫ Which camera feed to send out?

• Business analysis:

▫ What should be the correct price for iPhone?

• Earthquake rescue effort:

▫ Where to send out the next fire-fighter engine?

E2E communication: Project Overview

Sentient Information

System

Sentient Information

System

Environment 1 Environment 2

Web

Device to Device communication

Towards Environment to Environment (E2E) multimedia communication systems, in Multimedia Tools and Applications Journal, Springer Netherlands, 2009.

Also in: ACM Workshop on Semantic Ambient Media Experiences (SAME), ACM Multimedia workshop, 2008.

Network/Transmis

sion

Environment Model

Environment Server

Situation based

controller

Actuator / Presentation

Model

MMDB

Sensors

Actuators / Presentation

Devices

Physical Environment

EventBase

Environment: Node Architecture

Situation Calculus: Quick overview▫ enter(P1), startWork(P1)

▫ enter(P1), exit(P1), enter(P1), startWork(P1), stopWork(P1), startWork(P1)

- isInRoom(P1, s(k))- isWorking(P1, s(k))

isInRoom(P1, s) ˄~isWorking(P1, s) →

IncreaseMusicVolume()

isInRoom(P1, s) 0

isWorking(P1, s) 01

1

Situation = Not events , nor sequence of events, but their assimilated descriptor

Situation calculus

• Ω = {Actions, Situations, Objects, Fluents}

• Situation:

▫ “The set of necessary and sufficient world state descriptors for undertaking control decision”.

• D = Dfnd U Duna U ε U Dap U Dss U D0

▫ Precondition axioms

▫ Successor-state axioms

▫ Initial situation

• Do(action, situation): A X S → S

Control theoretic problem formulation

•

•

•

•

•

•

Loc 1: Desk Loc2: Whiteboard

Loc 3: EngineeringModel

Actions possible:

1. Work on PC

2. Work on Table

Conditions Actions

Move to location

Activity SelectedCam

Desired Volume

Desk WorkOnPC

1 1

Desk WorkOnTable

2 2

Whiteboard

- 3 3

Model - 4 4User

Situation modeling: E2E application

Situation based control for cyber physical environments, Accepted: IEEE workshop on situation management, MILCOM, 2009

Situ-itter: Large scale situations on

Twitter

• Looking beyond a room:

▫ Can an entire city or country be considered a cyber physical system.

• Humans as sensors:

▫ Everywhere !

▫ Perception, Censors, Rumors, Delays

• Data has salient features: Unstructured, Noisy, Humungous, Spatial semantics

• Event detection is not well studied!

Situ-itter: First steps

• Spatio-temporal visualization for insights

• Spatio-temporal analysis for event detection

• Combining with external sources of information for decision making

• Applications

▫ Event detection

▫ Should iPhone price be increased/decreased?

▫ Where and when to launch an ATT roadshow?

Comparison with external dataAggregate interest on iPhone, Current ATT store location data

Where to have an ATT roadshow?

(using spatial-temporal convolution)

<geoname><name>Sandy Big Bend Reservoir Number 1</name><lat>42.5191149</lat><lng>-109.4681887</lng><geonameId>5837570</geonameId><countryCode>US</countryCode><countryName>United States</countryName><fcl>H</fcl><fcode>RSV</fcode><fclName>stream, lake, ...</fclName><fcodeName>reservoir(s)</fcodeName><population>120,178<population/><alternateNames/><elevation>2194</elevation><continentCode>NA</continentCode><adminCode1>WY</adminCode1><adminName1>Wyoming</adminName1><adminCode2>035</adminCode2><adminName2>Sublette County</adminName2><timezone dstOffset="-6.0" gmtOffset="-7.0">America/Denver</timezone><distance>3.3639</distance></geoname>

Location has semantics

Future directions

• Tip of the iceberg:

▫ Spatio-temporal event detection in social media

• Reasoning/inference mechanisms

• Combining spatial, temporal and social semantics into decision making

• Considering multi-modal data, user and sensor based data

• A cyber-physical event-web which connects real users and environments