Project ‘aire’ and Semantic Networks Seminar ‘Multimodal Rooms’ Jochen von Spiczak.

Project ‘aire’and Semantic Networks

Seminar ‘Multimodal Rooms’

Jochen von Spiczak

/ 40

content:

overview

- project ‘aire’· overview / interesting examples

- semantic networks· introduction

· knowledge representation in IEs

· description of SNs,especially in intelligent rooms

· realization of SNs

· conclusion / future work

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction

- knowledge representation in IE

- SN in general

- SN in intelligent rooms

- realization of SN

- example of a SN

- conclusion /future work

summary

/ 40

content:

project ‘aire’

- ‘aire’ ~· Agent-based Intelligent Reactive

Environments

- MIT Artificial Intelligence Lab

- examining perceptual computing systems and applications

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

overview

- tools for collaboration support:· supporting interactions among people,

spaces and mobile devices

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

project ‘aire’project ‘aire’

coll. supportcoll. support

/ 40

content:

overview

- software infrastructure· communication

· resource management

· device control

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary


coll. supportcoll. support infrastructureinfrastructure

/ 40

content:

overview

- human-computer interfaces· new interfaces for people to interact

with their environments

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary


coll. supportcoll. support infrastructureinfrastructure HCIHCI

/ 40

content:

overview

- ‘aire’ spaces· large range of form factors and sizes

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary



conference rooms,personal workspaces,transitional spaces

conference rooms,personal workspaces,transitional spaces

/ 40

content:

examplesoverview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary



conference room,personal workspace,transitional spaces

conference room,personal workspace,transitional spaces

/ 40

content:

examplesoverview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary



- ‘Metaglue’· Java-based agent infrastructure

· providing communication and levels of abstraction

· base of the rest of the project

- ‘Metaglue’· Java-based agent infrastructure

· providing communication and levels of abstraction

· base of the rest of the project

/ 40

content:

examplesoverview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary



/ 40

content:

examples

- visit this very day:

http://www.ai.mit.edu/projects/aire

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary



/ 40

content:

semantic networks

- one outstanding aspect of the ‘aire’ project

- knowledge representation as a fundamental design question· semantic networks as such a

representation

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

introduction

- ever-increasing amounts of data about users· who they communicate with,

· their location,

· visited web pages, ...

- aim: · quick, correct assumptions about what

the user would like to do next

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

knowledge representation in IE

- five different roles that KRs play· surrogates for the real world

· ontological commitments

· theories of reasoning

· media for computation

· media for human expression

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

· » KR imposes a filter on what the

environment can see and act on

» example of camera-/microphone-signal

· » easily/less easily identifiable entities

» important for reasoning about items and making deductions based on their layout

· » KR defines how inferences are made by the

reasoning system

» constraints kinds of inferences which are allowed or recommended by the system

· » KR must be able to operate in an efficient

manner via computational processes

» without taking up inordinate quantities of time, memory, or disk space

· » KR represents the human view of the world

» must be simple to use

» turn into human-readable representations

/ 40

content:

knowledge representation in IE

- much literature about developing contextual systems for IE

- problems of other approaches· interpretation and inferencing is done

by the application

· application must collate and assemble disparate pieces of data

· no cohesive representation of the gained knowledge

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

SN in general

- need for a representation that…· can store information on objects

· can easily track and follow the relationships between them

- introducing semantic networks as one such representation

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

SN in general

- concept of semantic networks· information represented in a directed

graph:» nodes objects

» links relationships

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

SN in general

- variety of node and link types· nodes just like…

» people,

» meetings,

» documents,

» issues,

» discourse items,

» commitments,…

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

SN in general

- variety of node and link types· links just like…

» relationship between people,

» responsibilities for objects (groups/people),

» hierarchies between contributors,

» attendance to meetings,

» interest for objects, …

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

SN in general

- further link types:· endpoints for links not limited to

storage nodes

· can also be created to other links

- allows more complex interactions in the data model

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

SN in general

- easy to discover…· information about a particular object

· relationships between different objects

- little information stored in nodes· network relies more on the links to

define the relationships

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

SN in general

- some inherent advantages· adding data is highly straightforward

· changing data is highly localized

· making inferences is fast and easy

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

SN in general

- further advantages· great flexibility in terms of the data

which is captured and interpreted

· extremely efficient

· easy to bring into human-readable form

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

SN in intelligent rooms

- several areas for using a SN-based representation· user knowledge

· meeting management

· location management

- SN as an appropriate and valuable piece of infrastructure

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:


- user knowledge· knowledge of the users and

the individual spaces they work with

· simplistic level» simple set of objects, spaces and

the user’s current location

» queries like “Where is Steve located?”

· system as an “intelligent assistant”» far more information and relations must be

included

» more complex queries like“Who is Joe’s superior?”

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:


- meeting management· capturing meetings as the occur

· typical information» agenda topics,

» action items,

» supporting and dissenting arguments,

» documents such as presentations orweb references, …

· can be linked to a video or audio capture of the meeting

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:


- location management· location information important for IEs

· especially true when integratinghand-held devices

» require knowledge of the current location

» own coordinate framework of each system

· SN as a location infrastructure» integration and translation between

different coordinate systems

» specify own location descriptions,such as “at home”

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

realization of SNs

- technology of SNs deployed within the project ‘aire’

- current implementation· works on top of a SQL database

· easily extendable to use…» object-oriented databases

» resource description framework (RDF)back-ends

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

realization of SNs

- access to the semantic network· using the Metaglue agent

infrastructure of project ‘aire’

· several agents used to access the SN» MeetingModel agent

» GUI agent

» other viewing/editing agents

· network query language

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

realization of SNs

- semantic network itself· provides only basic interface methods

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

semantic networksemantic network

/ 40

content:

realization of SNs

- MeetingModel agent· serves merely as an interface to SN

· abstracts out some of the lookup tasks

· broadcast of any changes

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary


MeetingModelMeetingModel

/ 40

content:

realization of SNs

- network query language· allows user or agents to specify

queries in a formal language

· definition of rules in a syntax similar to Prolog

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary


MeetingModelMeetingModel query languagequery language

/ 40

content:

realization of SNs

- GUI agent· tree-structure view of stored

information

· creation/editing of information nodes

· currently the primary conduit for information to get into the SN

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary


GUI agentGUI agent


/ 40

content:

realization of SNsoverview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

/ 40

content:

realization of SNs

- other viewing/editing agents· providing different views of the data:

» web interfaces,

» graph networks,

» specialized meeting views which simply present the agenda or commitments, …

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary


GUI agentGUI agent


……

/ 40

content:

example of a SNoverview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

Stiefelhagen

McDonough

Rogina

Jochen

rest

face tracking

expert-in

speech recognition

expert-in

Mercedes SLK

expert-in

multimodal rooms

responsible-for

responsible-for

responsible-for

assisted

seminar talk

currently-doingbores-to-death

belongs-to

room 225

works-in

/ 40

content:

example of a SNoverview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

Stiefelhagen

Jochen

rest

face tracking

expert-in

Mercedes SLK

responsible-for

McDonough

speech recognition

expert-in

multimodal rooms

responsible-for

responsible-for

assisted

seminar talk

currently-doing

belongs-to

Rogina

room 225

works-in

McDonough

speech recognition

expert-in

multimodal rooms

responsible-for

Rogina

room 225

works-in

- “John McDonough.”- “In room 225.”

expert-in

bores-to-death

/ 40

content:

example of a SN

Stiefelhagen

Jochen

rest

face tracking

expert-in

Mercedes SLK

responsible-for

McDonough

speech recognition

expert-in

multimodal rooms

responsible-for

responsible-for

assisted

seminar talk

currently-doing

belongs-to

Rogina

room 225

works-in

expert-in

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

bores-to-death

/ 40

content:

conclusion / future work

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

- seems to be the most appropriate knowledge representation for IEs· flexibility of data

· adding/changing data highly localized

· inference generation extremely efficient

- future work· integration within other projects

· automatic sensor-based information recognizing

· incorporate user preference information

/ 40

content:

summaryoverview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

- project ‘aire’

- semantic networks· one approach to KR

· inherent advantages in general…

· … and in the context of IEs

· realization» implementation

» access hierarchy

· example

· most appropriate approach to KR

summary

- project ‘aire’

- semantic networks· one approach to KR

· inherent advantages in general…

· … and in the context of IEs

· realization» implementation

» access hierarchy

· example

· most appropriate approach to KRslide 40 / 40

content:

overview

project ‘aire’

- overview

- examples

semantic networks

- introduction


- SN in general


- realization of SN

- example of a SN


summary

discussion

…?

Project ‘aire’ and Semantic Networks Seminar ‘Multimodal Rooms’ Jochen von Spiczak.

Documents

Transcript of Project ‘aire’ and Semantic Networks Seminar ‘Multimodal Rooms’ Jochen von Spiczak.