SemTech 2011, Saltlux, Tony Lee

Geo-Social Semantics

and Hybrid Reasoning for

Smart Mobile ServicesSmart Mobile Services

SEMANTICTECHNOLOGY

6 June 2011 / SALTLUX, inc.

Tony LEE / [email protected]

Linked World

And SemanticsAnd Semantics

Communicating Knowledge 2

Scarecrow : I haven't got a brain... only straw.

Dorothy : How can you talk if you haven't got a brain?


Dorothy : How can you talk if you haven't got a brain?

Scarecrow : I don't know... But some people without brains do an awful lot of talking... don't they?

Dorothy : Yes, I guess you're right

Scarecrow : The sum of the square roots of any two sides of an isosceles triangle is equal to the square root of the remaining side. Oh joy! Rapture! I got a brain! How can I ever thank you enough?

Wizard of Oz: You can't.

Tony’s Brain and Knowledge

Neurons~100B #

~2x # ofWeb Pages


Synapse~100T #

~2# ofWeb Links

650ft (~200m)

650ft

1 : 10001 : 10001 : 10001 : 1000

KnowledgeNetwork

World Wide WebNetwork

InternetNetwork

PeopleNetwork

WordNetNetwork

Musical workNetwork

CommunicationNetwork

Linked DataNetwork

Data, Information and Knowledge

DATA

• symbol, a statement

• facts of the world

INFORMATION


By Gene Bellinger, Durval Castro and Anthony Mills

• collection of data, data in context

• answer about who, what, where, when

KNOWLEDGE

• contextualcontextual organization organization of information

• map of the world inside our brains

• answer about how and why

Knowledge Representations

Natural Language Human language written in letters: “The Earth orbits the sun in an ellipse”

Visual LanguageVisual expression of knowledge in picture, structure diagram, flow chart,and blueprint etc

TaggingKnowledge expressed in keywords, symbols and images related withobjects

Symbolic Language Knowledge expressed in mathematical symbols : x2/a2 + y2/b2 = 1

Decision Tree Tree-shaped graph structure for complex decision making

Combined expression in condition with various rules of human

Human


Rules LanguageCombined expression in condition with various rules of humanknowledge

Database SystemKnowledge expression system composed of objects and relations in atable format

Logical LanguageKnowledge expression of logical symbols and arithmetic operations:Woman ≡ Person ∩ Female

Frame LanguageKnowledge expression of values or pointers for other frames saved inslots

Semantic NetworkKnowledge expression of semantic relation between concepts in a graphstructure

Statistical KnowledgeAllows knowledge expression, machine learning technology combinationbased on probability and statistics

Machine

“Employees working for a company are humans; the company and the employees are legal

entities. The company is able to make a reservation for an employee’s trip. The trip is

available by plane or train that travels in cities within Korea or the U.S.. The companies

and destinations for business trip are located in the cities. Saltlux reserved OZ510 with a

round trip of Seoul and New York for Hong, Kildong.”

Natural Language



Rule Language

(Rule) If someone is flying, he must be on trip.

(Rule) If someone’s trip is reserved in a company, he is an employee of the company.

(+ Rule) For short trip in the same country, an employee should take a train.

(Deduction) Hong kil-dong whose flight is in reservation is an employee of Saltlux.

(Deduction) OZ510 is a flight for the U.S. and Korea.

Legal Entity

Person Company City

Location

Trip

kindOf

kindOf

endsIn

startFrom

books

Legal Entity

Person Company

subclssOf

instanceOf

Person Company

subclssOf

instanceOf

Legal Entity

Name

ID

Gender

Age

Industry

Address

Person Company

subclssOf

instanceOf

Legal Entity

Name (*)

ID (*)

Gender⊆{M,F}

Age > 25

Industry

Addr⊂Seoul

DISJOINT

Ontology



Semantic Network

Employee

Kildong

Saltlux

Airplane TrainKoreanCity

AmericanCity

New york

Seoul

OZ510

kindOf

instnaceOf

instanceOf

instanceOf

instanceOf

participatesIn

instanceOf

Employee

Kildong

Saltlux

subclssOf

instanceOf

instanceOf

Employee

#3502

#4831subclssOf

instanceOf

instanceOf

Position

Kildong

37Manager

P12345

Male

Saltlux

Seoul

C98765

Software

Employee

#3502

#4831

subclssOf

instanceOf

instanceOf

Pos ≠ Exec.

Kildong

37Manager

P12345

Male

Saltlux

Seoul

C98765

Software

(a) Semantic Network (b) (a) + Frame (Slots) (c) (b) + Logical Restrictions

Ontology

Five View Points for Semantic Technology

• URI/RDF based “Web of Data”

• Semantic annotations (RDFa)

• Ontology and Logics • Reasoning, Agent system


• Ontology and Logics

• OWL and RIF

• Reasoning, Agent system

• Personalized services

• Linked semantic data

• Data interoperability

• Semantic Search and Mining

• Recommendation and Discovery

Two Keywords, Today

1. HYBRID


2. MOBILE

Hybrid = Complementary


Strength + Strength

Weakness - Weakness

Hybrid Reasoning : (1) Mixed Method

Logical Reasoning Methods

• Deductive reasoningPremise 1: All humans are mortal. Premise 2: Socrates is a human. Conclusion: Socrates is mortal.

+

Ontology and Rules


Conclusion: Socrates is mortal.

• Inductive reasoningPremise: The sun has risen in the east every morning up until now. Conclusion: The sun will also rise in the east tomorrow.

• Abductive reasoning

• Analogical reasoning

+

Machine Learning

Hybrid Reasoning : (2) Mixed Formalism

+


The relationships among different formalisms(Benjamin Grosof)

Semantic Web Architecture

Hybrid Reasoning : (2) Mixed Formalism


Mobile Communication

Tele - Communication

Geo-Location Sociality


Geo-Semantics

GEO Data, GEO Information and GEO Knowledge

WHAT ISWHAT IS


www.ci.ferndale.wa.us/GIS/GIS.php www.ci.ferndale.wa.us/GIS/GIS.php www.ci.ferndale.wa.us/GIS/GIS.php www.ci.ferndale.wa.us/GIS/GIS.php

WHAT ISWHAT IS

GEOGEO--KNOWLEDGE ?KNOWLEDGE ?

An Evolution of Geo Ontology

Geo Tagging � GPS based POI processing� Connecting location coordinate with the relevant information

Geo Features� Applying classification system by domain� Referring to major geographic classification system


� Referring to major geographic classification systemsuch as GeoNames

Geo Ontology � Building/Applying ontology-based spatial information� Expressing Point/Line/Shape information

Geo Ontology + Rules � Utilizing Geo Ontology and rule-based inference� Applying deduction rules for intelligent spatial information processing

• GeoRSS

• Geo ontology

• Feature ontology

• Feature type ontology

• Spatial relationship ontology

Geospatial Ontologies


• Spatial relationship ontology

• Toponym ontology

• Coordinate reference/spatial

index ontology

• Geodata set/metadata ontology

• Spatial services ontology

• W3C Geospatial Ontology

Core Geographical Concepts: Case Finnish Geo-Ontology

Geo Ontology Examples


by R. Henriksson, 2008

http://www.seco.tkk.fi/publications/2008/henriksson-kauppinen-hyvonen-suo-2008.pdf

• Open data for Geo-

information

• TBL was involved in

• Supporting SPARQL

EndPoint

• Supporting RDF/XML,

Ordnance Survey : OpenData


http://www.ordnancesurvey.co.uk/oswebsite/opendata/http://www.ordnancesurvey.co.uk/oswebsite/opendata/

• Supporting RDF/XML,

Turtle, JSON

• Reference Ontologies- Spatial Relations Ontology- WGS84 Geo Positioning- Gazetteer Ontology- FOAF

Ordnance RDF Gazetteer


http://www.ordnancesurvey.co.uk/oswebsite/partnerships/research/pdf/RDFDescription.pdf

POI and Geo-data modeling by Saltlux

• 28# main category and 600 sub categories for POI classification

• Using SKOS for semantic classification

• Including named places and events

• All data set has its name space, http://www.saltlux.com/geospatial

• Coordination

- http://www.w3.org/2003/01/geo/wgs84_pos#lat

- http://www.w3.org/2003/01/geo/wgs84_pos#long

32

http://www.w3.org/2004/02/skos/core#ConceptScheme http://www.w3.org/2004/02/skos/core#Concept

http://www.saltlux.com/geospatial#Class http://www.saltlux.com/geospatial#Code

http://www.w3.org/2004/02/skos/core#hasTopConcept

http://www.w3.org/2004/02/skos/core#inScheme

http://www.w3.org/2004/02/skos/core#broaderhttp://www.w3.org/2004/02/skos/core#narrower

http://www.w3.org/2004/02/skos/core#broaderTransitivehttp://www.w3.org/2004/02/skos/core#narrowerTransitive

GEO Data ModelSKOS based Taxonomy

Communicating Knowledge

Taxonomy and Property modeling for POI

name

alternate name

description

street-adress

postal-code

categories

url

email

tel

latitude

longtitude

good for

products and services

specialities

brands

smoking

take-out

transit

wireless

reservations

best nights

alcohol

NO Main Classes

1 Arts & Entertainment

2 Adult Entertainment & Nightlife

3 Sports & Recreation

4 Media & Broadcasting

5 Religious Organizations

6 Transportation

7 Automotive

8 Education & Learning

9 Event Planning & Services

10 Manufacturing & Industry General

Arts & Entertainment Adult Entertainment & Nightlife Sports & Recreation

Arcades Adult Arcades, Casinos Archery

Art Galleries Adult Massage Badminton

Botanical Gardens, Arboretum Adult Novelties & Product Shop Baseball

Cinema Bars & Pubs Basketball

Music Concert Hall Business Clubs Billiards

Open air theatres, Festival Places Audult Comedy Clubs Boating

Theatres Dance Clubs Bowling

Museums Jazz & Blues Clubs Boxing

Astrologers & Psychics Audalt Karaoke Cricket

Social & Interests Clubs Hourse Racing Curling

Talent Agencies & Entertainers Boat Racing Cycling

Party Rentals Bycycle Racing Dance

Ticket Office Car Racing Equestrian

Aquariums Billiard Halls Fencing

Video/DVD, Game Rental Other Adult Entertainments Fishing

Comedy Clubs Fitness Clubs

Circus American Football

Amusement Parks Golf

Zoos Gun/Rifle Ranges

Cartoon Rental Gymnastics

Internet Café Hockey

Karaoke Horse Racing & Equestrian

lottery shop Hunting

Exhibition, show Lacrosse

Folk Village Taekwondo

Sub Classes

…

33

longtitude

device_lat

device_long

hours

fax

payment options

parking

ambiance

amentities

attire

price range

delivery

alcohol

reviews

photo

Year Established

seating

outdoor seating

chef

self service

languages spoken

music

associations

part_of

Communicating Knowledge

11 Financial & Legal Services

12 Health and Medical

13 Beauty and Spas

14 Other Professional Services

15 Travel & Tours

16 Home & Local Services

17 Shopping

18 Government & Public Services

19 Food & Drink

20 Restaurants

21 Other Artifacts

22 Other Natural Objects

23 Utility & Infrastructure

Folk Village Taekwondo

Other Arts Judo

Other Entertainment Kendo

Martial Arts

Motorsports & Racing

Paintball

Parachuting

Racquetball

Rafting/Kayaking

Ringette

Rugby

Running

Scuba

Skateboarding, inline skating

Ice Skating

Skiing

Skydiving

Soccer

Softball

Squash

Surfing

Swimming

Tennis

Track & Field

Volleyball

Wrestling

Aerobic

Yoga

Bungee Jump

Camping

Arenas & Stadiums

Playground

Other Sports

Taxonomy Modeling Property Modeling

Find good restaurants for dating that serves steaks and free parking near by GAGA gallery in Insa-dong.

PREFIX ns: <http://www.saltlux.com/geospatial#>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX dc: <http://purl.org/dc/elements/1.1/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> PREFIX wgs: <http://www.w3.org/2003/01/geo/wgs84_pos#>PREFIX f: <http://www.saltlux.com/geo/functions#>SELECT * WHERE {

?res rdf:type ns:NamedPlace;

Semantic queries for geo-data


?res rdf:type ns:NamedPlace;ns:name ?name; wgs:lat ?lat1; wgs:long ?long1;ns:address ?addr1.

OPTIONAL{?res ns:street-address ?straddr1.} ?rel rdf:type ns:NamedPlace;

ns:name ?relname; wgs:lat ?lat2; wgs:long ?long2; ns:address ?addr.

OPTIONAL{?rel ns:street-address ?straddr.} ?rel ns:category ?cate;ns:ambiance ns:ambiance_129;ns:parking ns:parking_9.

?cate rdfs:label ?catename. FILTER (f:distance(?lat1, ?long1, ?lat2, ?long2) <= 300 && ?cate = <http://www.saltlux.com/geospatial#code_655> && ?name = '가가갤러리') } ORDER BY ?relname

Demonstration


MOBILE APIs

• Supporting Mobile APIs by using SPARQL Endpoint

• Working on Android (and iPhone)

function description

findPOIbyAll(java.lang.String name, java.lang.String id, int dist, java.lang.String format)

� 4 argument(name, id, distance, format)

� format: xml, json, id: Category ID

findPOIbyAllCoordinate(java.lang.String id, int dist, double lat, double lon, java.lang.String format)

� 5 argument(id, distance, lat, long, format)

� id: Category ID, format: xml, json

findPOIbyCoordinate(java.lang.String id, int dist, double lat, double lon)

� 4 argument(id, distance, lat, long)

� id: Category ID, format: xml

findPOIbyDist(java.lang.String name, java.lang.String id, int dist) 3 argument(name, id, distance)


findPOIbyDist(java.lang.String name, java.lang.String id, int dist) � 3 argument(name, id, distance)

� name: name of PIO, id: Category ID

findPOIbyName(int dist, java.lang.String name) � 3 argument(distance, name)

� format: xml

findPOIbyFormat(int dist, java.lang.String name, java.lang.String format)

� 3 argument(distance, name, format)

� format: xml, json

findPOIbyID(java.lang.String id) � 1 argument(id)

� id: 특정 상점이 갖는 URI, return: meta data of POI, xml

geoQuery(java.lang.String query, java.lang.String format) � 2 argument(query, format)


� query for getting MAP data

query(java.lang.String query, java.lang.String format) � 2 argument(query, format)


� SPARQL query

protected void onStart() {

super.onStart();

// Service binding

Intent i = new Intent( this, SparqlEndpoint. class);

boolean ret = bindService(i, mConnection , Context. BIND_AUTO_CREATE);

private ServiceConnection mConnection = new ServiceConnection() {

// Service binding call

public void onServiceConnected(ComponentName name, IBinder servic e) {

// converting from service into ISparqlEndpoint interface

MOBILE APIs


mService = ISparqlEndpoint .Stub. asInterface(service);

mService.query( … );

}

// Closing service

public void onServiceDisconnected(ComponentName name) {

mService = null;

}

};

…

}

[DB]

OpenStreetMap[DB]

DBPedia

[Triple]

LinkedGeoData

(LGD)

Wikipedia

Mapping with

Owl:SameAs,

Jaro distance metric

Name comparing

Triplify

Schema : 24Class

Element : 320mega Nodes,

25mega Ways

Making XML data to Linked Datahttp://www.larkc.eu

Use-case : Urban Computing in LarKC project



Inconsistency check• Some POIs suddenly disappeared at a point and

reappeared at another point which make arrow directions are inconsistent.

Unsuitability check• some of road signs are not properly designed

and installed as specified by the regulation.


Inconsistent road sign Unsuitable road sign

New road sign covers another road sign

Go straight? Or turn right?

and installed as specified by the regulation.

• improperly located road signs such as those not installed within the specified range from junction points are not suitable one.

Continuous planning• Some of POI are destroys and moves or appears

from time to time. One of major POI movement(for example, city hall) may bring many road sign modifications.

[Node Element]

R K E

Node RS KPOI WPOI

[Road Sign & POI Insertion]

R

K

E

sameAs

[Junction & POI Finding]

[Link Element]

startNode endNode



[Way Element]

nextLink

link N

link M

[Road Element]

way N

way M

[Junction & POI Finding]

J1

R1

KE

R2

KE

searching range

J2

K

sameAs

Data Set Comments

Linked Geo Data(LGD)

� 1 billion triples in WGS84 coordinate� Loading LGD full and extracted part of Korea

Open Street Map(OSM)

� Extracting all way information in WGS84 coordinate� Selecting and importing 2 million triples for Seoul

Point Of Interest Data in Korea

(KPOI)

� 1 million POIs related with road signs� Around 4 million triples.



(KPOI)� Around 4 million triples.

Seoul road sign data (RSD)

� Diverse data of Seoul road sign in database� 9515 instance of direction road signs in Seoul� Converting TM coordinate into WGS84 coordinate� Converting RDB into RDF (0.5 million triples)

Korean road signregulations (RSR)

� Around 30 Regulations of road signs� Changing to SparQL for validation check

Mediate Ontology(MO)

� Ontology linking between OSM, KPOI, RSD or other data� Expressivity : subClassOf, subPropertyOf, sameAs, inverseOf

Total : about 1.1 billion triples

Linking, Mapping, Converting, Adjusting

LGD OSM KPOI RSD



Mediate Geo-Ontology

Finding the target POI around 500m from a node of road

PREFIX rsm: <http://www.saltlux.com/rsm#>

PREFIX owl: <http://www.w3.org/2002/07/owl#>

PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>

PREFIX sgf: <http://www.saltlux.com/geo/functions#>

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>



PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>

select distinct ?targetPOIID

where {

rsm:osmn_436718764 geo:lat ?endNodeLat .

rsm:osmn_436718764 geo:long ?endNodeLong .

rsm:kpoi_12720 geo:lat ?targetNodeLat .

rsm:kpoi_12720 geo:long ?targetNodeLong .

rsm:kpoi_12720 rsm:id ?targetPOIID .

filter ( sgf:distance(?endNodeLat, ?endNodeLong, ?targetNodeLat, ?targetNodeLong) <= 500 )

}

Data: Traffic Flow and Speed Prediction: Data from Milano

� Traffic data from Milano (Italy)

� Data ranging from Mar. 07 to July 09 (849 days)

� 5 min. sampling rate for flow & speed

Milano City Sensor Map



� 5 min. sampling rate for flow & speed

� Traffic flow & speed from

� 209 sensors that are able to classify vehicles, and

� 757 non classifying sensors

� Weather data provided fromhttp://www.ilmeteo.it

� 1 hour sampling rate for weather data

Sensors – Crossroads – Street Categories (multi-colored)

Problem Description: Traffic Flow and Speed Prediction

Traffic Flow [12:00; 12:05] (preprocessed)

Tra

ffic

Flo

w (

# v

ehic

les)



� Predict the traffic flow and speed for the next 24h based on a 5 min. time grid

� Traffic flow and speed forecasts are made on the sensor level for the whole traffic network

� Forecasts: inputs for optimal routing algorithms

Mar. 07 July 09

Mar. 07 July 09

Tra

ffic

Speed (

avera

ge)

Traffic Speed [12:00; 12:05] (preprocessed)

Context Awareness and Geo-Semantics


Source : Flickr.com, David Crow

• physical contexts

: location, time

• environmental contexts

Context Awareness and Geo-Semantics


• environmental contexts

: weather, light, sound levels

• informational contexts

: stock quotes, sports scores

• personal contexts

: health, mood, schedule, activity

• social contexts

: group activity, social relationship

• application contexts

: e-mail, websites visited

• system contexts

: network traffic, status of printers

PersonRelative

Person

personID

hasProfile

health

hasNick

belongTo

Health

disease

bloodType

heartBe

at

bodyTemperature

etcHealthInfo1

preference

interest

Profile

name

age

birth

Nick

nickName

calledBy

Interest

field

detailedField

etcHealthInfo2

job

cellularPhoneNum

email

Group

groupName

hasMemberuserPriority

privacyLevel

deviceAccessLevel

privacyLevel

height

weight

privacyLevel

privacyLevel

locationInfo

LocationSensor

contact

Contact

privacyLevel

hasSchedule

PolicysubClassOf subClassOf

Session

person

hasTask

Task

taskName

hasSubTask

timeInterval

SubTask

targetProperty

space propertyValue

OperationSubTask

taskPriority

DeviceSubTask

device

timeInstance

delayTime

hasOperation

SearchSubTask

searchWord

delayTime

timeInterval

subClassOfsubClassOf

deviceName

subClassOf

TimeRelation

before

after

during

at

Time

TimeWordMean

timeWord

before

after

at

serviceInterval

date

time

TimeInterval

dateInterval

to

from

timeInterval

subClassOfsubClassOf subClassOf

Space


OfficeSpace

이미지를 표시할 수 없습니다 . 컴퓨터 메모리가 부족하여 이미지를 열 수없거나 이미지가 손상되었습니다 . 컴퓨터를 다시 시작한 후 파일을 다시 여십시오 . 여전히 빨간색 x가나타나면 이미지를 삭제한 다음 다시 삽입해야 합니다 .

HomeSpace

spaceName

Event

timeInterval

actionTask

termsEventTerms

person

operation

deivceName

order

device

EventRelative

space


Space

Device

Person

Ontology for Context Awareness by Saltlux


EtcSensor

sensorID

sensingDataType

sensingTime

coordinate

status

Device

deviceID

remainBattery

coordinate

deviceDefault

owner

Actuator

Display

mode

controlColor

brightness

contras

OperationLevel

operationLevel

Temperature

degree

humidity

Sound

volume

soundType

modeEQ

actuator

Exclusive

device

deviceName

hasSensor

menuDisplayTime

Communication

powerStatus

Light

bright

color

PowerdeviceAccessLevel

LocationSensor

sensorID

sensingTime

coordinate

status


Sensor

sensingDataType

EntityRelative

alternativeDevice

Broadcast

station

channelisWork

Player

track

playerStatus

OpenClose

ocStatus

IPaddress

hasSchedule

registerService

DeviceAccessLevel

level

PrivacyLevel

level

Level


UserPriority

level

TaskPriority

level

subClassOf

Priority

subClassOf

subClassOf subClassOf

who

Schedule

title

to

withWhom

briefingContents

where

from

TaskType

RequestInformation

queryLiteral

deviceName

hasOperationO

DeviceTask

Operation

targetProperty

propertyValue

UserCommand

IPAdd

requestTime

hasServiceTime

hasTaskTypeO

O

TimeRelation

from

deviceID

subClassOf subClassOf

nodeType

moviePlace

personID

macID

to

at

UserCommandRelative

catagory

BatchTask

taskData

taskName

subClassOf

subClassOf

subClassOf

LocationSensor

Device

Person

etcSensor

spaceName

device

locationSensor

etcSensor

owner

Person

Dynamic Context

Inferred Context

Conte

xt

Model

Conte

xt

Rule

s

CONTEXT

DeviceUser

CONTEXT OWNER

QoC

CONTEXT MANAGERSENSOR / NETWORK

Filter

Collector

Concept of Context Driven Mobile Services


Smart Mobile Service

ServicePersonalization

CONTEXT-AWARE SERVICE

ServiceAdaptation

ServiceDiscovery

Context Aware Platform by Saltlux


Life Logging from Smart Phone

Use-case : Life Logging


Use-case : Life Logging

Life Logging as RDF data � Context Aware (Inductions: ML)


Sat Sun Sat Sun Sat Sun Sat Sun Sat Sun

Sickat Home

Use-case : Analysis of Life Pattern


at Home

WorkplaceWorkplaceWorkplaceWorkplace

Bus_BBus_BBus_BBus_B

Bus_ABus_ABus_ABus_A

Use-case : Trajectory Awareness

Hybrid Reasoning : DL Reasoning + Induction(Machine Learning)


HomeHomeHomeHome

Bus_EBus_EBus_EBus_EBus_DBus_DBus_DBus_D

Bus_CBus_CBus_CBus_C

Clustering CalculationSelection


Home Detection

Workplace Detection


SPARQL Query and DL Reasoning


GPS Log

Data

Clustering CalculationSelection

Cleansing MatchingSegmentation

Bus Stop Detection

SampledSampledSampledSampledMobileLogMobileLogMobileLogMobileLog

DataDataDataData

Main Line ExtractingMain Line ExtractingMain Line ExtractingMain Line Extracting

LineLineLineLineThickeningThickeningThickeningThickening

LineLineLineLineThinningThinningThinningThinning

LineLineLineLineGeneratingGeneratingGeneratingGenerating

Commute Trajectory LearningCommute Trajectory LearningCommute Trajectory LearningCommute Trajectory Learning

TrajectoryTrajectoryTrajectoryTrajectoryTracingTracingTracingTracing

TrajectoryTrajectoryTrajectoryTrajectoryClusteringClusteringClusteringClustering

TrajectoryTrajectoryTrajectoryTrajectoryGraphGraphGraphGraph




E1 � E3 � E5 � E7

Wed Nov 17 08:06:19 9min

Mon Nov 22 08:06:34 10min

Sat Nov 27 09:28:27 14min

E1�E2� E6 � E7

Thu Nov 18 09:52:35 13 min

Fri Nov 19 09:22:32 10 min

Sat Nov 20 09:44;30 13min

Thu Nov 23 08:18;39 6min

Wed Nov 24 12:35:52 15min

Thu Nov 25 09:31:23 14 min

Fri Nov 26 09:38:21 10 min

E4

V1

V4

E3

E1

E5

E7

V6

E6

E2

V3

V2

V5E4

V1

V4

E3

E1

E5

E7

V6

E6

E2

V3

V2

V5

1. No Accident and Disaster

Use-case : ITS of u-City

2. Finding Accident and Disaster : Recommending Detour Path


3. Finding Accident and Disaster but it could be recovered soon

Use-case : Ontology model for ITS


System Overview

• 4 System blocks

• OS : Windows Server

• Platform : J2EE based POJO

(Pure Object Java Object)

• Supporting Web service

Context Aware System

Triple Store

Rule Store

Query Engine

Reasoner

ServiceInterface

ServiceHandler

Situation ReasoningService Manager

Use-Case : Water and Gas Pipeline Management

Context-Aware Disaster Management POC by Saltlux


• Supporting Web service

ContextAwareSystem

DisasterCenter

DashBoard

ControlCenterWeb

ServiceSensorData

GIS Data

LogsData Layer

Rule StoreReasonerHandler

Context Filter

Context Collector

Instance Population

Instance Manager

Context ManagerContext Acquisition

Context-Aware Disaster Management POC by Saltlux

Use-Case : Water and Gas Pipeline Management


Sensor Monitoring Discover Leakage Area

Infer Leakage Pipe Link Automatic Alert Recom. Detour Path

Leakage Detection

Social Semantics

1. Structural features• Small world : six degree of separation

• Unfair world : governed by power law

• Strength of weak relationship

Characteristics of Social Network and Networking


• Strength of weak relationship

2. Synchronization and Amplification

3. Empowerment in Network

• Six degrees of separation

- Experiment : Six degrees of Kevin Bacon

• Does internet and SNS make it shorter?

Structure : Small World


make it shorter?

- Yes or No ?

- Average degrees in FB : 5.73

- Maximum degree in FB : 12

- Average degrees in TW : 4.67

- Average degrees in WP : 4.5

• Scale-Free Network

- Governed by power law

- Like Pareto and long tail principle

• Evolving to HUB network

- Portal(google), SNS(FB, TW)

Structure : Unfair World


- Portal(google), SNS(FB, TW)

- Airport, logistics(Fedex)

• Unfair world, reality

- men-women network

- The rich get richer and the poor get poorer

99

Structure : Unfair World


66

• Strong and Closed Network

- Mafia organization, trust network

- no secret, no new information

• Weak and Open Network

- a broker among heterogeneous netsVS.

Strength of Weak Relationship


- a broker among heterogeneous nets

- controller of network and info. flow

• Strength of Weakness- multiple and cross discipline become more important

- getting new job and ideas, building new business and innovation

VS.

Synchronization and Amplification


Infection of antismoking , fatness and etc like disease

Synchronization and Amplification : infection


Empowerment and democracy in Social network


Power Law VS. Power Dispersion

• Social Networks : networks based on the relation between people

• Semantic Social Network : RDF representations of social network and data

Foaf:knows

Abstraction stack for semantic SNA

Semantic Social Network Analysis


[Semantic Social Network Analysis, http://journal.webscience.org/141/2/websci09_submission_43.pdf]

Rich graph representations reduced to simple

untyped graphs in order to apply SNA

Foaf:interest

[Paolillo and Wright 2006]

Social NetworkSemantic Network

Text MiningText Mining SNASNA



Text Mining(Induction)Text Mining(Induction)

SNA(Deduction)

SNA(Deduction)

Semantic Social Network AnalysisSemantic Social Network Analysis

SemanticNetwork

Task

People

Org.

Service

Event

Place



Family, Colleagues , Community

InformationHub, Broker

Connecting Experts

73

vk : weighting of relation n : number of relations g : total number of entity

gjk : # of shortest path between j and kgjk(i) : # of path having i between j and k

Dijkstra’s algorithmDijkstra’s algorithm

O( | E | + | V | log | V | )

SAMZZIE : Social Semantic Platform by Saltlux

Knowledge Network Services

Knowledge Discovery Control

KDC

Knowledge Network Analysis

KNA

Integrated API

K-DicManager

Social Network Knowledge Knowledge Circulation

User Type Pattern

Social Network

User Type

Pattern

Web User Interface

Topic Rank

DA DAR KNA

Knowledge Circulation

Knowledge Trend

Contents Search

User & Admin Knowledge

Env.

Knowledge Discovery

Query

DAC Schedule


Data Aggregation

DA

Data Analysis and Reasoning

TREDAR

Meta Base Knowledge Base

Email Abst.

Email Aggr.Document

Aggr.Data

AbstractionKnowledge Population

Query Engine

Social Network Analysis

Knowledge Trend Analysis

Circulation Analysis

Pattern Analysis

K-Dic KDQ AuthorityPolicy

U & AEnv.

EmailContents

NE & Annotation

DA

Web Aggr.

ScheduleTopic Trend

Topic Ranker Engine

SN KT KC UTPTopic Rank

DA Scheduler

DAR Scheduler

KNAScheduler

TMS

FeatureExtraction

Use-case : Social Media Analytics


Use-case : VOC Sensing & Analysis (for KT)

Hybrid Reasoning : Induction(Machine Learning) + Deduction(Horn Logic)


Legacy System

MagicN

IPAS

AAA

SAS

Intra Portal Solution

FIMM MagicN

Intelligent Application Service (AS)

Personalized Personalized Search

Intelligent Intelligent Traveling Guide

Shopping Shopping Recommendation

Social Social Network

Local News Etc.,

Enabler

VME

ICE

PS

Internet Portal, BcN, All-IP N/W

empas

DAUM

CDE

ISMS

SICS

ICDS

O&M

Use-case : Intelligent Telco Platform


BGCF

Node-B(WCDMA)

RNC SGSN

SAS

JUICE

S(L)MSC

S(L)MSC

NAVER

InternetTV

BcN

All-IPNetwork

W-CDMA

PCRF

IMS-ALG

TrGW

HSSAuC

IMS Infra

GGSN

Simulation Server

P/I/S-CSCF

• ISMS: Intelligent Subscriber information Mgnt. Server• ICDS: Intelligent Content Delivery Server• SICS: Subscriber Information Collection Server• O&M: Operation & Management Server

Pay for

Call

Profile

•

Profile

• Name: Jerry

Obama

• Age: 12

• Sex: Woman

Profile

• Sex: Woman

Profile

• Name: Elizabeth

Cox

• Age: 12

• Sex: Woman

Profile

• Name: Jane Bush

• Age: 12

lives in

lives in

attend

attend

attend attend

attend

Major Residential Area

Major Activity Area

Use-case : Mobile Social Network Analysis


Call

CallCall

Call

• Age: 12

• Sex: Woman

Profile

• Sex: Woman

Profile

• Name: Edward

Adams

• Age: 11

• Sex: Woman

Profile

• Sex: Woman

Profile

• Name: Jessica

Bailey

• Age: 13

• Sex: Woman

Profile

• Name: Tom Obama

• Age: 16

• Sex: Man

Profile

•

Profile

• Name: Nancy

Obama

• Age: 42

• Sex: Woman

Call

lives in

Family Friends

User Profiling from Network

Use-case : Personal Profile Analysis


Bimodal

Normalμ1 = 38

σ1 = 4.2

w = 0.83

μ2 = 13

σ2 = 2.4

Φ=

각 나이대별 (SMS/VOICE), (성별),

(나이대) 120 클래스 패턴을 이용한 PI

결과

Ontology Population

Legacy Data

Ontology Mapping from Legacy Data

Use-case : Personal Profile Analysis - Semantics


50 Classes , 58 Relationships , 15 Properties , 57 Rules

Use-case : Hybrid Reasoning DL + Rules


Use-case : Discovering Social Relationship


Geo-Social Semantics

O2 Platform : Geo-Social Data Cloud by Saltlux


Semantic Geo-Social Platform

GEO Context Provider

SPARQL/XML, JSON

HTTP Engine

Query Engine Geo Context/Social Context Acquisition

Social Context Provider

Social Content EngineGeo Contextmanager

REST/XML, JSON

O2 Platform : Geo-Social Data Cloud by Saltlux


Geo Context Store(Place, Event, etc)

Acquisition

Wrapper Manager

Geo Context Manager

DataFormatter(RDF/XML)Query

Result Formatter

Query Executor

Wrapper

….

Result Formatter

Social ContextIndexer

Social ContextSearcher

Social Context Manager

manager(Admin Console)

Social Context Index

Weather

Traffic

Event

…. Blog News

Opinion Mining and Sentiment Analysis from Twitter


O2 Platform : Ontology model for twitter

sioc:UserAccountsioc:id(xsd:string)

sioc:creator_of

twd:retweet

sioc:has_creator

twd:followertwd:following

twd:post

twd:TwitterUsertwd:screenName(xsd:string)

twd:discuss


geo:SpatialThing geo:NamedPlace

twd:Tweettwd:messageID(xsd:string)

twd:messageTimeStamp(xsd:string)

twd:talksAbout

twd:reply

twd:talksAboutNeutrally

twd:talksAboutPositively

sioc:Postsioc:content(xsd:string)

twd:discuss

twd:talksAboutNegatively

Live Demo


Live Demo

PREFIX f: <java:ext.>SELECT ?poi1 ?poi2 ?user (f:similarWithProbability(data:Alice, ?user) AS ?p) WHERE {?user bot:posts ?t1 . ?t1 bot:talksAboutPositively ?poi1 . ?poi1 a bot:NamedPlace ;

geo:lat ?lat1 ;geo:long ?long1 ;skos:subject ?category .

data:Alice geo:lat ?givenLat ;geo:long ?givenLong ;

tweets about a given kind of POI of people similar to me that tweeted nearby in the last x minutes;

Hybrid Reasoning and Queries


geo:long ?givenLong ; bot:posts ?t2 .

?t2 bot:talksAboutPositively ?poi2 .?poi2 a bot:NamedPlace ;

geo:lat ?lat2 ;geo:long ?long2 ;skos:subject ?category .

FILTER(?t1!=?t2)FILTER(f:similarWithProbability(data:Alice, ?user)>0.5)FILTER((?lat1-?givenLat)<"0.1"^^xsd:float &&

(?lat1-?givenLat)>"-0.1"^^xsd:float &&(?long1-?givenLong)<"0.1"^^xsd:float &&(?long1-?givenLong)>"-0.1"^^xsd:float )

} ORDER BY DESC(?p)LIMIT 10

AR based Location SearchAR based Location Search Reputation AnalysisReputation Analysis

BOTTARI mobile App by Saltlux


Intro screenIntro screen Social Recommendationand Dynamic Social search

Expert Search andReal-time Q&A

DEMO Movie


Future Use-case : Disaster Management


Source : BBC, ESRI

Stream Data

Mobile phone Sensors Satellite/CCD images Sensor Networks

Hybrid Stream Reasoning

Mass

agin

g S

erv

ice

Social Media & Networks



Stream Data Collection

Deci

sion S

upport

ing

GIS and Geo-Data

Geo-SpatialData Collection

Hybrid Stream Reasoning

DisasterKnowledge

DisasterProcess

DisasterPolicy

DecisionSupporting

System

DashboardAnd

Controller

Mass

agin

g S

erv

ice

Citizen

Government ,Disaster Center

Intelligent Disaster Management System

Technical Tips

And the Company


And the Company

4 Dimensions of Semantic World

Sca

lability


Performace

Expressivity

DataDynamics

Current State of the Art of Technology

Sca

lability

Telco

UbiComp

Soci

al N

et

Year Performance

2005• 500M triples


Expressivity

EnterpriseSearch

Medical

Soci

al N

et

2005• 500M triples• OWL DLP

2010• 30B triples• OWL DL Horst

Sca

lability

Soci

al N

et Telco

UbiCompYear Performance

• 500M triples



Performance

Soci

al N

et Telco

EnterpriseSearch

Medical

2005• 500M triples• 1~40S (LUBM1000)

2010• 30B triples• 0.01~5S (LUBM1000)

Search

Perf

orm

ance

Telco

UbiComp

Year Performance

2005• 1~50S (LUBM1000)



Expressivity

Soci

al

Net

Medical

2005• 1~50S (LUBM1000)• OWL DLP

2009• 0.01~5S (L1000)• OWL DL Horst

How to move Maginot Lines?

Sca

lability

Sca

lability

?


Expressivity Expressivity

?

6 Solutions

Enhanced algorithm

Materialization

Distributed Computing

CurrentState of the Art

ImprovedResults


Distributed Computing

Approximation

Lean KR model

Query optimization+ Query/Data Cache

Wining Strategies

Algorithm Materialization

Query

Medical

E. Search

Social Net

Mobile

Ubiquitous


Distribution

ApproximationLean KR model

QueryOptimization(+ Cache)

Company Overview

Our Mission is

“Communicating Knowledge” for People.

� Company Name Saltlux, Inc. (Since 1979)

� President & CEO Tony (Kyung-il) Lee

� Headquarter Seoul, Korea Tel) +82 2 3402 0081


� Subsidiaries Japan(Tokyo), China(Chingtao), Vietnam(Hanoi)

� URL www.saltlux.com

� Research Lab HLT Laboratory (Member of EU FP6, FP7 projects)

� Investor JAFCO ASIA

�Main Products [IN2] : Semantic Search Platform

STORM : Semantic Business Platform

OWLIM : Semantic Web Search Service

Knowledge Communication Company

Tech/Biz Consulting Software Solution Innovative Service

Business Items


Web 3.0 &

Semantic Web

Ubiquitous

& MobileSearch 2.0

Text Mining

Semantic Search

Mobile Search

Semantic Web

Semantic Wiki and Blogs

Semantic Annotation

Context Awareness

Personalization

Intelligent Mobile Platform

Semantic Tech.

Good Software(GS) Cert.Asian Top 20 Tech. Comp. Best Software Awards

Trustable Company

International Certifications and Awards“


NationalStandard Product

DISCOVERY

NationalStandard Product

DOR & TMSTechnical

Innovation

INNO-BIZ comp. ISO9001:2000

QualityAssurance DOR &TMS

Digital InnovationAwards

20008HIT products

CustomerSatisfaction

TrasWiz

Best KoreanTech. Cert.

World TopWorld Top--1010 “Semantic Tech. Company” “Semantic Tech. Company” ((ZDnetZDnet))

Gov,, Public28%

Construction3%

Telco27%Law

1%

Manufacture10%

We have provided the unique solutions and services

to over 400 customers during last two years.

Customer Distribution


Finance9%

Univ.,Institute

19%

Enterprise49%

Steel2%

Manufacure11%

1%

Broadcasting5%

Portal13%

Electronics24%

Medical3%

Selected Customers

P A T


Core Technologies of Saltlux

Machine

Learning

Machine

LearningArtificial

Intelligent

Artificial

Intelligent

Natural Language Processing

Natural Language Processing

Text MiningText Mining ReasoningReasoningSemantic

Disambiguation

Semantic

DisambiguationSemantic

Annotation

Semantic

Annotation


Search 2.0

(Semantic Search)

Search 2.0

(Semantic Search)

Semantic Tech.with Ontology

Web 3.0

(Semantic Web)

Web 3.0

(Semantic Web)

Ubiquitous / Mobile

(Context Awareness)

Ubiquitous / Mobile

(Context Awareness)Semantic BPMSemantic BPM

Search & Mining Platform

“[“[IN2]”IN2]”Semantic Business Platform

““STORM”STORM”Semantic Web Search Service

““OWLIM”OWLIM”

Three Products

Saltlux has 3 product brands, [IN2], STORM and OWLIM.


Semantic Search and Mining Platform [IN2]

[IN2]Discovery 2

Semantic Search Engine

Cloud based[IN2]SSAMZIE

[IN2] Platform


[IN2]DOR

Cloud basedIntegratedSearch Engine

[IN2]SearchBox

ApplianceStyle SearchPortal

[IN2]SSAMZIE

Social Search& Mining Engine

[IN2]HBC

HybridClassification

Engine

112

Semantic Search & Mining : [IN2]Discovery


Knowledge Modeling : SBM5, COMET, OS

Triple storing and management : SOR, AG

Best Semantic Web platform in Asia

STORM : Semantic Business Platform


Semantic Annotation : SEMANO, OPTIMA

Semantic Query & Reasoning : SOR, COMET

Application Modules : SSAMZIE, SemEDIT

37

Characteristics

- Mass & real time semantic metadata

storage and management tool

SOR is integrated framework for semantic metadata searching and

management based on Ontology and reasoning technologies.

STORM : SOR


storage and management tool

- High scalability with distributed comp.

- Automatic data collection and filtering

- Context awareness by connecting

heterogeneous context information

- Including ontology and rule based

reasoning engines

- Ontology instantiation by [IN2]SEMANO

COMET is a collaborative framework allowing multiple clients to collect, store,

retrieve, edit, query and do reasoning, manage versions of stored ontologies.

Key Functions- Ontology Library & CVS

- Ontology Categorizing

- Supporting Ontology Engineering

STORM : COMET


- Supporting Ontology Engineering

- Query Management

- Server Reporting

- User Management

Future Works- Improving multi-user performances

- Web based client

- Improving ACL

- Developing plug-ins for Neon and

Topbraid Composer, etc.

Conclusion

• Open Linked Data

Commons forGeo-Social Semantics


• Open Linked Data

• Open Accessibility

• Open Knowledge Base

Thank

SemanticTechnologybecauseOf

interestedIn


You

BusinessInnovation

interestedIn

willAchieve

contributeTo

Q&A

@@TosajangTosajang

SemanticTechnology

SemTech 2011, Saltlux, Tony Lee

Technology

Transcript of SemTech 2011, Saltlux, Tony Lee