Epistenet: Facilitating Programatic Access & Processing of Semantically Related Data

Sauvik DasCarnegie Mellon University

Jason WieseUniversity of Utah

Jason HongCarnegie Mellon University

sauvik@cmu.edu wiese@cs.utah.edu jasonh@cs.cmu.edu

Overview

Motivation

Key features

Motivating Example

Conclusion

Goal

Facilitate the programmatic access to and processing of semantically related mobile personal data.

Goal

Facilitate the programmatic access to and processing of semantically related mobile personal data.

personal data other datacapturable data about the user

capturable data not about the user

call logslocationbrowser historyactivity

battery levelconnectivityprocessing load

Goal

Facilitate the programmatic access to and processing of semantically related mobile personal data.

Phone GPS People

Personal data is currently isolated in source-of-origin silos…

Epistenet, instead, semantically structures mobile personal data in knowledge graph…

Goal

Facilitate the programmatic access to and processing of semantically related personal data.

and provides a unified query interface to access this data.

Timestamp > 7/7/15

Attribute Constraint Filter

Ontology => SensorReading

Ontology Constraint Filter

Reference Node

Relative Filters

Related Node

Developers can query for personal data through expressive filter mechanisms.

Usage

EpistenetObject eo = uqi.getMostRecent( new OntologyClassFilter(OntologyClass.Communication));

Get a random correspondence

EpistenetObject[] rel = uqi.get( new RelativeFilter(eo).createdWithinRange(5*60*1000L);

Get all other data within 5 minutes of correspondence

Overview

Motivation

Key features

Motivating Example

Conclusion

Motivation

Mobile personal data underlies many of the most interesting application concepts of the present and future.

Examples

Smart app launchersIntelligent assistants

Life loggers eHealth

Personal data access

Location, calendar, interests, contacts

Application usage, time of day, activity,

communications

Location, time, application usage

Heart rate, activity logs, food logs, self-reports

Problem

Today, mobile personal data application development difficult is difficult because…

Queries based on source of origin, not

meaning

Semantic relationships between data lost

Data Silos

Persistence dramatically different between log

and sensor data

Programming complexity: event-based programming

vs. database queries

Sensor / log disparity

Data silos Sensor / log disparity

Queries based on origin, not

meaning

Semantic relationships

lost

Persistence vastly different,

lose context

Programming complexity

Challenges

Data silos Sensor / log disparity

Queries based on origin, not

meaning

Semantic relationships

lost

Persistence vastly different,

lose context

Programming complexity

Challenges

Epistenet

Knowledge graph

Relative, semantic and descriptive

query filters

Historical sensor data around log

data events

Unified query interface

Data silos Sensor / log disparity

Queries based on origin, not

meaning

Semantic relationships

lost

Persistence vastly different,

lose context

Programming complexity

Challenges

Epistenet

Knowledge graph

Relative, semantic and descriptive

query filters

Historical sensor data around log

data events

Unified query interface

Overview

Motivation

Key features

Motivating Example

Conclusion

Key features

Knowledge graph

Unified Query Interface

Relative, semantic and descriptive filters

Histories of sensor and log information

Knowledge graph

Epistenet organizes personal data into a single knowledge graph.

Ontology

The Epistenet ontology organizes personal data semantically. Data providers can contribute personal data under any ontology class.

MessageCommunic

ation

Person

Sensor reading

Phonecall Location

Accelerometer

Knowledge graph objects

Epistenet objects are individual personal data points (e.g., a phone call, a location ping, a set of accelerometer readings).

Knowledge graph objects

Descriptive attributes

Meta attributes

Ontology classes

(e.g., caller, duration)

(persistence, timestamp, permissions)

Unified query interface

Epistenet provides a single interface to access all personal data in the knowledge graph.

How can it be used?

uqi.get(filter) .getRandom(filter) .getN(filter, N) .getMostRecent(filter)

. . .

How can it be used?

uqi.get(filter) .getRandom(filter) .getN(filter, N) .getMostRecent(filter)

. . .

Filters

Filters are sieves on the personal data graph that developers use to isolate personal data points they want.

Attribute-constraint filters

Sender is not unknownnew AttributeConstraintFilter().addNonEqualsConstraint(“sender”, “unknown”)

Constrain any meta or descriptive attributes of an object.

Ontology-constraint filters

OntologyClass is “Location”

Constrain ontology class.

new OntologyConstraintFilter(OntologyClass.Location)

Ontology-constraint filters

OntologyClass is “SensorReading” or any subclass

Constrain ontology class. Can also traverse ontology.

new OntologyConstraintFilter(OntologyClass.SensorReading, true)

Relative filters

Constructs queries relative to other personal data points. Thus, any personal data point is an index into the knowledge graph.

Reference Node Related Node

+ - xunion (or) intersect (and) except (xor)

Filter compositions

Historical Sensor Data

Ontology-constraint filters

Epistenet also captures sensor histories around logged data points to bridge the persistence disparity. Also allows developers to get sensor data from before install-time.

Overview

Motivation

Key features

Motivating Example

Conclusion

Motivating Example: Autobiographical Authentication

DescriptionThree steps:

Find a “fact” to ask a question about

Create an answer list

Provide contextual hints

Find a fact

EpistenetObject fact =uqi.getRandom(sms.intersectCompositWith(atTime));

Filter sms =new OntologyConstraintFilter(OntologyClass.SMSMessage);

Filter atTime =new MetaConstraintFilter().timestampWithinRange(lt, ht);

Create answer listFilter similar =new RelativeFilter(fact).sameOntologyClass().createdBefore();

EpistenetObject[] answers = uqi.get(similar);

Provide contextual hintsFilter hintFilter =new RelativeFilter(fact).differentOntologyClass().createdWithinRange(10*60*1000L);

EpistenetObject[] hints = uqi.get(hintFilter);

Overview

Motivation

Key features

Motivating Example

Conclusion

Epistenet organizes mobile personal data into a knowledge graph.

Epistenet affords easy access to subgraphs of the knowledge graph via the UQI.

Epistenet bridges the data persistence and query gaps between sensor and log data.

Conclusion

Epistenet: Facilitating Programatic Access & Processing of Semantically Related Data

Sauvik DasCarnegie Mellon University

https://gitlab.com/scyrusk/epistenet

@scyrusk || sauvik@cmu.edu

http://sauvik.mew/ Jason Wiese and Jason Hong