Research of Database Group @ UNSW

Some slides are taken from memebers @DBG

Wenjie Zhang

Group Overview

• Research Field: core topics in DB, DM, IR, MM

• Group Size: 8 staff members; 20+ PhD students

• Research support: Consistent success in government research grant applications

Some recent research projects• Xuemin Lin and Wenjie Zhang: Efficiently Processing Pattern-based

Structure Queries over Large Graphs , ARC Discovery Grant (2015 - 2017 ), $397,500

• Wenjie Zhang and Lei Chen, Continuous Loyalty-based Similarity Queries over Moving Objects, ARC Discovery Project (2015-2017), $266,300

• Lijun Chang, Efficient Cohesive-Subgraph Search over Large Graphs, ARC Early Career Research Award (2015-2017), $372, 000

• Xuemin Lin, Probablistic Search Over Large-Scale Uncertain Graphs, ARC Discovery Project(2014-2016), $413,000

• Xuemin Lin and Wenjie Zhang, Ranking Complex Objects in a Multi-dimensional Space, ARC Discovery Project(2012-2014), $350,000

• Wenjie Zhang, Continuously Monitoring Uncertain Objects in a Multi-dimensional Space, ARC Early Career Research Award (2012-2014), $375,000

What is research ?

• Research comprises "creative work undertaken on a systematic basis in order to increase the stock of knowledge, including knowledge of humans, culture and society, and the use of this stock of knowledge to devise new applications”. ---- wikipedia

Research degrees & projects

• Master by Research

• PhD

• Research projects: 18UoC / 24UoC

Some research topics

• Location based services

• Preference queries on multi-dimensional data

Location based services

• Services that integrate a user’s location with other information to provide added value to a user.

Examples

Navigation and travelGeo-social networkingGamingRetailAdvertisement

and many many more…

Location-based services have a bright future

Number of mobiles > World’s population

24% use LBS and 94% of these find LBS valuable

LBS are a bonanza for start-ups (est. market $13B in 2014)

$21B in 2015

Past Research

Shortest Path Query Range Query k-Nearest Neighbors Query Reverse Nearest Neighbors Query k-Closest Pairs Query

and other similar queries…

Shortest path query

• What is the shortest path from here to airport

Range Query

• Return the coffee shops within 300 meters.

K Nearest Neighbor Queries

• Return the closest fuel stations.

Reverse Nearest Neighbor Query

• Return the cars for which my fuel station is the nearest fuel station.

K-Closest Pairs Return the closest pair of McDonald’s.

Variations

• Static queries VS continuous queries

• Euclidean distance VS network distance

Some research topics

• Location based services

• Preference queries on multi-dimensional data

Preference queries on massive multi-dimension data

DBG@UNSW 18

Massive multidimensional data are collected everyday

location data from various Observational Mechanisms.

- Smart Phone

0.36 billion this year in China – largest smart phone market , expect 0.45 billion next year. Baidu Location based service receives 3.5 billion location requests on average each day.

- Sensor

- Radio Frequency Identification (RFID)

- Global Position System (GPS)

Background

Other Multi-dimensional data from various applications - Environment monitoring Measure light, temperature, humidity…

- Finance and economic data purchase transactions, stock transactions …

- User behavior data click streams , shopping records, … - Network data Network monitoring data - etc.

DBG@UNSW 19

Problems Investigated

DBG@UNSW 20

Given a large number of multi-dimensional objects, we investigate the following representative and fundamental queries.

• Rank-based Queries

Top k query, Quantile query, Influence maximization

• Dominance-based Queries

Skyline query, representative skyline query, dominating queries

• Spatial Keyword queries

DBG@UNSW 21

Rank-based queries

1. Top k query

p2

p1

p3

X : academic score

p4p6

p5p7 p8

Y: rese

arch

score

f(p) = x + y

2. Φ-quantile : summarize score distribution

DBG@UNSW 22

Rank-based queries (cont.)

The first element in a sorted list with the cumulative weight not smaller than Φ, where Φ is a number in (0, 1].

Sorted elements:

3 3 6 7 8 9 12 13 15 20

0.5 quantile (median) 0.8 quantile

• Other Statistics

DBG@UNSW 23

Rank-based queries (cont.)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Find all elements with frequency > 0.1%

Top-k most frequent elements

What is the frequency of element 3? What is the total frequency

of elements between 8 and 14?

How many elements have non-zero frequency?

Rank-based queries (cont.)

• Reverse rank-based queries (ongoing….)– How can an object be the top-1 result ?– For most users ?–With minimum cost ?

Dominance-based queries

DBG@UNSW 25

n-dimensional numeric space D = (D1, …, Dn) on each dimension, a user preference ≺ is defined two points, u dominates v (u ≺ v), if

- Di (1 ≤ i ≤ n), u.Di ≺ = v.Di

- Dj (1 ≤ j ≤ n), u.Dj ≺ v.Dj

p2

p1

p3

p4p6

p5

p7p8

Y: rese

arch

score

X : academic score

DBG@UNSW 26

Dominance-based queries (cont.)Skyline : points not dominated by other points. - candidates of best options in multi-criteria decision applications.

Dominance-based queries (cont.)

• Top-k dominating queries: objects with the highest dominating ability

New challenges (1)

Massive Streaming data Arrive at high speed and the volume of the data is extremely large.

- Twitter : 140 million users and over 340 million tweets per Day

- 200Mb/sec from a single sensor node for reading of the weather data

- AT&T collects 600-800 Gigabytes of NetFlow data each day

- Square Kilometre Array (SKA) project : a few exabytes (1018 bytes) of data per day for a single beam per square kilometer,

Streaming Algorithm

DBG@UNSW 29

Stream processingEngine

Synopses in Memory

Data Streams

( Approximate ) Answer

One scan only Processing time ( fast ) Synopsis size ( small ) Accuracy ( a good tradeoff with synopsis size )

New Challenges (2)

DBG@UNSW 30

The data may be uncertain for various reasons.

Limits of the measuring devices Noise Delay or loss in data transfer. Privacy Data integration

The uncertainty of the data may be described continuously or discretely.

New Challenges (3)

DBG@UNSW 31

Enriched spatial data

Textual data - Twitter , Weibo, Fourquare

The user profile - age, gender, preference, etc.

Multimedia data - photos, videos

An enormous amount of spatio-textual objects

available in many applications• Online local search

e.g., online yellow pages Social network services

e.g., Facebook, Flickr, Twitter

Spatial-Textual Objects

Spatial keyword search

DBG@UNSW 32

Top k spatial keyword search

p1 (pizza,coffee,sushi)

p3 (pizza,sushi)

p2 (pizza,coffee,steak)

p4 (coffee,sushi)

p5 (pizza,steak,seafood)

pizza,coffee

DBG@UNSW 33

A little bit about BIG Data

• What is big data ?– Four Vs: Value, Velocity, Variety, Verocity

• How Big ?– Even scanning (linear algorithm) not

applicable

• How to handle ?– New computational paradigms

A little bit about BIG Data

• A recent Mckinsey Global Institute report forecasts a serious shortage of data science and engineering professionals in 2018.

• Data scientist: the sexiest job of the 21st century

Thank you!

Questions?