Spatio‐Temporal Analysis of Network Data d R d D l tand Road Developments

Dr Tao Cheng 

Dr Tao ChengDr Tao ChengSenior Lecturer in GeoInformaticsDepartment of Civil, Environmental & Geomatic Engineering, UCL

• Data quality and uncertainty of spatial objectsp j

• Multi-scale spatio-temporal data modelling and analysis Intelligent spatio temporal

[1][3]

• Intelligent spatio-temporal data mining

• Some relevant projects:– 4D GIS for decision support system

[1][ ]– Experimental modelling of changing

activity patterns using GIS (HK) [2]– Spatio-temporal data mining (PRC)

[3][3][2]

Artemis SkarlatidouArtemis SkarlatidouEngD student Department of Computer Science, UCL

Trust in web-based GIS/SDSS Case study: Site Selection of a nuclear waste repository in the UK

How trust can be built and enhanced in the web-based GIS/SDSS context and what elements affect its trustworthiness? - using Human-Computer Interaction Methodsg

Adel Bolbol Fernández PhD Student Department of Civil, Environmental & Geomatic Engineering, UCL

“Modelling the Changing Surroundings of Everyday Life”

Modelling

Modelling the Changing Surroundings of Everyday Life- Travel Mode Detection from Geotagged Data

by producingModelling• Spatio-Temporal Human Activity • & Travel Behaviour

Mode of Transport

by producingActivity Maps

Activity recognition• Diaries• Semi Supervised• Semi-Supervised• Unsupervised

Location InformationLocation Information• Sensor Data GPS,GSM, WiFi

& RFID

Spatio‐Temporal Analysis of Network DataSpatio‐Temporal Analysis of Network Data and Road Developments

Dr Tao Cheng gProf. Benjamin Heydecker , Mr Andy Emmonds, Team

OutlineIntroduction• Introduction– Background, aim and programme

• Methodology – integrated ST Data Mining– Statistical approach– Machining learning– Visualization– Simulation

• Working Plang• Progress

B k dBackground• Large cities are increasingly g g y

crowded - population & mobilityTraffic congestion affects • Traffic congestion affects both the economy and daily life.

• It is difficult and expensive to increase the capacity of the road networkthe road network.

City of London• Traffic levels in the Congestion • Traffic levels in the Congestion

Charging Zone are falling but congestion levels are risingcongestion levels are rising.

• cost of congestion £3 billi - £3 billion per year

• Mayor’s traffic priorities – reduce congestion and smooth traffic flows

• Remove western extension of CC (27/11/2008)

• Olympic Game 2012 Olympic Game 2012 – travel time to London Olympic sites

Why is this?Why is this?•Thought to be due to a reduction in network capacity?•Reallocations of capacity to other uses?R d ti i ili f th t k?

Aim•Reduction in resilience of the network?

• To quantitatively measure road network performancenetwork performance

• To understand causes of traffic congestioncongestion– association between traffic and

interventionsinterventions• traffic flow, speed/journey time• incidents, road works, signal changes and bus , , g g

lane changes

• Case study – London

Ch ll (1) N t k C l itChallenge (1) Network Complexity

1) D i1) Dynamics2) Spatial dependence3) S i l 3) Spatio-temporal

interactions4) H t g it4) Heterogeneity

To understand the traffic congestion in central London

/ /EP/G023212/1

Challenge (2) - Data issues

• massive – 20GB monthly• multi-sourced related to 5 different networks • different scales (density & frequency)• variable data qualityq y• contain conflicts, errors, mistakes and gaps

L d R d N t kLondon Road NetworksCordons Central, Inner, Outer Screenlines

Thames, ,Northern, five radialsfour peripherals

DATA COVERAGE

Traffic Flow SurveysTraffic Flow Surveys

• NMC (National manual count annual data)ATC (A i C ) 20 MB• ATC (Automatic Count) – 20 MB

• different time periods, intervals and accuracy

Traffic speed (and hence journey time) data

• MCOS (Moving Car Observer Surveys) – Centre, Inner, and Outer – least accurate of the datasetsleast accurate of the datasets

• ITIS (GPS vehicle tracking system) - 2GB – major A roads and bus routes in town with 2000 probes

di – medium accuracy• ANPR (Automatic Number Plate Reading) – 6 GB

– main roads in the central and west extensions of CCZs – 5-minute intervals, 5 vehicle groups,– high accuracy– available since March 2008available since March 2008

• At least 5 networks – boundaries do not fully align

LTIS i id t d t d t 20MBLTIS incident and event data - 20MB

• works, hazards, accidents, signal faults, special events, , , , g , p ,breakdowns, security, and other causes

• DfT have all these data as map or as text files

- Minimal, Moderate, Serious or Severe subjective ?subjective ?unrecorded?not geocoded?

not broadcast on the traffic Link website, creating problems in analysis and reporting.

There are uncertainties and gaps in the intervention data

Methodology - ISTDM

What’s new - (1) data-driven top-downTransition in data availability

D i

What s new - (1) data-driven, top-down

Data abundance:• Data scarcity:– High cost– Low volume

• Data abundance:– High volume– Multiple kinds and Low volume

– Intensive validation

psources

– Extensive application

Transition in modelling approach

• Bottom-up:• Top-down:

– PhenomenologicalDescribe system

Bottom up:– Mechanistic– Explicit

representation of – Describe system gross of all behavioural responses

representation of behaviour (origin, destn, model, time …) p

– Direct to objectives

)– System properties

by aggregation

What’s new: (2) integrated space and time

Existing ST analysis methods

What s new: (2) integrated space and time

• time series analysis + spatial correlation

i l i i h i di i

Existing ST analysis methods

• spatial statistics + the time dimension

• time series analysis + artificial neural networkstime series analysis artificial neural networks

ST dependence ≠ space + time

Integrated modelling of ST is needed –

• seamless & simultaneous

• ST-association/autocorrelation• ST-association/autocorrelation

What’s new: (3) Hybrid Approach

bi i l i ith hi

What s new: (3) Hybrid Approach

• combine regression analysis with machine learning

- improve the sensitivity and explanatory power- improve the sensitivity and explanatory power• study the heterogeneity and scale of road performance p

- optimal scale for monitoring• Quantitative assessment of network

fperformance- Sensible decision making & policy evaluation

Principle of ST Modelling

Space-time data = global (deterministic) space-time trends +

Principle of ST Modelling

)()()(

local (stochastic) space-time variations

)()()( tettZ iii +μ= Z(t)=u(t)+e(t)

• - the observation of the data series at spatial location i and at )(z ti

Zi=ui+ei

the observation of the data series at spatial location i and at time t;

• - space-time patterns that explain large-scale deterministic space-time trends and can be expressed as a nonlinear function in

)(i

)(tiμspace time trends and can be expressed as a nonlinear function in space and time.

• - the residual term, a zero mean space-time correlated error that explains small scale stochastic space time variations

)(teithat explains small-scale stochastic space-time variations.

Model 1 STARIMA Spatio Temporal AutoModel 1 - STARIMA - Spatio-Temporal Auto-Regressive Integrated Moving Average

∑ ∑∑∑= = ==

+−−−=p

k

q

l

n

h

hlh

m

h

hkhi

lk

tltWktzWtz1 1 0

)(

0

)( )()()()( εεθφ

(Pfeifer P E and Deutsch S J, 1980)

M d l 1 STARIMA

Our approach – Integrated modelling of ST

Model 1 – STARIMA

∑ ∑∑∑p q n

hm

hlk

tltWktWt )()( )()()()( θφ

• define weights based upon spatial distance and

∑ ∑∑∑= = ==

+−−−=k l h

hlh

h

hkhi tltWktzWtz

1 1 0

)(

0

)( )()()()( εεθφ

• define weights based upon spatial distance and spatial adjacency

id i t• consider anisotropy• able to model spatially continued phenomena

Tao Cheng Jiaqiu Wang Xia Li 2010 A hybrid approach for space-timeTao Cheng, Jiaqiu Wang, Xia Li, 2010 A hybrid approach for space-time series of environmental data, Geographical Analysis (subject to minor revision)

Model 2 ANN Artificial Ne ral Net orksModel 2 - ANN - Artificial Neural Networks

SFNN – spatial interpolation DRNN ti i l i

(Mandic D P and Chambers JA, 2001)

SFNN – spatial interpolation DRNN – time series analysis

( ) a static neuron ( ) neuronb dynamic

∑n

b1)(ˆl( )i)(ˆ∑=

+⋅=1j

jiji bziwz b1)(tzlwz(t)iw)t(z +−⋅+⋅=

• ANN for space-time trend analysis

)),(()(ˆ 0β+β=μ ∑ tifftn

ki )),(()( 01

ββμ ∑=

ffk

ki

Tao Cheng Jiaqiu Wang 2009 Accommodating Spatial Associations inTao Cheng, Jiaqiu Wang, 2009, Accommodating Spatial Associations in DRNN for Space-Time Analysis, Computers, Environment and Urban System, 33, 409-418.

Model 2 STANNModel 2 - STANN

∑ +−⋅+−⋅=n

i)0(

j)1(

jii b1)(tzlw1)(tziw(t)zSpace-Time Neuron ∑=

++1j

ijjii b1)(tzlw1)(tziw(t)zSpace-Time Neuron

• One step implementation of ANN+ STARIMAA d S i i i A• Accommodate ST associations in ANN

• Deal with nonlinearity & heterogeneity in BP learning

Jiaqiu Wang, Tao Cheng, STANN – Modeling Space-Time Series by Artificial Neural Networks, International Journal of Geographical Information Science, under review

3 S SModel 3 – SVM - Support Vector Machines

SVC & SVR (Vapnik et al, 1996)

Model 3 STSVR

J.Q. Wang, T. Cheng*, J. Haworth

Model 3 - STSVR

• Nonlinear Spatio-Temporal Regression by SVM

• Develop ST kernel function• Overcome over-fitting in STANN• Deal with errors• Model nonlinearity & heterogeneity

Jiaqiu Wang, Tao Cheng, James Haworth, Space-Time Kernels, submitted to Spatial Data Handling (SDH) 2010, Hong Kong, May 26-28.

Other methods

• Geographically Weighted Regression (GWR) – -> STWR?

• Permutation Scan Statistics (PSS) – –> STPSS? (or STC)> STPSS? (or STC)

• Exploratory Visualization + ST STEV?– -> STEV?

• Simulation (Multi-scales)– -> STMSS?

James HaworthJames HaworthDepartment of Civil, Environmental & Geomatic Engineering, UCL

Multi-scale analysis of road network performance

• Using spatio-temporal data mining techniques to look for • Using spatio temporal data mining techniques to look for patterns in congestion at varying spatial and temporal scales

• What patterns can be observed in inbound and outbound congestion...– Daily? Weekly? Seasonally?...y y y

• Identification of recurrent and non-recurrent congestion in LondonLondon

/ /EP/G023212/1

Berk AnbaroğluBerk AnbaroğluMphil/PhD Student

Detection of Emerging Spatio-Temporal Detection of Emerging Spatio Temporal Outliers on Network Data

Faulty Sensor Detection

B tt M d lliBetter Modelling

Garavig TanaksaranondGaravig TanaksaranondMphil/PhD Student

Visualization of dynamic traffic dataResearch Objectives

T id i li ti t l ith t ib ti th d t h l b tt

Visualization of dynamic traffic data

To provide visualization tools with contributing methods to help better understanding of traffic data. Visualization techniques should help to cope with the characteristics of traffic data that limit the ability to extract useful information and to understand traffic data.

How?M dif i i d i i h i•Modify existing depiction techniques

•Visualization + Other techniques:D i d i i•Data aggregation and summarization

•Extract patterns prior to visualization (ex. d i i h i )data mining techniques)

Ed ManleyEd ManleyEngD Candidate

Multi-Scale Traffic SimulationMulti Scale Traffic Simulation

Microscopic (Individual)

Mesoscopic (Link)

Macroscopic

Team (July 2009 – June 2012)Team (July 2009 – June 2012)• UCL

– Dr Tao Cheng (PI), Senior Lecturer in GISProf Benjamin Heydecker (Co I) Director of CTS– Prof. Benjamin Heydecker (Co-I), Director of CTS

– Dr Jingxin Dong, Pattern analysis (Congestion modelling)Dr Jiaqiu Wang Prediction (STARIMA STANN)– Dr Jiaqiu Wang, Prediction (STARIMA, STANN)

– Mr James Haworth, Scales in ST (SVM/GWR)– Mr Berk Anbaroglu, Outlier detection (STPSS)

M G i T k d D i i li i– Ms Garavig Tanaksaranond , Dynamic visualization– Mr Ed Manley, EngD, Simulation (Agent-based) – 3 visiting scholars, each 2 months

• TfL RNP&R– Mr Andy Emmonds, Principal Transport Analysty , p p y– Mr Mike Tarrier, Head of RNP&R– Mr Jonathan Turner, Performance Analyst

Working Plan

• Traffic pattern Analysis (recurrent/non recurrent)

Stage 1 -

• Traffic pattern Analysis (recurrent/non-recurrent)– GPS (formerly ITIS, now Traffic Master) - speed– ANPR (Automatic Number Plate Recognition) – journey ANPR (Automatic Number Plate Recognition) journey

time/speed – ATC (Automatic Traffic Counts) – flow

• The problems – how to associate ANPR data with the links defined in ITN map p

(GPS data).– Flow with speed?

Prediction (JW - STARIMA)Outlier (BA – Cluster, STARIMA)Visualization (GT – animation i b d/ b d d)inbound/outbound speed)Patterns (JD - Congestion movement)Scales in ST (JH - Kernel/GWR/SVM)Si l ti (EM fl +b h i )Simulation (EM – flow+behaviour)

Stage 2Stage 2• Cause-effect analysis of congestion (Spring 2010)

– LTIS (London Traffic Information System)• Time, Location, Event Type, Event description, Event

Duration

• Others for validation – Cordons and screen lines

• Geometric description of screen lines, preferably defined in core link map or ITN map.

• Flows crossing screen lines: Time, Traffic Counts, Vehicle g , ,type

– SCOOTTime Link ID Lane ID Traffic Volume Location• Time, Link ID, Lane ID, Traffic Volume, Location

LTIS data

Cordons and screen lines

SCOOT

Stage 3 -Stage 3 • Detailed traffic modelling (later stage)

OD t i ( lib ti f j ti )– OD matrix (calibration of journey time)• Observation Time, Origin, Destination, Number of Trips• the description of traffic zonesp

– Mobile phone data (traffic simulation)• Time, Current HandOff position ID

HandOff position ID Link ID Coordinates( )• HandOff position ID, Link ID, Coordinates(x,y)

GUI: A Web-base Platform for Dynamic Visualization, Simulation, Analysis (OLAP)

Tool Boxes: Integrated Spatio-Temporal Data Mining (Matlab+ ?)

PatternJH/BA

Clustering PatternI i

Tool Boxes: Integrated Spatio-Temporal Data Mining (Matlab+..?)

Clustering2.1 Transition

2.2

InterventionAnalysis

2.3

PerformancePrediction

2.4

ModelUpdating

2.5

STARIMADRNN SVMGWR

Database/Platform(Oracle + ArcGIS)(ANPR, GPS, ITLS, …. based on ITN)

STANDARD Platform Structure

Preliminary Results

St d areaStudy area

London Arterial Road Map

Pattern analysis of journey timePattern analysis of journey time4

in)

(a)2

3

ney

time

(mi

1

Jour

nm

in)

(b)

rney

tim

e (m

Jour

The distribution plot of 33 Mondays journey times link 605during 07:00 to 19:00

Periodic and Space-Time Autocorrelation AnalysisSpace-time Autocorrelation Function after one order non-seasonal differencex 106 (a) R605 Periodogram

Periodic and Space-Time Autocorrelation Analysis

0.2Space-time Autocorrelation Function after 48th order seasonal difference

0 03

0.04

10

15maximum at freq=0.020833

period=48

(a) 0.15

0 01

0.02

0.03

rela

tions

0 0 05 0 1 0 15 0 2 0 25 0 3 0 35 0 4 0 45 0 50

50.1

orre

latio

ns

0 01

0

0.01

me

Aut

ocor

r0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5frequency

1(b) R605. Cumulative Periodogram

0.05

ime

Aut

oco

0 03

-0.02

-0.01

Spa

ce-ti

m

0.5(b)

0

Spa

ce-t

-0.04

-0.03

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.50

-0.05

50 100 150 200 250 300 350 400 450Lag

frequency50 100 150 200 250 300 350 400 450Lag

Prediction results195

200Actual

195

200Actual200

l

200Actual200Actual200Actual

170

175

180

185

190

1955 min prediction

170

175

180

185

190

19510 min prediction

190

195

200Actual

5 min prediction

185

190

19510 min prediction

185

190

19515 min prediction

185

190

195Actual

20 min prediction

150

155

160

165

170

5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45

150

155

160

165

170

5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45175

180

185

175

180

185

175

180

185

175

180

185

5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45

7 8 9 10 7 8 9 10

2009‐Aug‐17 2009‐Aug‐24

5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45

7 8 9 10 7 8 9 10

2009‐Aug‐17 2009‐Aug‐24

200Actual 200

Actual165

170

175

160

165

170

160

165

170

165

170

175

180

185

190

19515 min prediction

175

180

185

190

19520 min prediction

150

155

160

150

155

160

150

155

160

150

155

160

150

155

160

165

170

5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45

150

155

160

165

170150

5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45

7 8 9 10 7 8 9 10

5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45

7 8 9 10 7 8 9 10

5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45

7 8 9 10 7 8 9 10

5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45

7 8 9 10 7 8 9 105 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45

7 8 9 10 7 8 9 10

2009‐Aug‐17 2009‐Aug‐24

5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45 5 25 45

7 8 9 10 7 8 9 10

2009‐Aug‐17 2009‐Aug‐242009‐Aug‐17 2009‐Aug‐242009‐Aug‐17 2009‐Aug‐242009‐Aug‐17 2009‐Aug‐242009‐Aug‐17 2009‐Aug‐24

Figure 1. Delay at 9 am on 12th April 2009g y p

Figure 2. Delay at 9:15am on 12th April 2009

Figure 3. Delay at 9:15 am and 3:45 pm on 12th April 2009

DATA COVERAGE

Wall Mapp

LCAP 15 January 2010 8:00-10:00 am

LCAP(Inbound) 15 January 2010 8:00-10:00 am

Bus lane users

Non bus lane usersNon bus lane users

Isosurface

High delay value (red color)

SideviewSideview

Topview

Traffic Master Data

SSTANDARD Website –http://standard.cege.ucl.ac.uk

•The STANDARD Project website is currently under development.

•It contains information regarding the objectives and progress of the project and details of the various methods that are used.

•There is a STANDARD blogThere is a STANDARD blog which will soon be made available for interested people to leave comments and engage in discussion.

C di t/ i t ( ti ) f R d N t k ?Can we predict/migrate emergence (congestion) of Road Network ?

Understand

Detect M d lModel

Simulation

AcknowledgementsAcknowledgements

National High-tech R&D Program (863 Program)

NSF China