Using Topic-Based Publish/Subscribe for Managing Real Time GPS Streams

Marlon Pierce, Galip Aydin, Zhigang QiCommunity Grids Lab

Indiana University

1

California Real Time Network

Network Data Rates Message Format

Time RYO ASCII GML

CRTN GPS Site Positions(9 Stations)

1 second 1.5KB 4.03KB 48.7KB

1 hour 5.31MB 14.18MB 171.31MB

1 day 127.44MB 340.38MB 4.01GB

1 month 3.8GB 9.97GB 123.3GB

1 year 45.8GB 119.67GB 1.41TB

Entire SCIGN Network (250

stations)1year 1.23TB 16.18TB 160TB

Continuous GPS Stations (CGPS) are depicted as triangles while the Real-Time stations are represented as circles. Image is obtained from SOPAC GPS Explorer at http://sopac.ucsd.edu/projects/realtime

How does one manage all the data generated by the 85 stations? How can you get just the data you want?

Note this is fundamentally different from traditional request/response style Web Services.

RDAHMM: GPS Time Series SegmentationSlide Courtesy of Robert Granat, JPL

Complex data with subtle signals is difficult for humans to analyze, leading to gaps in analysis

HMM segmentation provides an automatic way to focus attention on the most interesting parts of the time

GPS displacement (3D) length two years.

Divided automaticallyby HMM into 7 classes.

Features:• Dip due to aquifer

drainage (days 120-250)

• Hector Mine earthquake (day 626)

• Noisy period at end of time series

Use Case - GPS Sensors GPS is used to identify long-term tectonic deformation and static

displacements. SCIGN has 250 Real-Time GPS Stations. SOPAC GPS networks:

8 networks for 80 stations produce 1Hz high resolution data. Socket based real-time binary-RYO format access is available. We developed filters to provide multiple format (RYO, ASCII, GML) real-

time streaming access. OHIO principle (a general principle required by DOD) and chain of filters.

Our Architecture Uses publish/subscribe based NaradaBrokering for managing real-time

GPS streams Utilizes topics for hierarchical organization of the sensors Deploys successive data filters ranging from format translators to data

analysis codes Could potentially be used to run RDAHMM clones to monitor state

changes in the entire GPS network We are partner in a pioneering project to use the real-time GPS data for

the first time in this context.

4

Processing Real-Time GPS Streams

5

Raw Data

70107010

70117011

70127012

RYOPorts

NB Server

ScrippsRTD

Server

ScrippsRTD

Server

Raw Data

A Complete Sensor Message Processing Path, including a data analysis application.

GPS Networks

Application Integration with Real-Time Filters

Station Monitor Filter records real-time positions for 10 minutes and calculates position changes

Graph Plotter Application creates visual representation of the positions.

RDAHMM Filter records real-time positions for 10 minutes and invokes RDAHMM application which determines state changes in the XYZ signal.

Graph Plotter Application creates visual representation of the RDAHMM output.

6

GIS Grid Usage Model – Earthquake Science Supporting geophysical repositories and real-time

sensors is essential To analyze a typical earthquake it is important to access

to precise measurements of the initial earthquakes and aftershocks

To support earthquake forecasting and the time and spatial positions of the forecasts PI can be used with existing data RDAHMM can be used with the real-time data

Earth Science field is moving from a previously data poor field to a data rich world. We will have thousands of sensors spread around the world. (i.e. GPS sensors, InSAR satellites)

7

Recording and Replaying Sensor Streams

Filters can be used to record and replay scenarios, such as Earthquakes in GPS case.

We developed RYO Recorder and RYO Publisher Filters.

The RYO Recorder creates daily archives of the GPS Streams.

RYO Publisher can be used to play daily or certain segments of the records.

We replayed the 2004 Southern California Earthquake using Parkfield GPS network archive

These filters are used in the performance and scalability tests.

8

SensorGrid Performance Tests Two Major Goals: System Stability and

Scalability Ensuring stability of the distributed Filter Services

for continuous operation. Finding the maximum number of publishers

(sensors) and clients that can be supported with a single broker.

Investigate if system scales for larger number of sensors and clients.

9

Test Methodology

The test system consists of a NaradaBrokering server and a three-filter chain for publishing, converting and receiving RYO messages.

We take 4 timings to calculate mean end-to-end delivery times of GPS measurements.

The tests were run at least for 24 hours. GridFarm001-008 servers are used in these tests.

Ttransfer = (T2 – T1) + (T4 – T3)

10

NB Server

1

1 2

3

4

1- System Stability Test The basic system with

three filters and one broker.

The figure shows average results for every 30 minutes.

The average transfer time shows the continuous operation does not degrade the system performance.

11

2 – Multiple Publishers Test

We add more GPS networks by running more publishers. The results show that 1000 publishers can be supported with no

performance loss. This is an operating system limit.12

Topic 1A

Topic 1B

Topic 2

Topic n

3 – Multiple Clients Test

We add more clients by running multiple Simple Filters which subscribe to the same ASCII topic.

The system can support as many as 1000 clients with very low performance decrease.

13

Topic 1A

Topic 1B

1000 Clients

Extending Scalability

The limit of the basic system appears to be 1000 clients or publishers.

This is due to an Operating System restriction of open file descriptors (1024 for Red Hat Linux) which can be increased by changing OS parameters.

To overcome this limit we create NaradaBrokering networks with linking multiple brokers. NB supports scalable linkage of the brokers for building tree like architectures.

We run 2 brokers to support 1500 clients. Number of brokers can be increased indefinitely, so we

can potentially support any number of publishers and subscribers.

14

4 – Multiple Brokers Test NaradaBrokering allows creation of

Broker networks. We create a two-broker network. Messages published to first broker can

be received from the second broker. We take timings on each broker. We connect 750 clients to each broker

and run for 24 hours. We chose 750 clients to stay well below the saturation limit.

The results show that the performance is very good and similar to single broker test.

15

NB Server

1

NB Server

2

Topic 1A

Topic 1B

Topic 1B

NB Server

2

4 – Multiple Brokers Test

16

Real-Time Filters Test Results

The RYO Publisher filter runs at 1Hz and publishes 24-hour archive of the CRTN_01 GPS network, which contains 9 GPS stations.

The single broker configuration can support 1000 clients or publishers (GPS networks - 9000 individual stations).

The system can be scaled up by creating NaradaBrokering broker networks.

Message order was preserved in all tests.

17

Acknowledgement Galip Aydin: Designer and implementer ZhiGang Qi: SensorGrid Performance Tests We thank Prof. Yehuda Bock and his group at SIO for

their help with real-time GPS data streams. Robert Granat, JPL: RDAHMM developer

18