Center for Urban Transportation Research | University of South Florida

Performance Evaluation of Transit Data Formats on a Mobile Device

Sean J. Barbeau, Ph.D.Tatu Saloranta

10101

10101

10101

2

Overview

• Why does performance matter?

• Evaluation

• Conclusions

3

Choose an app for real-time info:

A B

SLOW

FAST

4

Performance matters

• Users can perceive time differences of 100ms when waiting for a response[1]

• Slower response => more communication with server => shorter battery life

• For time-sensitive tasks, and apps that are run frequently, we want good performance

[1] R. Jota, A. Ng, P. Dietz, and D. Wigdor, "How Fast is Fast Enough? A Study of the Effects of Latency in Direct-Touch Pointing Tasks," presented at the CHI 2013, Paris, France, 2013.

5

Typical architecture for real-time apps

Mobile device requests info from server, server responds

Request(StopId=1)

Response(Arrivals)

6

Typical architecture for real-time apps

Request(StopId=1)

Response(Arrivals)

• What are the response field names?• How is the response encoded?

7

Response Contents

• Field names– Define the names

of data elements– Usually dictated by

(defacto) standards• GTFS• GTFS-realtime• SIRI• TCIP

Bus Stop 1

stop_id

stopId

stop_id

stopId

Field names

8

Response Contents

• Encoding– Defines how the

data is formatted

Bus Stop 1

<stop_id>1</stop_id>

<stopId>1</stopId>

stop_id

stopId

XML

Field names Encodings

JSON

{stop_id: 1}

{stopId: 1}• Focus – XML vs. JSON

encoding• Use MTA Bus Time SIRI

StopMonitoring REST API [1]

[1] http://bustime.mta.info/wiki/Developers/SIRIStopMonitoring

9

Cold vs. Warm starts

• Cold start = first execution

• Warm start = subsequent executions

Initialize Make HTTP Request

Parse Response

Make HTTP Request

Parse Response

No initialization required for warm starts

10

Results

0

2

4

6

8

10

12

14

16

18

20

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Elap

sed

Tim

e (s

)XML vs. JSON Parsing Time - All Requests

JSON XML

Request #

Cold Start

11

Min.

Max

.Avg

.

50th perce

ntile

68th perce

ntile

95th perce

ntile

Std dev

.0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

JSON XML

Elap

sed

Tim

e (m

s)XML vs. JSON Parsing Time –

Samsung Galaxy S3

• ~4.3 times longer to parse the first response using XML

• Cold start time is critical -application state is often destroyed when user multitasks (checks email, etc.) in mobile apps

• Samsung Galaxy S5 – similar results

-Using Jackson 2.1.2-Using MTA SIRI REST API StopMonitoring

-See http://goo.gl/EhYSl for details

12

How can we improve cold starts?

• Cold start = first execution

• Instead of initializing, read prev. persisted data

Initialize Make HTTP Request

Parse Response

Read cached data structures

Make HTTP Request

Parse Response

“Pseudo-warm start”

13

0

5

10

15

20

25

Min. Max. Avg. 50th percentile 68th percentile 95th percentile Std dev.

Elap

sed

Tim

e (s

)

XML - Cold, Pseudo-Warm, and Warm Starts - Summary(n = 30)

Cold Starts Pseudo-Warm Starts Warm Starts

14

0

1

2

3

4

5

6

Min. Max. Avg. 50th percentile 68th percentile 95th percentile Std dev.

Elap

sed

Tim

e (s

)

JSON - Cold, Pseudo-Warm, and Warm Starts - Summary(n = 30)

Cold Starts Pseudo-Warm Starts Warm Starts

Noticeable difference to user!

15

Other benefits of Pseudo-warm starts

• Cold start

• Reading cache can begin when app starts!

Initialize Make HTTP Request

Parse Response

Read cached data structures

Make HTTP Request

Parse Response

Can be hidden from user!

Must be done at the time of request

16

0

1

2

3

4

5

6

Min. Max. Avg. 50th percentile 68th percentile 95th percentile Std dev.

Elap

sed

Tim

e (s

)

JSON - Cold, Pseudo-Warm without cache read time, and Warm Starts - Summary

(n = 30)

Cold Starts Pseudo-Warm Starts Warm Starts

17

Future work – XML improvements?

0

1

2

3

4

5

6

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

Elap

sed

Tim

e (s

)

XML vs. JSON Parsing Time - Cached XML List Wrapper Deserializers

JSON XML

Request #

Samsung Galaxy S5 using Jackson v2.4.1

1818

4 15 30 600

5

10

15

20

25

30

7.02

12.68

16.76

19.37

9.44

17.77 18.62

24.01

Using HTTP Increases Battery Life by 28% on Avg.

JAX-RPC HTTP-POSTInterval Between Wireless Transmissions (s)

Batt

ery

Life

(hou

rs)

SOAP

Energy overhead of XML

Motorola i580 phone -See http://goo.gl/hq6nE for details

19

Conclusions

• Performance matters (time and battery life)• Use efficient formats with mobile devices

(e.g., RESTful web services with JSON)• Further optimizations (pseudo-warm starts)

can help hide latency from users• Future work – further XML vs. JSON evaluation

20

Thanks!

Sean J. Barbeau, Ph.D.barbeau@cutr.usf.edu813.974.7208Center for Urban Transportation ResearchUniversity of South Florida

Tatu Salorantatatu@fasterxml.comFasterXML, LLC