Post on 20-Mar-2018
mobile measurements:the mobile app / OS perspective
Why understanding mobile app performance is hard
cellular performance matters to some apps
•most demanding apps• multiplayer gaming• VoIP• video chat
• somewhat demanding• web search• app interfaces with web services
• reasonably irrelevant• network transfers that aren’t in the critical path
26.March.2014 sharad.agarwal@microsoft.com 2
Internet
“Mitt Romney might vote for Obama as well”
“We recommend Mitt Romney for president”
“I would definitely trustMitt Romney with my money.”
“If I had a nickel for every time Mitt Romney said something stupid I'd be in his tax bracket”
Mitt Romney
MittRating
MittRating
Tweets
26.March.2014 sharad.agarwal@microsoft.com 10
Internet
“Mitt Romney might vote for Obama as well”
“We recommend Mitt Romney for president”
“I would definitely trustMitt Romney with my money.”
“If I had a nickel for every time Mitt Romney said something stupid I'd be in his tax bracket”
Mitt Romney
MittRating
MittRating
Tweets
26.March.2014 sharad.agarwal@microsoft.com 11
Internet
53%
“Mitt Romney might vote for Obama as well”
“We recommend Mitt Romney for president”
“I would definitely trustMitt Romney with my money.”
“If I had a nickel for every time Mitt Romney said something stupid I'd be in his tax bracket”
Mitt Romney
MittRating
MittRating
Tweets
26.March.2014 sharad.agarwal@microsoft.com 12
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 sharad.agarwal@microsoft.com 13
ClickHandler() {
}
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 sharad.agarwal@microsoft.com 14
ClickHandler() {
tweets = HttpGet(url);
}
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 sharad.agarwal@microsoft.com 15
ClickHandler() {
tweets = HttpGet(url);rating = ProcessTweets(tweets);
}
ProcessTweets(tweets){
...}
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 sharad.agarwal@microsoft.com 16
ClickHandler() {
tweets = HttpGet(url);rating = ProcessTweets(tweets);display.Text = rating;
}
ProcessTweets(tweets){
...}
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 sharad.agarwal@microsoft.com 17
ClickHandler() {
tweets = HttpGet(url);rating = ProcessTweets(tweets);display.Text = rating;
}
ProcessTweets(tweets){
...}
Click HandlerStart
Click HandlerEnd
Thread
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 sharad.agarwal@microsoft.com 18
ClickHandler() {
tweets = HttpGet(url);rating = ProcessTweets(tweets);display.Text = rating;
}
ProcessTweets(tweets){
...}
Click HandlerStart
Click HandlerEnd
Thread
User Perceived Delay
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 sharad.agarwal@microsoft.com 19
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...}
MittRating
MittRating
53%
Asynchronous Code
26.March.2014 sharad.agarwal@microsoft.com 21
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...} UI Thread
ClickHandlerEndClickHandler
Start
MittRating
MittRating
53%
User Click
Asynchronous Code
26.March.2014 sharad.agarwal@microsoft.com 22
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...} UI Thread
ClickHandlerEndClickHandler
Start
MittRating
MittRating
System
Async Get Call
53%
User Click
Asynchronous Code
26.March.2014 sharad.agarwal@microsoft.com 23
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...} UI Thread
Background Thread
ClickHandlerEndClickHandler
Start
MittRating
MittRating
System
Async Get Call
ProcessTweets
DownloadCallback
53%
User Click
Asynchronous Code
26.March.2014 sharad.agarwal@microsoft.com 24
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...} UI Thread
Background ThreadUI Dispatch
ClickHandlerEndClickHandler
Start
MittRating
MittRating
System
Async Get Call
ProcessTweets
DownloadCallback
53%
User Click
Asynchronous Code
26.March.2014 sharad.agarwal@microsoft.com 25
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...}
DisplayRating
UI Thread
Background ThreadUI Dispatch
ClickHandlerEndClickHandler
Start
MittRating
MittRating
System
Async Get Call
ProcessTweets
DownloadCallback
53%
User Click
Asynchronous Code
26.March.2014 sharad.agarwal@microsoft.com 26
Apps are highly asynchronous
30 popular apps167,000 transactions from user study
26.March.2014 sharad.agarwal@microsoft.com 27
Apps are highly asynchronous
30 popular apps167,000 transactions from user study
• On average, 19 asynchronous calls per user transaction• On average, 8 parallel threads per user transaction
26.March.2014 sharad.agarwal@microsoft.com 28
Apps are highly asynchronous
For each user transaction, what was the critical path, and did the network matter?
30 popular apps167,000 transactions from user study
• On average, 19 asynchronous calls per user transaction• On average, 8 parallel threads per user transaction
26.March.2014 sharad.agarwal@microsoft.com 29
UI Thread
Background Thread
Background Thread
Background Thread
User Transaction
Tweets
Posts
MittRating
MittRating
47%
26.March.2014 sharad.agarwal@microsoft.com 35
UI Thread
Background Thread
Background Thread
Background Thread
User Transaction
Tweets
Posts
MittRating
MittRating
47%
User Click
26.March.2014 sharad.agarwal@microsoft.com 36
UI Thread
Background Thread
Background Thread
Background Thread
Thread Blocked
User Transaction
Tweets
Posts
MittRating
MittRating
47%
User Click
26.March.2014 sharad.agarwal@microsoft.com 37
UI Thread
Background Thread
Background Thread
Background Thread
Thread Blocked
Fire
User Transaction
Callback
Tweets
PostsTwitter
Process Tweets
MittRating
MittRating
47%
User Click
26.March.2014 sharad.agarwal@microsoft.com 38
UI Thread
Background Thread
Background Thread
Background Thread
Thread Blocked
Fire
Fire
User Transaction
Callback
Callback
Tweets
PostsTwitter
Facebook Process Posts
Process Tweets
MittRating
MittRating
47%
User Click
26.March.2014 sharad.agarwal@microsoft.com 39
UI Thread
Background Thread
Background Thread
Background Thread
Thread WakeupThread Blocked
Fire
Fire
DisplayUser Transaction
Callback
Callback
Tweets
PostsTwitter
Facebook Process Posts
Process Tweets
MittRating
MittRating
47%
User Click
26.March.2014 sharad.agarwal@microsoft.com 40
UI Thread
Background Thread
Background Thread
Background Thread
User Click
Thread WakeupThread Blocked
Fire
Fire
Callback
Callback
Facebook Process Posts
Process Tweets
User Transaction Display
26.March.2014 sharad.agarwal@microsoft.com 41
UI Thread
Background Thread
Background Thread
Background Thread
User Click
Thread WakeupThread Blocked
Fire
Fire
Callback
Callback
Facebook Process Posts
Process Tweets
User Transaction Display
26.March.2014 sharad.agarwal@microsoft.com 42
UI Thread
Background Thread
Background Thread
Background Thread
User Click
Thread WakeupThread Blocked
Fire
Fire
Callback
Callback
Facebook Process Posts
Process Tweets
Critical Path
User Transaction Display
26.March.2014 sharad.agarwal@microsoft.com 43
UI Thread
Background Thread
Background Thread
Background Thread
User Click
Thread WakeupThread Blocked
Fire
Fire
Callback
Callback
Facebook Process Posts
Process Tweets
Critical PathOptimizing the critical path reduces the user perceived delay
User Transaction Display
26.March.2014 sharad.agarwal@microsoft.com 44
UI Thread
Background Thread
Background Thread
Background Thread
User Click
Thread WakeupThread Blocked
Fire
Fire
Callback
Callback
Facebook Process Posts
Process Tweets
Critical PathOptimizing the critical path reduces the user perceived delay
User Transaction Display
26.March.2014 sharad.agarwal@microsoft.com 45
key points
• app pages designed in GUI• async interfaces in code to UI elements• async interfaces to network• tendency to fetch many network objects at launch• fetch‐parse‐fetch pattern• critical path is hard to determine• in which user transactions does network matter?•of those, what caused network delay, if any?
26.March.2014 sharad.agarwal@microsoft.com 46
cellular network complicates this even more
• radio power states
26.March.2014 sharad.agarwal@microsoft.com 47
cellular network complicates this even more
• radio power states• latency varies by load
26.March.2014 sharad.agarwal@microsoft.com 48
cellular network complicates this even more
• radio power states• latency varies by load
26.March.2014 sharad.agarwal@microsoft.com 49
0
0.2
0.4
0.6
0.8
1
120 170 220
empi
rical
CD
F
RTT (ms)
cellular network complicates this even more
• radio power states• latency varies by load• latency varies
over space
26.March.2014 sharad.agarwal@microsoft.com 50
0
0.2
0.4
0.6
0.8
1
0 50 100 150 200
empi
rical
CD
F
RTT difference at 90th percentile (ms)
why am I telling you all this?•wireless BW, latency, RSSI, etc. data is useful for research
• relate it to user experience• this is hard because of how apps are built• need detailed & efficient app instrumentation
•our mobile OS throttles some network behavior• based on app performance need and cellular data limits• research systems to do this on a per app transfer basis, using detailed & lightweight instrumentation
•need detailed, predictive wireless performance info & how individual parts of apps are affected by net perf
26.March.2014 sharad.agarwal@microsoft.com 51
for more details
• cellular performance variations in short timescales• Switchboard paper in ACM MobiSys 2011
• asynchronous nature of apps & perf measurement• AppInsight paper in USENIX OSDI 2012
•network fetching behavior of apps• Procrastinator paper in ACM MobiSys 2014
26.March.2014 sharad.agarwal@microsoft.com 52