Balancing Interactive Performance and Budgeted Resources in Mobile Applications
-
Upload
kennan-jennings -
Category
Documents
-
view
27 -
download
0
description
Transcript of Balancing Interactive Performance and Budgeted Resources in Mobile Applications
Brett Higgins 4
The Most Precious Computational Resource
“The most precious resource in a computer system is no longer its processor, memory, disk, or network, but rather human attention.”
– Gollum Garlan et al.(12 years ago)
Brett Higgins 6
Balancing these tradeoffs is difficult!
• Mobile applications must:• Select the right network for the right task• Understand complex interactions between:
• Type of network technology• Bandwidth/latency• Timing/scheduling of network activity• Performance• Energy/data resource usage
• Cope with uncertainty in predictions
Opportunity:systemsupport
Brett Higgins 7
Spending Principles
• Spend resources effectively.
• Live within your means.
• Use it or lose it.
Brett Higgins 8
Thesis Statement
By:• Providing abstractions to simplify multinetworking,• Tailoring network use to application needs, and• Spending resources to purchase reductions in delay,
Mobile systems can help applications significantly improve user-visible performancewithout exhausting limited energy & data resources.
Brett Higgins 9
Roadmap
• Introduction• Application-aware multinetworking• Intentional Networking
• Purchasing performance with limited resources• Informed Mobile Prefetching
• Coping with cloudy predictions• Meatballs
• Conclusion
Brett Higgins 10
Diversity of networks
Diversity of behavior
Email• Fetch messages
The Challenge
YouTube•Upload video
Match traffic to available networks
Brett Higgins 11
Current approaches: two extremes
All details hidden All details exposed
Result: mismatched Result: hard for traffic applications
Please insert
packets
Brett Higgins 12
Solution: Intentional Networking
• System measures available networks
• Applications describe their traffic
• System matches traffic to networks
Brett Higgins 13
Abstraction: Multi-socket
• Multi-socket: virtual connection• Analogue: task scheduler• Measures performance of each alternative• Encapsulates transient network failure
Client Server
Brett Higgins 14
Abstraction: Label
• Qualitative description of network traffic• Interactivity: foreground vs. background• Analogue: task priority
Client Server
Brett Higgins 15
Evaluation Results: Email
• Vehicular network trace - Ypsilanti, MI
0
10
20
30
40
50
60
70
On-demand fetch time
Best band-widthBest latency
Aggregate
Intentional Networking
Tim
e (s
econ
ds)
0
50
100
150
200
250
300
350
400BG fetch time
Tim
e (s
econ
ds)
3%
7x
Brett Higgins 16
Intentional Networking: Summary
• Multinetworking state of the art• All details hidden: far from optimal• All details exposed: far too complex
• Solution: application-aware system support• Small amount of information from application• Significant reduction in user-visible delay• Only minimal background throughput overhead
• Can build additional services atop this
Brett Higgins 17
Roadmap
• Introduction• Application-aware multinetworking• Intentional Networking
• Purchasing performance with limited resources• Informed Mobile Prefetching
• Coping with cloudy predictions• Meatballs
• Conclusion
Brett Higgins 18
Mobile networks can be slow
$#&*!!
No need for profanity,
Brett.
Data fetching is slowUser: angry
Fetch time hiddenUser: happy
With prefetchingWithout prefetching
Brett Higgins 19
Mobile prefetching is complex
• Lots of challenges to overcome• How do I balance performance, energy, and cellular data?• Should I prefetch now or later?• Am I prefetching data that the user actually wants?• Does my prefetching interfere with interactive traffic?• How do I use cellular networks efficiently?
• But the potential benefits are large!
Brett Higgins 22
Informed Mobile Prefetching
• Prefetching as a system service• Handles complexity on behalf of users/apps• Apps specify what and how to prefetch• System decides when to prefetch
Application
IMP
prefetch()
Brett Higgins 23
Informed Mobile Prefetching
• Tackles the challenges of mobile prefetching• Balances multiple resources via cost-benefit analysis• Estimates future cost, decides whether to defer• Tracks accuracy of prefetch hints• Keeps prefetching from interfering with interactive traffic• Considers batching prefetches on cellular networks
Brett Higgins 25
Balancing multiple resources
• Performance (user time saved)• Future demand fetch time• Network bandwidth/latency
• Battery energy (spend or save)• Energy spent sending/receiving data• Network bandwidth/latency• Wireless radio power models (powertutor.org)
• Cellular data (spend or save)• Monthly allotment• Straightforward to track
3 GB$$$
Brett Higgins 26
Weighing benefit and cost
• IMP maintains exchange rates• One value for each resource• Expresses importance of resource
• Combine costs in common currency• Meaningful comparison to benefit
• Adjust over time via feedback• Goal-directed adaptation
Joules Bytes
Seconds Seconds
Brett Higgins 27
Users don’t always want what apps ask for
• Some messages may not be read• Low-priority• Spam
• Should consider the accuracy of hints• Don’t require the app to specify it• Just learn it through the API
• App tells IMP when it uses data• (or decides not to use the data)
• IMP tracks accuracy over time
Brett Higgins 28
Evaluation Results: EmailTi
me
(sec
onds
)
Average email fetch time500
400
300
200
100
0
8
7
6
5
4
3
2
1
0
Energy usage
Ener
gy (J
)
3G d
ata
(MB)
3G data usage5
4
3
2
1
0
Budgetmarker
~300ms
2-8x
Less energy than all others
(including WiFi-only!)
2x
Only WiFi-only used less 3G data(but…)
IMP meets all resource goalsOptimal
(100% hits)
Brett Higgins 29
IMP: Summary
• Mobile prefetching is a complex decision• Applications choose simple, suboptimal strategies• Powerful mechanism: purchase performance w/ resources
• Prefetching as a system service• Application provides needed information• System manages tradeoffs (spending vs. performance)• Significant reduction in user-visible delay• Meets specified resource budgets
• What about other performance/resource tradeoffs?
Brett Higgins 30
Roadmap
• Introduction• Application-aware multinetworking• Purchasing performance with limited resources• Coping with cloudy predictions• Motivation• Uncertainty-aware decision-making (Meatballs)
• Decision methods• Reevaluation from new information
• Evaluation
• Conclusion
Brett Higgins 31
Mobile Apps & Prediction
• Mobile apps rely on predictions of:• Networks (bandwidth, latency, availability)• Computation time
• These predictions are often wrong• Networks are highly variable• Load changes quickly
• Wrong prediction causes user-visible delay• But mobile apps treat them as oracles!
Brett Higgins 32
Example: code offload
99.9%
0.1%
Response time
10 sec100 sec
Likelihood of 100 sec response time0.1% 0.0001%
Brett Higgins 33
Example: code offload
Elapsed timeExpected response time
Expected response time (redundant)
0 sec10.09 sec10.00009 sec
99.9%
0.1%
Response time
10 sec100 sec
Brett Higgins 34
Example: code offload
Elapsed timeExpected response time
Expected response time (redundant)
9 sec1.09 sec1.00909 sec
99.9%
0.1%
Response time
10 sec100 sec
Brett Higgins 35
Example: code offload
Elapsed timeExpected response time
Expected response time (redundant)
11 sec89 sec10.09 sec
99.9%
0.1%
Response time
10 sec100 sec
Conditionalexpectation
Brett Higgins 36
Spending for a good cause
• It’s okay to spend extra resources!• …if we think it will benefit the user.
• Spend resources to cope with uncertainty• …via redundant operation
• Quantify uncertainty, use it to make decisions• Benefit (time saved)• Cost (energy + data)
✖
Brett Higgins 37
Meatballs
• A library for uncertainty-aware decision-making• Application specifies its strategies• Different means of accomplishing a single task• Functions to estimate time, energy, cellular data usage
• Application provides predictions, measurements• Allows library to capture error & quantify uncertainty
• Library helps application choose the best strategy• Hides complexity of decision mechanism• Balances cost & benefit of redundancy
Brett Higgins 38
Roadmap
• Introduction• Application-aware multinetworking• Purchasing performance with limited resources• Coping with cloudy predictions• Motivation• Uncertainty-aware decision-making (Meatballs)
• Decision methods• Reevaluation from new information
• Evaluation
• Conclusion
Brett Higgins 39
Deciding to operate redundantly
• Benefit of redundancy: time savings• Cost of redundancy: additional resources
• Benefit and cost are expectations• Consider predictions as distributions, not spot values
• Approaches:• Empirical error tracking• Error bounds• Bayesian estimation
Empirical error tracking
• Compute error upon new measurement• Weighted sum over joint error distribution• For multiple networks:
• Time: min across all networks• Cost: sum across all networks
ε(BP2)
.5 .6 .7 .8 .9 1 1.1 1.2 1.3 1.4
ε(BP1)
.5 .6 .7 .8 .9 1 1.1 1.2 1.3 1.4
40
observedpredicted
Brett Higgins
error =
Error bounds
• Range + p(next value is in range)• Student’s-t prediction interval
• Calculate bound on time savings of redundancy
9876543210
BP1 BP2
Band
wid
th (M
bps)
Network bandwidth
9876543210
T1 T2
Tim
e (s
econ
ds)
Time to send 10Mb
Max time savingsfrom redundancy
Predictor says: Use network 2
41Brett Higgins
Error bounds
• Calculate bound on net gain of redundancy max(benefit) – min(cost) = max(net gain)
• Use redundancy if max(net gain) > 0
9876543210
T1 T2
Tim
e (s
econ
ds)
Time to send 10Mb
Max time savingsfrom redundancy
Predictor says: Use network 2
9876543210
BP1 BP2
Band
wid
th (M
bps)
Network bandwidth
42Brett Higgins
Bayesian Estimation
• Basic idea:• Given a prior belief about the world,• and some new evidence,• update our beliefs to account for the evidence,
• AKA obtaining posterior distribution
• using the likelihood of the evidence
• Via Bayes’ Theorem: posterior = likelihood * prior p(evidence) Normalization factor;
ensures posterior sums to 1
43Brett Higgins
Bayesian Estimation
• Example: “will it rain tomorrow?”• Prior: historical rain frequency• Evidence: weather forecast (simple yes/no)• Posterior: believed rain probability given forecast• Likelihood:
• When it will rain, how often has the forecast agreed?• When it won’t rain, how often has the forecast agreed?
• Via Bayes’ Theorem: posterior = likelihood * prior p(evidence) Normalization factor;
ensures posterior sums to 1
44Brett Higgins
Bayesian Estimation
• Applied to Intentional Networking:• Prior: bandwidth measurements• Evidence: bandwidth prediction + implied decision• Posterior: new belief about bandwidths• Likelihood:
• When network 1 wins, how often has the prediction agreed?• When network 2 wins, how often has the prediction agreed?
• Via Bayes’ Theorem: posterior = likelihood * prior p(evidence) Normalization factor;
ensures posterior sums to 1
45Brett Higgins
Brett Higgins 46
Decision methods: summary
• Empirical error tracking• Captures error distribution accurately• Computationally intensive
• Error bounds• Computationally cheap• Prone to overestimating uncertainty
• Bayesian• Computationally cheap(er than brute-force)• Appears more reliant on history
Brett Higgins 47
Reevaluation: conditional distributions
99.9%
0.1%
Response time
10 sec100 sec
Decision
Elapsed Time
One server Two servers
0 10 …
Brett Higgins 48
Evaluation: methodology
• Network trace replay, energy & data measurement• Same as IMP
• Metric: weighted cost function• time + cenergy * energy + cdata * data
Low-cost Mid-cost High-cost
cenergy 0.00001 0.0001 0.001
Energy per 1 second delay reduction
100 J 10 J 1 J
Battery life reduction under average use (normally 20 hours)
6 min 36 sec 3.6 sec
Brett Higgins 49
Evaluation: IntNW, walking traceW
eigh
ted
cost
(nor
m.)
No cost Low cost Mid cost High cost0
0.20.40.60.8
11.21.41.61.8
2
2x
24%
Low-resource strategies improve
Meatballs matches the best strategy
Error boundsleans towardsredundancy
Simple Meatballs
Brett Higgins 50
Evaluation: PocketSphinx, server load
No cost Low cost Mid cost High cost0
0.2
0.4
0.6
0.8
1
1.2
1.4
23%
Meatballs matches the best strategy
Simple Meatballs
Error boundsleans towardsredundancy
Wei
ghte
d co
st (n
orm
.)
Brett Higgins 51
Meatballs: Summary
• Uncertainty in predictions causes user-visible delay• Impact of uncertainty is significant• Considering uncertainty can improve performance
• Library can help applications consider uncertainty• Small amount of information from application• Library hides complex decision process• Redundant operation mitigates uncertainty’s impact• Sufficient resources are commonly available
• Spend resources to purchase delay reduction
Brett Higgins 52
Conclusion
• Human attention is the most precious resource• Tradeoffs between performance, energy, cellular data• Difficult for applications to get right• Necessitates system support – proper abstractions
• We provide abstractions for:• Using multiple networks effectively• Budgeted resource spending• Hedging against uncertainty with redundancy
• Overall results• Improved performance without overspending resources
Brett Higgins 54
Future work
• Intentional Networking• Automatic label inference• Avoid modifying the server side (generic proxies)
• Predictor uncertainty• Better handling of idle periods
• Increase uncertainty estimate as measurements age• Add periodic active measurements?
Brett Higgins 55
Abstraction: Atomicity & Ordering Constraints
• App specifies boundaries, partial ordering• Receiving end enforces delivery atomicity, order
ServerChunk 1
Chunk 3
Chunk 2use_data()
Brett Higgins 56
Evaluation Results: Vehicular Sensing
• Trace #2: Ypsilanti, MI
0
1
2
3
4
5
6
7
Urgent update time
Best band-width
Best latency
Aggregate
Intentional Networking
Tim
e (s
econ
ds)
0
1
2
3
4
5
6
7
8BG throughput
Thro
ughp
ut (K
B/se
c)
48%
6%
Brett Higgins 57
Evaluation Results: Vehicular Sensing
• Trace #2: Ypsilanti, MI
0
1
2
3
4
5
6
7
Urgent update timeBest band-width
Best latency
Aggregate
Intentional Networking
Intentional Networking, two-process
Tim
e (s
econ
ds)
0
1
2
3
4
5
6
7
8
BG throughput
Thro
ughp
ut (K
B/se
c)
Brett Higgins 58
IMP Interface
• Data fetching is app-specific• App passes a fetch callback• Receives a handle for data
• App provides needed info• When prefetch is desired• When demand fetch occurs• When data is no longer wanted
Application
IMP
prefetch()
Brett Higgins 59
How to estimate the future?
• Current cost: straightforward• Future cost: trickier• Not just predicting network conditions• When will the user request the data?• Simplify: use average network conditions
• Average bandwidth/latency of each network• Average availability of WiFi
• Future benefit• Same method as future cost
Brett Higgins 60
Balancing multiple resources
• Cost/benefit analysis• How much value can the resources buy?• Used in disk prefetching (TIP; SOSP ‘95)
• Prefetch benefit: user time saved• Prefetch cost: energy and cellular data spent• Prefetch if benefit > cost• How to meaningfully weigh benefit and cost?
Error bounds
• How to calculate bounds?• Chebyshev’s inequality?
• Simple, used before; provides loose bounds• Bounds turned out to be too loose
• Confidence interval?• Bounds true value of underlying distribution• Quantities (e.g., bandwidth) neither known nor fixed
• Prediction interval• Range containing the next value (with some probability)• Student’s-t distribution, alpha=0.05
61Brett Higgins
Brett Higgins 62
How to decide which network(s) to use?
• First, compute the time on the “best” network T1 = Σ s/B1 * posterior(B1| BP1 > BP2)
• Next, compute the time using multiple networks Tall = Σ min(s/B1, s/B2) * posterior(B1, B2 | BP1 > BP2)
• Finally, do the same with resource costs.• If cost < benefit, use redundancy
Tall = Σ min(s/B1, s/B2) * posterior(B1, B2 | BP1 > BP2) L(BP1 > BP2|B1, B2) * prior(B1, B2)
p(BP1 > BP2)
Here is where each component comes from.
= Σ s/max(B1, B2) *
1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 10
B1 = 1
B2 = 4
B1 = 8
B1 = 9
B2 = 5 B2 = 6
likelihood(BP1 > BP2|B1, B2)
p(BP1 > BP2) p(BP1 < BP2)
prior(B1)
prior(B2)
Brett Higgins 64
The weight of history
• A measurement’s age decreases its usefulness• Uncertainty may have increased or decreased• Meatballs gives greater weight to new observations• Brute-force, error bounds: decay weight on old samples• Bayesian: decay weight in prior distributions
Brett Higgins 65
Reevaluating redundancy decisions
• Confidence in decision changes with new information• Suppose we decided not to transmit redundantly• Lack of response is new information
• Consider conditional distributions of predictions• Meatballs provides interface for:• Deferred re-evaluation• “Tipping point” calculation: when will decision change?
Brett Higgins 66
Applications
• Intentional Networking• Bandwidth, latency of multiple networks• Energy consumption of network usage• Reevaluation (network delay/change)
• Speech recognition (PocketSphinx)• Local/remote execution time• Local CPU energy model• For remote execution: network effects• Reevaluation (network delay/change)
Brett Higgins 67
Evaluation: IntNW, driving trace
No cost Low cost Mid cost High cost0
0.5
1
1.5
2
Not much benefitfrom using WiFi
Simple Meatballs
Wei
ghte
d co
st (n
orm
.)