Mantis: Automatic Performance Prediction for Smartphone Applications
description
Transcript of Mantis: Automatic Performance Prediction for Smartphone Applications
Mantis: Automatic Performance Prediction for Smartphone ApplicationsYongin Kwon, Sangmin Lee, Hayoon Yi, Donghyun Kwon, Seungjun Yang, Byung-Gon Chun,Ling Huang, Petros Maniatis, Mayur Naik, Yunheung Paek
USENIX ATC’13
Performance Prediction ProblemPredict the execution time of a
program on a given input before running it.
Two kinds of ApproachesMost existing techniques can be
classified into two broad categories.◦Domain-specific programs,
automatically-extracted features.◦General-purpose programs,
manually-specified features.
MantisA new framework to
automatically predict the performance of general-purpose byte-code programs on given inputs.
Four components:◦Feature instrumentor◦Profiler◦Performance model generator◦Predictor code generator
Architecture
Feature InstrumentorInstruments the program to
collect the values of feature (f1, … , fM) as per feature schemes.
Feature scheme◦Branch counts◦Loop counts◦Method-call counts◦Variable values
Examples
ProfilerOutputs a data set
◦ti: the ith observation of execution time.
◦vi: the ith observation of the vector of M
features.
Niiit 1, v
],...,[ 1 iMii ffv
Performance ModelingPerforms a sparse nonlinear
regression on the feature values and execution time.
Produces a function ◦ is the approximation of
execution time◦ is a subset of
In practice, K << M.
f)'(vf
],...,[' 1 Kffv ],...,[ 1 Mffv
Performance Modeling(Cont.)However, regression with best
subset selection is NP-hard.◦Find the subset of size K that gives
the smallest Residual Sum of Squares(RSS).
◦Discrete optimization problem.
SPORE-FoBaSparse POlynomial REgression –
FoBa.◦A feature from the candidate set is
added into the model if and only if adding it makes the RSS decrease a lot. If the drop is greater than ε.
◦Remove a feature from the active set if deleting it makes the RSS increase the least. If the increment is smaller than ε’.
ExampleDegree-2 polynomial with
◦Expand (1+x1+x2)2 to get 1, x1, x2, x1
2, x1x2, x22.
◦Construct the following function for regression
],[ 21 xxv
225214
21322110)( xxxxxxf v
Predictor code generatorProduce a code snippet, called slice, for each chosen features.
◦Slice: an executable sub-programs that yields the same value v of a feature at a program point p as the given program on all inputs.
Automatically evaluate feature values for each input by executing slices.
Example
Prototype Toolchain
Experiment SetupA machine runs Ubuntu 11.10 64-
bit with a 3.1GHz quad-core CPU, and 8GB of RAM.
A Galaxy Nexus running Android 4.1.2 with dual-core 1.2Ghz CPU and 1GB RAM.
Six CPU-intensive Android applications.◦Each with 1,000 randomly generated
inputs.◦Train the predictor on 100 inputs.
Experimental Results
Features and Models
Effect of the Number of Training Inputs
Compare with Linear Model
Prediction Time of Mantis and PE
Prediction Error of Mantis and BE
Prediction on Different Hardware Platform
Prediction under Background Load
Offline Stage Processing Time
ConclusionMantis is a framework that
automatically generates program performance predictors.◦Combines program slicing and
sparse regression in a novel way.Evaluation shows that the
generated predictors estimate execution time accurately and efficiently for smartphone applications.