Mantis: Automatic Performance Prediction for Smartphone ApplicationsYongin Kwon, Sangmin Lee, Hayoon Yi, Donghyun Kwon, Seungjun Yang, Byung-Gon Chun,Ling Huang, Petros Maniatis, Mayur Naik, Yunheung Paek

USENIX ATC’13

Performance Prediction ProblemPredict the execution time of a

program on a given input before running it.

Two kinds of ApproachesMost existing techniques can be

classified into two broad categories.◦Domain-specific programs,

automatically-extracted features.◦General-purpose programs,

manually-specified features.

MantisA new framework to

automatically predict the performance of general-purpose byte-code programs on given inputs.

Four components:◦Feature instrumentor◦Profiler◦Performance model generator◦Predictor code generator

Architecture

Feature InstrumentorInstruments the program to

collect the values of feature (f1, … , fM) as per feature schemes.

Feature scheme◦Branch counts◦Loop counts◦Method-call counts◦Variable values

Examples

ProfilerOutputs a data set

◦ti: the ith observation of execution time.

◦vi: the ith observation of the vector of M

features.

Niiit 1, v

],...,[ 1 iMii ffv

Performance ModelingPerforms a sparse nonlinear

regression on the feature values and execution time.

Produces a function ◦ is the approximation of

execution time◦ is a subset of

In practice, K << M.

f)'(vf

],...,[' 1 Kffv ],...,[ 1 Mffv

Performance Modeling(Cont.)However, regression with best

subset selection is NP-hard.◦Find the subset of size K that gives

the smallest Residual Sum of Squares(RSS).

◦Discrete optimization problem.

SPORE-FoBaSparse POlynomial REgression –

FoBa.◦A feature from the candidate set is

added into the model if and only if adding it makes the RSS decrease a lot. If the drop is greater than ε.

◦Remove a feature from the active set if deleting it makes the RSS increase the least. If the increment is smaller than ε’.

ExampleDegree-2 polynomial with

◦Expand (1+x1+x2)2 to get 1, x1, x2, x1

2, x1x2, x22.

◦Construct the following function for regression

],[ 21 xxv

225214

21322110)( xxxxxxf v

Predictor code generatorProduce a code snippet, called slice, for each chosen features.

◦Slice: an executable sub-programs that yields the same value v of a feature at a program point p as the given program on all inputs.

Automatically evaluate feature values for each input by executing slices.

Example

Prototype Toolchain

Experiment SetupA machine runs Ubuntu 11.10 64-

bit with a 3.1GHz quad-core CPU, and 8GB of RAM.

A Galaxy Nexus running Android 4.1.2 with dual-core 1.2Ghz CPU and 1GB RAM.

Six CPU-intensive Android applications.◦Each with 1,000 randomly generated

inputs.◦Train the predictor on 100 inputs.

Experimental Results

Features and Models

Effect of the Number of Training Inputs

Compare with Linear Model

Prediction Time of Mantis and PE

Prediction Error of Mantis and BE

Prediction on Different Hardware Platform

Prediction under Background Load

Offline Stage Processing Time

ConclusionMantis is a framework that

automatically generates program performance predictors.◦Combines program slicing and

sparse regression in a novel way.Evaluation shows that the

generated predictors estimate execution time accurately and efficiently for smartphone applications.