© 2013 IBM Corporation A. Craik, C Black, P Doyle 4-Nov-14 Mincer: a distributed automated problem...
-
Upload
christine-fields -
Category
Documents
-
view
223 -
download
0
description
Transcript of © 2013 IBM Corporation A. Craik, C Black, P Doyle 4-Nov-14 Mincer: a distributed automated problem...
© 2013 IBM Corporation
A. Craik, C Black, P Doyle 4-Nov-14
Mincer: a distributed automated problem determination tool
© 2013 IBM Corporation
Debugging Optimizing Compilers
Nearly every optimizing transformation is optional
Adopt the scientific method – run experiments varying compilation control parameters to find failure
Problems
Time consuming & tedious to configure & run
Resource constraints limit number of experiments
Experience needed to choose the best experiments
Environment may cause problem to occur intermittently
Resources wasted repeatedly running the same experiments
© 2013 IBM Corporation
Objectives
Tool to run tests in a distributed machine farm
Distributed experiment selection and execution logic for scalability
Configurability & extensibility to support new transformations and new debugging techniques
Reduce resource waste – gain knowledge from every experiment run
Tolerate false-positive & false-negative results and handle variable failure rates
© 2013 IBM Corporation
Problem Conceptualization
A problem determination tactic aims to isolate a problem by varying an experimental parameter and observing the effect
Problem has two discrete components:– Deciding which problem determination tactics to use– Applying chosen problem determination tactics to failure
Each experimental parameter can be mapped back to an integer search space
Mathematically model the selection of experiments and the results they produce
© 2013 IBM Corporation
Mincer Architecture
mincer.pl
DataLayer
Sequential Subset
LastOptIndexOptLevel
TacticExecutionEngine
GatherLogs
Solvers
Tactics
Results DB
LastOptSubIndexLimitFile
© 2013 IBM Corporation
Experiment Selection Logic
© 2013 IBM Corporation
The Numerical Solvers
Solvers use Bayesian Inference to update the underlying hyperparameters of the experiments
Solvers use Information Theory to make optimal test parameter choices
Solve to a user-defined confidence level (-c option)
Currently using 2 solvers: - sequential (contiguous subranges from 1 to some n)
- subset (unordered subsets)
Solvers also track failure rate to guide experiment selection
© 2013 IBM Corporation
Sequential Solver
© 2013 IBM Corporation
Sequential Solver
© 2013 IBM Corporation
Sequential Solver
© 2013 IBM Corporation
Sequential Solver
© 2013 IBM Corporation
Sequential Solver
© 2013 IBM Corporation
Sequential Solver
© 2013 IBM Corporation
Sequential Solver
© 2013 IBM Corporation
Sequential Solver
© 2013 IBM Corporation
Sequential Solver
© 2013 IBM Corporation
Current Status
Mincer being used to debug problems in production
We have implemented tactics for several different experimental parameters on our Java JIT
Working on Infinite Sequential Solver to allow us to isolate intermittent failures to specific changesets
© 2013 IBM Corporation
Q & A
© 2013 IBM Corporation
Sequential Solver
TheoryHave a finite, ordered set, T
Random Variable P, support on TRandom Variable f, support on [0, 1]Random Variables {Xi}, binary
© 2013 IBM Corporation
Sequential Solver
TheoryEach Xi is associated with an index ni in TPr(Xi passing | f, P) = (1-f) if ni ≥ PPr(Xi passing | f, P) = 1 if ni < P
Choose index to associate with next Xi optimally
© 2013 IBM Corporation
Subset Solver
TheoryEach Xi is associated with a subset Ni of TPr(Xi passing | f, P) = (1-f) if P ϵ Ni
Pr(Xi passing | f, P) = 1 otherwise
Choose next subset optimally
Faster convergence than possible with Sequential Solver