London, 21.01.2020 [email protected]...
Transcript of London, 21.01.2020 [email protected]...
![Page 1: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/1.jpg)
Actionable Performance AnalysesActionable Performance Analyses
Marija [email protected]
London, 21.01.2020
1
![Page 2: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/2.jpg)
Software PerformanceSoftware Performance
One of the most important aspects of software quality
Efficiency
Responsiveness
Scaling
Throughput
User satisfaction
2
![Page 3: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/3.jpg)
Software PerformanceSoftware Performance
One of the most important aspects of software quality
1 http://www.fastcompany.com/1825005/how-one-second-could-cost-amazon-1.6-billion-sales
Efficiency
Responsiveness
Scaling
Throughput
User satisfaction
2
![Page 4: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/4.jpg)
Software PerformanceSoftware Performance
One of the most important aspects of software quality
1 http://www.fastcompany.com/1825005/how-one-second-could-cost-amazon-1.6-billion-sales 2 http://www.cedexis.com/blog/for-google-400ms-of-increased-page-load-time-results-in-044-lost-search-sessions/
Efficiency
Responsiveness
Scaling
Throughput
User satisfaction
2
![Page 5: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/5.jpg)
Approaches for ImprovingApproaches for ImprovingSoftware PerformanceSoftware Performance
CPU profiling Compiler optimizations
Manual tuning
code optimizedcode
3
![Page 6: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/6.jpg)
Approaches for ImprovingApproaches for ImprovingSoftware PerformanceSoftware Performance
CPU profiling Compiler optimizations
Manual tuning
code optimizedcode
4Actionable Performance Analyses
code actionablesuggestions
![Page 7: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/7.jpg)
Actionable Performance AnalysesActionable Performance Analyses
Suggest concrete code changes
Demonstrate the impact of applying optimizations
Optimizations that are:
exploitable - easy to understand and apply
effective - lead to significant performanceimprovements
recurring - applicable across multiple projects
5
![Page 8: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/8.jpg)
This TalkThis Talk
Reordering opportunities [1]
Method inlining in Big Data system [2]
6
![Page 9: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/9.jpg)
This TalkThis Talk
Reordering opportunities [1]
Method inlining in Big Data system [2]
[1] Selakovic et al. An Actionable Performance Profiler for Optimizing the Order of Evaluations (ISSTA'17)
[2] Selakovic et al. Cross-Language Optimizations in Big Data Systems: A Case Study of SCOPE (ICSE-SEIP'18) 6
![Page 10: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/10.jpg)
Inefficient Order of EvaluationsInefficient Order of Evaluations
expensiveAndUnlikely() && cheapAndLikely()
7
![Page 11: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/11.jpg)
Inefficient Order of EvaluationsInefficient Order of Evaluations
expensiveAndUnlikely() && cheapAndLikely()
7
![Page 12: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/12.jpg)
Inefficient Order of EvaluationsInefficient Order of Evaluations
expensiveAndUnlikely() && cheapAndLikely()
Analysis of all conditions in in logical expressionsor switch statements
Assessment of the computational cost
Safe to apply and beneficial optimizations7
![Page 13: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/13.jpg)
DecisionProf: An Analysis forDecisionProf: An Analysis forOptimizing Orders of EvaluationsOptimizing Orders of Evaluations
Programtransformation
Performanceevaluation
Profiler
Program + input
Optimization candidates
P ... P1 n
Optimizationopportunities
Static preprocessing
Dynamic analysis
8
![Page 14: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/14.jpg)
DecisionProf: An Analysis forDecisionProf: An Analysis forOptimizing Orders of EvaluationsOptimizing Orders of Evaluations
Programtransformation
Performanceevaluation
Profiler
Program + input
Optimization candidates
P ... P1 n
Optimizationopportunities
Static preprocessing
Dynamic analysis
8
Dynamic analysis
![Page 15: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/15.jpg)
Dynamic AnalysisDynamic Analysis
Collecting runtime data:
Cost - number of branching pointValue - true/false
Assessing the optimal order
9
![Page 16: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/16.jpg)
Dynamic AnalysisDynamic Analysis
a() && b()
var x;
Collecting runtime data:
Cost - number of branching pointValue - true/false
Assessing the optimal order
9
![Page 17: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/17.jpg)
Dynamic AnalysisDynamic Analysis
a() && b()
var x;
Collecting runtime data:
Cost - number of branching pointValue - true/false
Assessing the optimal order
a() && b()
cost value
c v
c v
c v
a1 a1
a2 a2
a3 a3
cost value
c v
c v
c v
b1 b1
b2 b2
b3 b3 9
![Page 18: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/18.jpg)
Dynamic Analysis: ExampleDynamic Analysis: Example
_.isNumber(input) && isNaN(input)
cost value
3 true
2 true
4 true
cost value
1 false
1 false
1 true
Execution 1
Execution 2
Execution 3
10
![Page 19: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/19.jpg)
Dynamic Analysis: ExampleDynamic Analysis: Example
_.isNumber(input) && isNaN(input)
cost value
3 true
2 true
4 true
cost value
1 false
1 false
1 true
Overall cost = 12
Execution 1
Execution 2
Execution 3
10
![Page 20: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/20.jpg)
Dynamic Analysis: ExampleDynamic Analysis: Example
_.isNumber(input) && isNaN(input)
cost value
3 true
2 true
4 true
cost value
1 false
1 false
1 true
Overall cost = 12
Execution 1
Execution 2
Execution 3
10
![Page 21: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/21.jpg)
Dynamic Analysis: ExampleDynamic Analysis: Example
isNaN(input) && _.isNumber(input)1
Cost Value
1 false
1 false
1 true
Cost Value
3 true
2 true
4 true
Execution 1
Execution 2
Execution 3
11
![Page 22: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/22.jpg)
Dynamic Analysis: ExampleDynamic Analysis: Example
isNaN(input) && _.isNumber(input)1
Cost Value
1 false
1 false
1 true
Cost Value
3 true
2 true
4 true
Overall cost = 7
1 See pull request #2496 of Underscore.js
Execution 1
Execution 2
Execution 3
11
![Page 23: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/23.jpg)
Pruning Non-CommutativePruning Non-CommutativeConditionsConditions
Non-commutative conditions : change program semantics
Two approaches:
Dynamic: write to the samememory location
Static: known patternsif (a && a.x) {...} y = x || "abc"
123
var x = 0;function a(){ x++;} if (a() && x) {...}
123456
12
![Page 24: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/24.jpg)
Performance EvaluationPerformance Evaluation
Program transformation for each optimization candidateMethodology by Georges et al. [1]
Original program + input Optimized program + input
t-test
VM instances
Warm up
Measure
[1] Georges et al. Statistically rigorous Java performance evaluation (OOPSLA 07) 13
![Page 25: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/25.jpg)
DecisonProf: EvaluationDecisonProf: Evaluation
Subject programs:
9 JavaScript libraries and test suites
34 benchmarks from JetStream suite
Results:
23 opportunities across libraries
29 opportunities across benchmarks
Performance improvements: 2.5% - 59% (function level)and 2.5% - 6.5% (application level)
14
![Page 26: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/26.jpg)
Examples of ReorderingExamples of ReorderingOpportunitiesOpportunities
Cheerio library:
Gbemu benchmark:
//code before isTag (elem) && elems.indexOf(elem) === -1 //code after elems.indexOf(elem) === -1 && isTag (elem)
//code before numberType != "float32" && GameBoyWindow.opera && this.checkForOperaMathBug () //code after GameBoyWindow.opera && numberType != "float32" && this.checkForOperaMathBug ()
tests: 26%, 34%
application:5.8%
15
![Page 27: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/27.jpg)
DecisionProf: SummaryDecisionProf: Summary
The first profiler to detect inefficient orders of evaluations
Simple and easy to exploit optimizations
Suggests program refactorings
Guaranteed improvements for given inputs
16
![Page 28: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/28.jpg)
Cross-language OptimizationsCross-language Optimizationsin Big Data Systemsin Big Data Systems
SCOPE - Structured ComputationsOptimized for Parallel Executions
Relational (SQL) + non-relational (C#)
SCOPE job is DAG where:
Vertices - processes
Edges - data flows
V1
V2 V3
V4
Stream
Output
17
![Page 29: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/29.jpg)
Performance Problem in SCOPEPerformance Problem in SCOPE
data
vertex
data
data = SELECT * FROM inputStream; C++
Cross runtime interaction
Intrinsics vs non-intrinsics
18
![Page 30: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/30.jpg)
Performance Problem in SCOPEPerformance Problem in SCOPE
data
vertex
datadata = SELECT * FROM inputStream WHERE !String.IsNullOrEmpty(A);
Cross runtime interaction
Intrinsics vs non-intrinsics
data = SELECT * FROM inputStream; C++
intrinsic (has c++ impl.)
19
![Page 31: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/31.jpg)
Performance Problem in SCOPEPerformance Problem in SCOPE
data = SELECT * FROM inputStream WHERE A.Equals('ColumnA');
data
vertex
data
deserialize
serialize
Cross runtime interaction
Intrinsics vs non-intrinsics
non-intrinsic
C#
20
![Page 32: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/32.jpg)
Method Inlining: ExampleMethod Inlining: Example
data = SELECT * FROM inputStream WHERE filter(JobID); #CS bool filter (string s) { return (!string.IsNullOrEmpty(s) && s.StartsWith("07")); } #ENDCS
C#
Replacing function call with the body of a function
21
![Page 33: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/33.jpg)
Method Inlining: ExampleMethod Inlining: Example
Replacing function call with the body of a function
data = SELECT * FROM inputStream WHERE !string.IsNullOrEmpty(JobID) && JobID.StartsWith("07");
C++
22
![Page 34: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/34.jpg)
Method Inlining: ExampleMethod Inlining: Example
Replacing function call with the body of a function
data = SELECT * FROM inputStream WHERE !string.IsNullOrEmpty(JobID) && JobID.StartsWith("07");
C++
~ 4X faster
22
![Page 35: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/35.jpg)
Static AnalysisStatic Analysis
data
vertex
data
deserialize
serialize
Goal: C# to C++ translation for a vertex
Only calls to intrinsics
No loops
No try-catch blocks
No new and cast operations
No arguments passed by reference
Inlineable methods:
23
![Page 36: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/36.jpg)
Static Analysis: EvaluationStatic Analysis: Evaluation
Optimizable vertices:
0
0.05
0.1
0.15
0.2
0.25
0.3
Cosmos8
Cosmos9
Cosmos11
Cosmos14
Cosmos15
Total
Prop
ortio
n (%
)
Data center
24
![Page 37: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/37.jpg)
Static Analysis: EvaluationStatic Analysis: Evaluation
Optimizable vertices:
0
0.05
0.1
0.15
0.2
0.25
0.3
Cosmos8
Cosmos9
Cosmos11
Cosmos14
Cosmos15
Total
Prop
ortio
n (%
)
Data center
0.13
24
![Page 38: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/38.jpg)
Static Analysis: EvaluationStatic Analysis: Evaluation
Optimizable vertices:
0
0.05
0.1
0.15
0.2
0.25
0.3
Cosmos8
Cosmos9
Cosmos11
Cosmos14
Cosmos15
Total
Prop
ortio
n (%
)
Data center
0.13
0.13% ~ 40,000 hours 24
![Page 39: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/39.jpg)
Static Analysis: EvaluationStatic Analysis: Evaluation
Job CPU Time Throughput
A 23% 30%
B no change no change
C 25% 38%
E 4.7% 5%
F no change 115%
Case studies:Optimizable vertices:
0
0.05
0.1
0.15
0.2
0.25
0.3
Cosmos8
Cosmos9
Cosmos11
Cosmos14
Cosmos15
Total
Prop
ortio
n (%
)
Data center
0.13
0.13% ~ 40,000 hours 24
![Page 40: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/40.jpg)
Static Analysis: EvaluationStatic Analysis: Evaluation
Optimizable vertices:
0
0.05
0.1
0.15
0.2
0.25
0.3
Cosmos8
Cosmos9
Cosmos11
Cosmos14
Cosmos15
Total
Prop
ortio
n (%
)
Data center
25
![Page 41: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/41.jpg)
Static Analysis: EvaluationStatic Analysis: Evaluation
Optimizable vertices:
0
0.05
0.1
0.15
0.2
0.25
0.3
Cosmos8
Cosmos9
Cosmos11
Cosmos14
Cosmos15
Total
Prop
ortio
n (%
)
Data center
0.13
25
![Page 42: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/42.jpg)
Static Analysis: EvaluationStatic Analysis: Evaluation
Optimizable vertices:
0
0.05
0.1
0.15
0.2
0.25
0.3
Cosmos8
Cosmos9
Cosmos11
Cosmos14
Cosmos15
Total
Prop
ortio
n (%
)
Data center
0.13
0.13% ~ 40,000 hours 25
![Page 43: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/43.jpg)
Static Analysis: EvaluationStatic Analysis: Evaluation
Job CPU Time Throughput
A 23% 30%
B no change no change
C 25% 38%
E 4.7% 5%
F no change 115%
Case studies:Optimizable vertices:
0
0.05
0.1
0.15
0.2
0.25
0.3
Cosmos8
Cosmos9
Cosmos11
Cosmos14
Cosmos15
Total
Prop
ortio
n (%
)
Data center
0.13
0.13% ~ 40,000 hours 25
![Page 44: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/44.jpg)
Case Study in SCOPE: ExampleCase Study in SCOPE: Example
//optimization opportunity SELECT url_id, url, t_url_id, CommonMethod.GetTargetUrl(url,t_url) AS t_url FROM SSTREAM @VLPMapSS; //optimized code SELECT url_id, url, t_url_id, string.isNullOrEmpty(t_url)?url:t_url AS t_url FROM SSTREAM @VLPMapSS;
Vertex level improvement: 42%
Job level improvement: 25%26
![Page 45: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/45.jpg)
Cross-language Optimizations in BigCross-language Optimizations in BigData Systems: SummaryData Systems: Summary
Static analysis for method inlining opportunities
Only optimizations that reduce cross-runtime interactions
Large scale evaluation
27
![Page 46: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/46.jpg)
ConclusionsConclusions
expensiveAndUnlikely() && cheapAndLikely()
data
serialize
vertex
deserialize
data
Actionable performance analyses
Easy to exploit classes of optimizations
Future work: automatically inferring optimization patternsacross and within different domains
28
![Page 47: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/47.jpg)
Huawei Dresden Research CenterHuawei Dresden Research Center
Research topics:
Design and development of embedded systemsProgram analysis and formal verificationSoftware testing: fuzzing and automated test case generation
29
![Page 48: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/48.jpg)
Huawei Dresden Research CenterHuawei Dresden Research Center
Research topics:
Design and development of embedded systemsProgram analysis and formal verificationSoftware testing: fuzzing and automated test case generation
We are hiring!
29
![Page 49: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/49.jpg)
Questions?Questions?
30
![Page 50: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/50.jpg)
DecisionProf: Static PreprocessingDecisionProf: Static Preprocessing
startDecision; startCheck: a(); endCheck(); startCheck: b(); endCheck(); startCheck: c(); endCheck(); endDecision; if (a() && b() && c()) { ...
Hoists leaf expressions
Beginning and end of each decision
Beginning and end of each checks31
![Page 51: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/51.jpg)
DecisionProf: Safe Check EvaluationDecisionProf: Safe Check Evaluation
Collect and undo all writes to variables and object propertiesthat may affect code after check evaluation
var x = 0; function a () { x++; var y=1; ...... } startCheck: a(); startCheck: b(); //reset all side effects if (a () && b()) ...
32
![Page 52: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/52.jpg)
DecisionProf: Safe Check EvaluationDecisionProf: Safe Check Evaluation
Collect and undo all writes to variables and object propertiesthat may affect code after check evaluation
var x = 0; function a () { x++; var y=1; ...... } startCheck: a(); startCheck: b(); //reset all side effects if (a () && b()) ...
write to x affectsprogram state
32
![Page 53: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/53.jpg)
DecisionProf: Safe Check EvaluationDecisionProf: Safe Check Evaluation
Collect and undo all writes to variables and object propertiesthat may affect code after check evaluation
var x = 0; function a () { x++; var y=1; ...... } startCheck: a(); startCheck: b(); //reset all side effects if (a () && b()) ...
write to x affectsprogram state
program state is changedoutside normal execution
32
![Page 54: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/54.jpg)
DecisionProf: Safe Check EvaluationDecisionProf: Safe Check Evaluation
Collect and undo all writes to variables and object propertiesthat may affect code after check evaluation
var x = 0; function a () { x++; var y=1; ...... } startCheck: a(); startCheck: b(); //reset all side effects if (a () && b()) ...
write to x affectsprogram state
program state is changedoutside normal execution
dynamically execute x = 0;
32
![Page 55: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/55.jpg)
DecisionProf: Finding Optimal OrderDecisionProf: Finding Optimal Order
33
![Page 56: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/56.jpg)
DecisionProf: Actual vsDecisionProf: Actual vsEstimated CostEstimated Cost
05000
100001500020000250003000035000400004500050000
0 50000100000
150000
200000
250000
300000
Exec
utio
n tim
e (m
s)
Estimated cost
34
![Page 57: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/57.jpg)
Job ArtifactsJob Artifacts
Job Algebra
Vertices - processes
Edges - data flow
Runtime statistics
Script source code
Generated C# and C++ code
V1
V2 V3
V4
Stream
Output
35
![Page 58: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/58.jpg)
Profiling InfrastructureProfiling Infrastructure
V1 V2 V3
Job algebra
Analysis of C++ code
Analysis of C# code
Class names
Sources of C# code +Inlinealbe methods
.NET framework methods
User-written methods
Processors and reduces
36
![Page 59: London, 21.01.2020 marija.selakovic@huawei.com …crest.cs.ucl.ac.uk/fileadmin/crest/COWphotos/Other/...A p p roac h e s for I m p rov i n g Soft wa r e Per for m a n c e CPU profiling](https://reader033.fdocuments.us/reader033/viewer/2022050220/5f65ac6d7774f263ef25bf22/html5/thumbnails/59.jpg)
Native vs Non-Native TimeNative vs Non-Native Time
0
20
40
60
80
100
Cosmos8
Cosmos9
Cosmos11
Cosmos14
Cosmos15
Total
Prop
ortio
ns re
l. to
dat
a ce
nter
tim
e(%
)
Data center
37