On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02)...
Transcript of On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02)...
![Page 1: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/1.jpg)
1
On Utilizing Experiment Data Repository for Performance Analysis
of Parallel Applications Hong-Linh Truong1, Thomas Fahringer2
1 Institute for Software Science,University of Vienna, Austria
2 Institute of Computer Science,University of Innsbruck, Austria
www.par.univie.ac.at/project/scalea
EURO-PAR 2003, Klagenfurt, August 26th, 2003
![Page 2: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/2.jpg)
2
OutlineOutlineØØMotivationMotivationØØSCALEASCALEAØØExperiment Data RepositoryExperiment Data Repository
ExperimentExperiment--related Datarelated DataImplementation OverviewImplementation OverviewExperimentExperiment--related APIsrelated APIs
ØØAchievementsAchievementsSearch and Filter Performance DataSearch and Filter Performance DataMultiMulti--Experiment AnalysisExperiment AnalysisTools IntegrationTools Integration
ØØConclusions and Future WorkConclusions and Future Work
![Page 3: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/3.jpg)
3
MotivationMotivationØØThe need to collect and archive performance data The need to collect and archive performance data forfor
MultiMulti--experiment analysisexperiment analysisPerformance comparisonPerformance comparison
ØØLimitation on supporting code complexity:Limitation on supporting code complexity:Handle largeHandle large--scale performance datascale performance dataBasic search on performance dataBasic search on performance data
ØØLack of capabilities to export, share, and exchange Lack of capabilities to export, share, and exchange performance data and tools interoperationperformance data and tools interoperation
![Page 4: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/4.jpg)
4
SCALEA (SC01, SCALEA (SC01, EuroEuro--Par02Par02, PVMMPI02) , PVMMPI02) OpenMP,MPI,HPF, hybrid programs
Instrumented programs
Instrumentation description file
Executable Programs
System Sensors
Instrumentation Control
SIS Instrumentation Libraries:
SISPROFILING
Instrumentation
Compilation
Execution
EnvironmentPerformance
Analysis
Experiment Data
Repository
Target machines
SCALEA Sensor Manager
SCALEA Instrumentation System(SIS)
SCALEA Performance Analysis & Visualization
SCALEA Runtime System
![Page 5: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/5.jpg)
5
ExperimentExperiment Data RepositoryData RepositoryØ For comparing performance across experiments with large amounts of data
Need to archive performance dataRequire a very strong and flexible database system
ØData repository is first step required by mining, knowledge discovery in performance dataØExperiment Data Repository:
Stores information about application, source code, machine information, and performance dataAssociates performance results with source code and machine informationSupports sophisticated analysis and easy integration with other tools
![Page 6: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/6.jpg)
6
Data ModelData Model
1:n1:n 1:n
1:n
1:nn:1
1:n
m:n
1:n1:n
n:1 1:n
1:n
1:n
1:n1:n
![Page 7: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/7.jpg)
7
Performance Metric CatalogPerformance Metric CatalogØWhy performance metric catalog:
Documentation about metrics providedHelp tools/users understanding and interpreting performance data
ØPerformance metric has 4 propertiesunique name is a value that distinguishes this metric from all othersdata type describes data type of measurement which describes the value of the metric.measurement unit is the unit of measurementsemantics (or well-defined meaning)
ØAll metrics are stored in a catalog in experiment data repository
![Page 8: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/8.jpg)
8
Implementation Overview Implementation Overview
ØØPowering search and archive with great flexibility and robustnesPowering search and archive with great flexibility and robustnesssRelational database based on Relational database based on PostgreSQLPostgreSQL
ØØTaking advantages of portability and network capabilityTaking advantages of portability and network capabilityAll components written in JavaAll components written in JavaDatabase connection powered by JDBC Database connection powered by JDBC
![Page 9: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/9.jpg)
9
ExperimentExperiment--related Data APIsrelated Data APIs
public class PerformanceMetric { private String metricName;private Object metricValue;public String getMetricName(){…};public void setMetricName(String metricName){…}; public Object getMetricValue(){…};public void setMetricValue(Object metricValue){…};... }
public class ProcessingUnit {private String computationalNode;private int processID ;private int threadID ;…public ProcessingUnit(String node,int process, int thread) {...}…}
ØØWellWell--defined APIs for accessing data in defined APIs for accessing data in Experiment Data Repository Experiment Data Repository ØØ Java implementationJava implementation
![Page 10: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/10.jpg)
10
ExperimentExperiment--related Data APIs (const.)related Data APIs (const.)
public class ExperimentData {DatabaseConnection connection; ...public ProcessingUnit[] getProcessingUnits(Experiment e){...}public RegionSummary[] getRegionSummaries(CodeRegion cr, Experiment e){...}public RegionSummary getRegionSummary(CodeRegion cr, ProcessingUnitpu, Experiment e) {...}public RegionSummary getRegionSummary(CodeRegion cr, Experiment e,ProcessingUnit pu, RegionSummary parent) {...} public RegionSummary[] getChildOfRegionSummary(RegionSummary rs){...} public RegionSummary getParentOfRegionSummary(RegionSummary rs){...}...
}
public class RegionSummary {... public PerformanceMetric[] getMetrics(){...}public PerformanceMetric getMetric(String metricName){...}...}
![Page 11: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/11.jpg)
11
CodeRegion cr = new CodeRegion(“IONIZE_MOVE”);Experiment e = new Experiment(“9Nx4P,P4”);ProcessingUnit
pu =new ProcessingUnit(``gsr402.vcpc.ac.at'',2,0);
ExperimentData ed = new ExperimentData(new DatabaseConnection(...));
RegionSummary rs = ed.getRegionSummary (cr,pu,e);
PerformanceMetric overhead =rs.getMetric(''oall_ident'');
PerformanceMetricwtime =rs.getMetric(``wtime'');
double overheadRatio=((Double)overhead.getMetricValue()).doubleValue()/
((Double)wtime.getMetricValue()).doubleValue();
Example of Using APIsExample of Using APIs
![Page 12: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/12.jpg)
12
XML DataXML Data
ØFacilitate information exchange between toolsØAllow tools interoperation in a uniform way ØWith/without source code ØCall-graph, selective metrics
Performance Metric Catalog
![Page 13: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/13.jpg)
13
Search and Filter CapabilitiesSearch and Filter Capabilities
ØExisting problems: - difficult to find interesting events via process time-lines, zooming, scrolling, call-graph-search and filter data based on files are not efficient, robust and fast
ØSearch and filter data based relational database and SQL: flexibility and robustnessØWe support generic search and filter
Picture taken from D. Picture taken from D. KranzlmuellerKranzlmueller ´s ´s talktalk
![Page 14: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/14.jpg)
14
Search and Filter CapabilitiesSearch and Filter Capabilities
![Page 15: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/15.jpg)
15
Search and Filter Capabilities (const.)Search and Filter Capabilities (const.)
![Page 16: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/16.jpg)
16
Search and Filter Capabilities (const.)Search and Filter Capabilities (const.)ØDefining an expression of performance metrics ØDiscretizing the expression into quantitative characteristicsØSearch based on selected quantitative characteristics
![Page 17: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/17.jpg)
17
MultiMulti--Experiment AnalysisExperiment AnalysisØMany ways to specify what you want to analysis
ExperimentsCode regionsPerformance metricsStatistic methods
ØVarious analysesPerformance comparison for different sets of experimentsOverhead analysis for multi-experimentPerformance speedup/improvement at both program and code region level
![Page 18: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/18.jpg)
18
Select what to be analyzedSelect what to be analyzed
![Page 19: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/19.jpg)
19
Different Sets of ExperimentsDifferent Sets of Experiments
![Page 20: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/20.jpg)
21
Improvement/Speedup, EfficiencyImprovement/Speedup, Efficiency
![Page 21: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/21.jpg)
22
System InformationSystem Information
![Page 22: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/22.jpg)
Tools IntegrationTools Integration
ArchitecturesArchitecturesq NOWsq PC-Clustersq SMP Clustersq GRID Systemsq DM/SM Systems
q Parameter Studiesq Experiment
Management
ZenturioZenturio
Programming Programming ParadigmsParadigmsq MPI,GlobusMPIq OpenMP/MPIq HPF/OpenMPq JavaSymphony
ExperimentData
Repository
http://www.par.univie.ac.at/project/askalon/
q Instrumentationq Measuring q Performance Analysis
SCALEASCALEA
q Modelingq Simulationq Performance
Prediction
PerformancePerformanceProphetProphet
q AutomaticBottleneckAnalysis
AksumAksum
![Page 23: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/23.jpg)
24
SCALEA
AKSUM
Experiment Data Repository
Applicationfiles
Instrument application
files
Files were instrumented
AKSUM employs SCALEA to:
• instrument files given an arbitrary code region
• transfer application files to the repository
• transfer the data generated by the monitoring to the repository
•use data provided by SCALEA for property analysis
Case 1: AKSUM ØØ URL: URL: http://www.par.http://www.par.univieunivie.ac.at/project/.ac.at/project/aksumaksum//
ØØFahringerFahringer and and SeragiottoSeragiotto, Jr., Jr.
![Page 24: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/24.jpg)
Case 2: Performance ProphetCase 2: Performance ProphetØØ URL: URL: http://www.par.http://www.par.univieunivie.ac.at/project/pro.ac.at/project/prophet/phet/
ØØ FahringerFahringer, , PllanaPllana, and , and Testori Testori
ØØPP is a performance modeling and PP is a performance modeling and prediction systemprediction system
ØØ PP utilizes SCALEA to obtain timing PP utilizes SCALEA to obtain timing parameters for application.parameters for application.
ØØPP uses performance data in XML format PP uses performance data in XML format exported by SCALEA for automatic building exported by SCALEA for automatic building of cost functionsof cost functionsØØ Cost functions are used to develop a Cost functions are used to develop a hybrid analytical and simulation model of the hybrid analytical and simulation model of the applicationapplication
Instrument the application
Execute the application
Set the problem and/or machine size
Store the data in the performance database
[All experimentsdone]
[Else]
Build the cost functions
![Page 25: On Utilizing Experiment Data Repository for Performance ...SCALEA (SC01, Euro-Par02, PVMMPI02) OpenMP,MPI,HPF, hybrid programs Instrumented programs Instrumentation description file](https://reader036.fdocuments.us/reader036/viewer/2022071411/6106469fa476dc6a4d391299/html5/thumbnails/25.jpg)
26
ConclusionsConclusions and Futureand Future WorkWorkØØConclusionsConclusions
Design of experiment data repository for performance Design of experiment data repository for performance analysis toolanalysis toolDemonstration of achievements gaining when employing Demonstration of achievements gaining when employing experiment data repositoryexperiment data repositoryààData repository has increasingly supported the automation Data repository has increasingly supported the automation
of performance analysis and optimization processof performance analysis and optimization process
ØØOngoing workOngoing workWorking on simple and efficient way to search on Working on simple and efficient way to search on performance dataperformance dataApplying automatic scalable analysis techniquesApplying automatic scalable analysis techniquesSemantic representation of performance dataSemantic representation of performance data
www.par.www.par.univieunivie.ac.at/project/.ac.at/project/scaleascalea