1

Independent Study Summary3/22/05

B. Golden

2

Presentation Methodology: Introduction Why use visualizations?

To facilitate user comprehension To convey complexity and intricacy of performance data Help bridge the gap between raw performance data and

performance improvements When to use visualizations?

On-line: visualization while the application is running (can slow down execution significantly)

Post mortem: after execution (usually based on trace data gathered at runtime)

What to visualize? Interactive displays to guide the user, not rationalize Default visualizations should provide high-level views Low-level information should be easily accessible

3

Visualization Concepts and Principles Context

Perspective: the point of view from which information is presented Semantic Context: the relationship between performance information and user data constructs Sub-view Mapping: a mapping between a subset of graphical views

Scaling Multidimensional/Multivariate representation: a representation of data with many attributes per data point Macroscopic/Microscopic views: the level of detail represented by a given view Micro/Macro composition: showing both local detail and global structure Adaptive display: the adjustment of a display’s characteristics in response to data size Display manipulation: interactive modification of a display (i.e. zooming, scrolling) Composite view: synthesis of two or more views into a single view

Comparison Multiple views: the presentation of data from multiple perspectives Small multiples: a series of images indexed by changes in other performance data (e.g. animation) Cross-execution views: comparison of performance information from various program executions

Extraction of information Reduction and filtering: representing raw data by statistical summaries Clustering: multivariate statistical analysis and presentation techniques for grouping or categorizing related

data points Encoding and abstracting: using graphical attributes (color, shape, size, etc) to convey information Separating information: differentiation among layers of information through color highlighting

foreground/background, etc.

4

General Approaches to Performance Visualization General Categories

Program specific: application specific way to show how computation progresses by animating data structures

System oriented: focus on the impact the application has on the system

System/Application independent: depict performance data for a variety of systems and applications. See figure

Meta-tools: facilitate the development of custom visualization tools

Other Categories On-line: visualization during execution

Can be intrusive Volume of information may be too large

to interpret with out playback functionality Allows the user to observe only the

interesting parts of execution Post mortem: visualization after execution

Large trace files Easier to implement Users are accustomed to this format

5

Specific Features of Existing Visualization Strategies Animation

Has been employed by various tools to assist in the program execution replay Communication operations are the most commonly animated events Viewing data dynamically may illuminate bottlenecks more efficiently

Program graphs A generalized picture of the whole system Paradyn has this functionality

Gantt charts De facto standard for displaying inter-process communication

Data access displays Each cell of the 2D display is devoted to an element of the array Color distinguishes between local/remote and read/write

Critical path analysis Concerned with identifying the program regions which most contribute to program

execution time Construct a graph which depicts synchronization and communication dependencies

among the processes in the program

6

Summary of VisualizationsVisualization Name Advantages Disadvantages Include in the PAT Used For

Animation Adds another dimension to visualizations

CPU intensive Yes Various

Program Graphs

(N-ary tree)

Built-in zooming; Integration of high and low-level data

Difficult to see inter-process data

Maybe Comprehensive Program Visualization

Gantt Charts

(Time histogram; Timeline)

Ubiquitous; Intuitive Not as applicable to shared memory as to message passing

Yes Communication Graphs

Data Access Displays

(2D array)

Provide detailed information regarding the dynamics of shared data

Narrow focus; Users may not be familiar with this type of visualization

Maybe Data Structure Visualization

Kiviat Diagrams Provides an easy way to represent statistical data

Can be difficult to understand

Maybe Various statistical data (processor utilization, cache miss rates, etc.)

Event Graph Displays

(Timeline)

Can be used to display multiple data types (event-based)

Mostly provides only high-level information

Maybe Inter-process dependency

7

Evaluation of User Interfaces

General Guidelines Visualization should guide, not rationalize Scalability is crucial Color should inform, not entertain Visualization should be interactive Visualizations should provide meaningful labels Default visualization should provide useful information Avoid showing too much detail Visualization controls should be simple

GOMS Goals, Operators, Methods, and Selection Rules Formal user interface evaluation technique A way to characterize a set of design decisions from the point of view of the user A description of what the user must learn; may be the basis for reference

documentation The knowledge is described in a form that can actually be executed (there have been

several fairly successful attempts to implement GOMS analysis in software, ie GLEAN) There are various incarnations of GOMS with different assumptions useful for more

specific analyses (KVL, CMN-GOMS, NGOMSL, CPM-GOMS, etc.)

8

Simple GOMS Example

GOMS model for OS X Method for goal: delete a file.

Step 1. Accomplish goal: drag file to trash. Step 2. Return with goal accomplished.

Method for goal: move a file. Step 1. Accomplish goal: drag file to destination. Step 2. Return with goal accomplished.

GOMS model for UNIX Method for goal: delete a file.

Step 1. Recall that command verb is "rm -f". Step 2. Think of directory name and file name and retain as first filespec. Step 3. Accomplish goal: enter and execute a command. Step 4. Return with goal accomplished.

Method for goal: copy a file. Step 1. Recall that command verb is "cp". Step 2. Think of source directory name and file name and retain as first filespec. Step 3. Think of destination directory name and retain as second filespec. Step 4. Accomplish goal: enter and execute a command. Step 5. Return with goal accomplished.

9

Conclusion

Plan for development Develop a preliminary interface that provides the

functionality required by the user while conforming to visualization guidelines presented previously

After the preliminary design is complete, elicit user feedback

During periods where user contact is unavailable, we may be able to use GOMS analysis or another formal interface evaluation technique

10

Other Tasks

Usability Survey Update Posts on MPI user discussion lists Email to all the UPC/SHMEM users we know One response so far We should be able to get paper copies of the 30 responses

from the APART survey Gmail invites used to elicit more responses Other potential avenues

Existing performance tool mailing lists Use our other contacts (developers, etc.)

Literature Search Usability Regarding tool development