Using Visualization to Understand the Behavior of Computer Systems
description
Transcript of Using Visualization to Understand the Behavior of Computer Systems
![Page 1: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/1.jpg)
Using Visualization to Understand
the Behavior of Computer Systems
Robert P. Bosch Jr.Stanford University
May 3, 2001
![Page 2: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/2.jpg)
2
Motivation
• Explosion in complexity of computer systems
• Development of rich data collection tools– Complete Machine Simulation: SimOS– Software Monitoring: Performance Co-Pilot
(PCP)– Firmware Instrumentation: FlashPoint– Hardware Monitoring: DCPI
• Challenge: how do we fully exploit the large, detailed data sets these tools can generate?
![Page 3: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/3.jpg)
3
Data Analysis Challenge
• How do we typically handle this data?– Visual inspection of huge log files– Summarize through statistics and aggregation– Focus on restricted data subsets
• Alternative approach: data visualization– Display large amounts of data at once– Enable interactive exploration of entire data set
• Overview, zoom and filter, details-on-demand
– Use human perception to discover patterns, trends, and interesting information
![Page 4: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/4.jpg)
4
Computer Systems Visualization:Existing Work
• Pedagogical examples– Processors: DLXView, Pentium Pro Tutorial– Memory: Cache Visualization Tool (CVT)
• Parallel systems performance– AIMS, Pablo, Paradyn, ParaGraph, PARvis,
StormWatch, VAMPIR…
• Other examples– Network performance, file systems, etc.
![Page 5: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/5.jpg)
5
Computer Systems Visualization:Existing Work
• Demonstrate potential of visualization• Limitations
– Focused on particular system components– Integrated with specific data collection tools– Limited to a fixed set of visual
representations
• Conclusion: rich, flexible data sources require an equally powerful and flexible visualization system
![Page 6: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/6.jpg)
6
Outline
• Motivation• The Rivet visualization environment
– Architecture– Implementation
• Focused visualization systems– SUIF Explorer– Thor– PipeCleaner– Visible Computer
• Ad hoc analysis and visualization• Contributions
![Page 7: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/7.jpg)
7
Rivet Architecture: Goals
• Learn once, apply to wide range of problems– Decouple visualization and data collection
• Interactive exploration of large data sets– Couple visualization and analysis
• Rapid prototyping of visualizations• Extensibility
– Allow users to add new components
![Page 8: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/8.jpg)
8
Rivet Architecture: Approach
• Identify visualization “building blocks”• Mix and match to create visualizations• Users can add new components as
needed• Three basic object types:
– Data management: tuples, tables, and transforms
– Visual representation: primitives and metaphors
– Mapping from data to visual: encodings
![Page 9: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/9.jpg)
9
Data Model
• Simplified relational model– Tuple: collection of data fields– Table: set of tuples with common data format– Familiar, homogeneous model
• Load data by parsing text files• Directly save/load tables in binary format
Procedure:PID:Page Faults:
Redraw()1717
129...
Tuple Tabl
e
![Page 10: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/10.jpg)
10
.
.
.
Data Transforms
• Transforms enable users to operate on data• Can be composed to form data networks
– Active: data changes are propagated
• Rivet includes a set of standard transforms– Filter, sort, group, merge, join, aggregate, etc.
• Users may design and incorporate their own– Example: clustering algorithms
.
.
.......
GroupBy
Transform
![Page 11: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/11.jpg)
11
Visual Objects and Data Mapping
• Primitives draw individual tuples• Metaphors draw entire tables
– Draws table attributes: axes, labels, etc.– For each tuple: compute bounding box, select primitive
• Encodings map tuple contents to visual properties– Metaphors use spatial encodings
• Map tuple fields to bounding box parameters
– Primitives use attribute encodings• Map tuple fields to retinal properties: color, fill pattern,
size…
• Encodings encapsulate datavisual mapping– Metaphors and primitives are data independent
![Page 12: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/12.jpg)
12
Display
The Encoding Process: Example
X
Y
Graph Metaphor:
Spatial EncodingsProcedure:
PID:Page Faults:
Redraw()1717
129
Tuple
C
F
S
Rectangle Primitive: Attribute Encodings
0.5
![Page 13: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/13.jpg)
13
Coordination and Interaction
• Implicit coordination: sharing of objects– Examples:
• Primitives share attribute encodings: brushing• Metaphors share primitives: common appearance• Metaphors share spatial encodings: common axes
– Listener mechanism
• Explicit coordination: events and bindings– Rivet objects can raise events– User can bind actions to these events– Example: details-on-demand
![Page 14: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/14.jpg)
14
Rivet Implementation
• Goal: balance performance and flexibility– Interactive visualizations of large data sets– Rapid prototyping of visualizations
• C++ and OpenGL for performance– Also provides platform independence
• Scripting language interfaces for flexibility– Simplified Wrapper and Interface Generator (SWIG)– Create visualizations by writing scripts
• Interpreter is not in the main loop– Listener mechanism for high-frequency events– Event bindings for low-frequency user interaction
![Page 15: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/15.jpg)
15
Sample Rivet Scriptset table [DataVector]set parser [CSVParser -args $table]rparse $parser poletops.csv Import data into
tableCreate point primitive
set primitive [GLPoint]
Create graph metaphor
rwindow .radiosrgeometry .radios W 500 H 675rglob Graph .radios.map.radios.map SetData $table
Encode Longitude as X
set gran [expr 1.0 / 3600.0]set lmin [$table GetMin Longitude]set lmax [$table GetMax Longitude]set long [QUniformRangeMap –args $lmin $lmax $gran].radios.map EncodeAsXPosition \ [QRangeEncoding -args $long Longitude]
Encode # Radios as Color
set hue [list 0.0 0.5]set sat [list 0.0 1.0]set val [list 0.6 1.0]set ramp [IsomorphicColorMap –args $hue $sat $val]set rmin [$table GetMin Radios]set rmax [$table GetMax Radios]set radios [QNumberMap -args $Log $rmin $rmax]$ramp SetDomainMap $radios$primitive EncodeAsColor \ [QColorEncoding -args $ramp Radios]
Encode Latitude as Y
set lmin [$table GetMin Latitude]set lmax [$table GetMax Latitude]set lat [QUniformRangeMap –args $lmin $lmax $gran].radios.map EncodeAsYPosition \ [QRangeEncoding -args $lat Latitude]
![Page 16: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/16.jpg)
16
Outline
• Motivation• The Rivet visualization environment
– Architecture– Implementation
• Focused visualization systems– SUIF Explorer: Interactive user-directed
parallelization– Thor: Detailed memory profiling on FLASH– PipeCleaner: Superscalar processor pipelines– Visible Computer: System and cluster monitoring
• Ad hoc analysis and visualization• Contributions
![Page 17: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/17.jpg)
17
SUIF Explorer:Interactive Parallelization
![Page 18: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/18.jpg)
18
SUIF Explorer: Background
• Goal: enable sequential codes to run fast on MPs
• Compiler parallelizes loops when possible• Otherwise:
– Compiler presents loop analysis results to user– User applies application knowledge to assist compiler
• Data source: SUIF dynamic analyzers– Performance metrics for all loops in program– Coverage, granularity, loop level
• Visualization– Displays data in context of program source code– Focuses on loops most deserving of user attention
![Page 19: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/19.jpg)
19
SUIF Explorer: Discussion
• Benefits– Combines data with source, instead of loop
IDs– Allows user to filter uninteresting lines of
code– Facilitates comparisons between runs
• Limitations– Loosely coupled with rest of SUIF Explorer– Short loops less prominent than long loops
• Add sortable table of results as a linked view
![Page 20: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/20.jpg)
20
Thor: Detailed Memory Profiling
![Page 21: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/21.jpg)
21
Thor: Background
• Memory getting slower relative to CPU• High remote access latencies on NUMA systems• Memory system is often performance bottleneck• Data source: FlashPoint protocol on FLASH
– Collects all cache and TLB misses in firmware– Classified as local/remote, read/write– Attributed to CPU, procedure, data structure
• Visualization– Stacked bar charts showing miss counts– Collection of UI controls for configuring the display
![Page 22: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/22.jpg)
22
Thor: Discussion
• Both post-mortem and real-time analysis– Live connection to FLASH via socket– Useful for multi-phase applications
• Simple, familiar visualization• Interactive filtering, sorting,
aggregation• Drill down from overview to data of
interest
![Page 23: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/23.jpg)
23
PipeCleaner: Superscalar Processors
![Page 24: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/24.jpg)
24
PipeCleaner: Background
• High peak performance• Complex implementation techniques
– Speculation, multiple functional units, out-of-order execution
• Intended to be transparent to the programmer– True for correctness, not necessarily for performance
• Data sources: MXS and MMIX simulators– Detailed superscalar processor pipeline models
• Visualization: three linked views– Overview: occupancy strip charts– Detail: animated pipeline display– Context: program source code
![Page 25: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/25.jpg)
25
PipeCleaner: Discussion
• Applications of PipeCleaner– Program development– Compiler optimizations– Hardware design– Education – Simulator development
• Possible extensions– Other computer pipelines: graphics
hardware– Physical pipelines: assembly lines, etc.
![Page 26: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/26.jpg)
26
Visible Computer:System and Cluster Monitoring
![Page 27: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/27.jpg)
27
Visible Computer: Background
• Real-time analysis of system behavior– Observe behavior of system in its entirety– Explore interesting phenomena in detail
• Data sources: PCP, SimOS– Comprehensive data sources– Collect low-level hardware events
• Cache misses, disk requests, CPU utilization, etc.
– Classify using high-level structures• Process name, user/group, CPU mode, etc.
![Page 28: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/28.jpg)
28
Visible Computer: Visualization
• Organizes data using nested physical hierarchy
• Provides overviews at each level of detail– Active icons representing system components– User defines data ranges of interest for each
component– Icons activate when components are in range
• Allows users to “remove the cover”, show next level
• Displays detailed charts on demand– Data filtered/colored using high-level classifiers
![Page 29: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/29.jpg)
29
Visible Computer: Discussion
• Unified interface for computer systems data– Focus-plus-context view of system– Physical layout, virtual classification
• Provides an overview of system behavior
• Draws attention to potential problem areas
• Suggests targets for further exploration
![Page 30: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/30.jpg)
30
Outline
• Motivation• The Rivet visualization environment
– Architecture– Implementation
• Focused visualization systems– SUIF Explorer– Thor– PipeCleaner– Visible Computer
• Ad hoc analysis and visualization• Contributions
![Page 31: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/31.jpg)
31
Ad hoc Analysis and Visualization
• SimOS: Complete machine simulator– Full non-intrusive access to HW, OS, SW
state– Flexible data collection mechanism– Deterministic execution
• Combine with Rivet– Flexible data import mechanism– Rapid prototyping of visualizations
• Result: powerful analysis framework
![Page 32: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/32.jpg)
32
Change HW model or SW
Simulation & Visualization Cycle
Configure simulated machine and
software
Perform simulation
Visualize results
Change annotations
Change visualization
![Page 33: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/33.jpg)
33
Case Study: Argus
• Parallel, multithreaded graphics library– NURBS rendering application – Implemented using fork & shared memory
region
• Performance limitations on SGI Origin– Linear speedup up to 26 processors– Rapid performance falloff beyond 26 CPUs
• Imported into SimOS environment– Same performance characteristics observed
• Expectation: memory system is the problem
![Page 34: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/34.jpg)
34
Memory Visualization
![Page 35: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/35.jpg)
35
Memory Visualization
1. Histograms of memory stall time vs. physical/virtual address
![Page 36: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/36.jpg)
36
Memory Visualization
2. Memory stall time incurred by each line of source code
![Page 37: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/37.jpg)
37
Memory Visualization
3. Local/remote stall time and idle time for each process
![Page 38: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/38.jpg)
38
Memory Visualization
Large amounts of unexpected idle timeVery little memory stall time in process view
![Page 39: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/39.jpg)
39
Visualization of Process Data
Processes go idle in kernel pfault/vfault calls
![Page 40: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/40.jpg)
40
Process Scheduling & Kernel Locks
CPU schedulingkernel lock
If the lock is unavailable, the process is descheduledIf the lock is available, it is immediately granted
![Page 41: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/41.jpg)
41
Process Scheduling & Kernel Locks
kernel lock is held
process is descheduled
Kernel lock is heavily contended
![Page 42: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/42.jpg)
42
Results With Processes Pinned
Lots of idle time and kernel lock contention still remain
![Page 43: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/43.jpg)
43
Results Using sproc
• Minimal kernel time• No idle time at all• Completes nearly
twice as fast as original version
• 95% parallel efficiency
![Page 44: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/44.jpg)
44
Argus: Conclusion
• Five simulation & visualization iterations• Initial suspicions were totally incorrect• Flexibility of Rivet & SimOS enabled us
to follow leads• Visualizations led us to the bottleneck
– Single scheduling event out of over 50,000 events
![Page 45: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/45.jpg)
45
Contributions
• Rivet computer systems visualization environment– Rapid prototyping support
• Quick ‘rough cut’ visualization of data• Incremental development of sophisticated data displays
– Modular architecture• Mix and match basic building blocks to create visualizations• Enables coordination through object sharing
– Data transforms as first-class citizens in viz environment• Unifies the analysis and visualization process
• Collection of computer systems visualizations– Emphasize interactive data exploration– Use relatively conventional visual representations– Provide extensive support for filtering, sorting, brushing
![Page 46: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/46.jpg)
46
Acknowledgments
• Orals committee: Joel Ferziger, Mendel Rosenblum, Pat Hanrahan, Mark Horowitz, Monica Lam
• Mendel• Visualization collaborators
– PipeCleaner: Donald Knuth– Visible Computer: John Gerth– Argus: Gordon Stoll– Thor: Jeff Gibson– SUIF Explorer: Shih-Wei Liao
![Page 47: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/47.jpg)
47
Acknowledgments
• Groupmates: Rivet, SimOS, FLASH, Graphics
• Studio 354– Steve Herrod and John Heinlein– The foosball table– Chris Stolte and Diane Tang
• Staff: – John, Charlie, Thoi, Kevin– Ada, Heather, Chris, …
• Funding agencies: ONR, [D?]ARPA, ASCI• Robert Bosch Corporation
![Page 48: Using Visualization to Understand the Behavior of Computer Systems](https://reader034.fdocuments.us/reader034/viewer/2022051820/56814a8e550346895db79ca4/html5/thumbnails/48.jpg)
48
Acknowledgments
• Rains 7A: Steve, Hoa, Marco• Friends• Family• Ming