1 Ashish Gupta Bin Lin Peter Dinda Department of Computer Science Northwestern University.
-
Upload
easter-palmer -
Category
Documents
-
view
216 -
download
1
Transcript of 1 Ashish Gupta Bin Lin Peter Dinda Department of Computer Science Northwestern University.
1
Ashish Gupta
Bin Lin
Peter Dinda
Department of Computer Science
Northwestern University
3
Outline
• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled User Study
• Its Results
• Conclusions
4
New Models of Distributed Computing
BreakScreenSaver time
Working
9 am 11 am 1 pm 3 pm 4 pm 5 pm 9 am
5
Observation and ideas
Idea: Why not borrow the unused resources ?
Unused Resources
Problem: Performance Slowdown
6
Effect of performance slowdown
7
Effect of performance slowdown
8
The question we want to answer
Depends on Other Factors as well…
What level of resource borrowing leads to user
discomfort for significant fraction of users ?
9
CPU RAM Disk
10
Context Person Hardware
Others ?
11
Power Users Typical Users Beginners
12
13
Outline
• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled Study
• Its Results
• Conclusions
14
How can we find this out ?
Actually test it with user feedback
A system for emulating resource borrowing and measuring user discomfort using direct feedback
http://comfort.cs.northwestern.edu
15
How can we find this out ?
1Measure existing contention
16
How can we find this out ?
Emulate Resource Borrowing
Real Contention
Borrowed Contention
Final Contention
2 % CPU Usage vs. Time
17
How can we find this out ?
Allow User Feedback
3
F11
18
How can we find this out ?
Find MAX Level of tolerable contention
Borrowed Contention
Final Contention4
MAX
% CPU Usage vs. Time
19
How is it useful ?
Comfort = f (Contention)
To find a quantitative relationship between resource borrowing and user comfort
20
How is it useful ?
Enhances the degree of workstation sharing
21
Interactivity Aware Scheduling
Interactive
More intelligent scheduling of interactive tasks possible
via user feedback and our results
22
Outline
• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled Study
• Its Results
• Conclusions
24
The client design
25
Testcases
• Each testcase stores details of resource borrowing• Testcase consists of exercise functions for each resource
• Exercise function is a vector of contention values• Example:
CPU contention: [0,0.5,1.0,1.5,2.0] at 1 Hz
27
Flowchart of Testcase Execution
No Testcase running
Resource Borrowing Phase Begins
(Testcase Runs)
No user feedback User expressed discomfort
Testcase Exhaustsafter some time
Testcase Ends immediately
User feedback
28
Resource Exercisers
• CPU Exerciser– Contention describes the expected extra number of
threads in ready queue– Fractional resource borrowing using stochastic scheduling
methods– Validated to contention level of 10
• Disk Exerciser– Random seek and read/write in a large file (twice the
memory)– Validated to contention level of 7
• Memory Exerciser– Borrows a fraction of physical memory: from 0 to 1
29
Outline
• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled Study
• Its Results
• Conclusions
30
The control study description
• 33 undergrad and grad students
• 1.5 hrs each
• Each user was assigned 4 popular tasks to do– MS Word– MS Powerpoint– MS Explorer searching and saving information– Quake III
31
Machine Configuration
32
Testcases for the Control Study
Three types of testcases: step, ramp and blank
• Ramp allows us to know the point of discomfort for each resource• Step and Ramps: comparing the manner of borrowing resources• Blank: To test the background level of discomfort
34
Testcases for the Control Study
• Testcases different for each application–Some apps more intensive than
others
• Example : Word and Quake
• Requires Calibration
35
Outline
• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled Study
• Its Results
• Conclusions
36
Resource borrowing vs User Discomfort
• CDFs for discomfort contention level
• Not all contentions cause discomfort: exhausted region
37• C0.05 = 0.35 ( aggregated over all applications)
Exhausted Region
Discomfort Region
Aggregated Results
38
Resource borrowing vs User Discomfort
• CDFs for discomfort contention level
• Not all contentions cause discomfort: exhausted region
• CDFs allow us to read %age of people discomforted for a given contention
–Metric c0.05 : At what contention do we
discomfort only 5% of the people ?
39
Results overview
Aggregated Results
Dependence on Resource
Dependence on Context
Dependence on Users
Dependence on Step/Ramp
40
Aggregated Results
41• C0.05 = 0.35 ( aggregated over all applications)
Exhausted Region
Discomfort Region
Aggregated Results
42
• Can borrow a lot, discomforting only a small fraction\
• C0.05=0.33
Aggregated Results
Exhausted Region
Discomfort Region
43
• Can borrow a lot, discomforting only a small fraction• C0.05 = 1.11
Aggregated Results
Exhausted Region
Discomfort Region
44
Dependence on Resource
45
An overview of factors Dependenceon Resource
User sensitivity by task and resource (Low, Medium, High)
Users more tolerant for disk and memory borrowing, than CPU
Results Qualitative , but the testcase range is wide
46CPU
Dependenceon Resource
47Memory
Dependenceon Resource
48Disk
Dependenceon Resource
49
Dependence on Context
50
Comparison of CPU discomfort
WordIE
Powerpoint
Quake
Dependenceon Context
51
Dependence On Context - CPU
Word
Powerpoint
IE
Quake
Dependenceon Context
52
Dependence on Context - Memory
Word
Powerpoint
IE
Quake
Dependenceon Context
53
Dependence on Context - Disk
Word
Powerpoint
IE
Quake
Dependenceon Context
55
Internet Wide-Area Study
• No controls here, hope to get more general and complete data
• Users can download a client from http://comfort.cs.northwestern.edu
• Clean installation/un-installation and privacy options
Join us !
56
Outline
• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled Study
• Its Results
• Conclusions
57
Conclusion
Comfort = f(contention)
Disk and memory can be borrowed more aggressively, CPU less so
A throttle for resource borrowing very useful
CDFs can be used to set the throttle
58
Conclusion
Comfort = f(contention)
Knowing the user context helps
Using Direct user feedback can be useful
59
Current and Future Work
User feedback for interactive VM Scheduling
Real time Scheduling to give exact resources to VMs
Minimal resources required to keep the user happy
60
Acknowledgements
http://comfort.cs.northwestern.edu
Special Thanks to:
Don Norman
Andrew Ortony
Ben Watson
Jack Tumblin
Leslie Lamport
and
All our beta testers
Study participants
61
Conclusion
Comfort = f(contention)
Disk and Memory surprisingly easy to borrow, CPU borrowing also liberal
Set of empirical CDFs provided
Depends of other factors: Resource, Context and Users
62
Dependence on User
63
Dependence on User
Power
Typical
Beginner
Dependenceon User
67
The placebo effect
• Users express discomfort even when no testcase is running• Can never guarantee that the user will never express
feedback
68
Memory Exerciser
• Contention indicates fraction of total memory allocated in addition to occupied memory
• Pages are touched frequently to keep them in physical memory
0MB 1024 MB
Occupied Mem. 0.5
Memory Contention
Clipping Point
69
70
Related Work
• User Frustration in HCI• Computer Response to User Frustration [K99]• Sensing and measurement of frustration with
computers [R01]
• Impact of latency of user utility• Behavioral aspects of text editors [EN81]• Psychological Upper and lower limits of
system response time and user’s preference on skill level [K97]
71
Virtual Machine Scheduling
• Virtuoso: Distributed Computing with Virtual Machines
• User pays for the resources he uses
• Need to know minimal resources needed to keep an interactive VM user happy
72
Questions and Answers• Other types of irritation• Bursty natures of Desktop Computing, bursts indicate sudden action-response tasks• Large number of factors for irritation• Since there is usually a power law for process run times, why do we need this model ? (CPU
is idle most of the time)• Why can’t we just run the bg app at a lower priority ? It may benefit much more than just
running at a fixed contention.• Memory exerciser: What does contention mean ? How do you find out free physical memory
?• How do you avoid thrashing ? Why do you avoid thrashing ?• How does the CPU exerciser work ? • Why do you disable buffering in the Disk exerciser ?• The aggregated graphs are not useful. Memory usage may be much more dynamic then in
your tests due to user multitasking etc. So this may not represent reality ?– Our study is a start in this direction and we realize that there are limitations to the data we have
collected. But we expect to collect and analyze more results from the Internet Wide area study and generalize our results to more real environments. There are no controls in the internet wide area study. The control study was meant to specifically answer the questions we have posed, like the affect of other factors in user discomfort and resource borrowing.
73
• Lower priority ?– Disk and memory ?
– There is a limit (max) possible amount of cycles stolen. Our goal is to s=estimate that. Seperately, there are many possible sched mech that would get closer to that max. Lower priority is an example. Linger longer is an example. Screen saver is an example.
– Quake example
– Don’t argue for/against a particular scheduling scheme
– It is a step in the right direction… (for results)
74
Testcase Transfer
Results Transfer
Server
Client
Resource borrowing using testcase profile
75
The client-server system
• Data Exchange• Hotsync
– Results transfer– Testcase update
• Testcase Tools
76
77
78
An irritating Interface
79
Interactive Computation
Bursty Nature of Interactive Computation
80
81
82
Understanding Testcases
CPU
ramp 1ramp 2ramp 3step 1step 2
…..
MEM
ramp 1ramp 2ramp 3step 1step 2
…..
Ramp Step
83
Combination of Resources
• CPU Usage• Memory Usage• Hard Disk activity• Network Traffic
Comfort Zone surface
85