The Impact of Soft Resource Allocation on n-tier Application Scalability Qingyang Wang, Simon...
-
Upload
reilly-lansdowne -
Category
Documents
-
view
212 -
download
0
Transcript of The Impact of Soft Resource Allocation on n-tier Application Scalability Qingyang Wang, Simon...
The Impact of Soft Resource Allocation on n-tier Application
Scalability
Qingyang Wang, Simon Malkowski, Yasuhiko Kanemasa, Deepal Jayasinghe, Pengcheng Xiong,
Motoyuki Kawaba, Lilian Harada, Calton Pu
2225th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
3325th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Cloud Computing Environment
Good performance + Cost efficiency
Scaling applications on demand
BottleneckBottleneck
High throughput + low response time
High resource utilization
4425th IEEE International Parallel & Distributed Processing Symposium
Soft resources in n-tier systems Threads, database connections, TCP
connections, locks, etc.
Soft Resource Allocation
19 May 2011
Bottleneck
Thread pool
Thread pool
Thread pool
Connection pool
Is it okay to duplicate the same configuration of soft resource allocation?
Thread pool
5525th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
6625th IEEE International Parallel & Distributed Processing Symposium
RUBBoS benchmark Bulletin board system like Slashdot
(www.slashdot.org) Typical 3-tier or 4-tier architecture Two types of workload
Browsing only Read/Write mix
24 web interactions
C-JDBC Middleware for database scale-out
Read: act as a load-balancer Write: send a request to all
databases to keep consistency
Experimental Environment (1)
19 May 2011
7725th IEEE International Parallel & Distributed Processing Symposium
Emulab (http://www.emulab.net) Relatively modest testbed originally for
network research Virtual network & physical machines (not VM)
Experimental Environment (2)
19 May 2011
Hardware
Specifications
Server type
PC3000 in Emulab
Processor Xeon 3GHz 64bit
Memory 2GB
Network 1Gbps
Disk 2 x 146GB 10,000rpm
8825th IEEE International Parallel & Distributed Processing Symposium
Software setups
Experimental Environment (3)
19 May 2011
Function Software
Web server Apache 2.0.54
Application server Apache Tomcat 5.5.17
DB clustering middleware
C-JDBC 2.0.2
Database server MySQL 5.0.51a
Java Sun jdk1.6.0_14
Operating system Redhat FC4
System Monitor Sysstat 7.0.2
9925th IEEE International Parallel & Distributed Processing Symposium
Notation
Experimental Environment (4)
19 May 2011
400-6-2001 / 3 / 1 / 2 configuration
101025th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Result of Motivational Experiment
5000
5200
5400
5600
5800
6000
6200
6400
6600
6800
7000
7200
7400
7600
7800
600
650
700
750
800
850
900
950
1000
1050
Workload [# Users]
Thro
ughp
ut [R
eqs/
s]
400-150-60
400-150-60
400-6-6
400-6-6
1/2/1/2
1/4/1/4
Scale out
111125th IEEE International Parallel & Distributed Processing Symposium
Hardware configuration
2
19 May 2011
Challenge
Hardware configuration
Soft resource allocation
Hardware configuratio
n 1
Soft resource allocation 1Thread pool,
DB connection pool
Thread pool, DB connection pool
Soft resource allocation 2
Good performance
Bad performance
Good performance
Scale out
How to choose a reasonable soft resource allocation to match the hardware configuration?
121225th IEEE International Parallel & Distributed Processing Symposium
Evaluate two important soft resources
Threads Database connections
Show their performance impact on n-tier applications
Over-allocation & under-allocation cases
Special case of under-allocation
Introduce a practical way to choose a reasonable allocation of soft resources
Focus of This Paper
19 May 2011
131325th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
141425th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Performance Loss due to Soft Resource Over-allocation (1)
400-200-6
Sensitivity analysis: change DB Connection pool size in Tomcat
151525th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
50
6
100200
Perfo
rman
ce
deg
rad
atio
n
Goodput 1/4/1/4 CPU utilization in CJDBC
6
200
Throughput with response time boundary
Performance Loss due to Soft Resource Over-allocation (2)
High allocation of DB connections in Tomcat degrades the system performance
161625th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
JVM Garbage Collection Costs
Over-allocation of soft resources causes waste of critical hardware resources
200
6
8% more time used for GC over experimental time
JVM Garbage Collection in CJDBC
171725th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
181825th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Performance Loss due to Soft Resource Under-allocation (1)
400-6-200
Sensitivity analysis: change thread pool size in Tomcat
191925th IEEE International Parallel & Distributed Processing Symposium
6
200
19 May 2011
Performance Loss due to Soft Resource Under-allocation (2)
Goodput 1/2/1/2
20
10
CPU utilization of Tomcat
20
6
Under-allocation of soft resources causes inefficient utilization of hardware resources
202025th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
212125th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Special Case of Soft Resource Under-allocation
30-6-100
Sensitivity analysis: change thread pool size in Apache
222225th IEEE International Parallel & Distributed Processing Symposium
30-6-100
400-6-100
50-6-100
100-6-100
19 May 2011
Performance Loss due to Under-allocation of Apache Threads
Goodput 1/4/1/4 CPU utilization in CJDBC
30-6-100
400-6-100
Low allocation of Apache threads degradesthe system performance
232325th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Non-Trivial Correlation between Apache and Tomcat Threads (1)
30-6-100
30 > 6 * 4
Why are 30 threads in Apache not enough?
242425th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Non-Trivial Correlation between Apache and Tomcat Threads (2)
1: HTTP request
2
34: HTTP response
5: FIN reply Waiting for TCP FIN reply from the client
252525th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Non-Trivial Correlation between Apache and Tomcat Threads (3)
1: HTTP request
2
34: HTTP response
5: FIN reply
Apache thread active period
Communicating with Tomcat
The long waiting time for FIN reply from clients is unpredictable and frequently happens under high workload
Waiting for TCP FIN reply from client
262625th IEEE International Parallel & Distributed Processing Symposium
Concurrency for 30-6-100
Concurrency for 400-6-100
Large number of soft resources in front tier acts as a buffer providing stable workload for lower tiers
19 May 2011
Concurrency of Apache Threads
Connecting to Tomcat
272725th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Summary of Experiments
1. Over-allocation of soft resources causes waste of critical hardware resources
2. Under-allocation of soft resources causes inefficient utilization of hardware resources
3. Large number of soft resources in front tier acts as a buffer providing stable workload for downstream tiers
282825th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
292925th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Soft Resource Allocation Algorithm
Key idea: allocating soft resources globally to utilize the critical hardware resource as efficiently as possible
303025th IEEE International Parallel & Distributed Processing Symposium
1. Identifying the critical hardware resource first
Soft Resource Allocation Algorithm (1)
19 May 2011
5000
5600
6200
6800
7400
600
750
900
1050
WorkloadTh
roug
hput
Throughput vs. Work-load(1)
Bottleneck server
313125th IEEE International Parallel & Distributed Processing Symposium
5000
5600
6200
6800
7400
600
750
900
1050
WorkloadTh
roug
hput
Throughput vs. Work-load
1. Identifying the critical hardware resource first
2. Allocating proper soft resources for the bottleneck server
Soft Resource Allocation Algorithm (2)
19 May 2011
Throughput knee
Minimum Saturation workload(2) Ave.
number of active threads
(1) Bottleneck server
323225th IEEE International Parallel & Distributed Processing Symposium
(3) Dependency between current tier and bottleneck tier
1. Identifying the critical hardware resource first
2. Allocating proper soft resources for the bottleneck server
3. Allocating proper amount soft resources in other tiers
Soft Resource Allocation Algorithm (3)
19 May 2011
(2) Ave. number of active threads
(1) Bottleneck server
RTTTPL *
)Re/(* ratioratiocjdbcapache qRTTLL
ratiocjdbcapache qTPTP Re/
333325th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
3434
Achieving good performance by scaling n-tier applications in Cloud requires a unified exploration of both hardware and software
Contributions: We showed allocation of soft resources has a big impact
on the total performance of an N-tier system
We showed to decide a proper soft-resources allocation is a complex problem, especially in cloud environments which requires dynamic scaling-out/in
We gave a practical algorithm for proper soft resource allocation
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Conclusion
353525th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Future Work
Explore more soft resources such as locks, buffer/queue
Explore more efficient ways to find proper soft resource allocation
Explore the impact of soft resource allocation in virtualized environment
Thank You. Any Questions?
Qingyang [email protected]