The Impact of Soft Resource Allocation on n-tier Application Scalability
description
Transcript of The Impact of Soft Resource Allocation on n-tier Application Scalability
![Page 1: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/1.jpg)
The Impact of Soft Resource Allocation on n-tier Application
Scalability
Qingyang Wang, Simon Malkowski, Yasuhiko Kanemasa, Deepal Jayasinghe, Pengcheng Xiong,
Motoyuki Kawaba, Lilian Harada, Calton Pu
![Page 2: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/2.jpg)
2225th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
![Page 3: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/3.jpg)
3325th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Cloud Computing EnvironmentGood performance + Cost efficiency
Scaling applications on demand
BottleneckBottleneck
High throughput + low response time
High resource utilization
![Page 4: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/4.jpg)
4425th IEEE International Parallel & Distributed Processing Symposium
Soft resources in n-tier systems Threads, database connections, TCP
connections, locks, etc.
Soft Resource Allocation
19 May 2011
Bottleneck
Thread pool
Thread pool
Thread pool
Connection pool
Is it okay to duplicate the same configuration of soft resource allocation?
Thread pool
![Page 5: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/5.jpg)
5525th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
![Page 6: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/6.jpg)
6625th IEEE International Parallel & Distributed Processing Symposium
RUBBoS benchmark Bulletin board system like
Slashdot (www.slashdot.org) Typical 3-tier or 4-tier architecture Two types of workload
Browsing only Read/Write mix
24 web interactions
C-JDBC Middleware for database scale-out
Read: act as a load-balancer Write: send a request to all
databases to keep consistency
Experimental Environment (1)
19 May 2011
![Page 7: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/7.jpg)
7725th IEEE International Parallel & Distributed Processing Symposium
Emulab (http://www.emulab.net) Relatively modest testbed originally for
network research Virtual network & physical machines (not VM)
Experimental Environment (2)
19 May 2011
Hardware
Specifications
Server type
PC3000 in Emulab
Processor Xeon 3GHz 64bitMemory 2GBNetwork 1GbpsDisk 2 x 146GB
10,000rpm
![Page 8: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/8.jpg)
8825th IEEE International Parallel & Distributed Processing Symposium
Software setupsExperimental Environment (3)
19 May 2011
Function SoftwareWeb server Apache 2.0.54Application server Apache Tomcat 5.5.17DB clustering middleware
C-JDBC 2.0.2
Database server MySQL 5.0.51aJava Sun jdk1.6.0_14Operating system Redhat FC4System Monitor Sysstat 7.0.2
![Page 9: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/9.jpg)
9925th IEEE International Parallel & Distributed Processing Symposium
NotationExperimental Environment (4)
19 May 2011
400-6-2001 / 3 / 1 / 2 configuration
![Page 10: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/10.jpg)
101025th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Result of Motivational Experiment
5000
5200
5400
5600
5800
6000
6200
6400
6600
6800
7000
7200
7400
7600
7800
600650700750800850900950
10001050
Workload [# Users]
Thro
ughp
ut [R
eqs/
s]
400-150-60
400-150-60
400-6-6
400-6-6
1/2/1/2
1/4/1/4
Scale out
![Page 11: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/11.jpg)
111125th IEEE International Parallel & Distributed Processing Symposium
Hardware configuration
2
19 May 2011
Challenge
Hardware configuration
Soft resource allocation
Hardware configuratio
n 1
Soft resource allocation 1Thread pool,
DB connection pool
Thread pool, DB connection pool
Soft resource allocation 2
Good performance
Bad performance
Good performance
Scale out
How to choose a reasonable soft resource allocation to match the hardware configuration?
![Page 12: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/12.jpg)
121225th IEEE International Parallel & Distributed Processing Symposium
Evaluate two important soft resources
Threads Database connections
Show their performance impact on n-tier applications
Over-allocation & under-allocation cases
Special case of under-allocation Introduce a practical way to choose a
reasonable allocation of soft resources
Focus of This Paper
19 May 2011
![Page 13: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/13.jpg)
131325th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
![Page 14: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/14.jpg)
141425th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Performance Loss due to Soft Resource Over-allocation (1)
400-200-6
Sensitivity analysis: change DB Connection pool size in Tomcat
![Page 15: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/15.jpg)
151525th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
50
6
100200
Performanc
e degradation
Goodput 1/4/1/4 CPU utilization in CJDBC
6
200
Throughput with response time boundary
Performance Loss due to Soft Resource Over-allocation (2)
High allocation of DB connections in Tomcat degrades the system performance
![Page 16: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/16.jpg)
161625th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
JVM Garbage Collection Costs
Over-allocation of soft resources causes waste of critical hardware resources
200
6
8% more time used for GC over experimental time
JVM Garbage Collection in CJDBC
![Page 17: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/17.jpg)
171725th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
![Page 18: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/18.jpg)
181825th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Performance Loss due to Soft Resource Under-allocation (1)
400-6-200
Sensitivity analysis: change thread pool size in Tomcat
![Page 19: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/19.jpg)
191925th IEEE International Parallel & Distributed Processing Symposium
6
200
19 May 2011
Performance Loss due to Soft Resource Under-allocation (2)
Goodput 1/2/1/2
20
10
CPU utilization of Tomcat
20
6
Under-allocation of soft resources causes inefficient utilization of hardware resources
![Page 20: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/20.jpg)
202025th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
![Page 21: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/21.jpg)
212125th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Special Case of Soft Resource Under-allocation
30-6-100
Sensitivity analysis: change thread pool size in Apache
![Page 22: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/22.jpg)
222225th IEEE International Parallel & Distributed Processing Symposium
30-6-100
400-6-100
50-6-100100-6-100
19 May 2011
Performance Loss due to Under-allocation of Apache Threads
Goodput 1/4/1/4 CPU utilization in CJDBC
30-6-100
400-6-100
Low allocation of Apache threads degradesthe system performance
![Page 23: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/23.jpg)
232325th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Non-Trivial Correlation between Apache and Tomcat Threads (1)
30-6-100
30 > 6 * 4Why are 30 threads in Apache not enough?
![Page 24: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/24.jpg)
242425th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Non-Trivial Correlation between Apache and Tomcat Threads (2)
1: HTTP request
2
34: HTTP response
5: FIN reply Waiting for TCP FIN reply from the client
![Page 25: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/25.jpg)
252525th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Non-Trivial Correlation between Apache and Tomcat Threads (3)
1: HTTP request
2
34: HTTP response
5: FIN reply
Apache thread active period
Communicating with Tomcat
The long waiting time for FIN reply from clients is unpredictable and frequently happens under high workload
Waiting for TCP FIN reply from client
![Page 26: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/26.jpg)
262625th IEEE International Parallel & Distributed Processing Symposium
Concurrency for 30-6-100
Concurrency for 400-6-100
Large number of soft resources in front tier acts as a buffer providing stable workload for lower tiers
19 May 2011
Concurrency of Apache Threads
Connecting to Tomcat
![Page 27: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/27.jpg)
272725th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Summary of Experiments
1. Over-allocation of soft resources causes waste of critical hardware resources
2. Under-allocation of soft resources causes inefficient utilization of hardware resources
3. Large number of soft resources in front tier acts as a buffer providing stable workload for downstream tiers
![Page 28: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/28.jpg)
282825th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
![Page 29: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/29.jpg)
292925th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Soft Resource Allocation Algorithm Key idea: allocating soft resources
globally to utilize the critical hardware resource as efficiently as possible
![Page 30: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/30.jpg)
303025th IEEE International Parallel & Distributed Processing Symposium
1. Identifying the critical hardware resource first
Soft Resource Allocation Algorithm (1)
19 May 2011
5000
5600
6200
6800
7400
600
750
900
1050
WorkloadTh
roug
hput
Throughput vs. Work-load(1)
Bottleneck server
![Page 31: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/31.jpg)
313125th IEEE International Parallel & Distributed Processing Symposium
5000
5600
6200
6800
7400
600
750
900
1050
WorkloadTh
roug
hput
Throughput vs. Work-load
1. Identifying the critical hardware resource first2. Allocating proper soft resources for the bottleneck
server
Soft Resource Allocation Algorithm (2)
19 May 2011
Throughput knee
Minimum Saturation workload(2) Ave.
number of active threads
(1) Bottleneck server
![Page 32: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/32.jpg)
323225th IEEE International Parallel & Distributed Processing Symposium
(3) Dependency between current tier and bottleneck tier
1. Identifying the critical hardware resource first2. Allocating proper soft resources for the bottleneck
server3. Allocating proper amount soft resources in other
tiers
Soft Resource Allocation Algorithm (3)
19 May 2011
(2) Ave. number of active threads
(1) Bottleneck server
RTTTPL *
)Re/(* ratioratiocjdbcapache qRTTLL
ratiocjdbcapache qTPTP Re/
![Page 33: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/33.jpg)
333325th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Outline
Background & Motivation Background Motivational experiment
Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation
Solution A practical algorithm for good soft resource allocation
Conclusion & Future Works
![Page 34: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/34.jpg)
3434
Achieving good performance by scaling n-tier applications in Cloud requires a unified exploration of both hardware and software
Contributions: We showed allocation of soft resources has a big impact
on the total performance of an N-tier system We showed to decide a proper soft-resources allocation is
a complex problem, especially in cloud environments which requires dynamic scaling-out/in
We gave a practical algorithm for proper soft resource allocation
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Conclusion
![Page 35: The Impact of Soft Resource Allocation on n-tier Application Scalability](https://reader035.fdocuments.us/reader035/viewer/2022062315/5681637b550346895dd459a1/html5/thumbnails/35.jpg)
353525th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Future Work
Explore more soft resources such as locks, buffer/queue
Explore more efficient ways to find proper soft resource allocation
Explore the impact of soft resource allocation in virtualized environment