Micro Servers in Big Data
-
Upload
aater-suleman -
Category
Technology
-
view
3.573 -
download
0
Transcript of Micro Servers in Big Data
Presentation by: Aater Suleman, PhD
BACKGROUND
Hardware designer come parallel programmer
Core microarchitecture and many core design
Worked on parallel programming, compilers, task scheduling
Distributed application performance
The views are my own and do not represent those of my current and
past employers
PERFORMANCE OPTIMIZATION
CPU RAM Disk NIC
Optimization
App
Middleware (Hadoop,
Sector/Sphere, etc)
OS/Hypervisor
System Hardware
MICROSERVERS IN BIGDATA
AND THE MICROSERVER WORLD IS JUST
AROUND THE CORNER
Shipments of microservers will rise threefold this year
Microservers would change the face of computing
Estimates of adoption between now and 2015 vary, but
are as high as 49% compound growth rate for Micro
server adoption
BIG DATA NEWS AND GOSSIP WORLD EXCLUSIVES
THE
BIG DATA news
Going green with micro
servers
AV
AIL
AB
LE
MIC
RO
SE
RV
ER
S
MICROSERVERS
NE
ED
FO
R U
SE
MICRO SERVERS AVAILABLE
TODAY
Marvell, TI, nVidia
Intel ATOM based servers
Intel Server Calxeda Server
HOW ARE THEY DIFFERENT? N
EE
D F
OR
US
E
MICROSERVERS A
VA
ILA
BL
E M
ICR
O S
ER
VE
RS
Power-efficient
cores
Disk BW/comp
ute
Network bandwidth/compute
Computes/TCO-$
It is not that simple …
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
16 32 64 128 256 512 1024 2048 4096 8192 16384Bandwidth Requirement
NB
CB
Micro server becomes feasible due to cost
* CB – CORE BOUND ; NB – NETWORK BOUND
Traditional Server
Micro server
Syst
em
Ca
pa
city
WHEN TO USE MICROSERVERS?
When app is bandwidth bound and not CPU bound
When app scales well
When cost and throughput are more important
than latency
Data Size
CPU/
Memory
x
Data Size z
Disk
Bandwidth
Data Size y
Network
BW
Capacity =MIN x, y, z( )
1. Difference between x, y, z represents inefficiency
2. Traditional servers had these fixed
3. Microservers will have more choices
BENCHMARKS
Porting is not always feasible
Use performance monitoring to characterize app
Architecture independent benchmarks that test
sub-systems in isolation
SPECInt Rate for CPU/Memory
FIO (JBOD configuration) for disk
Iperf for Network
Compare Actual Cost (dollars)
Number servers = Requirement/capacity
Total Cost of ownership = (cost per server) x
number of servers
Don’t forget to future-proof the analysis
The requirements will change
What looks good today won’t look good tomorrow
EXPECT
Lots of differentiated platforms
New approaches
Asymmetric Clusters
Dedicated Networks
Shared local disks with remote cores
Optimized appliances
GPGPUs
Hardware accelerators
RECOMMENDATIONS
Keep Microservers on your Big Data roadmap
Keep their strengths and weaknesses in your mind
while you code
Keep your eyes and ears open to things that can
make a good benchmark
More on this on my blog :
futurechips.org
I am just a click away on
www.linkedin.com/pub/aater-suleman/3/21b/ba4