Going Modular?

Lessons from 500 modular data

center implementations around the

world

John M. Kundtz

Data Center Services Manager

IBM Global Services

jkundtz@us.ibm.com

© Copyright IBM Corp. 2011All Rights Reserved

Boston Seminar: Financial Services Infrastructure Management

Today’s Agenda

• Background – what are we seeing

• Definition of Modular Data Center

• Lessons Learned

• Summary

• More Information

Change is accelerating …

IT demand

82M Servers installed by 2013

20% Server workloads have been virtualized

650% Storage growth by 2012

<30% Disk storage is used effectively

Cost pressures

75% Of CIOs anticipate a strongly centralized infrastructure in 5 years

2-3% Increase forecast for IT spending

Need for flexibility

70¢ Of every $1 is spent on maintaining existing environment

71% Of data centers are > 7 years old

5-60% Of IT workloads may be cloud-enabled

Our research suggests that Clients can respond to these

challenges in 4 ways.

Future proof existing data center

infrastructureDouble IT capacity or reduce

operational expenses by 50%

Rationalize the data center infrastructure across the company

Improve operational efficiencies while reducing operational

expenses by 50%

Flexible design to be responsive to change

Pay as you grow by deferring 40-50% of capital and

operational costs

Integrated management of IT and data center operations Lower operational costs up to 20%

Capital cost gets 100% of the focus & are 15% of the total cost.

Data center capital costs60% costs from mechanical /

electrical systems

Shell 7%

Mechanical20%

Fees24%

Fit-Up Costs

9%

Instrumentation & Controls4%

Power36%

Source: IBM Estimates

Operating costs3- 5X capital costs

Cum

ulat

ive

Cos

t of O

pera

tions

($M

illio

ns)

Year

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Energy Cost

Staffing

Bldg. Maint. & Mgmt.

R E Tax

$100

$150

$200

$250

$50

0

Cum

ulat

ive

Cos

t of O

pera

tions

($M

illio

ns)

Year

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Energy Cost

Staffing

Bldg. Maint. & Mgmt.

R E Tax

Energy Cost

Staffing

Bldg. Maint. & Mgmt.

R E Tax

$100

$150

$200

$250

$50

0

Major cost savings is the deferral of 40-50% of the electrical and mechanical capacity until it is required.

We have analyzed over 500 global modular data center

design and builds in the last 3 years ...

Custom data centers

Scalable modular data centerSCALABLE MODULAR DATA CENTER

Enterprise modular data center data centerENTERPRISE MODULAR DATA CENTER PORTABLE MODULAR DATA CENTER HIGH DENSITY ZONE

Past industry definition of “ Modular Data Center”

• Modular data center = Shipping container

• Modular = subsystems such as UPS systems, rack/row level cooling components

• Prefabricated “modules” that provide the ability to easily add space, power and cooling

More flexible definition of modular data center

• Plan for unpredictability in requirements

• Provide capacity when you need it

• Upgrade with no disruption to operations

• Maximize flexibility for future technology and computing models

• Tailored to each client’s requirements, not a fixed architectural answer

• Suitable for mid-size and enterprise clients

Definitions of modular data centers …

1. Data center projects often go off track …

• No governance process to make decisions based on facts

• Operating vs. capital costs are not a standard part of the design and evaluation process

• The real estate, technology and operational management teams are not all equal players

Decisions made on

wants versus needs

• Plans need to reflect unpredictability of business and technology over 10-30 year period

• No clear or detailed statement of requirements

• Clients take on the responsibility by themselves with minimal current data center skills or experience

Unclear control over

how decisions are made

Not finding a trusted

partner from concept to completion

Business challengeServer power density trends1: 20x

Data Center challengeSupport expected IT kW/rack power density

2. Plan to maximize capacity & flexibility for technology &

computing models, investment is an upfront statement of

requirements …

1. ASHRAE Publication: Datacomm Equipment Power and Cooling Applications, 2005, page 24

Source: Page 4, Figure 1.3, Best Practices for Datacom Facility Energy Efficiency Second Ed. (c) 2009.

Kilo

wat

ts/r

ack

0

10

20

30

40

2000 2002 2004 2006 2008 2010 2012 20140%

5%

10%

15%

20%

25%

30%

35%

0 5 10 15 20 25 30 35

h

Kilowatts/rack

• Support 5 kw/rack and 32 kW/Rack for high density servers• Design for air cooling as primary; provide liquid cooling without disrupting operations• Able to support 2.5x power density growth by pre-plumbing infrastructure• Leverage IBM Rear Door Heat Exchanger as most cost-effective high density support

3. Plan for unpredictability to manage the lifecycle of a

data center. Dodge the next 2-3 retrofits by smart planning

for unknowns …

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

0 5 10 15 20 25 30 35

h

Based on IBM estimates from client engagements

Today 50% of server density is < 5 kw/Rack or less

202060% of server density is > 10-20 kw/rack

Kilowatts/rack

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

0 5 10 15 20 25 30 35

h

Racks loaded at 40%Racks at 90%

Kilowatts/rack

• Unknown loads. Create a density requirement model which plans for changes over time

• Unknown capacity. Support 3-5x power density over the 10 year timeframe

• Predicted load density is a pre-requisite to a better design

• Design included efficient dual zone capacity ; saving $1M USD per year for client

4. A Modular design allows for expansion of power &

cooling density with minimal disruption to operations …

05

101520253035

40

1x 3x

Pay now Pay later

• Size electrical and mechanical systems to meet future capacity needs

• Provision for potential water cooling

• Modular UPS design and chiller plant design

• Electrical switchgear sized to grow. Electrical distribution scalable per module

$ (M

illio

ns)

Power Density Growth (# times)

Piping for water cooling

Capital Costs5-10% up front to support 3x growth versus 60% higher cost & disruptive retrofit later

Good design is an insurance policy with a 5-10% premium versus paying 60% later.

5. No single electrical and mechanical approach is cost effective for all sizes and densities …

Example: 20,000 square foot data center, computer room air conditioners

Capital CostsCapital costs for computer room air

conditioners can vary by 5x

Under-floorCooling

CombinedSystems

Close CoupledCooling

$2

1.5

1

.5

0

$ M

illio

ns

Operational CostsOperational cost for computer room air

conditioners can vary by 140%

2

1.5

1

.5

0

Year

1 2 3 4 5 6 7 8 9 10

2.5

$3

$ M

illio

ns

Under-floor Cooling

Combined Systems

Close Coupled

Close Coupled, RDHX

• Although highly efficient, close coupled cooling doesn't always provide the lowest TCO.• Rear door heat exchanger is the lowest cost solution to support high density equipment.• Raised floor cooling is cost effective for low density environment.

Source: IBM Estimates

In Summary: Lessons learned from 500 modular data

center implementations include …

• Good modular design is an insurance policy with a 5% premium versus 60% later.

• Major cost savings is the deferral of electrical and mechanical capacity until it is required.

• No single electrical and mechanical approach is cost effective for all sizes and power densities.

Decisions made on

wants versus needs

• Insure your data center strategy can meet business and IT requirements over the long term.

• Best investment is an upfront statement of requirements.

• Plan for unpredictability. Dodge the next 2-3 retrofits by smart planning for unknowns.

• There is not a one-size-fits-all strategy.

• Find a partner who doesn’t use a fixed architectural approach.

Not finding a trusted

partner from concept to completion

Unclear control over

how decisions are made

Scalable modular data center.

Enterprise modular data center.

Portable modular data center.

High densityzone.

• Turnkey center for 500-2,500 sq ft.

• Implement in 8-12 weeks.

• Standardized design starting with modules as small as 5,000 sq ft.

• Save up to 50% operational costs.

• Rapidly deploy in 12-14 weeks.

• Without impacting operations.

Watch a video at :http://www-03.ibm.com/systems/data/flash/dynamicinfrastructure/datacenterdesignsolutions/

15-25% lower TCO than traditional data centers

Defer 40-50% of capex and opex cost

Fully functional data center

35% lower cost than site retrofit

IBM has designed over 500 modular data centers in the

last 3 years to be responsive to change …

Design new infrastructure for flexibility with modularity IBM’s data center familyTM solutions align to your business and cost objectives.

Portable modular data

center (PMDC)

Enterprise modular

data center (EMDC)

Scalable modular data

center (SMDC)

Modular data centers

Data center

consolidation and

relocation

Data center

strategy

Extend the life of an

existing data center

IBM site and facilities

services

For more information on “going modular”