Kirk A. Killian. Corporate America Needs New Data Centers 88% of Mid-market Companies (< $500...

Kirk A. Killian

Strategic Data Center Procurement

Corporate AmericaNeeds NewData Centers

88% of Mid-market Companies (< $500 million revenues) plan to acquire/build or expand data centers in the next 4 years.

91% of Mid-market Companies expect equal or greater annual expenditures next year than last year for data center facilities.

• Most corporate IT departments will procure (build or buy/lease) additional data center capacity in the next few years.

Strategic Data Center Procurement 2

Demand Driversfor AdditionalData Center Capacity

46% - Consolidate infrastructure (shrink # of facilities) 41% - Replace aging facilities38% - Support organic growth31% - Expand geographic coverage29% - Improve business continuity25% - Improve energy efficiency21% - Change sourcing models (colocation, cloud, etc.)16% - Reduce operating costs

• Needs vary by organization; many companies have multiple reasons for needing new data centers.

• Cost reduction is always important, but is not usually the most significant factor.


Importance toIT Procurement Professionals

Cross-functional team (procurement, IT, facilities, legal, CFO)• Cross-Organizational Visibility

Expensive projects (often $20 - $250 million)• Upper Management Scrutiny

Infrequent purchases (once every few years vs. many purchases every year for hardware & software)• Process Unfamiliarity

This session will inform the IT Procurement Pro about this niche, and help you strategically provide value to your organization, avoiding expensive, high-visibility missteps.


Overview –Data Center Delivery Models

Design, Build & Self-Operate (traditional corporate model)• Company builds its own non-shared data center and

operates it for many years. All hardware and software is owned.

Colocation Suite• Company leases a non-shared private suite within a

large colo data center. All hardware and software is owned.

Cloud• Company gets use of specific hardware/software owned

by the provider. Location and operations practices determined by provider. Hardware (maybe software) owned by cloud provider.


Brief DiscussionOn Cloud

Cloud delivery model not discussed in depth in this session

• Rapidly growing sector – many companies are moving part of their computing to the cloud, especially test/dev, storage, backup, and special projects with extreme scaling or short durations

• Can be economical, flexible, rapidly deployed• Control measures less defined than in corporate data

centers• Can present control, reliability, audit and security issues

(reliability varies widely by provider, location and service level)

• Embraced warmly by internet/emerging industries; cautiously by heavily regulated industries (finance, healthcare, insurance)

• Incorporating cloud into the overall mix is a powerful tool for adding flexibility and controlling costs


Overview – Traditional Delivery Model: Design,Build & Self-Operate

Company designs a data center facility, builds it, and self-operates it

Design can start with raw land or “shell building” suitable for data center operations

Design Decisions: Tier Rating, Electrical Density, Raised Flooring, Modular

Tier Rating (reliability) and Electrical Density are biggest variables affecting construction cost

• Owner/operator company maintains maximum control over its investment level, operating costs, and facility managementStrategic Data Center Procurement 7

Construction Costs

Data center facilities are very expensive

Large companies commonly spend $10 million on land and >=$100 million on buildings and critical systems (plus hardware/software) with future phases costing even more • $5-30 million is more common for mid-sized Fortune

1000

Data center developers can construct “turnkey” build-to-suit data centers pursuant to a long-term lease, converting a CapEx spend to a long-term lease of building

• To maximize negotiations leverage, keep multiple locations, developers and delivery paths open for consideration


Example Budget:Building a Ground-UpData Center

Example: (1) 10,000 SF data hall w/ 1,200 kW, expandable into (2) 10,000 SF halls with 1,200 – 2,400 kW each

Land $ 2,100,000 (8 acres at ~ $6.00/SF)Building/Site Work $ 4,500,000 (60,000 SF at $75/SF)Critical Systems $10,800,000 (1,200 kW @ $9,000/kW) Design/Fees $ 800,000Contingencies (~10%) $ 1,800,000Total (approx.) $20,000,000 = $16,667/kW (first 1.2 mW)

• Above example design allows physical expansion of raised floor area within building, and separately, increase in power/cooling density within both data center phases = Future Flexibility


Design/Build Timeline –New ConstructionTakes a Long Time!

Example schedule below doesn’t include internal scoping & approvals (inherently political), which can add months/years!

6 months Site Selection/Approvals3 Months Site/Building Acquisition6 months Design & Permitting9 Months Construction3 Months Commissioning & Move-In27 Months Total – Start to Finish

• Converting an existing building to data center use can usually save 3-9 months

• Really agile companies can compress and overlap tasks to complete new project in 18 months, but that is rare

• Most common source of delays is getting internal approvals!


Facility Reliability& Tier Ratings

The Uptime Institute has established the most widely used data center reliability rating system, with (4) tier levels:

• Most corporate and colo data centers built since 2005 are Tier 3

• Most 10-20 year old corporate data centers are Tier 2• Most small data centers within office buildings are Tier 1• Higher Tier levels cost more to construct and operate• Few data centers get formal TUI certification, but are

instead designed in “substantial compliance” w/ TUI guidelines


Electrical Density/PhysicalExpandability

Most corporate IT departments are deploying computing equipment at 100 – 150 watts/SF (some much higher)

Most 1990’s era data centers provide +/- 75 watts/SF

Most new data centers designed to deliver 150 – 300 watts/SF, including colocation facilities

• “Value engineer” new design/build to initially deliver 150-175 watts/SF (or current needs + 20%), then add more power later

• Design new data center to allow physical addition of future additional data halls within the original building structure


New Trend –Modular Data CenterConstruction

Data Center “Modular Kits” have factory-built major components, shipped to site and inter-connected

• Less customization (standard sizes and configurations)• 10-15% lower upfront costs• Higher initial reliability (in theory: factory built vs. site

assembled)• Faster timeline – can reduce schedule by 6-12 months if

no customization desired• Long-term reliability and expandability TBD• Vendor-specific reliance risks TBD

• Modular components can be a great solution to construct a new data center quickly within a building you already own!


Geography inNew Data CenterSite Selection

Preference by many in IT to locations near existing data centers or operations staff (“server huggers”)

For “active/active” data center pairs, proximity to existing data centers important for telecom latency (<= 40 miles apart)

Lower construction costs in rural and southern/midwestern metros; research resource is Means Construction Cost Index

Cold, dry climates maximize potential for “free cooling”, reducing electricity load by 10-20%.

Lower electricity costs in south/midwest, higher in northeast and California; “hydro” utilities are lowest

Green “fuel mix” important to some companies


Avoidance ofHazards/Risks

Avoid sites that are:• <= 12 miles to paired “active/active” data center• <= 100’ to nearby roads• In seismically active areas (UBC zones 3 & 4)• Within 100-year and 500-year flood zones• < .5 mile to freight railroad ROW or HazMat or explosive

storage/production facilities• < .25 mile to major interstate highway• <= 2 miles to commercial airport & flight paths

Note: These are guidelines only; many data centers are near airports, railroads and highways, and in seismic zone 4 (California)Strategic Data Center Procurement 15

EconomicIncentives

Available in many forms for new data center projects:• Property Tax Abatements (real and personal property

taxes)• Sales Tax Waivers (for building systems and computers)• Utility Enhancements (e.g. redundant feeders)• Utility Tariff Subsidies (typically 1-5 year rate discounts)• Cash Grants (difficult to get)• Job Grants & Training Programs (small, but easy to get)• Expedited Permitting (accelerates construction)• Free/Discounted Land Sites

Strategically negotiate incentives; amounts can be millions!

Keep multiple locations in consideration to maximize value


Site InfrastructureIssues

Desirable site attributes include:• Electricity from redundant sources (2 feeders, preferably

from 2 substations)• Power availability at medium voltage (12,470v –

50,000v), which is better than 480v• Multiple redundant fiber optic networks nearby• Dark fiber optic network nearby• Light Industrial zoning to allow diesel backup generators• >= .25 mile to residential neighborhoods, where

homeowners can protest and delay data center development


Data Center Build-Outin Existing Buildings

Seek buildings with:• >= 16’ clear height• >= 100 pounds/SF floor loading capacity• Concrete panel or concrete block exterior construction• Single-story building preferable• No hazardous co-tenants (single-tenant buildings

preferable)• No co-tenants above data center (water penetration risk)

Suggest phased build-out (multiple “data halls” within building) to conserve capital and reduce operating costs


Summary: Design,Build & Self-OperatedData Centers

Advantages• Lower operating costs if long-term needs predictable &

stable• Maximum control over operations, audits & reporting (“It’s

Yours – Operate It How You Want”)• Incentives offset some upfront costs

Disadvantages• Slow to construct & deploy• Expensive to build (large CapEx investment)• Inflexible specifications/quantities (“You Built It – Live

With It”)


“Top 10 List”:Procuring Self-Operated Data Centers

Adding Value as an IT Procurement Pro

1. Allow ample project time; start planning 3-5 years before new data center must be operational. If less time is available, you may need to construct in an existing building or procure colo.

2. Carefully define facility needs: space, power, reliability, and design preferences. Force users to forecast growth to avoid overbuilding or under-expandability. Resist “Taj Mahal” criteria.

3. Create an effective multi-disciplined project team (IT, engineering, real estate, legal). If needed to supplement in-house knowledge, hire expert consultants (architects, engineers, site selectors, incentives advisors, legal counsel).

4. Select location wisely to reduce utility costs, improve telecom connectivity and reduce business continuity risks.


“Top 10 List”:Procuring Self-Operated Data Centers

5. Challenge “server hugger” mentality that dictates new data center location must be near existing facilities.

6. Suggest phased build-out allowing future expansion phases as space needs dictate.

7. Suggest phased power increases; similar to space phasing methodology in #6 above.

8. Investigate and procure economic incentives, often overlooked by IT departments - can be worth millions!

9. Evaluate modular “plug-in” data center systems components, and suggest multi-vendor component consideration.

10. Explore developer sale/leasebacks to shift CapEx expenditures to OpEx rent if desired by corporate finance.


2nd DeliveryModel: Colocation

In data center Colocation, a third party provider constructs and operates a data center (similar in design to corporate data centers), leasing private suites to corporate IT departments - similar to an office building with different tenants

• In the colo model, the corporate IT department owns its hardware and software, but places it within a private suite or cage, which is inside the larger colo data center building


Modern Colo: DedicatedInfrastructure

Trend: “Private Suite” colocation = less infrastructure sharing• Key feature for corporate customers - suites unlikely to

share PDUs, CRACs, UPS (“data center within a data center building”)

Example: Colo suite provides 1,200 kW to (1) customer• PDUs and CRACs non-shared within suite• N+1 UPS modules (3 x 750 kVa) non-shared serving this

suite• Primary switchgear, generators, chillers not shared


Key Colo Terminology

Provider – the company operating the colocation facility Suite – the premises occupied solely by one customer, often with hard walls

and non-shared infrastructure components Cage – same space concept as “Suite”, but smaller and less likely to have

non-shared CRACs and UPS modules Critical Load – the IT equipment load, typically backed up by UPS and

generators Carrier Neutral – customer selects whichever 3rd party telecom carriers it

chooses to serve its suite; offers cost advantage. Managed Services – available from colo provider in addition to the basics of

space, power and telecom circuits; may include server re-boots, rack and stack, server builds, tape changes, tape shipping, SAN/NAS usage, firewall, load balancing, intrusion detection, and “cloud”. Managed services leverages the colo provider’s staff for users too small for local FTEs.


Telecom Optionsin Colo Facilities

Key competitive advantage of many colocation facilities is carrier selection and carrier neutrality• Colo may provide access to 6-30 telecom carriers’

networks, offering wide choice of carriers, services and pricing

• Circuit installations are relatively fast (+/- 60 days) and cheap – they’re already in the colo’s “meet-me” room!

• Most colos (except those operated by telecom companies) are “carrier neutral”; each customer can choose its carriers

• Select a colocation facility with at least one dark fiber option

Carrier variety creates negotiating leverage; savings and contract flexibility can be huge for IT Procurement Pros!


Colo Industry Development:1998 to Present

Industry founded in 1998; early pioneers were startups, and many crashed with “dot-com” bust. Huge growth since 2005.

Newer facilities have high reliability - mostly TUI Tier 3

High electrical densities, mostly 125 – 250 watts/SF

Experienced, sophisticated providers, SSAE16 audits, PCI compliance, and professional operations/reporting

Financially stable providers, many now publicly-traded

Occupancy costs have decreased steadily since 2007

More dedicated (non-shared) infrastructure now

• Greater acceptance by Fortune 1000 IT departments!


ColocationGeographicAvailability

While colocation is available in all major cities, the largest clusters are in (10) U.S. metro areas:• San Jose, Washington, NY/NJ, Dallas, Chicago (the “Big

5”), plus Los Angeles, Boston, Atlanta, Phoenix, and Seattle

• Emerging market leaders are Austin, Las Vegas, Houston, Portland, Minneapolis, Denver and Miami

• Providers include telecom companies, colo specialists, and publicly-traded Real Estate Investment Trusts (REITs)

Colo provider variety creates negotiating leverage; savings and contract flexibility in most competitive markets is significant!

Keep multiple markets under consideration, and include at least one “discount” provider in selection process


InternationalColocationProcurement

Many US companies procure colo space internationally to support local or regional IT initiatives• The largest international “money center” cities offer a

variety of modern colocation data centers• Telecom latency and taxation issues can significantly

affect the location decision (e.g. Dublin popular for tax reasons)

• International data ownership and data privacy laws (e.g. China) also affect the data center location decision.

• Data center space/power tends to be 10-40% more expensive overseas than in the US; London is particularly pricey

• A “last resort” colo option in most countries is the local phone company; usually very expensive and inflexible


ColocationPricing Models

The most common colo pricing models are:• Wholesale: Rent in $/kW of Capacity + Metered

Power; typically for critical loads >= 150 kW. Metered power includes “uplift” of ~ 60% for cooling loads.

• Retail: Cost/SF + Electrical Circuits; typically for critical loads below 150 kW.

Retail colo placements can be effective for small corporate deployments (2-10 cabinets) or those in remote locations

Wholesale colo pricing models tend to be ~ 30% cheaper than Retail, and customized for corporate IT customers

Seek “Wholesale” pricing even if quantity is “Retail”


Example Colocation Pricing

Example: 4,000 SF data center suite with 500 kW of critical power (125 watts/SF) for (125) 42u IT equipment cabinets• Typical wholesale pricing is $150/kW/month for the

installed critical power capacity (including raised floor space), plus a separate charge for the electricity consumed (kW hours x $.06/kWh), including a 60% cooling uplift factor.

$75,000 Rent/month (500 kW x $150/kW)$34,560 Metered Electricity (500 kW x 720 hours/month

x $.06/kWh x 1.60 “cooling uplift factor”)$109,560 Occupancy costs (monthly for 5 years)

• In a “lease vs. buy” analysis, the $900,000 example annual rent ($75,000/month x 5 years) compares to a ~ $12 million construction cost to build a data center of similar capacity


Service LevelAgreements(SLA)

Colo operators provide an SLA stipulating downtime allowed before financial compensation is provided to the customer

Colo SLAs are skimpy; one day’s fees credit for each downtime event. No customer wants downtime triggering SLA credits (downtime risks/costs exceed the compensation)

One of the most powerful incentives toward proper facility maintenance is the provider’s reputation, and uptime performance is generally excellent at new colo facilities

• Negotiate early termination options (plus financial credits) for repeated/extended events of downtime

• Negotiate that any downtime event of any duration triggers a credit, not only events that exceed a minimum duration


“Chemistry”With Colo ProviderIs Important

Procuring a colo data center involves a continuous cycle of requests from both parties; getting along well with the provider is key to a successful long-term relationship

Invest time during the selection process to insure that your IT team communicates well with the vendor’s team, in addition to verifying competency from the vendor’s staff

Some companies want a formal, all business approach; others appreciate a looser “silicon valley” vendor style

It’s a painfully long contract term if either party doesn’t like doing business with the other side

• Check customer references from finalist providers to learn contract terms existing customers wish they had obtained


ColocationCompared ToSelf-Operated

Fast Deployment. Users can build-out and occupy colo suites/cages in months instead of years.

Low CapEx Costs. Colo users only pay upfront for physical construction of the suite/cage plus electrical and network distribution.

Higher Occupancy Costs. Compared to owned data centers, colo occupancy cost (rent) tends to be a little higher.

Flexible Contracts. Colo contracts offer expansion options for “Pay As You Grow”, and short lease terms.

Managed Services Options. At most colos, managed services are optional and ala carte, and less expensive than hiring FTEs.

Less Operational Control. Colo customers have less control over critical systems operations, audits, security, and reporting.

Lower Staffing Requirement. The colo provides all facility and critical systems maintenance.Strategic Data Center Procurement 33

Colo ProcurementProcess/RFPs

Colo procurement is similar to other IT procurement: Project Scoping, Vendor Candidate Selection, RFPs, Facility Tours, Proposal Analysis, Negotiations, and Contract Review.

RFPs should be sufficiently detailed to state the need, solution criteria, contract term, and preferred pricing models.

Use a “short form” RFP (6-8 pages) for small cages and a standard RFP (20-25 pages) for larger colo suites.

Most IT Procurement Pros distribute 4-8 colo RFPs.• RFPs should request flexibility in contract term, early

termination, expansion/contraction of scope, and ala carte managed services.

• To create negotiating leverage, send an RFP to at least one vendor in the most competitive geographic market under consideration, and at least one “competitively priced” vendor.


Baker’s Dozen: (13) Recommendationsfor Procuring Colo

Adding Value as an IT Procurement Pro

1. Allow ample project time; start planning 1-2 years before new data center must be operational, even though colo can be procured in as little as 60 days.

2. Carefully define facility needs: space, power density, reliability, systemic redundancies & design preferences). Force IT users to forecast growth to avoid over- or under-provisioning.

3. Create an effective multi-disciplined project team (IT, engineering, real estate, legal). If needed to supplement in-house knowledge, engage expert consultants (architects, engineers, site selectors, incentives advisors, legal counsel).

4. Select locations strategically to meet project objectives, reduce occupancy/utility costs, improve telecom connectivity, and include colo providers best suited to meet the company’s long-term needs.



5. Challenge colo vendors (and your IT team) to creatively devise contract flexibility – future changes in suite and power size, multiple locations within their portfolio, and switching services across their organization; e.g. substitute cloud spend for space/power spend.

6. Incorporate a combination of “must take” space/power increases with optional increases to best address expansion flexibility.

7. In addition to negotiating aggressively on initial pricing, get expansion provisions with multiple pricing options, such as current rate, ROFR on adjacent space, and “most favored nation”.

8. Get fixed “rate card” pricing for optional managed services, and early termination options on all managed services and cloud services.

9. Obtain sublease and contract assignment rights.



10. Get the right to bring new telecom carriers into the colo’s “meet-me” room, especially if you already have carrier discounts. Pick a colo with dark fiber, so you can bring in new cheaper carriers easily.

11. At the closing table, when you’re about ready to sign the new colo contract, don’t be bashful in asking the colo provider to cover all or part of your installation costs. (Many providers are willing to make one-time cost concessions to win contracts.)

12. Similar to other IT vendor relationships, end-of-quarter and end-of-year can be great times to sign new colo contracts, especially with publicly-traded providers eager to “make their numbers”.

13. Carefully scrutinize any under-construction facility not yet fully operational – demand detailed construction and commissioning schedules. Some colo providers “over promise” delivery dates.


Follow-Up Questions?


Kirk A. Killian. Corporate America Needs New Data Centers 88% of Mid-market Companies (< $500...

Documents

Transcript of Kirk A. Killian. Corporate America Needs New Data Centers 88% of Mid-market Companies (< $500...