W H I T E PA P E R

£

€€

¥

$£

¥

€

$

$

¥

¥

$

£

A Quick Primer on Data Center Tier Classifi cations

Bob Landstrom: CGEIT, CISSPDirector of Customer Solutions Engineeringblandstrom@switchanddata.com

W H I T E PA P E R D ATA C E N T E R T I E R C L A S S I F I C AT I O N S

Table ofContents

Today a number of companies are either consolidating their data centers or implementing new data center projects. Their decisions are often based on the appropriate “tier level” of the IT facility. The tier level is determined by an industry standard classifi cation system for infrastructure performance. A four-tier system rates a facility’s ability to support uninterrupted operation of its data processing activities.

The tier classifi cation model provides an academic and objective benchmark that helps describe and compare the functionality, capacity, and cost of data center infrastructures. At times, the drive to align the uptime of the IT facility with the business becomes bogged down in focusing on tier levels. Other factors beyond tier level compliance can impact uptime performance. In this paper, we will fi rst review data center tier levels, then comment briefl y on what is and is not critical to tier level attainment, and conclude with some pragmatic advice about facility assessment.

As in most areas of business, we rely on industry standards as guidelines to help us avoid the dangers and ambiguities of qualitative references. Instead of “very robust,” “fully fault-tolerant,” “top-notch,” or “Class-A,” we use industry standards to articulate quantifi able measurements.

The American National Standards Institute (ANSI) and the Telecommunications Industry Association (TIA) are examples of organizations that formulate standards for the industry to follow. The TIA developed a specifi cation entitled “TIA-942 Telecommunications Infrastructure Standard for Data Centers.” This is the most widely referenced standard when talking about data center facility availability. From the title, you might be inclined to think this is a specifi cation of telecom for data centers. It is that, of course, and more. The broad standard also includes cabling specifi cations, space layout recommendations, site selection criteria, and infrastructure tiers/availability guidelines. This last point, represented by “Annex G” in the TIA-942 standard, references data center tiers.

The TIA relied upon an organization called “the Uptime Institute” (or, “the Institute,” for short) to develop this part of the standard. The Institute provides research-based information on high-density computing and mission-critical facilities in a vendor-neutral manner. It has become the industry’s trusted source of data in this regard. The Institute continually gathers information by surveying existing data centers and data center projects. Its research includes models for estimating implementation costs and Total Cost of Ownership (TCO) for data center projects. Perhaps the one piece of research that most strongly defi nes the Institute’s work is the defi nition of tier classifi cations for data center performance — a four-tier system for categorizing data center capabilities.

Thus, when a facility is labeled a “Tier-1, 2, 3 or 4 data center” the numbers refer to the classifi cations from TIA-942 or the Uptime Institute. The rating helps companies manage their own information, ensuring accuracy and relevancy for queries and guided search navigation.

A Bit of History on Standards for Data Center Facilities

A Bit of History on Standards for Data Center Facilities 3

The Tier Classifi cations for Data Centers 4

Tier-1: Basic Data Center Infrastructure 4

Tier-2: Data Center with Redundant Capacity Components 5

Tier-3: Concurrently Maintainable Data Center 5

Tier-4: Fault-tolerant Data Center 6

Notes of Caution About Tier Classifi cations 6

Data Center Tier Ratings, Still Misunderstood 7

Tier Classifi cations and Our Customers 8

2 3

W H I T E PA P E R D ATA C E N T E R T I E R C L A S S I F I C AT I O N S

The tier classifi cation system measures the performance of a site’s operating infrastructure, which includes power, cooling, emergency backup, and fi re suppression. The power and cooling capabilities of a facility are delivered by its Mechanical, Electrical, and Plumbing (MEP) infrastructure. The mechanical systems provide cooling to the environment in which the data processing equipment is installed. These systems are composed of air handlers, air conditioners, chillers, plenums to channel air fl ow, and so on. The electrical systems provide the power to the data processing equipment. These systems are composed of the utility service to the facility, transfer switches, generators and Uninterruptible Power Supplies (UPS), batteries, Power Distribution Units (PDUs), load banks, breaker panels, copper cabling, etc. The plumbing systems support the mechanical and electrical systems by routing cabling, air, water, fi re suppression gases, and so on. There are multiple plumbing circuits in a facility; these are analogous to the vascular system of the human body.

Very simply put, the tier classifi cations refer to the degree of resilience the facility has to failures of its MEP systems. Redundancy and topology of

the infrastructure’s design provide resilience to failures. In the tier classifi cation model, a Tier-1 facility is the “least resilient” and a Tier-4 is the “most resilient.” Therefore, a Tier-1 facility has the “lowest availability” and a Tier-4 has the “highest availability.”

Next let’s discuss each of the four tier levels and compare their capabilities.

A Tier-1 facility has no redundant capacity components. It provides basic power and coolingto the data processing footprint with no excess capacity for backup or failover, and has noredundancy in its MEP distribution paths.

In this type of facility, any unplanned outage or failure of a capacity component or distributionelement will impact the data processing equipment and end users. Whenever maintenance is needed for the MEP infrastructure (e.g., utility work, component replacement, certifi cation testing,preventative maintenance, etc.), the impact is the same as if there were an unplanned outage. All systems and users are affected.

Per Institute benchmark data, Tier-1 sites typically experience two separate 12-hour, site-wideshutdowns per year for repair work. In addition, Tier-1 sites typically experience 1.2 equipmentor distribution component failures on average each year. Statistically, this means 28.8 hours ofdowntime per year, or 99.67% availability.

What kind of business is suited for this type of facility? In general, a Tier-1 facility is suitable for small businesses or start-ups where IT is an enhancement to internal business processes, where the principal use of a Web presence is for passive marketing, or where there is no enforceable fi nancial penalty for failing to meet customer quality-of-service commitments.

The Tier Classifi cations for Data Centers

Tier-1: Basic Data Center Infrastructure

A Tier-2 data center has redundant capacity components, but only a single, nonredundant distribution path serving the data processing equipment. The benefi t of this level is thatany redundant capacity component can be removed from service on a planned basis (e.g., for preventative maintenance) without causing the data processing to be shut down. However, any unplanned outage, failure of any capacity component, or disruption to the distribution pathmay impact the computer equipment.

On average, Tier-2 sites have one unplanned outage per year, and schedule three maintenance activities over a two-year period. The annual impact to operations is 22 hours of downtime peryear or 99.75% availability.

Businesses appropriate for Tier-2 facilities are small businesses with IT requirements that are limited to the traditional 9-5 business hours, companies without serious fi nancial penalties for failing to meet customer quality-of-service commitments, services without real-time delivery obligations, and call centers with multiple sites.

A Tier-3 data center has redundant capacity components and multiple independent distribution paths serving the data processing footprint. There is suffi cient MEP capacity to meet the needs of the data processing systems, even when one of these redundant MEP components has been removed from the infrastructure. In a Tier-3 data center, maintenance activities and certain unplanned events can occur without interruption to the computing systems.

Because of the concurrently maintainable characteristic of Tier-3 facilities, no annual shutdowns for routine maintenance are required. This allows for very aggressive preventative maintenance programs to be implemented, further extending the operational duty of the MEP components. The Institute has concluded that Tier-3 data centers have unplanned events totaling only 1.6 hours per year. Tier-3 sites then deliver 99.98% availability.

Notice that both the Tier-1 and Tier-2 levels deliver “two nines” availability, but the step to Tier-3 delivers “three nines.” This is a big improvement in uptime, and comes with a cost, which we will discuss later.

Businesses supported by a Tier-3 data center are companies that serve both internal and external customers 24x7. Their IT resources support automated business processes so that the customer impact of short shutdowns is manageable. Tier-3 is appropriate for businesses that span multiple time zones and have corresponding geographic diversity of employees and customers. Businesses that have signifi cant fi nancial exposure due to customer quality-of-service issues are well supported by Tier-3 facilities.

Tier-3: Concurrently MaintainableData Center

Tier-2: Data Center with Redundant Capacity Components

Very simply put, the tier classifi cations refer to the resilience the facility has to failures of its MEP systems.

“

”4 5

W H I T E PA P E R D ATA C E N T E R T I E R C L A S S I F I C AT I O N S

6 7

Tier-4 facilities have multiple, independent, and physically separate systems that each have redundant capacity components and multiple, independent, diverse, AND active distribution paths supporting the data processing footprint. In a Tier-4 data center, any single failure of an MEP component or distribution path has no negative impact on the data processing systems, and the infrastructure automatically responds to the failure to prevent further impact to the facility.

Because of the degree of redundancy and fault-tolerance in Tier-4 infrastructures, facility-related failures that impact the data processing equipment

are statistically reduced to 0.8 hours per year. This yields 99.99% availability.

Companies that operate in international markets, with a “24 by forever” services commitment,and in a market space that requires nonstop processes are well served by Tier-4 facilities. This also includes businesses that are based upon market transactions, fi nancial settlement services, e-commerce sites, or any business for which ensuring uninterrupted customer access to applications or employee access to IT is competitively advantageous.

Tier-4: Fault-tolerant Data Center

Those in the data center and mission-critical facility fi elds are regularly exposed to the TIA-942/Uptime Institute tier classifi cation guidelines. Some additional background and words of caution are important to keep in mind.

There is a temptation to stretch the defi nition of the Institute’s tier classifi cations. You may hear of a “strong Tier-2” data center or a “Tier-3+” data center. In actuality, there is no such thing. The best one can conclude from a term “like Tier-3” is that the facility qualifi es as a Tier-3 facility and has some other availability-supportive features. If any single system in a Tier-n data center does not meet the Tier-n requirements, then the facility as a whole is not Tier-n. There is no such thing as a fractional tier rating. Only whole numbers are valid when talking about tier levels.

Finally, it’s important to mention the subject of cost. The cost of any mission-critical facility is driven predominantly by the MEP infrastructure.

The power and cooling capacity necessary to support the data center are the dominant drivers of cost in implementing a new facility. Similarly, because the tier classifi cations imply redundancy, the higher the intended tier rating, the higher the capital investment in the construction and operation of the facility. This cost does not increase linearly, and is a major factor in the decisions involved in facility operation.

Notes of Caution About Tier Classifi cations

There is a temptation to stretch the defi nition of the Institute’s tier classifi cations.

“

”

The tier classifi cation system defi nes the requirements for data centers to meet a certain level of availability. What is not so well understood is which “critical to quality” features of a data center characterize it as being of one tier level rather than another.

We receive questions from customers about whether or not the height of the raised fl oor, a fence for perimeter security, or the number of parking spaces, etc. will impact the assessment for the tier rating of the facility. These questions are often based on an individual’s interpretation of TIA-942 guidelines, which list numerous features typically found in the various tier levels of mission-critical facilities. These points, though, are not “critical to quality” in the true spirit of what the tier levels are intended to deliver, which is availability.

It is the level of capacity and redundancy of MEP infrastructure components, as well as the topology of confi guration, that determines the tier level rating. The Institute guidelines cite a 30-inch raised fl oor as typical for Tier-3 data centers. This is based on the assumption that the data center is using the raised fl oor as a plenum for cool air distribution to IT equipment, and as such needs a certain height to maintain the anticipated volume of airfl ow. However, there are ways to cool IT equipment without using a raised fl oor as an air plenum. If the facility uses one of these alternate techniques and doesn’t have a raised fl oor at all, it can still achieve Tier-3 capabilities if the capacity, redundancy, and topology of MEP systems are

correct for that tier. Similarly, a facility doesn’t require a fence to qualify as a Tier-4 site. Perimeter security is important for your investment in facilities and systems, but technically, a fence is not required to deliver that level of availability. If there is an identifi ed risk to the facility (such as physical trespass, in the case of absent fences) then the impact to availability should be considered in the design of the facility. The same holds for the other ancillary characteristics that the Institute’s research has shown to typify sites of various tier levels.

The core factors contributing to the tier level rating of mission-critical facilities are the following:

These factors describe the level of redundancy and topology of the MEP systems, and are coreto determining tier level availability ratings for the facility. Ensuring suffi cient availability in support of business operations is the reason a certain tier level goal is targeted for the enterprise’s mission-critical facilities. The main features impacting the facility’s uptime are those listed above and they are the most important ones used to establish a tier level.

Data Center Tier Ratings, Still Misunderstood

• Active Capacity Components to Support IT

Load (N, N+1, etc.)

• Distribution Paths (single, single +

alternate, dual)

• Concurrent Maintainability

• Fault-tolerance (to a single event)

• Compartmentalization

• Continuous Cooling Capability

What is not so well known are the critical features necessary to differentiate one tier from another.

“

”

W H I T E PA P E R D ATA C E N T E R T I E R C L A S S I F I C AT I O N S

Data center consolidation and data center outsourcing are top of mind for many CIOs these days. Many companies have ’90s-era, vintage IT facilities that not only lack the availability to align with the business’ operating model, but also struggle with the power and cooling demands of contemporary computing systems. The deployment of multi-core processors and blade-based systems concerns many facility managers. Rapidly growing power consumption and energy costs are at the top of IT issues.

Whatever the motivation for a data center project, it helps to be familiar with the spirit of the TIA-942 tier classifi cations as well as the nuances so that you can exercise the proper degree of due care and deliver value to the customer. In many cases, the enterprise may be focused on the minutia of TIA-942 because of the desire to align the business with the proper availability of the IT facility, but this may take place at the expense of practicality.

The Uptime Institute has done a great job of providing a benchmark for defi ning the topologyand architecture of the MEP infrastructure of a data center in order to yield a certain level of facility availability. However, some facilities that score high on the tier-rating scale fail to achieve the target availability performance. Similarly, there are many facilities that score low on the tier-rating scale that have exceptional availability performance. Many Tier-2 sites exceed Tier-4 availability performance and many Tier-3 and Tier-4 sites fail to meet their availability targets.

Why is this so? Well, it’s one thing to have the right levels of redundant paths and backupcomponents, but it’s another thing to operate them in a way that ensures high availability. Many data centers have suffered outages because equipment settings were defective. Inadequate equipment maintenance and operational procedures also account for higher frequencies of outages and extended downtime. In other availability issues, it often comes down to the “3 Ps” — people, processes, and procedures put in place to ensure the quality of operation in mission-critical facilities.

A recent survey of AFCOM members found that 81% of respondents had experienced a data center failure in the past fi ve years, and 20% had sustained at least fi ve failures. It’s common to fi nd reports of high-profi le data centers that endure unexpectedly long outages because of backup generators that fail to start, improperly confi gured ATS equipment, contamination of diesel fuel and water, and many other similar nuances of system confi guration and operation. The majority of failures in data centers are caused, triggered, or exacerbated by human error and process gaps. These exposures are mitigated through disciplined operational standards.

Tier Classifi cations and Our Customers

We use the terms “SOPs and MOPs.” These are the Standard Operating Procedures and Methods of Procedures that form the principle guidelines for the facility staff. The exceptional performance of high-availability sites is a testament to the quality of the staff managing and operating those sites.

In 2008, Switch and Data scored an impressive fi ve nines or 99.999% for facility uptime, and some individual sites even exceeded that metric, which surpasses Tier-4 guidelines. This accomplishment is the result of strong people, processes, and procedures.

It’s important to move the tier level discussion to one of availability, performance history, and value. The interest in knowing a prospective colocation site’s tier level is driven by a desire to obtain a minimum threshold of facility availability in order to align with business goals. But this goal can be accomplished outside of the context of tiers.

The right to certify the tier level of data centers is reserved by the Uptime Institute, and those services are performed by a specifi c contractor. Certifi cation is an expensive and very involved process. As of this writing, there are only 13 Tier-3 and two Tier-4 facilities that have been certifi ed. While a consulting fi rm can do a thorough assessment to determine a facility’s tier rating, only the Institute can “certify” it. All of the above-mentioned certifi ed sites are enterprise facilities. None of Switch and Data’s competitors have certifi ed colocation sites. Many data centers are “said to be” of a certain tier level, but the extent to which these designations are accurate can only be determined by thorough analysis of the MEP infrastructure.

Why is it that colocation providers are not rushing to achieve tier certifi cation? The reasons gobeyond the simple cost of undergoing certifi cation. The levels of MEP infrastructure necessary toachieve the highest tier levels are extremely capital intensive. The cost/benefi t and ROI outcomes are diffi cult to justify. Given that equally high availability is achievable through strong operational ethics, a company in the business of providing mission-critical IT facilities can provide exceptional value to its customers without securing tier certifi cation.

Switch and Data has chosen not to undergo tier level certifi cation of its facilities (as have othercolocation providers), in spite of continued investment in high-availability MEP systemsinfrastructure. The company has a widely recognized history of excellence in facility uptime, exceeding the highest tier level guidelines of the Uptime Institute. This is a testament to the company’s highly resilient facility infrastructure, and to the quality of the people, processes, and procedures it has in place to manage site operations.

Availability often comes down to the ‘3 Ps’ — People, Processes, and Procedures put in place to ensure the quality of operation inmission-critical facilities.

“

”

In 2008, Switch and Data scored an impressive fi ve nines for facility uptime, surpassing Tier-4 guidelines.

“

”

8 9

W H I T E PA P E R D ATA C E N T E R T I E R C L A S S I F I C AT I O N S

10 11

Notes Notes

D ATA C E N T E R T I E R C L A S S I F I C AT I O N S

Switch and Data is a premier provider of network-neutral data centers that house, power, andinterconnect the Internet. Leading content companies, enterprises, and communicationsservice providers rely on Switch and Data to connect to customers and exchange Internettraffi c. Switch and Data has built a reputation for world-class service, delivered across thebroadest colocation footprint and richest network

of interconnections in North America. Switch and Data operates 34 sites in the U.S. and Canada, provides one of the highest customer satisfactionscores for technical and engineering support inthe industry, and is home to PAIX® – the world’sfi rst commercial Internet exchange. For more information on Switch and Data, please visitwww.switchanddata.com.

About Switch and Data

www.switchanddata.com

£

€€

¥

$£

¥

€

$

$

¥

¥

$

£

1715 North Westshore Boulevard, Suite 650

Tampa, Florida 33607

tel 813.207.7700 | fax 813.207.7701