Case

Virtualization Is Happening Today

MAY / JUNE 2009 / VOLUME: 2 ISSUE 3

Making the Case for Data Virtualization 3 Virtualization Experts in High Demand DAVID LINTHICUM

5 Taming the Butterfl y Effect in Virtualized Environments HEZI MORE

14 Solving the Virtual Machine Backup Conundrum LAURA BUCKLEY

18 Enterprise Data Storage DAVE PISTACCHIO

20 Memory: The Real Data Center Bottleneck CLIVE COOK

24 Server Virtualization Technology Makes Its Way into Storage Systems LEE CASWELL

Robert Eve

http://virtualization.sys-con.com2 MAY / JUNE 2009 Volume: 2 Issue: 3 3http://virtualization.sys-con.com MAY / JUNE 2009 Volume: 2 Issue: 3

PRESIDENT & CEOFuat Kircaali [email protected]

Senior VP, Editorial & EventsJeremy Geelan [email protected]

EDITORIALExecutive EditorNancy Valentine [email protected]

Research EditorBahadir Karuv, PhD [email protected]

ADVERTISINGSenior VP, Sales & MarketingCarmen Gonzalez [email protected]

Advertising Sales DirectorMegan Mussa [email protected]

Associate Sales ManagerCorinna Melcon [email protected]

EVENTS

Events ManagerSharmonique Shade [email protected]

PRODUCTIONLead DesignerAbraham Addo [email protected]

Associate Art DirectorsLouis F. Cuffari [email protected] Beatty [email protected]

CUSTOMER RELATIONSCirculation Service CoordinatorEdna Earle Russell [email protected]

SYS-CON.COMWeb DesignersRichard Walter [email protected]

ACCOUNTINGFinancial AnalystJoan LaRose [email protected]

Accounts PayableBetty White [email protected]

SUBSCRIPTIONS

[email protected]

Call 1-888-303-5282For subscriptions and requests for bulk orders, please send your letters to Subscription Department $49/yr (6 issues)

EDITORIAL OFFICESSYS-CON MEDIA577 Chestnut Ridge Rd., Woodcliff Lake, NJ 07677Telephone: 201 802-3000 Fax: 201 782-9601Virtualization Journal (ISSN #1549-1331)is published bimonthly (6 times a year)

Postmaster: send address changes to: Virtualization JournalSYS-CON MEDIA 577 Chestnut Ridge Rd., Woodcliff Lake, NJ 07677

©Copyright 2009 by SYS-CON Publications, Inc. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy or any information storage and retrieval system, without written permission. All brand and product names used on these pages are trade names, service marks or trademarks of their respective companies.

FOR LIST RENTAL INFORMATION:Kevin Collopy: 845 731-2684 [email protected] Cipolla: 845 731-3832 [email protected]

REPRINTSFor promotional reprints, contact reprint coordinator Megan Mussa [email protected]

SYS-CON Publications, Inc., reserves the right to revise, republish and authorize its readers to use the articles submitted for publication.

Those with virtualiza-tion skills are in high demand right now. That makes them

just as difficult to retain as it is to find and hire them. Indeed, as virtualization becomes more important to enterprise IT, those who understand VMware, and

other virtualization technology, and understand virtualization in general are commanding top sala-ries and they are often recruited away from current employers. Personally, I’m getting a few calls a week from headhunters looking for design and operational virtualization talent, even in the downturn when data centers are shrinking. At the heart of the is-sue is that virtualization now touches all parts of the data center, the use cases are becoming more complex, and the operational value is mission-critical. In responding to this need, many organiza-tions have created virtualization centers of excellence with the hopes of creating new talent and becoming more attractive to virtualization experts. Typically this means an investment that the IT leaders hope comes back to them quickly as virtualization becomes a larger part of the in-frastructure, and the use of cloud computing and private clouds begins to become more pervasive. So, how can you attract new virtualization talent? The trick is to understand first that you have to pay for talent, and perhaps pay more than some of the existing players. Talk to recruiters and can-didates to understand more about expectations. Beyond that, plan on paying retention and sign-ing bonuses, and being creative around work/life issues, such as allowing a liberal telecommuting policy and a relaxed dress code. So, if you’re a virtualization expert, how can you find the best places to work? Look for orga-nizations that invest in your talent as well as IT. Those organizations that seem to have a lot of turnover typically have that turnover for a good

reason. They’re either too slow to promote or don’t provide competitive wages. The core issue around virtualization is that virtualization goes well beyond simple net-work and system administration. Many seek traditional “network administration” types, and they quickly find that the skills go far above the network. There are many abstraction layers to consider, and virtualization is much more com-plex than traditional systems administration. To this end, virtualization skills require that the ideal candidate be a jack-of-all-trades and a master of VMware. As the use of virtualization rapidly expands, the need for talent will expand with it. At this point there are more positions chasing too few candidates, and that could mean salaries go up quickly for the core talent, or they settle for those who don’t fully fit the bill. Either way, it’s going to cost more to maintain virtualization in 2009 and 2010.

About the Author

David S. Linthicum, Editor-in-Chief of SYS-CON’s Virtualization

Journal, is an internationally known cloud computing and service-

oriented architecture (SOA) expert. He is a sought-after consultant,

speaker, and writer, and formed Blue Mountain Labs, a leading

cloud computing advisory, validation, and consulting organization.

In his career, Linthicum has formed or enhanced many of the ideas

behind modern distributed computing including EAI, B2B Application

Integration, and SOA, approaches and technologies in wide use

today. For the last 10 years, he has focused on the technology and

strategies around cloud computing, including working with several

cloud computing startups. His industry experience includes tenure

as CTO and CEO of several successful software and cloud computing

companies, and upper-level management positions in Fortune 500

companies. In addition, he was an associate professor of computer

science for eight years, and continues to lecture at major techni-

cal colleges and universities, including University of Virginia and

Arizona State University. He keynotes at many leading technology

conferences, and has several well-read columns and blogs. Dave has

authored 10 books, including the ground-breaking “Enterprise Ap-

plication Integration” and “B2B Application Integration.” You can

reach Dave at [email protected].

[email protected]

Virtualization Experts in High Demand

BY DAVID LINTHICUM

Be prepared to pay

EDITORIAL

Harness the power of Lean IT to drive business value, reducewaste and maximize productivity.Virtualization management solutions from CA help increase the value IT delivers to thebusiness by enabling reliable, high-quality services, improving operational efficiency andagility, and reducing costs through automation and integration with CA enterprise managers.

CA virtual management solutions can help you:

• Centralize virtual and physical management and provisioning

• Leverage repeatable and extensible process automation

• Audit system and application configurations in minutes not hours or days

• Create a self-service provisioning model

• Enhance agility with automated resource allocation based on business needs

For additionalinformationplease visitca.com/virtual.

Copyright © 2009 CA. All rights reserved. MP338650409

When the going gets tough,the tough get LEAN.

http://www.ca.com/us/


In 1969 Edward Lorenz, the famed meteorolo-gist and Chaos Theory proponent, introduced the concept now known as the “Butterfly Effect” when he posed the famous question: “Does the

flap of a butterfly’s wings in Brazil set off a tornado in Texas?” The Butterfly Effect theory describes the phenomenon of how small variations in a dynamic system can subsequently cause larger and more complex variations over the long term. Today’s modern virtualized data center repre-sents a new kind of dynamic system. Just as the Brazilian butterfly may set off a tornado a world away, any change to the virtualized objects that are the components of the virtual infrastructure can have severe and unintended consequences to virtualized systems, networks, and data centers. As virtualization technologies are deployed in live production environments, the potential for small changes at the virtual layer disrupting the virtual and physical network resources are becoming a real concern.

Seeing Through the Fog of Virtualization The evolution of IT has been marked by several distinct cycles. The adoption of virtualization in the data center is proving to be the next major paradigm shift as was the change from main-frame to client/server computing in years past. The next advance in the corporate infrastructure is now underway with the broad embrace of cloud computing, marked by the use of Software-as-a-Service (SaaS). Cloud computing will become in-creasingly more prevalent once more servers and desktops are virtualized and migrating servers in the cloud is as easy as dragging and dropping an icon onto a remote server. Clearly, the virtual data center is quickly be-coming a reality and organizations that deploy virtualization are now coming to grips with a new set of management challenges. Jim Frey, a

senior analyst at Enterprise Management Associ-ates, coined the phrase “Fog of Virtualization” to describe this new challenge. Since one cannot manage or control what they can’t see, virtual data center administrators are experiencing man-agement and policy enforcement limitations. Visibility into both the virtual and physical infrastructure is becoming a crucial operating requirement. Administrators need a visual repre-sentation of their virtual environment to under-stand the entire virtual network, track changes, and manage both their physical and virtual infrastructure. Virtualization creates an invisible line where servers and applications are easily moved and managed by multiple cross-functional organizational departments. Not only can serv-ers, storage, and networks can be dynamically managed as a single pool of resources but servers, switches, and networks can be created at the click of a mouse with applications moving seamlessly from one physical server to another. Consequently such a dynamic environment can create issues uncommon in traditional physical environments. Policies can be defined to dynamically move virtual machines based on resource utilization or for disaster recovery and business continuity. This automated mobility means that administrators only know the current location of a virtual machine. Of course, with vir-tualization it’s fast and easy to deploy new servers. Perhaps a little too easy, as administrators struggle to control this new breed of virtual server sprawl which adds another layer of complex-ity and decreases the cost savings to the organization. Virtual environments are subject to an ex-treme number of infrastruc-

Taming the Butterfly Effect in Virtualized EnvironmentsHow small inconsequential events can have radical consequences down the line

BY HEZI MOORE

STRATEGY

INSIDE

FROM THE EDITOR

3 Virtualization Experts in High Demand DAVID LINTHICUM

STRATEGY

5 Taming the Butterfly Effect in Virtualized Environments HEZI MOORE

FEATURE

8 Making the Case for Data Virtualization: Hard Metrics for Hard Times ROBERT EVE

VIRTUALIZATION 2.0

14 Solving the Virtual Machine Backup Conundrum LAURA BUCKLEY

DATA STORAGE

18 Enterprise Data Storage DAVE PISTACCHIO

NETWORK RESOURCES

20 Memory: The Real Data Center Bottleneck CLIVE COOK

AGGREGATION

24 Server Virtualization Technology Makes Its Way into Storage Systems LEE CASWELL

10

http://www2.sys-con.com/virtualization/pdfs/DataVirtualization_Print.pdf


ture configuration changes that tend to obfuscate the data center as a whole – understanding how components are connected, and the consequenc-es of changes, can mean the difference between success and failure. To maintain proper and secure configuration in a virtual environment, organizations must be able to track all infrastructure changes in real-time to maintain an accurate model of the virtual environment. If Murphy were a data center administrator, his law might read: “A misconfig-ured device possesses a risk equal to one that’s unpatched for known vulnerabilities.” Virtualiza-tion makes configuration of virtual infrastructure a point-and-click operation, opening the doors for simple human error.

The Unintended Consequences of Interdependent Systems Large virtual infrastructures have grown so complex and interdependent that the Butterfly Effect is emerging as a real threat. There are a number of dynamic forces at work in virtual systems, and changes to one part of the infra-structure can have far-reaching consequences. The downside of these complex interdependen-cies is that human mistakes, even small errors in configuration, can manifest themselves as major problems down the road. These forces conspire to put pressure on both the system and administrators, increasing the probability of mistakes and making their impact more far-reaching and costly. IT specialists attempting to configure, change, release, and support critical applications can find themselves facing a daunt-ing reality. The Butterfly Effect in the virtual data center is exacerbated by a complex infrastructure designed to support multi-tiered networks, interdependen-cies between different layers of infrastructure, fewer IT resources, and more applications that require changes and demands to support new initiatives like service-oriented IT and distrib-uted architectures. Administrators are struggling to manage the Butterfly Effect while trying to meet the conflicting challenges of providing the continuous availability of critical applications. But achieving continuous availability in complex virtual environments is no small task. Virtual environments are subject to an extreme number of infrastructure configuration changes that increase the complexity in the IT enterprise. Knowing how network resources are connected, and the consequences of change, can make all the difference. Seemingly small changes to the virtual

environments now bring potential risk to enter-prise operations. Complex virtual environments coupled with high end-user expectations, the pressures to meet SLA agreements, and a focus on cost reduction have significantly increased the probability of disruption. For example, virtual machines are typically connected to shared networks. Any virtual ma-chine or group of virtual machines connected to a common network can communicate across those network links and can, therefore, still be the target of network attacks from other virtual machines on the network. As a result, adminis-trators apply network best practices to harden the networks and isolate them to minimize the risks of data leaks from one virtual machine zone to the next across the network. The com-mon approach used in implementing segmen-tation is the use of virtual local area networks (VLANs). This approach is based on VLANs providing almost all of the security benefits inherent in implementing physically separate networks. Two virtual machines in the network are unable to transmit packets back and forth unless they’re part of the same VLAN. Proper segmentation of the virtual data center using VLANs requires careful planning of both the physical and the virtual network components. VLANs segmentation increases the network’s complexity and creates environment depen-dencies between the physical and the virtual network. Administrators require vigilant monitoring and tight coordination of configuration changes across both the virtual and physical network. A simple change to the VLAN configuration of the virtual infrastructure can impact the security level and the performance of VMs that will fail to perform the business task it was designed to do. Small variations of the initial configuration of the virtual network’s VLANs structure or VLANs configuration changes to the physical network can produce large variations in the long-term behavior of the virtual and the physical environ-ment. Just as Lorenz’s Brazilian Butterfly has the potential to set off a tornado thousand of miles away, frequent changes that network VLAN configuration impose on the infrastructure can have serious consequences and negatively impact the security and performance of the entire data center. Another example is VMotion migration tech-nology, which adds a much needed dynamism to virtual environments. Virtual machines and their applications are moved between physical serv-

STRATEGY

ers with features such as Distributed Resource Scheduling (DRS) and High Availability (HA) that are based on VMotion’s enabling technology. DRS continuously optimizes resource alloca-tions based on defined rules and resource utili-zation. The changes to resource allocation can be automatically executed by performing live migration of virtual machines through VMotion. The frequency of moves depends on the aggres-siveness of the policy chosen. A more aggressive DRS implementation would set off a flurry of VM migrations on a frequent basis. Like Lorenz’s Brazilian Butterfly, frequent changes that DRS imposed on the infrastructure can have serious consequences. If the butterfly hadn’t flapped its wings and DRS stopped migrating VMs at such a high frequency, the trajectory of the system might have been vastly different. While neither the butterfly nor the DRS activities directly caused the tornado, the flap of its wings and the initial configuration of the virtual environment are an essential part of the initial conditions that presage disaster. Recurrence – the approximate return of a system toward its initial conditions – together with sensitive dependence on initial conditions are the two primary ingredients for chaotic motion. The Butterfly Effect is a real phenomenon in virtual data centers that can increase risk, drive up costs, negatively affect availability, and frac-ture IT’s credibility. Several core problems faced by nearly every large IT organization have magni-fied these burdens. Information about virtual configuration interdependencies is poorly under-stood, tracking it is inefficient, and mapping out a dynamic virtual infrastructure is a complicated and time-consuming endeavor. But organizations can mitigate these effects by automating their virtual infrastructure management. IT organizations need a new breed of manage-ment tools that understands and takes advan-tage of virtualization’s unique requirements and capabilities. Virtual Infrastructure Management is now being embraced by organizations around the world as a way of achieving large increases in availability and reliability for relatively modest investments in services and tools. Organizations implementing tools like Reflex’ management solution for virtual environment have achieved these goals by reducing the time required to resolve problems, improving capacity planning, monitoring configuration changes, increasing business resiliency, and better managing their virtual data center assets. Those enterprises that have made the leap to effective Virtual Infrastruc-

ture Management no longer fear the winds of change or Lorenz’s Butterfly Effect. Leveraging the Virtual Infrastructure Management tools, IT departments can evolve and add the following capabilities:

• Gain visibility and assert control across virtual environments. Since you can’t control what you can’t see, visibility is key in managing and monitoring the virtual infrastructure. Admin-istrators need a visual representation of their virtual environment to understand the entire virtual network, track changes, and address vir-tualization challenges that have management implications.

• Track all infrastructure changes in real-time and maintain an accurate model of the virtual environment. With Virtual Infrastructure Man-agement tools at their disposal, administrators can track changes to assets and alert stakehold-ers when critical assets are modified. In addi-tion, the administrator has access to historical changes to identify unauthorized configuration changes, configuration errors, and enforce policies across the entire virtual infrastructure. Each time a change is detected, the change and all relevant details (who, when, why) are recorded

• Monitoring and managing virtual performance enables organizations to increase business efficiency and assure business continuity in the virtual data center. The ability to detect network bottlenecks inside the virtual infra-structure, identify over-utilized VMs, improve the performance of critical applications, and avoid network outages will allow administra-tors to improve the delivery of these critical applications, quickly troubleshoot issues, and optimize their virtual infrastructure.

From a philosophical standpoint, the Butterfly Effect provides a framework for understanding how seemingly small and inconsequential events can have radical consequences down the line. This theory can be applied to the virtual data cen-ter, a system that has become exponentially more complex and unpredictable as it grows in scope and gets beyond our direct control. While such systems will always possess a measure of uncer-tainty, we will require new tools and diagnostics that let us look deep into the black box.

About the Author

Hezi Moore is the CTO and co-founder of Reflex Systems, a pro-

vider of virtualization management solutions based in Atlanta, GA


Data Virtualization Data virtualization is used to integrate data from multiple disparate sources – anywhere across the extended enterprise – for consumption by front-end business solutions, including por-tals, mashups, reports, applications, and search engines. (See Figure 1.) As middleware technology, data virtualization or virtual data federation has advanced beyond high-performance query or enterprise informa-tion integration (EII). As IT architecture, data virtualization is implemented as a virtualized data layer, an information grid, an information fabric, or a data services layer in Service Oriented Architecture (SOA) environments. It can also be deployed on a project basis, for business intel-ligence (BI) and reporting, portals and mashups, and industry-focused single views.

Data Virtualization’s Five Value Points The many ways data virtualization delivers value to business functions and IT operations can be categorized as: • Sales Growth• Risk Reduction• Time Savings• Technology Savings• Staff Savings

Converting these to hard metrics requires an understanding of the relationships between specific data virtualization capabilities and the IT and business value they deliver. Value calcula-tions are made using one or more forms of re-turn-on-investment (ROI) calculators. Examples of the five value points and their metrics along with actual customer case studies are provided below.

Sales Growth As an important indicator of an enterprise’s success (or, in the public sector, as an indica-tor of service growth or mission effectiveness), sales growth results from business strategies such as improved offerings, better customer support, and faster market response. Data virtualization supports these strategies by pro-viding data federation, on-demand data access and delivery, and automated data discovery and modeling.

More Complete Data Data federation capabilities enable the integra-tion of disparate data on-the-fly without physical data consolidation, making more complete data

available to revenue-producing and customer-facing staff for better sales-related business deci-sions. Hard metrics include:• The number of decisions within the project’s

scope; and• The revenue-enhancing value of improving

each decision based on the availability of more complete data.

An energy provider used data federation to increase oil production from its 10,000 wells. The data included complex surface, subsurface, and business data in high volumes from many disparate sources. The data virtualization solution federated actionable information to automate maintenance and repair decisions made through-out the day, while relieving key resources for other value-adding tasks. This increased both staff and repair rig productivity, which were key factors in the 10% increase achieved in well revenue perfor-mance and efficiency.

Fresher Data Data virtualization’s on-demand data access and delivery capabilities reach difficult-to-access data and deliver it to consuming applications in near-real-time. Fresher data means more timely and accurate decision-making, often yielding sales growth. The hard metrics include:• The number of decisions in the project’s scope;

and• The revenue-enhancing value of improving

each decision based on the availability of more timely data.

A leading marketing information company used on-demand data access and delivery to grow sales by providing its large consumer goods clients with more timely access to its huge col-lection of consumer trends and demand infor-

FEATURE

Source: Composite Software, Inc.Figure 1 Data virtualization at a glance

Making the Case for Data Virtualization: Hard Metrics for Hard Times

BY ROBERT EVE

Increased risk and an opportunity for better security

Achieving compelling value from infor-mation technology is critical because IT is typically an enterprise or govern-ment agency’s largest capital expense.

Increasing business complexities and technology choices create greater demands for justification when making IT investments Cambridge, MA-based analyst firm Forrester Research recently reported that, “Business and government’s purchases of computer and com-munication equipment, software, IT consulting, and integration services and IT outsourcing will decline by 3% on a global basis in 2009 when

measured in U.S. dollars, then rise by 9% in 2010.”With smaller budgets, IT must validate purchases by correlating tangible business and IT returns that align with corporate strategic objectives. This validation should come early in the acquisition process as well as after the implementation to demonstrate actual value and justify expanded adoption. Evaluating data virtualization first requires understanding how it specifically delivers value. This understanding can then be used to calculate value and provide the hard metrics required for hard economic times.


FEATURE

mation. The data virtualization layer enabled simplified and rapid development of the real-time queries required by the customers’ self-service reporting tools. This capability was the key factor behind a 2% increase in revenue.

Quicker Time-to-Solution Data virtualization’s automated data discovery and rapid modeling capabilities reduce the time typically wasted on searching for relationships among data tables. These capabilities automate many of the detailed modeling and validation activities. With quicker time-to-solution, new sales-impacting applications and their associated revenues are available sooner. The hard metrics include: • The number of months the project can be

delivered earlier; and• The revenue-enhancing value associated with

each month where value was accelerated.

An investment bank used data discovery and modeling to increase revenues by improving its trade order management, debt/equity market research, and risk management applications. The abstracted data layer in the SOA environ-ment enabled rapid modeling and complex query creation that was shareable across the bank. The resulting 60% reduction in integration design and development time on revenue-enabling applica-tions and portals contributed to a 2% revenue increase at the bank.

Risk Reduction Risk reduction has become increasingly important as a result of greater complexity and regulation. Becoming more agile in response to risk, improving predictability in light of risk, and ensuring compliance with changing regulations and reporting mandates are a few of the strategies to reduce risk. Data virtualization supports these strategies through its data federation, on-demand data access and delivery, and data discovery and modeling capabilities.

These data virtualization capabilities and IT benefits are similar to those driving sales growth. However, for risk reduction, the business benefit is better risk visibility and faster problem reme-diation. In both cases, quicker time-to-solution helps get new or improved applications online faster. However, in the case of risk reduction, these might be applications for risk management or compliance reporting, rather than sales or customer management.

More Complete Data Data federation provides more complete data to risk and compliance management staff, thereby improving data visibility and reducing overall risk. The hard metrics include:• The number of risk decisions in the project’s

scope; and• The risk reduction value of improving each de-

cision based on the availability more complete data.

A global pharmaceutical company used data federation to shorten lengthy R&D cycles and re-duce the risk of new product delays. Its Research Scientists’ Workbench solution combined dis-parate structured and semi-structured research data from across the enterprise. Armed with more complete information, researchers were able to resolve problems faster, resulting in 60% fewer new product delays.

Fresher Data and Quicker Time-to-Solution Data virtualization’s on-demand data access and delivery capabilities improve the timeliness of data so risk issues can be remediated faster. Data virtualization’s automated discovery and modeling accelerates new risk management and compliance reporting application development, thereby delivering their associated risk-reducing benefits sooner.

Time Savings New information systems must deliver

“Evaluating data virtualization first requires understanding how

it specifically delivers value.”

the data needed while reducing the latency between business event and response. So IT is under constant pressure to provide these new systems and their associated information more quickly. Strategies for saving develop-ment and deployment time as well as decreas-ing data latency are crucial. Data virtualiza-tion supports these strategies by providing a data services library; installation wizard, manager and clustering; and query optimiza-tion capabilities.

Less New Code, Greater Reuse The data services library holds complete collec-tions of reusable views and data services. By using these existing objects, the need for new coding efforts is lessened, permitting greater reuse across multiple applications. This saves project develop-ment time for both new applications and existing ones when changes are needed. The hard metrics include:• The number of views or services planned;• The savings per view or service; and • The percentage reuse factor for a specified time

frame.

A major investment bank wanted to build new applications faster, but it couldn’t be-cause key reference data, such as counter-par-ty accounts, was duplicated across multiple applications. Other than slowing develop-ment, this proliferation contradicted good banking practices and data governance. The bank shaved 25% off its average development time by creating a shared data services library to house Web Services for sharing counter-party master reference data.

Easy Installation & Reliable Operation Various installation wizards along with man-ager and clustering capabilities accelerate and automate the installation and runtime of data virtualization solutions. As a result, new solutions are deployed faster. The hard metrics include:• The number of months a project can be deliv-

ered earlier; and• The assignable value associated with each

month the value was accelerated.

A leading life sciences R&D organization needed to quickly prototype, develop, and de-ploy the new information solutions required to support strategic decisions by business executives. It used data virtualization to build and deploy virtual data marts in support of

multiple data consumers including Microsoft SharePoint, Business Objects Business Intel-ligence, Tibco Spotfire, Microsoft Excel, and various Web portals. This resulted in a 90% reduction in the time required to deploy new information sets.

High-Performance Data Delivery Data virtualization’s query optimization and caching capabilities help eliminate data latencies, speeding the delivery of critical information to users and applications, thereby shortening the time between business events and response. A North American telecom chip maker tar-geted faster responses to customer requests. To do this, its sales force management analyt-ics required up-to-the-minute data from the packaged Salesforce.com CRM application as well as other systems. The manufacturer used data virtualization to optimize query per-formance, ultimately cutting average report runtimes from four minutes to 30 seconds or less.

Technology Savings Just as storage, server, and applications’ virtualization have demonstrated huge tech-nology savings, data virtualization has proven to provide similar savings by requiring fewer physical data repositories along with the systems required to operate and manage them. Many users find that these technology infra-structure savings alone justify the investment. This is frequently a natural place to start a data virtualization deployment.

Fewer Physical Repositories, Lower Hardware, Software, & Facilities Costs Data virtualization doesn’t require replica-tion in intermediate physical data repositories. Fewer physical data marts and operational data stores mean less supporting hardware and software. This, in turn, means less rack space, electricity, air conditioning, manage-ment software, and other facilities’ costs. The hard metrics include:• The number of servers reduced due to virtual

federation rather than physical consolidation; and

• The assignable hardware, software and facili-ties cost associated with each server.

A leading computer maker wanted to re-duce the cost of its supply chain and custom-


FEATURE

er management operational BI applications, which included more than 50 intermediate data marts. Each mart required a server, resulting in lifecycle hardware infrastructure costs of $20,000 each. It used data virtualiza-tion to provide a virtual supply chain data hub that replaced the physical data marts. This resulted in $1 million in infrastructure cost savings.

Staff Savings The cost of internal and outsourced staff is typically the largest IT expenditure. Substitut-ing automated tools for labor is one strategy for cutting staffing costs. Increasing existing staff productivity by improving SDLC process-es is another. Simplifying the work to employ lower-skilled (and therefore less costly) staff is a third. Data virtualization supports these strategies by delivering a user-friendly GUI development environment, along with auto-mated data discovery and data services library capabilities.

Fewer Skills Required Data virtualization’s GUI development envi-ronment simplifies and automates the detailed design and development work that would otherwise require more technically capable staff. Enterprises find they spend less money on expensive consultants. Enterprises also find that they can redeploy their highly skilled staff to other more critical work. The hard metrics include:• The number of consulting staff hours being

reduced; and• The cost per hour.

A global money manager wanted to reduce the effort required by 100 financial analysts who build the complex portfolio models used by fund managers. Its solution was to build a virtualization layer surrounding the ware-house to abstract away the complexity of the underlying data. This simplification resulted in a financial analyst productivity increase of 25%, allowing many to be redeployed to develop additional financial analytics useful to the firm.

Greater Collaboration Discovery capabilities within today’s data virtualization platforms quickly validate new reporting solutions with end users using live data early in the development process.

Furthermore, analysts and data designers can transfer models to application developers and operations teams to complete the process. This encourages team collaboration and helps reduce rework. The hard metrics include:• The number of hours saved; and• The cost per hour.

The same global money management firm cited above wanted to improve the collabora-tion of its 100 financial analysts. Many of its financial models relied on similar data and data models, but technology hindered these analysts from effectively sharing their work. A common virtualization layer over the finan-cial research data warehouse provided the financial analysts with reusable data views that could be shared for the first time. In addition, IT provided a dedicated DBA and data architect who created the new views as needed. The improved collaboration resulted in higher portfolio returns and a 150% ROI in six months.

Metrics for Hard Times By understanding the specific contributing value factors of data virtualization, C-level executives and IT managers can more easily calculate both the estimated and actual value of each data virtualization implementation under consideration, be it architecture or project. The resulting hard metrics that clearly contribute to enterprise-wide goals arm bud-get decision makers with the data they need to make confident decisions in hard economic times.

About the Author

Robert Eve has held executive-level marketing and busi-

ness development positions at several leading enterprise

software companies. At Informatica and Mercury Interactive,

he helped penetrate new segments as vice-president of

market development. Bob also ran marketing and alliances

at Kintana (acquired by Mercury Interactive in 2003) where

he defined the IT governance category. As VP of alliances at

PeopleSoft, he was responsible for more than 300 partners

and 100 staff members. In his nearly 10 years at Oracle,

Bob held positions such as Oracle manufacturing’s first

product manager and head of its revolutionary software

integration program, CAI. Bob has an MS in management

from MIT and a BS with honors in business administration

from the University of California at Berkeley. He is a frequent

contributor to publications including SYS-CON’S SOA-

World and Virtualization magazines.

Mark Your Calendar & Join SYS-CON Events’ 2009 Conferences & Expos

October 6, 2009 Washington D.C.

For over a decade, SYS-CON Events has delivered the world’s leading i-technology conferences, seminars, and expos, including Virtualization Conference, Cloud Computing Summit, SOA World, AJAXWorld, Enterprise Open Source Conference, “Real-World Flex” Seminar Series, iTVcon, Real-World Java One-Day Seminar, and Real-World Ruby-on-Rails Seminar Series. Our world-class conference programs, roster of exhibitors/sponsors, the

industry’s most-respected faculty, and paid delegates solidify our position as the largest, most intelligent, and well-attended i-technology conferences, seminars, and expos in the world.

REGISTER TODAY AND SAVE CALL 201 802-3021 OR EMAIL [email protected]

November 2-4, 2009 Santa Clara, CA

Spring 2010 Europe

http://events.sys-con.com/


possible resource degradation during concurrent backup operations of multiple virtual machines on a single host server. Licensing, maintenance, and scalability issues may also be encountered.

Method 2: Backup Agent Installed on ESX Server Service Console Another method that can be used for backing up and restoring virtual machines is to install a backup agent on the ESX Server service console. The backup agent can then be configured to backup the virtual disk files and virtual machine configuration files (.vmdk and .vmx). These files contain the virtual machine operating system, memory, and all data files. The entire virtual machine can then be restored by simply restor-ing these few files. However, it’s not possible to restore individual data files and directories from this backup. This method is typically faster than creating a virtual machine, installing the operat-ing system, installing the backup agent, and re-storing all of the data files as described in Method 1, so it’s preferred for disaster recovery and virtual machine relocation. To ensure a consistent and recoverable backup of the full virtual machine, it’s recommended that you pause, suspend, or freeze the virtual machine with one of the following approaches:• Back up the inactive virtual machine. This is

considered the best way to ensure a consistent backup; however, it requires that the virtual machine be shut down during the backup. This may require pre- and post-backup schedule scripts that execute the commands to shut down the machine, back up the .vmdk files, and the restart the machine.

• Back up a suspended virtual machine. Again, you’d need to script pre- and post-backup schedule commands to suspend the machine. This method can minimize machine downtime, but restores should be tested and verified to be sure that the backup of the .vmdk files are consistent and restorable.

• Back up an active virtual machine. Again, using scripted commands in pre- and post-backup schedule jobs; you can make a snapshot of a virtual machine and backup the .vmdk files. This approach essentially freezes the virtual machine, but allows it to continue operating with all updates being sent to a snapshot file. When the backup is complete, the post-schedule command script unfreezes the machine and the changes in the snapshot file are committed to the virtual machine. This method provides for “hot backups” of the

virtual machine; however, it adds complexity to the backup procedure. Restores should also be tested and verified to ensure consistency of the backups.

Common disadvantages to all of the approach-es described above include possible resource contention and degradation during concurrent backup operations of multiple virtual machines on the same ESX server host. In addition, there are maintenance and complexity issues involved in writing, scheduling, and maintaining the scripts required, as well as the inability to restore individual files and directories from the backed-up .vmdk files.

Method 3: Using VMware Consolidated Backup VMware Consolidated Backup provides a cen-tralized backup facility for virtual machines. It’s not a complete enterprise backup solution and is intended only as an enabling technology to be used in conjunction with enterprise backup software. VCB is essentially a set of scripts and driv-ers that enables third-party backup software to protect virtual machines through a centralized backup “proxy” server that runs on a Windows server. Doing backups “by proxy” removes the backup workload from VMware ESX servers and virtual machines. VMware supplies different sets of integration module scripts for different third-party backup applications. You execute these scripts as pre- and post-process commands for the backup agent on the proxy server. During the pre-processing, the script calls the VCB proxy utilities either to create a snapshot of the virtual machine and mount it to the proxy server (for file-level backups) or export the virtual machine to a set of files on the proxy server (for full machine backups).

VIRTUALIZATION 2.0

Just when it seems that you’ve finally met and solved your data backup and recovery challenges, along comes another kink to complicate the problem.

While server virtualization provides many benefits, it may create a conundrum when deter-mining an organization’s best backup and restore strategy. In a physical server environment, some com-bination of full and incremental backups of all files, volumes, and servers are most likely being performed. Perhaps you’ve included a product that provides deduplication and an incremental-only backup strategy that reduces the need to do weekly or monthly full-system backups. Or, maybe you’ve decreased your reliance on tape for off-site backups and have included a data replication or snapshot technology that will help to quickly restore critical applications or systems in the event of a failure. But now that your servers have been virtualized there are new challenges: What should be backed up? You’ll certainly want to continue to have backups of all files and application data as you’ve done in the physical server environment. You’ll also want to be able to restore any individual file, directory, or file system from these current backups. But virtualization offers another level of consideration for backup. Since the “guest” virtual machines are presented by the “host” server as files, you can back up the entire virtual machine as just one or several individual file objects. This means that if backed up correctly, you could restore an entire virtual machine by restoring just a file or two. Let’s explore the best practices for a backup-and-restore strategy in a VMware virtual infra-structure environment. When deciding how best to protect VMware virtual machines, it’s impor-tant to consider your business requirements for

data recovery. It’s possible a restore of an individ-ual file, file system, or database may be needed. It’s also possible a rapid restore of the entire ma-chine is required in the event of a disaster, or you may want to move a virtual machine to a different physical machine for resource load balancing. There are three basic approaches that can be combined to meet different backup-and-restore objectives: • Install a backup agent in the virtual machine.• Install a backup agent on the ESX Server service

console.• Configure a VMware Consolidated Backup

(VCB) proxy server and install a backup agent on it to centralize the backups of many virtual machines.

Method 1: Backup Agent Installed on the Virtual Machine To achieve file level, directory level, and volume level backup and restore, a backup agent in the virtual machine can be installed. This is the same as installing a traditional backup agent for a cen-tralized network backup application on a physical machine, and provides the same capabilities (i.e., incremental backups, full backups, file level restores, and full volume restores), but does not provide for full virtual machine backups and restores. In the event of a complete virtual machine recovery, the first step would be to install the virtual machine operating system and backup agent. Next, restore all of the volumes backed up for the virtual machine from a full backup. This would be a directory-by-directory and file-by-file restore. This is similar to a bare machine recovery process of a physical server and is typically more time consuming than restoring from a full virtual machine backup as described in Method 2. Common disadvantages to this method include

Solving the Virtual Machine Backup Conundrum

BY LAURA BUCKLEY

Best practices for a backup-and-restore strategy in a VMware environment


VIRTUALIZATION 2.0

For fi le-level backups, the virtual machine continues to operate without interruption during the snapshot process. Changes to the virtual machine while it’s in snapshot mode are written to a snapshot fi le that’s stored in the virtual machine. These changes are ap-plied to the machine when the snapshot is dismounted. For full machine backups, the virtual machine is suspended during the export. When the export is complete, the machine resumes and the backup agent can then backup the exported fi les. After the pre-processing step is complete, users must run the backup agent on the proxy server to per-form the backup of a particular drive or drives on the proxy server where the snapshot or exported machine fi les reside. When the backup is complete, users must execute the post-processing scripts to dismount and delete the snapshot or virtual machine fi les. The advantages of using VCB to consolidate back-ups of virtual machines are that it reduces resource utilization on production virtual machines and hosts and allows virtual machines to continue to operate with little to no impact during backup operations. Ad-ditionally, using the central VCB proxy server means it’s not necessary to install individual backup clients on each virtual machine. However, VCB can be complex to implement using the supplied integration module scripts. In many cases, the scripts will need modifi cation. It may also be diffi -cult to schedule the scripts for individual pre- and post-processing of each virtual machine, especially for large numbers of virtual machines. Finally, it may be diffi cult to troubleshoot anomalies when using the scripts. Recognizing these diffi culties, some backup appli-cation suppliers have developed their own integration utilities or agents for VCB. These products simplify the implementation of VCB by providing easy-to-use scheduling and reporting functions, while eliminating the need to use the VMware integration modules. There are many considerations for backing up virtual environments. Just as when solving the backup needs of your physical environment, you should care-fully defi ne your business requirements for backing up and restoring data in the virtual infrastructure. Once you understand recovery time and recovery point objectives, you can begin to implement and test your strategy.

About the Author

Laura Buckley is president of Colorado Springs-based STORServer,

Inc. STORServer was founded in 2000 to provide simple, adaptable,

and scalable solutions for data protection. A privately held corpora-

tion with locations in the United States and Europe, STORServer

is the manufacturer of STORServer Appliances. Each appliance is

an enterprise-wide comprehensive solution that installs quickly and

takes just minutes a day to manage..

RE

PR

INTS

!R

EP

RIN

TS!

contact MEGAN MUSSAtel [email protected]

VIRTERA is the leading independent virtualization consulting firm with proven experience in enabling a highly-adaptive and agile IT infrastructure.

The economic downturn has companies focused on ways to reduce costs and improve their bottom line. In many cases budget cuts are the first steps towards achieving this goal.

IT executives are having to deal with reduced budgets and headcount with an expectation to innovate in order to drive further costs out of their operation. They are being asked to provide increased level of service to the business and to support new business initiatives. This challenge leaves IT executives in an almost “paralyzed” state as they try to balance budget, resources and the risk of undertaking a complex project such as virtualization.

VIRTERA understands these pressures and provides clients with solutions that:• Help identify new cost reduction initiatives• Provide IT service delivery cost transparency• Deliver a strategic virtualization roadmap with supporting tactical deliverables• Realize benefits in as soon as a business-quarter• Support corporate sustainability initiatives

800.848.7185 virteratech.com

The Promise of Virtualization

CostReduction

Experience

Scalability

AgilityAdaptability

Transparency

Innovation

Call or visit our website for more information

DELIVERING

MARKYOURCALENDAR& JOINSYS-CONEVENTS’2009CONFERENCES&

EXPOSvisit www.events.sys-con.com

tel 201-802-3021

http://www.virteratech.com

http://events.sys-con.com/


and diverse alternative to the traditional phone company network is ideal. It ensures data protec-tion and access in the event of an emergency. Eth-ernet is rapidly becoming the technology of choice to meet these mission-critical needs because of its inherent ability to provide the best connectivity over a wide area. Furthermore, it’s important that enterprises have point-to-point Ethernet connectivity to their data storage facility, ideally via a fully fiber-optic connection. This will let the data be transported instantaneously, securely, and reliably. Without this type of network, the data center’s range of critical business services to its clients won’t be delivered effectively – including business continuity and disaster recovery. Leading data centers in the New York metro-politan area have chosen carrier-class Ethernet to support their business. This advanced network enables these companies to provide a range of critical business services to their clients including business continuity/disaster recovery, production data center outsourcing, and Internet application hosting solutions all delivered over a highly intel-ligent, reliable, and secure network. Many of these data centers service customers in highly regulated industries, such as financial services, where reli-ability is paramount. There is no crystal ball for anticipating growth and technological changes in your business. The data center, as well as its network, should be able to respond to growth and changes in equipment, standards, and demands all while remaining man-ageable and of course, reliable. IT managers need to seriously consider every option available to them. As businesses evolve, de-

cision makers need to constantly re-evaluate how they deploy technologies and how their network will be there for them, no matter what decision they make. As IT managers make decisions during the next years regarding servers, virtualization software, and traditional data networking to control their data, be sure that a fiber network is top of mind in that decision process. Today, most high-end enterprise-size companies house their data in their own in-house data centers and replicate their data with providers such as Cervalis (using them for strictly backup purposes). Small companies today are using service providers or virtual storage services. Moving forward, we may see larger companies begin to virtualize their data centers rather than investing in in-house solutions, or taking advantage of private cloud environments where companies have multiple connections to multiple service providers. As we continue to watch how these new trends shape the enterprise data storage landscape, one thing remains evident – reli-able high-bandwidth connectivity will be essential to overall success.

About the Author

Dave Pistacchio is president of Optimum Lightpath. A 29-year

veteran of the cable industry, he is responsible for driving Optimum

Lightpath’s growth as the premier provider of data, Internet, voice

and video transport solutions and managed services for mid-sized

and larger businesses across the New York metropolitan area. He

joined Lightpath in 2005 and guided the company as it switched its

entire offering to Metro Ethernet services delivered over its fully fiber

optic Intelligent Enterprise Network. This “game changing” move

has resulted in accelerated business growth and significant industry

recognition.

DATA STORAGE

Business continuity and disaster recovery continue to be top priorities for enter-prise IT decision makers. Big or small, data storage is a critical issue impacting

every business in every market segment. A survey of 80 large U.S. companies conducted by Infonetics Research last year indicated they had an average of 501 hours of network downtime a year, and this cost them almost 4% of their revenue totaling millions of dollars. It’s not very difficult to see that downtime directly translates into dol-lars, lots of dollars. Businesses trying to survive in today’s economic environment can’t take this lightly. Infonetics also found that data centers can rep-resent 50% of an IT budget for an organization. As more and more companies add branch offices and servers, with their top priority being cutting costs, backup plans are becoming extremely difficult. A host of state and industry regulators require the development and implementation of business continuity plans to effectively and efficiently back up, protect, and recover data. Threats such as an exponential increase in the volume of data, space demands, and logistics of physical storage, the increasing difficulty and effort required for manual discovery and the escalating liabilities and legal risks for IT and business managers are key drivers in businesses that chose the data center for their backup plan. In developing their plans, businesses look for services designed to support a range of storage hosting and backup services including business continuity disaster recovery (BCDR), remote rep-lication, backup and recovery, data center storage area network and disk and transaction mirroring. Independent data centers have made tremen-dous strides in providing companies with viable, hearty solutions for their growing data stores and requirements. They continue to work to reduce

costs and risks associated with protecting, destroy-ing, storing, and recovering physical data assets. Alternatives such as virtualization, storage-as-a-service, and cloud computing are also gaining traction among enterprise decision makers due to their inherent cost-savings. Regardless of the service customers are looking for from their data center provider, the biggest thing setting one pro-vider apart from another is the network on which it relies. A data center’s network should serve as an en-abler for virtualization, SaaS, and/or cloud comput-ing technologies. High-speed fiber-based networks with large amounts of bandwidth are ideal for powering these technologies. Companies leverag-ing SaaS offerings buy bulk bandwidth and require “ultra-high” availability. Carrier-class Ethernet lets these companies do this effectively and reliably. The network should serve as the backbone of a company’s data storage solutions – strong enough to handle the demands of emerging technologies and flexible enough to scale seamlessly alongside a companies growing needs. Carrier-class Ethernet networks offer the ideal solution. No matter what method of storage is chosen, companies on fiber gain tremendous advantages in scalability, virtual-ization, and reliability. Data centers are a growing piece of enterprise data management architectures due to significant improvements in data transport reliability, security, and scalability. Growing administrative frustrations, shrinking disk storage costs, and rising tape backup expenses are fueling greater interest in data centers. Additionally, data centers continue to be more at-tractive to enterprise IT managers concerned with entrusting highly sensitive data to the cloud. A key element of making these facilities so ap-pealing is the IT infrastructure they’re built on. A data center with a highly reliable and secure fiber-optic network that provides a fully separate

Enterprise Data Storage

BY DAVE PISTACCHIO

A data center’s network should serve as an enabler for virtualization, SaaS, and/or cloud computing technologies


to make money, and it’s a race to performance. The fastest trading desks will profit most. Alongside the significant impact of peak perfor-mance is the need for certified messaging. Trading data streams must be certified – reliably stored for record keeping and rollback. Current solutions to the trading message problem are difficult to integrate, expensive, and cannot meet the performance re-quirements of the algorithmic trading desk. A leading vendor’s message bus solution has transac-tion latencies in the millisecond range, and reaches maximum throughput at close to 5,000 transactions per second. This performance hampers algorithmic trading, and throughput is not enough to meet the peak trading volumes at the opening bell, closing bell, or during market-moving events.

Memory Virtualization – Breaking the Memory Barrier The introduction of memory virtualization shat-ters a long-standing and tolerated assumption in data processing – that servers are restricted to the memory that is physically installed. Until now, the data center has been primarily focused on server virtualization and storage virtualization. Memory virtualization is the key to overcoming physical memory limitations, a common bottle-neck in information technology performance. This technology allows servers in the data center to share a common, aggregated pool of memory that lives between the application and operating sys-tem. Memory virtualization is logically decoupled from local physical machines and made available to any connected computer as a global network resource. This technology dramatically changes the price and performance model of the data center by bringing the performance benefits of resource virtualization, while reducing infrastructure costs. In addition, it eliminates the need for changes to applications in order to take advantage of the pool. This creates a very large memory resource that is much faster than local or networked storage. Memory virtualization scales across commod-ity hardware, takes advantage of existing data center equipment, and is implemented without application changes to deliver unmatched trans-actional throughput. High-performance com-puting now exists in the enterprise data center on commodity equipment, reducing capital and operational costs.

Memory Virtualization in Action – Large Working Data Set Applications Memory virtualization reduces hundreds to

thousands of reads from storage or databases to one, by making frequently read data available in a cache of virtualized memory with microsecond access speeds. This decreases reliance on expensive load balancers and allows servers to perform opti-mally even with simple, inexpensive round-robin load balancing by linking into common file system calls or application-level API integration. Any server may contribute RAM into the cache by using a command-line interface or a configuration and management dashboard that sets up and controls the virtualized memory pool through a web-based user interface. Memory virtualization then uses native high-speed fabric integration to move data rapidly between servers. For applications with large working data sets, larger than will fit in physical memory, such as those found in high-volume Internet, predictive

NETWORK RESOURCES

ACIOs and IT managers agree that mem-ory is emerging as a critical resource constraint in the data center for both economic and operational reasons. Re-

gardless of density, memory is not a shareable re-source across the data center. In fact, new servers are often purchased to increase memory capacity, rather than to add compute power. While storage capacity and CPU performance have advanced geometrically over time, memory density and storage performance have not kept pace. Data center architects refresh servers every few years, over-provision memory and storage, and are forced to bear the costs of the associated space, power and management overhead. The result of this inefficiency has been high data center costs with marginal performance improvement. Memory: Where Are We? Business-critical applications demand high perfor-mance from all network resources to derive value out of the ever-increasing volume of data. Memory is one of the three key computing resources, along with CPU and storage, which determine overall data center performance. However, memory has lagged far behind the advances of both processors and storage in capacity, price, and performance ratios. While processor vendors assert that data centers are processor-bound and storage vendors

imply that data centers are storage-bound, in many cases the true performance barrier is memory. To that end, both a major network vendor and a domi-nant server vendor have recently made announce-ments about dramatic increases in the memory footprint of servers to better support data center virtualization. The major network vendor built their first-ever blade server with custom developed hardware to support a larger memory footprint (up to 384 GB) for one dual-processor blade. This is significantly larger than the 144 GB maximum that is typical in high-end systems. The dominant server vendor enables individual VMs to use more of the local system memory.

Industry Challenges Memory constraints continue to impact ap-plication performance for a number of industries. For example, data for seismic processing of oil and gas extraction, flight and reservation information, or business analytics quickly add up to terabytes, much too large to fit in even large-scale (and expensive) local RAM. These growing data sets cre-ate huge performance slowdowns in applications where latency and throughput matter. Multi-core processors are underutilized, waiting for data they can’t get fast enough. And, currently available solutions are inefficient and don’t entirely solve the problem. Latency, or the delay in delivering the initial piece of data, is critical to application performance in areas such as manufacturing, pharmaceuticals, energy, and capital markets. As an example, algo-rithmic traders can execute hundreds of thousands of trades per day. Twenty-five percent of securities trades are now algorithmic trades – trades initiated by computers in response to market conditions or trading in patterns and sequences that generate profits. These trades leverage trade execution speed

Memory: The Real Data Center Bottleneck

BY CLIVE COOK

Memory virtualization solves the key barrier to increasing the efficiency of existing network resources

Figure 2: Benefits of Memory Virtualization Compared to Over-provisioning

Figure 1:


NETWORK RESOURCES

analytics, HPC and oil and gas, memory virtualiza-tion brings faster results and improves end-user experiences. In capital markets, memory virtualiza-tion delivers the lowest trade execution latencies, includes certified messaging, and integrates simply as demanded in this competitive market. The associated performance gains relative to traditional storage are huge. NWChem is a compu-tational chemistry application typically deployed in an HPC environment. In a 4 node cluster with a 4 GB / node running NWChem, memory virtualiza-tion cut the test run time from 17 minutes down to 6 minutes 15 seconds with no additional hardware, simply by creating an 8 GB cache with 2 GB con-tributed from each node.

Alternatives Fall Short Attempts to address these challenges include scaling out (adding servers), over-provisioning (adding more storage or memory than is needed), scaling up (adding memory to existing or larger servers), or even designing software around the current constraints. Larger data centers draw more power and re-quire more IT staff and maintenance. For example, a 16-server data center with 32 GB RAM/server costs $190,000 in capital and operational expense over two years. Scaling out that data center to 32 servers would double the cost to $375,000 (see Fig-ure 1). Scaling up the servers to 64GB RAM/server would raise the cost to $279,000 (data center costs based on the cost of scaling up a 16-node cluster from 32GB to 64GB per server, and scaling out a 16-node cluster to 32-nodes, two years operational expense). What does this investment buy you? You get more servers to work on the problem – but per-formance has not improved significantly because they aren’t working together; each server is still working only with its own local memory. By try-ing to divide and conquer your data set, you’ve fragmented it. Like fragmented drives, fragmented data sets restrict the flow of data and force data to be replicated across the network. The overhead of drawing data into each server consumes resources that should be focused on one thing – application performance. By sharing memory, data centers require less memory per server because they have access to a much larger pool of virtualized memory. Memory virtualization also enables fewer servers to accom-plish the same level of application performance, meaning less rack space, power consumption, and management staff (see Figure 2). Additional cache capacity can be added dynami-

cally with no downtime, and application servers can easily connect to virtualized network memory to share and consume data at any time without re-provisioning. High Availability features eliminate data loss when servers or networks go down by keeping multiple copies of data in the cache and employing persistent writes to comply with certified messag-ing standards. In the storage area, SAN and NAS have decou-pled storage from computing, but storage is not the place for the active working data set. Storage ac-celeration can only marginally improve application performance because it connects too far down the stack and is not application-aware (understands state). The millisecond latencies of storage requests are unacceptable bottlenecks for business and mis-sion-critical applications. In some cases, data center architects have turned to data grids in the search for performance. Data grids impose a high management overhead and performance load and tend to replicate the working data set, rather than truly share it. These solutions are difficult to integrate, debug, and optimize, and remain tightly coupled to your application, reducing flexibility. Architects who have imple-mented these solutions complain of the “black box” software to which they have tied their applica-tions’ performance and disappointing acceleration results.

Conclusion Memory virtualization has solved the key barrier to increasing the efficiency of existing network resources in order to improve the performance of business-critical applications. This capability decouples the memory from its physical environ-ment, making it a shared resource across the data center or cluster. Addressing today’s IT perfor-mance challenges, virtualized memory enables new business computing scenarios by eliminating application bottlenecks associated with memory and data sharing. Currently available, memory virtualization is delivering optimized data center utilization, performance and reliability with mini-mum risk and immediate business results.

About the Author

Clive Cook is CEO of RNA networks (http://www.rnanetworks.

com), a leading provider of memory virtualization software that

transforms server memory into a shared network resource. He

has a track record of success building and leading technology

businesses including VeriLAN and Elematics, and holds an

MBA from the Ivey School of Business, University of Western

Ontario.

http://www.appsense.com


tion technology because there just aren’t enough controllers to make it technically viable or com-mercially beneficial. A traditional storage system contains only one or two RAID controllers that provide data protection for many drives as shown in Figure 1. Besides providing dedicated data protection for many drives, each controller must have adequate resources to protect all of the drives in case the other controller becomes disabled. The scarcity of controllers also limits the end-customer benefit of sharing the x86 CPU resources since consoli-dating one or two servers has a nominal impact on a system containing hundreds of disk drives.

Clustered Storage Systems Are Uniquely Suited for Server Virtualization Clustered storage architectures are character-ized by many controllers working in parallel to provide data protection. In a clustered archi-tecture, each appliance contains an x86-based controller with relatively few disk drives (see Figure 2). Data protection and controller failover responsibilities are evenly distributed across all of the appliances, so that no one appliance is heav-ily loaded. More resources, such as a second x86 control-ler, more RAM, and more NIC ports, can be easily added to each controller appliance since the clus-tered approach relies on industry-standard server technology. Thus, the performance of both the underlying storage system and the shared virtual machine server environments can be ensured.

Serverless Computing Versus Conventional Server Consolidation In traditional server consolidation, applications running at low CPU utilization are consolidated on beefy servers platforms dedicated to server-only tasks. Storage in the traditional model has generally moved external creating a massive market opportunity for easy-to-use IP SAN solu-tions. However, while server consolidation leads to widespread use of external shared storage, the server and storage platforms are discrete, man-aged separately, and based on different hardware platforms.

Customer Benefits of Serverless Computing Serverless computing allows, for the first time, the consolidation of server applications on to a common platform that runs both server apps and a clustered IP SAN ( see Figure 3). The benefits are startling. • Reduced Cost: The most obvious benefit of

consolidation servers into the SAN platform is reduced acquisition and management costs derived from the eliminated physical servers. The consolidated physical servers are normally quad-core Xeon processor machines with 2GB

AGGREGATION

Storage engineers have a long-standing tradition of pilfering innovations from the server market to create disruptive prod-uct improvements in storage systems.

This tried-and-true method works because the server market is larger than the storage market and, correspondingly, collects more investment dollars and has more engineers working on server innovations. Storage engineers constantly look for ways to reuse or repurpose server innovations that reduce development costs, speed product introductions, or, in extraordinary cases, break open new product opportunities.

Hardware Disruption Example: Repurposing x86 CPUs as Storage Controllers The use of server processor chips in high-end storage systems is an obvious example of this borrowing trend. Most high-end storage systems from companies such as EMC, Network Appli-ance, and Pivot3 use general-purpose x86 CPUs for advanced RAID controller functions, although this silicon was designed for general-purpose server applications. By repurposing this high-per-formance server silicon as high-end SAN and NAS storage controllers, storage companies eliminate internal chip development expense and leverage the massive engineering investments made by silicon specialists Intel and AMD. This move to off-the-shelf server silicon allows these same stor-age companies to redirect engineering dollars and engineers to work on software innovations that offered more direct product differentiation. Software Disruption Example: Repositioning NTFS as a NAS Offering Server software is also frequently reused to meet the needs of the storage market. A good

example would be the NTFS file system designed by Microsoft for the server market. Microsoft realized that the core NTFS functionality could be shipped independently of the more general Windows Server operating system to proliferate NTFS across volume servers, and reach a lower price point without cannibalizing the mainstream server OS. High-volume server companies, such as HP and Dell, resell the repurposed file systems a mainstream NAS offering, which has quickly become the unit volume leader in the NAS space.

Server Virtualization Disruption: Unleashing the Serverless SAN Platform It would be difficult to name a more disruptive technology in the server market than server virtu-alization. The $9 billion market value of VMware alone speaks to the immediate concrete savings that customers realize by replacing standalone physical servers with virtual machines that run x86-based hardware concurrently. This powerful value proposition has made server virtualization one of the most quickly adopted technologies in recent memory. So it should be no surprise to see storage architects looking for creative ways to deploy this technology innovation to storage systems.Why Haven’t Conventional SANs Adopted Server Virtualization? It turns out a conventional storage system can’t easily or practically integrate server virtualiza-

Server Virtualization Technology Makes Its Way into Storage Systems

BY LEE CASWELL

A common platform for both server apps and a clustered IP SAN

Figure 1 Conventional systems have few controllers

Figure 2 Serverless computing combines clustered storage & server virtualization

Figure 3 Serverless computing consolidation benefits

SERVER INNOVATION STORAGE APPLICATION STORAGE MARKET DISRUPTION

x86 CPU Replaced proprietary controllers Differentiation moved to software

Windows File System Windows NAS platform Wide-scale deployment of NAS

Server Virtualization Host servers on SAN hardware Serverless Computing

Table 1


AGGREGATION

of RAM, two NIC ports, and dual redundant power supplies in a 2U form factor, such as a Dell 2950. By consolidating the physical servers into a serverless computing platform, the exter-nal servers are not required.

Table 2 provides an example of how a recent customer using 1.2 petabytes of clustered SAN storage was able to eliminate more than 90 Dell servers by deploying serverless computing:

• Improved Failover: The use of a combined SAN/server platform also introduces a new stan-dard for failover protection for both server and storage workloads. On the storage front, a failed

appliance is treated as a simple RAID failure so that data is preserved and volume access is unimpeded. At the same time, the virtual ma-chine that was running on the failed appliance can be automatically restarted on any available appliance, so that the application can restart. This is possible because both LAN connections and storage connections are virtual and can be automatically reestablished once the applica-tion restarts on a resident appliance. This level of application failover is normally found only in the most expensive fault-tolerant server solutions.

• Simplifi ed Management: Applications run on one of the physical appliances, but these same applications have access to the aggregated ca-pacity and performance of all the appliances in the clustered storage array. Since load balanc-ing of performance and provisioning of capac-ity is automated, system confi guration is much simpler to design and ongoing management is simplifi ed since system administrators manage server and storage resources at an aggregate level instead of managing physical attributes of individual appliances.

Summary The application of server technologies to stor-age architectures has generally led to monumen-tal shifts in value. Server consolidation offers these types of benefi ts and is uniquely suited to clustered storage architectures. Serverless computing holds the promise of extending the reach of server virtualization technology to a large market that could benefi t from consolidating servers. This is a market that until now could not have benefi ted from consolidation.

About the Author

Lee Caswell is the co-founder and chief marketing offi cer of Pivot3.

He is an experienced technology executive with more than 20 years

of marketing and management experience in the storage, networking,

and digital video markets. Prior to Pivot3, Caswell held positions as

EVP of marketing and business development at VMware and was a

corporate offi cer at Adaptec, where he was VP and general manager

of Adaptec’s $350 million Storage Solutions Group. Prior to Adaptec,

he was VP of sales and marketing at Parallax Graphics where he

successfully closed funded development relationships with HP and

IBM. Caswell was also a co-founder and director of marketing at

Kodiak Technology, a LAN start-up, and was a marketing analyst at

SEEQ Technology, an Ethernet semiconductor company. He spent the

fi rst fi ve years of his career in management programs at GE. Lee has

a BA from Carleton College where he was a Wingspread Fellow and

an MBA from the Tuck School at Dartmouth College where he was a

Tuck Scholar.

Advertiser Index ADVERTISER URL PHONE PG

General Conditions: The Publisher reserves the right to refuse any advertising not meeting the standards that are set to protect the high edito-rial quality of .Net Developer’s Journal. All advertising is subject to approval by the Publisher. The Publisher assumes no liability for any costs or damages incurred if for any reason the Publisher fails to publish an advertisement. In no event shall the Publisher be liable for any costs or damages in excess of the cost of the advertisement as a result of a mistake in the advertisement or for any other reason. The Advertiser is fully responsible for all financial liability and terms of the contract executed by the agents or agencies who are acting on behalf of the Advertiser. Conditions set in this document (except the rates) are subject to change by the Publisher without notice. No conditions other than those set forth in this “General Conditions Document” shall be binding upon the Publisher. Advertisers (and their agencies) are fully responsible for the content of their advertisements printed in .Net Developer’s Journal. Advertisements are to be printed at the discretion of the Publisher. This discretion includes the positioning of the advertisement, except for “preferred positions” described in the rate table. Cancellations and changes to adver-tisements must be made in writing before the closing date. “Publisher” in this “General Conditions Document” refers to SYS-CON Publications, Inc. This index is provided as an additional service to our readers. The publisher does not assume any liability for errors or omissions.

Appsense http://www.appsense.com 23

CA http://www.ca.com/us/ C2

JavaOne http://java.sun.com/javaone C3

SYS-CON Events http://events.sys-con.com/ 201-802-3020 13 & 17

SYS-CON Media http://www.sys-con.com 201-802-3020 C4

Virtera http://www.virteratech.com 800-848-7185 17

JAVAONE CONFERENCE: June 2–5, 2009 | The Moscone Center, San Francisco, CA | THE PAVILION: June 1–4

Save $200on Conference registration!

Register by June 1st atjava.sun.com/javaone

The JavaOneSM conference brings together a vibrant developer community actively creating the innovations that will shape many industries. Come to the Conference, and learn how Java™ technology is helping with server, storage, and desktop virtualization. Here are 10 reasons why the JavaOne conference should be your must-attend conference of 2009:

Platinum Cosponsor

Cosponsors

© 2009. All rights reserved. Sun, Sun Microsystems, the Sun logo, Java, JavaFX, and JavaOne are trademarks or registered trademarks of Sun Microsystems, Inc. or its subsidiaries in the United States and other coutnries. Information subject to change without notice.

VIRTUALIZATION AT THE JAVAONESM CONFERENCE

Reason 1: Gain hands-on experience with Java technology as part of an amazing range of embedded devices you can carry, share, and drive.

Reason 2: Network with peers, and influence the future of the Java platform, including Java SE, Java Development Kit 7, JavaFX™, Java ME technology, and more.

Reason 3: See what’s new on the Java EE 6 platform and the GlassFish™ v3 application server.

Reason 4: Learn how to use the MySQL™ database with theGlassFish application server.

Reason 5: Hear about lightweight and RESTful approaches toservices.

Reason 6: See what’s new with contemporary integration inProject Fuji.

Reason 7: Learn about cloud services.

Reason 8: Sit down with Sun’s Java technology architects, and create applications and content together for the Java and JavaFX platforms.

Reason 9: Develop JavaFX Mobile applications, and run themon a nifty new handset.

Reason 10: Build interactive content for TV that you can watchon that new big screen or Blu-ray player you’ve been wanting.

Nowadays money’s tight. That’s why it’s more important than ever to attend the one conference that delivers everything you want to see, learn, and experience—all under one big roof. And that’s the JavaOne conference.

Save Your Spot—Register Today!

By the way, check out CommunityOne, Sun’s conference on open-source innovation and implementation, colocated with the JavaOne conference.

developers.sun.com/events/communityone

ATTRIBUTE TRADITIONAL SERVERLESS Savings

Server Acquisition Costs $276k 0 $276k

Server Management Costs $92k/year $0 $92k/year

Power/Cooling 92 KWH/yr 40KWH/yr 52KWH/yr

Server Rack Space 4 Racks 0 4 Racks

Table 2

http://java.sun.com/javaone/index.jsp?cid=J9JMT56

http://virtualization.sys-con.com28 MAY / JUNE 2009 Volume: 2 Issue: 3

��

24/7

Visit the ��

Website Today!

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

24/7

http://www.sys-con.com

Case

Technology

Transcript of Case