Open Storage
description
Transcript of Open Storage
-
Abstract
Demands for data storage continue to increase, and enterprises are looking for ways to address this challenge
while keeping costs down. Instead of buying and then maintaining more and more storage devices, enterprises
now have another option: open storage. With open storage, businesses can leverage industry-standard
components and open-source software to build highly scalable, reliable, and flexible storage systems. And
now, Sun Open Storage provides a flexible option that can significantly lower costs, provide high performance,
simplify storage management, and provide access to the latest innovative capabilities.
SUN OPEN STORAGE:A LOWER-COST, HIGHER- PERFORMANCE ALTERNATIVE FOR THE ENTERPRISEOpen StorageWhite PaperMay 2009
-
Sun Microsystems, Inc.
Table of Contents
Executive Summary ........................................................................................1
Storage Challenges in a Data-Intensive World ...................................................1
Open Storage in the Enterprise .......................................................................2
Open Source Delivers Software Building Blocks for Open Storage ........................ 3
Solaris ZFS Software .......................................................................................... 3
OpenSolaris COMSTAR ......................................................................................... 4
Solaris DTrace ..................................................................................................... 4
Solaris Fault Management Architecture ................................................................ 4
Extending the Value of Open Storage to Hardware Components.......................... 5
Complete, Enterprise-Ready Storage Solutions: Hardware Innovation That Fully Leverages Open Source ..................................................................................6
Sun Fire X4540 Storage Server: Tapping the Power of ZFS ................................. 8
How to Get Started with Sun Open Storage ......................................................9
Sun Open Storage Customer Success Story ........................................................9
Wikimedia Foundation Ensures Data Integrity for Ever-Expanding Online Resources 9
-
Sun Microsystems, Inc.1 Sun Open Storage
1 Source: IDC White Paper sponsored by EMC, The Diverse and Exploding Digital Universe, March 2008
.
Executive SummaryThe amount of data that enterprises need to store, manage, and protect is increasing
exponentially. Enterprises traditionally address increasing storage needs by
purchasing more storage devices from proprietary vendors. However, these closed
systems have a number of disadvantages, including:
High cost. In a proprietary environment, all disk drives, controllers, and software
typically have to come from the same vendor and are sold at premium prices.
The hardware often severely lags in capability behind industry-standard systems
components at similar price levels. And while the hardware may not seem
expensive, the cumulative cost associated with required and proprietary software
makes such a storage system much more costly.
Lack of flexibility. Enterprises cannot add or modify software to suit their specific
needs or adapt and expand a storage system to meet growing data demands.
Today, enterprises have another option: open storage, which some industry analysts
estimate will make up almost 12% of the storage market by 2011. Using industry-
standard components and open-source software, open storage enables enterprises
to build highly scalable, reliable, and flexible storage systems to fit their growing
demandsall at a fraction of the cost of traditional, proprietary storage.
Responding to this market need, Sun Open Storage brings together leading open-
source software and industry-standard, proven servers to provide enterprises with a
real alternative. Enterprises that leverage Sun Open Storage can enjoy:
Lower costs. With open-source software and industry-standard systems, Sun
Open Storage delivers enterprise reliability and scalability at one-tenth the cost of
closed, proprietary storage systems.
A high degree of flexibility. With Sun Open Storage, enterprises are not locked in
to a single vendor; they can use the components that fit their needs and add new
capabilitiesfrom Sun or third partieswithout paying a premium or annuity for
proprietary products.
High performance, simplified management. Sun Open Storage leverages
sophisticated tools for optimizing performance and dramatically reducing the
complexity of storage management.
Innovation. With a large community of developers focused on creating new
capabilities that meet enterprises storage needs, Sun Open Storage can quickly
bring the best features to market.
Storage Challenges in a Data-Intensive WorldWith more and more users sharing content and collaborating, communicating, and
conducting business across the globe, enterprises today must ensure that their
storage infrastructure is flexible and scalable enough to handle these rapidly
growing demands. In 2008, IDC1 reported that the amount of new digital information
-
Sun Microsystems, Inc.2 Sun Open Storage
created, captured, or replicated in 2007 was 281 exabytes and is expected to grow
to 1773 exabytes by 2011. (An exabyte is equal to 1 quintillion bytes.) Approximately
70% of this data will be created by individuals or non-enterprises, but enterprises
and organizations will be responsible for storing, securing, and protecting 85% of
this new digital data.
When faced with increasing storage needs, the default strategy for many years
has been for enterprises to simply buy more storage devices. However, almost all
modern disk arrays and storage systems available from traditional storage vendors
are closed, proprietary systems; their customers are locked into buying disk drives,
controllers, and proprietary software features from only themtypically at premium
prices. Additionally, their customers cannot add their own drives or add or modify
software beyond what management configuration tools allow.
The rapid growth of new digital data, the management challenges that accompany
this growth, and the increasing costs of storage management point to the need for a
radically different storage pricing structure, as well as for new storage architectures
that offer more flexibility.
With the continuing growth of open-source software, community development, and
industry-standard hardware, open-storage architectures have become a viable
alternative for enterprises.
Open Storage in the Enterprise Open storage uses open-source software and industry-standard system components
to build highly scalable, reliable, and affordable enterprise storage systems, leverag-
ing the rapid innovation and flexibility of open communities and architectures.
Enterprises using open storage solutions can quickly benefit from the new
capabilities developed in a growing community of open-source software developers
rather than depend upon a proprietary vendors business goals or R&D budget.
They gain more freedom to expand their storage solutions in cost-effective ways.
For example, enterprises undertaking Web 2.0 initiatives need a high degree
of flexibility. Web 2.0 applications are dynamic, and they depend on open stan-
dards and open-source software to reduce costs and give developers the ability
to differentiate by adding their own custom software. Open storage meets these
requirements better than any other storage infrastructure or architecture avail-
able. Google and Amazon could not exist in their current forms if they hadnt built
their own storage infrastructures based on open storage principles, rather than
traditional, proprietary storage architectures.
On the hardware side, there are a number of technical advantages to building
storage devices out of industry-standard system components and servers. One of
-
Sun Microsystems, Inc.3 Sun Open Storage
the performance bottlenecks in a data center has been the network; by using open
storage, applications can be brought closer to the data by running them on the
same system dedicated to storage functions. Industry-standard servers also provide
more processing capability and more cache memory than is available in a traditional
modular storage array. And finally, an open-storage architecture allows enterprises
to easily add CPUs, cache, and I/O connectivity, providing multi-dimensional
scalability.
By heavily investing in the development of the most comprehensive set of open-
storage products, solutions, and services, Sun offers enterprises proven, scalable
technology with a level of flexibility and cost-effectiveness not found in proprietary
systems.
Open Source Delivers Software Building Blocks for Open StorageSun Open Storage offers enterprises the ability to build the storage infrastructure
that suits their needs using Suns proven, open-source software and industry-
standard hardware components (either from Sun or certified hardware from third
parties).
The foundation of Sun Open Storage is the OpenSolaris Operating System, one of
the most robust, reliable, and innovative OSs available. The OpenSolaris storage
community, launched in 2007 when Sun donated a vast amount of storage technol-
ogy to the OpenSolaris project, is now the worlds leading centralized open-source
storage community. Sun plays an active role in the OpenSolaris storage community
by contributing code and working with open-source developers around the world to
design or enhance storage capabilities.
Storage industry leaders such as Hitachi Data Systems, QLogic, Emulex, and Brocade
have also contributed their software to the OpenSolaris community, and companies
across the IT landscape are using OpenSolaris technology in their storage product
offerings. Sun remains active in the community, providing expert support and service
to Sun Open Storage users, and Sun engineers continue to build upon the work of
the open-source community to deliver additional features and capabilities that
businesses need.
OpenSolaris features that are central to the Sun Open Storage solution are:
Solaris ZFS SoftwareSolaris ZFS technology offers a dramatic advancement in data management with
a virtual storage pool design, integrated volume manager, and data services that
provide an innovative approach to data integrity.
-
Sun Microsystems, Inc.4 Sun Open Storage
1 The $2 million claim is based upon Sun consolidation customers average savings over an 18-24 month period. Cus-tomers calculated savings based on energy, space, licensing, services contract costs, increased utilization and reduction of management of systems.
Sun Open Storage:
Cost-Saving Opportunities
for the Enterprise
Sun Open Storage delivers breakthrough
economics, saving enterprises up to 90%
over closed, proprietary storage.
Licensing fees. OpenSolaris and related
data services software, such as snapshot
and remote replication, are free to use
without licensing fees. On the other
hand, technologies from competitors like
EMC or NetApp carry expensive license
fees, often carrying list prices in the tens
of thousands of dollars, significantly
increasing the initial cost of their
solutions.
Acquisition cost. Proprietary disks such
as the EMC CLARiiON are marked up 5x
compared to market-priced disks. Enter-
prises can save up to 90% with a Sun Fire
x4500 ($1.29/GB) vs. comparable disks in
EMC Symmetrix with SATA ($11.78/GB) or
EMC CLARiiON tier 3 SATA ($3.44/GB) for
a 62% savings. (Note: this is only
acquisition cost.)
Investment protection. With open
source, there is no cost to acquire and
no vendor lock-in. Because enterprises
pay at the point of value (hardware, war-
ranty, and services), this model protects
investments and turns software into
operational costs.
Virtualization. With Sun Open Storage,
enterprises can apply server virtualiza-
tion techniques to storage infrastructure,
allowing flexibility and choice for where
they run data services and saving energy,
space, and expense while increasing
utilization rates. Suns virtualization
solutions combine energy-efficient Sun
servers, storage, software, and profes-
sional services which have been proven
to reduce IT costs by as much as $2M/
year1, achieve 99.99% availability, and
improve system utilization to as high as
85%.
(Continued on Page 5)
ZFS software allows for more efficient use of storage devices and enables accelerated
performance of Web applications without changes to the application. Physical
storage can be added to or removed from storage pools dynamically without inter-
rupting services, providing new levels of flexibility, availability, and performance.
In terms of scalability, ZFS software is a 128-bit file system that provides 16 billion,
billion times the capacity of 32- or even 64-bit file systems. It supports more file
systems, snapshots, and files in a file system than can possibly be created in the
foreseeable future.
ZFS software also provides the data services needed to protect data far beyond what
exists today in traditional storage systems. It saves enterprises money by maintain-
ing data redundancy on commodity hardware through the delivery of basic mirror-
ing, compression, and integrated volume management. It supports new hybrid disk-
storage devices that include Flash technology for greater performance. And it also
combines proven, cutting-edge technologies to ensure data integrity; while most file
systems do nothing to check the integrity of data on the disks, ZFS software checks
the data continuously to guard against silent corruption.
Vendors of closed, proprietary storage appliances typically charge customers extra
software licensing fees for data management services like administration, replica-
tion, and volume management. Because ZFS software is part of the OpenSolaris OS,
those additional costs, complexities, and unnecessary layers are eliminated.
OpenSolaris COMSTARWith OpenSolaris COMSTAR (Common Multiprotocol SCSI TARget), an enterprise IT
department can turn any standard server into a storage device, delivering significant
cost savings. COMSTAR is a software framework that offers the functionality of a
SCSI target device that can be accessed over the network, essentially allowing an
enterprise to create a storage array from standard computing server and storage
components.
Solaris DTraceSolaris DTrace is a comprehensive, advanced tracing tool for troubleshooting
systemic problems in real time. With DTrace, administrators, integrators, and
developers can tune applications for performance and troubleshoot production
systemsall without requiring the production system to be taken offline and
without impacting performance.
Solaris Fault Management ArchitectureAs storage data grows, there is an increasing chance of media and subsequent
hardware failures. The OpenSolaris community is continuing the development and
refinement of the Solaris 10 Fault Management Architecture (FMA) by adding disk
-
Sun Microsystems, Inc.5 Sun Open Storage
2 To find out more, see http://www.sun.com/customers/storage/knowledgebase.xml
(Continued from Page 4)
Eco Efficiency. Sun offers systems,
storage, software, and services that
yield tremendous savings in terms of
energy, materials, space, and labor. For
example, an OpenSolaris hybrid stor-
age pool (HSP) running on a four-way
quad-core Intel Xeon platform with
five 400GB 5400 RPM drives, one 80GB
read-optimized flash drive, and one 32GB
write-optimized flash drive, used 4.9x
less power than an equivalent server
platform running seven 146GB 10,000
RPM SAS drives. The HSP also delivered
a 3.2x improvement in read IOPS and an
11% improvement in write IOPS. (Perfor-
mance numbers will vary depending on
the specific workload.)
Assessment Services. Suns storage
experts can help architect cost-effective
storage infrastructures tailored to each
organizations specific needs. For exam-
ple, by utilizing Sun Storage Assessment
services, KnowledgeBase Marketing
reduced storage TCO by 40%, administra-
tion time by 50%, and transaction costs
by 86%.2
failure diagnosis, which includes quick identification of faulty drives and
preventative measures.
FMA provides automatic monitoring and diagnosis of I/O subsystems and hardware
faults and facilitates a simpler and more effective end-to-end experience for system
administrators. This is achieved by isolating and disabling faulty components and
then continuing the provision of service through a reconfiguration of redundant
paths to dataeven before an administrator knows there is a problem.
Extending the Value of Open Storage to Hardware ComponentsSun Open Storage solutions leverage the influence of a community of passionate
open-source software developers as well as more than 25 years of Suns on-going
innovation in server, storage, and software technologies. With Sun Open Storage,
enterprises can freely mix, match, and maximize software and datacenter compo-
nents and even repurpose and reuse hardware simply by adding new software.
Because its not always economically feasible to swap out old systems in order to
upgrade performance or add capacity, Sun Open Storage solutions offer enterprises
virtually unlimited file system capacity (through the open-source ZFS software)
and can scale throughput well beyond that of traditional, proprietary storage
solutionsat a fraction of the cost.
By combining proven open-source software with industry-standard hardware, Sun
Open Storage enables enterprises to:
Selectthesoftwareandhardwarecomponentsthatmeettheirneeds
Reducetheirrelianceonhigh-pricedsystemsandsaveupto90%onstoragecost
GainmassivescalabilitytoenablefastresponsetounpredictableWeb
data growth
Simplifydeploymentandmanagement
Improveservicelevelsthroughtuningandreal-timediagnosticsandworkload
analysis
Quicklyreconfiguresystemstoindependentlyincreasecomputepower,storage
capacity, or performance
-
Sun Microsystems, Inc.6 Sun Open Storage
Sun Open Storage solutions allow enterprises flexibility in designing their storage
infrastructures as well as cost savings compared to proprietary storage solutions.
Businesses can leverage the innovation available in Suns open-source software,
including advanced tools for optimizing performance and simplifying storage
management, use only the components that fit their needs, and add new capabili-
ties easily and without the high prices charged by proprietary vendors. And because
industry-standard hardware can adopt next-generation technologies fasterwith
the additional benefit of fair market pricing through competitionenterprises can
realize consistently higher performance at a lower cost with open-storage systems
than with closed systems.
To accommodate different business and technical needs, Sun offers both industry-
standard open-storage components and, for enterprises that want a complete
solution out of the box, fully integrated and optimized storage appliances such as
the Sun Storage 7000 Unified Storage Systems, which include one of the industrys
most powerful, easy-to-use management software solutions.
Complete, Enterprise-Ready Storage Solutions: Hardware Innovation That Fully Leverages Open Source While some enterprises may want to build their own storage infrastructure using
Suns open-source software and industry-standard hardware, others want the cost-
effectiveness and flexibility of open-source technologies in a complete, enterprise-
ready storage offering. The Sun Storage 7000 Unified Storage System offers a turnkey
storage solution specifically designed to leverage the powerful features of the Open-
Solaris OS, bringing the promise and benefits of open storage to the enterprise.
Sun Storage 7000 unified storage systems are powerful storage appliances that
utilize low-cost commodity disks, enabling enterprises to scale performance along
with capacity without expensive, high-speed disk drives. These appliances are the
only storage systems that also fully exploit Flash storage technology by taking
advantage of hybrid storage pools in Solaris ZFS software as well as a range of other
unique technologies in the OpenSolaris OS.
Unlike other storage solutions that charge for each additional storage feature, all
Sun Storage 7000 unified storage systems include comprehensive data services at
no extra cost, including snap/clone, restore, mirroring, RAID-5, RAID-6, replication,
active-active clustering, compression, thin provisioning, CIFS, NFS, iSCSI, HTTP/FTP,
and WebDAV. The unique design of these systems also enables enterprises to benefit
from continuous innovation when new data services and features are available from
Sun through included automated updates.
-
Sun Microsystems, Inc.7 Sun Open Storage
Sun Storage 7000 unified storage systems provide:
Simplified management. Sun Storage 7000 series products are fully integrated
appliances that can be installed and configured in less than five minutes in some
cases. The graphical user interface is extremely intuitive and simple to use and
administer.
Breakthrough economics. By leveraging open-source software and open-standards
hardware components, the Sun Storage 7000 series achieves breakthrough
economics for enterprise customers in todays turbulent economy with:
Lower cost
Lower power consumption
No software licensing fees
Faster innovation compared to proprietary systems
DTrace Analytics. The industrys only comprehensive and intuitive analytics
environment, DTrace Analytics provides all the tools administrators need to quickly
identify and diagnose system performance issues, perform capacity planning, and
debug live storage and networking problems before they become challenging for
the entire network. Available only with Sun Storage 7000 unified storage systems,
DTrace Analytics gives administrators the ability to easily drill down for in-depth
analysis of Web infrastructure storage subsystems. Built-in instrumentation
provides real-time visibility throughout the data path, making it much faster and
easier to pinpoint performance bottleneckseven those that may not be related
to the storage system.
Hybrid storage pools. Sun Storage 7000 series products are the only storage
systems with hybrid storage pools that combine DRAM, write-optimized solid-state
drives (SSDs), and read-optimized SSD with high-capacity, low-cost, enterprise-
class SATA drives. Hybrid storage pools use Solaris ZFS software to optimize and
manage them transparently as a single hierarchy. This new storage design uses the
low latency of SSDs to increase performance for both write- and read-oriented
applications, delivering superior performance and high capacity with lower cost
and energy consumption.
Self-healing storage. The Sun Storage 7000 series includes Failure Management
Architecture (FMA), which provides the capability to detect and take faulty hard-
ware components offline in order to prevent system disruption. FMA combined
with ZFS data integrity facilities makes the Sun Storage 7000 the most
comprehensive self-healing unified storage system available.
-
Sun Microsystems, Inc.8 Sun Open Storage
SuN STORAGE 7000 EMC CELERRA NETAPP FAS
OPERATING SySTEM Open-source, enterprise-class OpenSolaris
Proprietary DART Proprietary Data ONTAP
DATA SERVICES SOFTWARE
No separate licensing fees
Additional license fee for most options
Additional license fee for most options
FLASH/SSD OPTIMIzED
Yes, through ZFS and hybrid storage pools
Yes: Flash as Tier 0 (no hybrid storage pool)
Not available
ANALyTICS Robust, real-time analytics through DTrace
Limited reporting through Celerra Manager
Limited reporting available
Sun Storage 7000 vs. Proprietary Systems
Sun Fire X4540 Storage Server: Tapping the Power of ZFSThe Sun Fire X4540 Storage Server is the first open-storage server that integrates
industry-standard server hardware and storage components in an open architecture
and leverages the open source-based Solaris ZFS software RAID, which maintains
data redundancy without the expense of a hardware RAID controller or NVRAM. The
Sun Fire X4540 server delivers the highest storage density and data throughput with
a built-in, high-performance compute engine at half the cost and up to 50% power
savings compared to traditional servers and storage arrays.
When compared to a typical storage controller, the Sun Fire X4540 server offers
significant cost savings and more power:
PROPRIETARy 2u STORAGE CONTROLLER 2u x64 SuN FIRE x4540 SERVER
1-2 CPU sockets 4 CPU sockets
2-4 CPU cores 16 CPU cores
4 GB DRAM 64-128 GB DRAM
1 GB NVRAM
4x1GB Ethernet 4x1GB Ethernet
4-6x PCI-X or PCIe 4-6x PCIe
Cost: $40,000 to $50,000 Cost: $12,000 to $24,000
-
Sun Microsystems, Inc.9 Sun Open Storage
The Sun Fire X4540 storage server is a great fit for applications that take advantage
of the close proximity of data to compute resources, such as HPC storage clusters
utilizing Lustre (an open-source file system available from Sun designed for high-
performance Linux compute clusters), high-performance data warehousing, video
streaming and video surveillance, and data backup on disk. A very versatile
storage system, the Sun Fire X4540 storage server fits seamlessly in file-based
storage architectures with NFS and CIFS, iSCSI storage systems, and FC-based SAN
storage environments.
How to Get Started with Sun Open Storage
1. If they havent already, ask your developers to download and experiment with
OpenSolaris through the storage community at opensolaris.org/os/community/
storage/. They can also join the community and help create new innovations in
storage software.
2. Try out Sun storage solutions free of charge through the Sun Try and Buy
program at sun.com/tryandbuy/.
For more information on Sun Open Storage, see sun.com/openstorage/.
Sun Open Storage Customer Success Story
Wikimedia Foundation Ensures Data Integrity for Ever-Expanding Online Resources Founded in 2001, the Wikimedia Foundation hosts a variety of wikis, including
Wiktionary, Wikiquote, and Wikimedia Commonsbut by far it is best known for
Wikipedia, the free online encyclopedia. Among the 10 most visited Web sites in the
world, Wikipedia had nearly 250 million unique visitors in October 2008. Articles are
written and edited by more than 100,000 volunteer editors worldwide and in total,
the site hosts 11 million articles in 265 languages.
The Web site receives between 25,000 and 60,000 page requests per second,
depending on the time of day. Wikimedia needed to update its infrastructure to
handle this huge volume of traffic and ensure that its systems were reliable, highly
available, and easily scalable. It also wanted to expand its upload file limit from 20
MB to 100 MB to accommodate rich media (audio and video) contentbut before it
could do that, Wikimedia needed to expand its storage capacity.
The foundation acquired Sun Fire X4150, Sun Fire X4240, and Sun Fire X4540 servers
running the Solaris 10 Operating System and MySQL open-source database soft-
ware. The Sun Fire 4540 storage server provides low-cost, high-density storage and
eight-way processing in a single enclosure. We looked at other vendors, but they
couldnt get up to the scale we wanted, says Brion Vibber, chief technology officer
-
Sun Microsystems, Inc.10 Sun Open Storage
at the Wikimedia Foundation. They didnt offer support like ZFS backups, snapshot-
ting and self-healing; or they were just incredibly expensive SAN systems that were
totally proprietary.
Data integrity was one of the Wikimedia Foundations primary concerns. On their
Linux servers, data was backed up to direct-attached RAID arrays. If a disk failed, we
could replace it, but if a whole machine died, we were out of luck. We needed a way
to keep a standby disk active, says Vibber. On the Sun Fire X4540 servers, which are
configured in pairs, Wikimedia uses Solaris ZFS replication to ensure data integrity.
With ZFS, were able to replicate in real time and store multiple backup snapshots
on each server, Vibber says. Its a big improvement because we no longer worry
about losing all of our work if a system fails.
It was also very difficult for Wikimedia to expand storage with the configuration
it hadbut Suns Open Storage approach changed that, too. When we needed
more storage, we would add another Linux box with a big RAID array, and then copy
directories around and split up everything manually, says Vibber. Now with ZFS,
we can add disks and extend the ZFS pool, and were donethe space is instantly
available. It doesnt require us to replace everything if we need to expand, and were
very happy about that.
-
Sun Microsystems, Inc. 4150 Network Circle, Santa Clara, CA 95054 USA Phone 1-650-960-1300 or 1-800-555-9SUN (9786) Web sun.com
Sun Microsystems, Inc.Sun Open Storage
2009 Sun Microsystems, Inc. All rights reserved. Sun, Sun Microsystems, the Sun logo, OpenSolaris, Solaris, ZFS, Sun Fire, and MySQL are trademarks or registered trademarks of Sun Microsystems, Inc. or its subsidiaries in the United States and other countries. Printed in USA 05/09 #XXXXX