EMC-Isilon__CAUDIT-RDSI_Nov2013

7/25/2019 EMC-Isilon__CAUDIT-RDSI_Nov2013

http://slidepdf.com/reader/full/emc-isiloncaudit-rdsinov2013 1/33

© Copyright 2013 EMC Corporation. Company Confidential, do not distribute without explicit permission

ThePOWE

ofEMC ISITo TranseResea

Greg Rogers, RTMEMC ISILON Australia

Charles Sevior, CTOEMC ISILON Asia-Pac

ISILON – CAUDIT presentation to RDSI

http://localhost/var/www/apps/Videos/Isilon/EMC%20Isilon%20Scale-Out%20NAS%20for%20Media%20&%20Entertainment.mp4



© Copyright 2013 EMC Corporation. All rights reserved.


Agenda – Topics to be covered

1. Isilon eResearch customer references - Charles

2. Isilon NAS brief overview - Charles

3. Aspera integration into Isilon OneFS – JC

4. Hadoop analytics integration with Isilon - Charles

5. Isilon OneFS software / hardware roadmap – Charles

6. Next steps and partnerships - Greg



© Copyright 2013 EMC Corporation. All rights reserved.ISILON – CAUDIT presentation to RDSI

UCLA

EMC Isilon Established LeadershipHealthcare, Life Sciences, eResear

http://images.google.com/imgres?imgurl=http://www.logoblog.org/wp-images/logo-designs/general-electric-logo.png&imgrefurl=http://www.logoblog.org/wordpress/creative-logos/&usg=__PSrbEu5MnGFRSR32bTvEKhxhXvc=&h=173&w=171&sz=13&hl=en&start=4&tbnid=ST59Z-5Yi1BQUM:&tbnh=100&tbnw=99&prev=/images?q=general+electric+logo&gbv=2&hl=en&sa=G

http://www.mpg.de/english/portal/index.html

http://www.jhu.edu/

http://www.broad.mit.edu/index.html




Columbia UniversityCenter for Computational Biology and Bioinformatics (“

Challenge Using a traditional NAS system that strug

huge amounts of input/output demands oour computing infrastructure

Imminently outgrowing their infrastructu

Solution X-Series, NL-Series

SmartPools

SmartQuotas

Results Now supports some 4,000 CPUs, which ca

heavy I/O and data analysis demands of

Supports the storage needs of three addiwithin the Columbia University network

Applicati Genomic

“After switching to Isilon, we no longer had to worry tcouldn’t handle our research demands. We knew thatindependently scale capacity and performance, so tha

we need, when we need it.”

JOHN LOWELLWOFFORD

Director IT Services




3TIERTurns Big Data Into Accurate Renewable Energy Intelli

Challenge Traditional RAID array couldn’t scale to m

of its big data workflow

Complicated management

Escalating costs

Solution X-Series

NL-Series

SmartPools

Results Single file system and point of managem

performance tiers

Unifying all operations on a single, shared

Simplifying Big Data management

Applicati energy a

forecasti

“Before Isilon, our team had to do everything from mamigrations to mapping directory paths, which simply for a business growing as quickly as ours. With Isilonsimple. We’ve eliminated data management and stora

freeing up resources to focus on the services that delour clients.”

PAUL ENGLISHDirector of IT at 3TIER


http://slidepdf.com/reader/full/emc-isiloncaudit-rdsinov2013 6/33© Copyright 2013 EMC Corporation. All rights reserved.


RENCIRenaissance Computing Institute of the University of N

• Turning Big Data into

Insight in the lab andtherapy in the clinic

• Knowledge based medicineprograms for epilepsy &prostate cancer

• Secure medical workspace

• Informatics for GeneticSequencing (IGS)

• Blending traditional clustercomputing with iRODS andHadoop analytics

http://www.emc.com/collateral/white-papers/h11692-life-sciences-renci-big-data-manage-info-wp

http://www.emc.com/collateral/white-papers/h11692-life-sciences-renci-big-data-manage-info-wp.pdf






http://slidepdf.com/reader/full/emc-isiloncaudit-rdsinov2013 7/33© Copyright 2013 EMC Corporation. All rights reserved.ISILON – CAUDIT presentation to RDSI

eResear

Challeng



User requirements for RDSI?

Growing data sets – mostly unstructured

Reliable storage integrity / preservation Manage migration issues – active and cold d

Leverage MapReduce and Hadoop analytics

Ease of use and simple management Trust / Security / Immutability

Sharing / Collaboration / Cloud – REST API



EMC IsilUnique Differentia



Challenges arising from standard

•Utilisation rates varyacross islands

•QOS headroom

•Performanceconsiderations

= Storage Efficiency50% - 65%



Clustered StorageLayer Com

Servers/Clients

Client/Application Layer Ethernet Layer

Servers/Clients

Servers/Clients

Compute Nodes

EMC Isilon – NODE-Based Architect



Focus on the d

not the stora




AutoBalance: Automated data balancing across nodereduces costs, complexity and risks for scaling storage

• AutoBalance migrates constorage nodes while systeand in production

• NO manual intervention

• NO reconfiguration

• NO server or client mountapplication changes

• Eliminates “Hot Spots”

EMPTY

EMPTY

EMPTY

EMPTY

EMPTY

FULL

FULL

FULL

FULL

BALANCED

BALANCED

BALANCED

BALANCED

BALANCED

EMC Isilon – On-The-Fly Expansio




Data Layout - protection and throu

Data is striped across the nodes

– Not across disks – FEC not RAID

Data breakdown:

– 8 KB blocks

– 16 blocks per stripe unit

– 128 KB stripe width per drive

Cli entsFile

WriteFile




With N+2, N+3, andN+4 protection,data is 100% availableif multiple drives ornodes fail together

With n+1 protection,

data is 100% availableeven if one driveor one node fails

Built-in high availability clustered architecture – No RA

100%

100%

100%

100%

100%

100%

100%

100%

FAILED

FAILED

Fastest rebuild time, andwith Isilon, the more

nodes in the cluster, the

faster drive rebuild time

EMC Isilon – Leading Data Protect




20120

20

20

“No Node Left Behind”

12

12

72

12

72

288

Generations of Isilon nodes can coexist

– Heterogeneous nodes mix and match

– Storage investment protected

Rapid introduction of the latest technology

– Faster, denser, greener storage each year

– New capabilities like Hadoop and MobileIQ

– No server, network or application changes

Push-button node retire – SmartFail

– Storage older than 5 years is a waste of space!

EMC Isilon – Storage Never Obsole

No More Data Migration. Keep up with Moore




Isilon+AspeStore & Move Big

JC DiomaAspera




EMC Isilon with integrated Aspera




Aspera and Isilon have partnered for many

create a predictable, non-disruptive, high-performance wide area file and data deliversolution specifically for moving large data selong distances at the fastest possible speed

Aspera’s cluster-aware high-speed transfer coupled with EMC Isilon OneFS operating syand scale-out NAS architecture is a powerfucombination for distributed storage environ

The Benefits of Isilon + Aspera




The Benefits of Isilon + Aspera

Maximum cluster-to-cluster performance ov

campus and wide-area networks Massive concurrency

Easy scalability of WAN transfer performanc

No single point of failure Single point of management

Web services and industry-standard API sup




EMC IsilHadoop Analy




ANALYTICSMOBILE

Key

B

Hadoop 2

Native HD

Pivotal H

Simultane& HDFS 2

Distribute

Support FHadoop A

No Single

Improved

Next-Gen Analytics

HDFS St d d H d Cl t




HDFS: Standard Hadoop Cluster

Data

CompuCompu

Data

Name node

3X

NFS

Name node

OLAP

HTTP

CIFS

FTP

NFS

Landing ZoneServers

Step 1:Data is copied intothe Landing Zone

StepData is locafrom a node

Cluster (3

SMapRed

Log Files

Decision SupportDatabases

User/WEBClick data

3X

3X

Step 2:Data is copied fromlanding zone onto anode in the Cluster

HDFS: Integ ated Isilon and Hado




SMB, NFS,HTTP, FTP,

HDFS

HDFS: Integrated Isilon and Hado

name no

name no

name no

name no

Step 1:Much or all of the Data

lives on the Isilon Cluster

Step 2:Jobs are run

HadNFS

OLAP

Log Files

Decision SupportDatabases

User/WEBClick data

ARCHITECTURE FOR HIGH PERFORMANCE COMPUTING WITH INTEGRA




ARCHITECTURE FOR HIGH PERFORMANCE COMPUTING WITH INTEGRA




Future Directio




OneFS 7.0 - Maveric

Scalability Enterprise Mana Performance

– Improved Throughput

– Reduced Latency

Capacity– SnapshotIQ File Clones

– 20 PB File System Ready

Extensibility– REST Platform API

Data Pr– SyncIQ:

– Snapsho

AutomaProvisio– Improve

– Higher R

Availabi– Upgrade

Multi-Tenancy– Authentication Zones

– Role Based Administration

VMware– Reduced I/O Latency

– VAAI, VASA, SRM

Security and Archive– SmartLock v2 (SEC17a-4)

2H2012

2H




OneFS 7.1 - Waikiki

Efficiency Scale Deduplication

– Post process, block leveldeduplication

– Increased data efficiency

– Multi-level directoryconfiguration

– Dry-run dedupe savingsestimation tool

– SmartPools Enhancements

– CLI and API management

– RBAC integration

– User defined node-pools

Perfo SyncIQ &

– “Continuo

– ~1 minut

Job Engine

– Improved

– Multiple s

– Enhanced

monitorin

Breaking b

– 3 simulta jobs

Enterprise Security and Archive

– CEE support. Varonis firstpartner for Audit

– Improved RBAC Coverage

– Encrypted Nodes

Backup

– Faster incrementals andincrementals forever

Manageability & Extensibility

– RBAC improvements to API

– Broaden pAPI coverage incl.SyncIQ AND JobEngine

– ESRS Gateway Integration

2H2013




Growth In Applications

Sources: IDC, Gartner, AWS Workload Estimates

Next Gen Cloud Ap

2016 48M

2012 6M

Traditional Applications

2016 141M

2012 83M 70%




Making Compute the Bottleneck

A Giant Leap in Capacity

Around-the-Clock, Around-the-Globe

Absolute Freedom and Control

Isilon’s OneFS Operating System

StrategicVectors

O FS St




NEXT GEN APPLICATIONS

Hybrid

Private(Enterprise)

OneFS StorageIsilon, The Big Data Platform

VERTIA

Public

I il h i R h




Isilon strengths in eResearchArea Strength

Onsite data collection • Multi-protocol support for instruments & workstations• CIFS (SMB), NFS, FTP, HTTP

Repository•

SmartPools (Automatic storage tiering)• GNS (Global Namespace, with metadata acceleration)

Workspaces • SmartQuotas (Policy-based storage management)• Home Directories

Collaboration • Aspera FASP protocol integrated into OneFS• Aspera Enterprise Server and Connect Server hosted within the

HPC/Hadoop • HDFS (native support)• 10GbE (2/node)•

SmartConnect• Concurrent File Access support (OneFS 7.0)

Archival • SnapshotIQ• WORM – immutable writes• NL Series

EMC-Isilon__CAUDIT-RDSI_Nov2013

Documents

Transcript of EMC-Isilon__CAUDIT-RDSI_Nov2013