The Future of Network-Based Storage

INTEL High Performance Data Division 1

The Future of Network-Based Storage

Brent Gorda

January 2013


(Almost) Everyone is talking exascale

Background

Driving forces

Our thoughts and our R&D activities

The future of network based storage


Background and Justification (why I/O)

• HPC pushes the edge – every edge – of technology

• Storage is a bottleneck

• A tax on compute per Mark Seager (LLNL -> Intel)

• ASCI systems were designed for < 10% CR I/O

• Memory gets all the glory

• Horst Simon: the Memory Wall

• But I/O really matters to users

• To persist state of computation


#include <stdisclaimer.h>

• Intel continues the Whamcloud business model:

• Multi-vendor Lustre development

• Open source, community friendly, single source tree

• Staff retention at 98% since acquisition

• Intel supports all Whamcloud business (incl. Fastforward)

• A member of EOFS (Whamcloud’s position)

• Recently joined OpenSFS as a promoter ($500K)

• Yes, this work is being done with Lustre


US Dept Of Energy – Fastforward Challenge

• FF RFP provided funding for exascale R&d

• Sponsored by 7 Leading US Labs (DOE)

• Aims to solve the currently intractable problems of

Exascale to meet the 2020 goal of an exascale machine

• Awards in: CPU, Memory and I/O (Filesystem)

• Whamcloud won the I/O portion (with EMC, Cray, DDN, HDF)

• Renegotiated after acquisition to add:

• DDN (versioning OSD)

• Intel (arbitrary connected graph application)


Exascale I/O technology drivers


Exascale I/O Drivers


Exascale I/O Architecture


Fastforward I/O Arcitecture


Exascale filesystem


Epoch’s

INTEL High Performance Data Division

Thank You

13

The Future of Network-Based Storage

Documents

Transcript of The Future of Network-Based Storage