Voltaire The Grid Backbone™

18
Voltaire Voltaire The Grid Backbone™ The Grid Backbone™ InfiniBand CERN Seminar Asaf Somekh VP Strategic Alliances June 2006

description

Voltaire The Grid Backbone™. InfiniBand CERN Seminar Asaf Somekh VP Strategic Alliances June 2006. Agenda. About Voltaire Products Update InfiniBand – Ethernet connectivity. Voltaire: The Company. 2005 Sales: 4x over 2004, 15x over 2003 Over 50,000 Ports in 2005 Solid Financing - PowerPoint PPT Presentation

Transcript of Voltaire The Grid Backbone™

Page 1: Voltaire The Grid Backbone™

VoltaireVoltaireThe Grid Backbone™The Grid Backbone™

VoltaireVoltaireThe Grid Backbone™The Grid Backbone™

InfiniBand CERN Seminar

Asaf Somekh

VP Strategic Alliances

June 2006

Page 2: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 2

AgendaAgenda

About Voltaire

Products Update

InfiniBand – Ethernet connectivity

Page 3: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 3

Voltaire: The CompanyVoltaire: The Company

2005 Sales: 4x over 2004, 15x over 2003- Over 50,000 Ports in 2005

Solid Financing- Recent Round: $17M, Summer 2005

- Sufficient funds to breakeven

Global Locations- Business HQ: Boston

- R&D Center: Hertzliya, Israel

- Sales Offices: San Francisco, Philadelphia, New York, Frankfurt, London, Tokyo

- Headcount : 120

Page 4: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 4

Deploying Grids Globally Across IndustriesDeploying Grids Globally Across Industries

Government Education & ResearchManufacturing

Financial Srv.EntertainmentOil & Gas

Page 5: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 5

Expanding ISV And Solution PartnershipsExpanding ISV And Solution Partnerships

Infrastructure

Application and Middleware Storage Software

Management and Scheduling ServerOEMs

Page 6: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 6

Voltaire Products SummaryVoltaire Products Summary

Grid Directors

ISR 9096 - 96 4X (DDR or SDR) or 32 12X ports

ISR 9288 – 288 4X (DDR or SDR) or 96 12X ports

Copper and optical support

Integrated GbE and/or FC ports (up to 144)

Full redundancy (PS, FAN, Fabric, Management, ..)

Blade Solutions

High performance blade solutions for IBM BladeCenterH

Blade Server Connectivity Solutions

Voltaire ISR 9096 Voltaire ISR 9288

Page 7: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 7

Voltaire Products SummaryVoltaire Products Summary

Grid switches- ISR 9024 - 24 port IB (SDR and DDR)

- ISR 6000 - Multi-Protocol (IP / FC / IB)

Management & Virtualization- Embedded in switches

- Highly scalable Grid management

Host software

- Unique High-Performance offeringsfor HPC, Financials, and Storage

ISR 9024

ISR 6000

GridVision™ Management Suite

Page 8: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 8

Voltaire Top500 LeadershipVoltaire Top500 Leadership

Largest number of Top500 InfiniBand sites

Broadest and Strongest Channels- Joint Installations with 7 OEMs and Resellers

Top InfiniBand Site: NASA Ames #4

Most network-efficient InfiniBand site

Number of InfiniBand sites doubled from previous list, tripled in a year

Proprietary interconnects keep going down

GbE pushed lower on the list

Interconnect Change from Nov 2004

InfiniBand +200%

GbE +38%

Quadrics -19%

Myrinet -23%

Page 9: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 9

Voltaire InfiniBand = Network Efficiency = $$$Voltaire InfiniBand = Network Efficiency = $$$

Assumptions:

- Cluster Size: 256 Nodes

- 2P Opteron Server Price: $3,000

- InfiniBand Price/Node: $900 80% Network Efficiency

- Myrinet Price/Node $800 65% Network Efficiency

- GbE Price/Node: $100 55% Network Efficiency

Actual Performance

Myr GbE

TF

lop

s

2

1.5

1

2

1.61.4

Cost Per TFlop

Myr GbE

$500K

$600K

$400K

507

609587

Example:

Page 10: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 10

Multi-Protocol SwitchingMulti-Protocol Switching

Same Switch, Multiple Protocol and I/F Combinations

Page 11: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 11

Voltaire IP Router Unique CapabilitiesHardware Configuration and Chassis ManagementVoltaire IP Router Unique CapabilitiesHardware Configuration and Chassis Management

ISR 9096/9288, as well as ISR 6000 chassis are supported

- sRBD is required for holding up to 3 routers (any mix)

- sRBD-D is required for DDR configurations, although the IPR is an SDR device

Copper as well as optic-fiber SFPs are supported

- IPRs are provided with 4 copper SFPs by default (included in price)

Cluster mode is supported (multiple devices per chassis, any mix)

Voltaire unique InfiniBand-Ethernet Silicon

Link aggregation

Page 12: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 12

Voltaire IP Router Unique CapabilitiesNetworking ManagementVoltaire IP Router Unique CapabilitiesNetworking Management

Centralized management – single point representation

- Feedback from the field leads to an improved mechanism - in development

CLI and GUI based management support

- Including full TFM (Traffic Flow Management) and partitioning support

- Click on Routers launches their VDM

Automated configuration and application transparency by cluster learning and ARP Proxy

Monitoring - management access to programmable hardware classification counters (CLI)

Page 13: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 13

Healthcare GridHealthcare Grid

Voltaire Grid Director ISR 9096

Native InfiniBand Storage with FalconStor IPStor

Local Users

InfiniBand VTL

ApplicationServers

Oracle 10g RAC Cluster

Data from Hospitals

WAN Router

IPoIBRDS

IPoIBSDP

iSERTCP/IP

Page 14: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 14

32 Gb Ethernet Connectivity with InfiniBand 32 Gb Ethernet Connectivity with InfiniBand

ISR 9288

ISR 9288

ISR 9288

ISR 9288

ISR 9288

ISR 9288

ISR 9288

240 4X links to 120 compute

nodes

240 4X links to 120 compute

nodes

240 4X links to 120 compute

nodes

240 4X links to 120 compute

nodes

240 4X links to 120 compute

nodes

194 4X links to 55 compute nodes

+ 42 4X links to 90 storage nodes

48 4X links to each lower tier

ISR 9288

Upper tier blocking (over-subscription) factor = 5:1Node to node hops (worst case) = 9

ISR 9288

IP Router

IP Router

IP Router

IP Router

IP Router

IP Router

IP Router

IP Router

Multiple trunks of 4 x 1 GbE 16 GbE Links 16 GbE Links

Page 15: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 15

TITECH Grid - ProtocolsTITECH Grid - Protocols

650 16-Core NodesEach with 2 HCAs

Stretching over 3 floors

1PB InfiniBand Storage

with Lustre

Legacy NAS Storage

650 16-Core NodesEach with 2 HCAs

Stretching over 3 floors

1PB InfiniBand Storage

with Lustre

Legacy NAS Storage

MPI(MVAPICH)Lustre

NFS

300 Opticallinks

300 Opticallinks

iSER

Page 16: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 16

Enabling High Performance Commodity Storage with iSCSI RDMA (iSER) and InfiniBand Enabling High Performance Commodity Storage with iSCSI RDMA (iSER) and InfiniBand

iSER Target BOM Qty Cost Sum

Target platform (PC) 1 $1-2K $1-2K

SATA drives 16 $0.7 * GB 16*300GB*.7 = $3.4K

SATA RAID Adapter 1-2 $300 $300-600

InfiniBand Adapter 1 $500 $500

FCFC

PerformancePerformance

400 MB/s

400 MB/s

900 MB/s

900 MB/s

iSERiSER

Delivering unprecedented cost/performance, leveraging on industry standard components

Commercial grade target

1 $2K -5K $2K-5K (config dependent)

Total (4.8 TB) ~$2-3/GB

Page 17: Voltaire The Grid Backbone™

Thank YouThank YouThank YouThank You

Page 18: Voltaire The Grid Backbone™

© 2006 Voltaire, All Rights Reserved. 18

1,024 Compute Nodes on A Single Global File Systemwith Voltaire Grid Backbone™ and HP SFS ™1,024 Compute Nodes on A Single Global File Systemwith Voltaire Grid Backbone™ and HP SFS ™

87 Voltaire ISR 9024Grid Switches

87 Voltaire ISR 9024Grid Switches

4 Voltaire ISR 9288 Grid Directors

4 Voltaire ISR 9288 Grid Directors

1,024 Compute Nodes4 Login Nodes2 Admin Nodes

1,024 Compute Nodes4 Login Nodes2 Admin Nodes

12 HP SFS OSS Nodes (Lustre Servers) with 132 TB

OSS Throughput 900 MB/s1 SFS Admin Node

1 MDS Node

12 HP SFS OSS Nodes (Lustre Servers) with 132 TB

OSS Throughput 900 MB/s1 SFS Admin Node

1 MDS Node

Client Throughput:

350 MB/s

Client Throughput:

350 MB/s

Customer: D.O.DUS Air ForceProject TI-05

Customer: D.O.DUS Air ForceProject TI-05

Fully Non-blocking10 Gbps InfiniBand Fabric

Fully Non-blocking10 Gbps InfiniBand Fabric