© 2013 Mellanox Technologies 1 NoSQL DB Benchmarking with high performance Networking solutions...

© 2013 Mellanox Technologies 1

NoSQL DB Benchmarking with high performance Networking solutions

WBDB, Xian, July 2013


Leading Supplier of End-to-End Interconnect Solutions

Host/Fabric SoftwareICs Switches/GatewaysAdapter Cards Cables

Comprehensive End-to-End InfiniBand and Ethernet Portfolio

Virtual Protocol Interconnect

StorageFront / Back-EndServer / Compute Switch / Gateway

56G IB & FCoIB 56G InfiniBand

10/40/56GbE & FCoE 10/40/56GbE

Fibre Channel

Virtual Protocol Interconnect


Motivation to Accelerate Data Analytics

Data Analysis Requires Faster Network• Hadoop Map Reduce Framework is a network

intensive workload- Mapped data is shuffled between nodes in the cluster

• Data Replication - A high availability event triggers Multi-Tera of data

movement

Provide Higher Data Value• Expose SSD’s low latency capabilities• Better server/CPU utilization

* Data Source: Intersect360 Research, 2012, IT and Data scientists survey

Big Data Applications Require High Bandwidth and Low Latency Interconnect


Cassandra Database enables update capabilities Latency factors

• Commit-log settings• Workload

Cassandra, Update Latency


Cassandra Database Read Latency factors

• Media used• Workload

Cassandra, Read Latency


5 Nodes in the Ring 64GB RAM

• 8 x 8GB DDR3 1333MHz

2 x E5-2670• 8 Cores per socket

5 x Seagate® Constellation® ES SATA 6Gb/s 2TB Hard Drive• 7200 RPM

NIC: Mellanox Technologies MT27500 Family [ConnectX-3]• 10Gb Ethernet• FW_VER=2.11.500

Switch SX1036 OS: RH 6.3

• MLNX_OFED_LINUX-1.5.3

Apache Cassandra 1.1.12, 2 seeds

System Used for Cassandra Benchmark


SSDs Become De-Facto standard in HDFS deployment• Read capability is a critical factor for application performance

E-DFSIO, Part of Intel’s HiBench test suite, profiles aggregated throughput on the cluster• 1GbE network impede any performance benefit from SSD deployment

Unlocking the Power of SSDs In Hadoop Environment

E-DFSIO, Showing the Power of SSD @ HDFS


Updates are made to server memory• Extreme low latency for HBase- Java GC policy hurting on large throughput

HBase Benchmarking, Update Latency


Hitting the media capabilities

HBase Benchmarking, Read Latency


4 Region servers, 1 Master, 3 Zookeeper quorum servers 64GB RAM

• 8 x 8GB DDR3 1333MHz

2 x E5-2670• 8 Cores per socket

5 x Seagate® Constellation® ES SATA 6Gb/s 2TB Hard Drive• 7200 RPM

NIC: Mellanox Technologies MT27500 Family [ConnectX-3]• 10Gb Ethernet• FW_VER=2.11.500

Switch SX1036 OS: RH 6.3

• MLNX_OFED_LINUX-1.5.3

Apache Hbase 0.94.9, Zookeeper 3.4.5, Apache Hadoop 1.1.2

System Used for HBase Benchmarks


EMC 1000-Node Analytic Platform Accelerates Industry's Hadoop Development 24 PetaByte of physical storage Mellanox VPI Solutions

Test Drive Your Big Data

2X Faster Hadoop Job Run-TimeHadoop

Acceleration

High Throughput, Low Latency, RDMA Critical for ROI


The Great Things in Hadoop Distributed File System

• HDFS is a block storage solution• Block size can be modified to provide efficient solutions for very large files• Inherent reliability, no need for high end storage solution to make sure data is there!• Tuned for Hadoop work loads, write one and read many


The Less Great Things in HDFS

It’s hard to manage the different settingto get the right nodes into the right capabilities.

Ingress and extraction of data requires additional tools.

Small files or latency sensitiveDefault 3x ReplicationMetadata Server Failure


Local Disks – The Common Practice


Other Distributed Storage Solution for Hadoop, Really?!


OrangeFS as Hadoop Storage Solution


Lustre as Hadoop Storage Solution

Source: Map/Reduce on Lustre, Hadoop Performance in HPC Environments, Nathan Rutman, Senior Architect, Networked Storage Solutions, Xyratex


CEPH as Hadoop Storage Solution

Generating lot of Interest since the Ceph kernel client was pulled into Linux kernel 2.6.34• Object-based parallel file system• Scalable metadata server• Each file can specify it’s own striping strategy and object size• Automatic rebalancing of data with minimal data movement• Hadoop module for integrating Ceph has been in development since 0.12 release

Benchmarks on Ceph is still WIP• We are currently working on using running benchmarks on Ceph – Stay tuned!!


Thank You

© 2013 Mellanox Technologies 1 NoSQL DB Benchmarking with high performance Networking solutions...

Documents

Transcript of © 2013 Mellanox Technologies 1 NoSQL DB Benchmarking with high performance Networking solutions...