NoSQL DB Benchmarking with high performance Networking solutions
description
Transcript of NoSQL DB Benchmarking with high performance Networking solutions
© 2013 Mellanox Technologies 1
NoSQL DB Benchmarking with high performance Networking solutions
WBDB, Xian, July 2013
© 2013 Mellanox Technologies 2
Leading Supplier of End-to-End Interconnect Solutions
Host/Fabric SoftwareICs Switches/GatewaysAdapter Cards Cables
Comprehensive End-to-End InfiniBand and Ethernet Portfolio
Virtual Protocol Interconnect
StorageFront / Back-EndServer / Compute Switch / Gateway
56G IB & FCoIB 56G InfiniBand10/40/56GbE & FCoE 10/40/56GbE
Fibre Channel
Virtual Protocol Interconnect
© 2013 Mellanox Technologies 3
Motivation to Accelerate Data Analytics
Data Analysis Requires Faster Network• Hadoop Map Reduce Framework is a network
intensive workload- Mapped data is shuffled between nodes in the cluster
• Data Replication - A high availability event triggers Multi-Tera of data
movement
Provide Higher Data Value• Expose SSD’s low latency capabilities• Better server/CPU utilization
* Data Source: Intersect360 Research, 2012, IT and Data scientists survey
Big Data Applications Require High Bandwidth and Low Latency Interconnect
© 2013 Mellanox Technologies 4
Cassandra Database enables update capabilities Latency factors
• Commit-log settings• Workload
Cassandra, Update Latency
© 2013 Mellanox Technologies 5
Cassandra Database Read Latency factors
• Media used• Workload
Cassandra, Read Latency
© 2013 Mellanox Technologies 6
5 Nodes in the Ring 64GB RAM
• 8 x 8GB DDR3 1333MHz 2 x E5-2670
• 8 Cores per socket 5 x Seagate® Constellation® ES SATA 6Gb/s 2TB Hard Drive
• 7200 RPM NIC: Mellanox Technologies MT27500 Family [ConnectX-3]
• 10Gb Ethernet• FW_VER=2.11.500
Switch SX1036 OS: RH 6.3
• MLNX_OFED_LINUX-1.5.3 Apache Cassandra 1.1.12, 2 seeds
System Used for Cassandra Benchmark
© 2013 Mellanox Technologies 7
SSDs Become De-Facto standard in HDFS deployment• Read capability is a critical factor for application performance
E-DFSIO, Part of Intel’s HiBench test suite, profiles aggregated throughput on the cluster• 1GbE network impede any performance benefit from SSD deployment
Unlocking the Power of SSDs In Hadoop Environment
E-DFSIO, Showing the Power of SSD @ HDFS
© 2013 Mellanox Technologies 8
Updates are made to server memory• Extreme low latency for HBase- Java GC policy hurting on large throughput
HBase Benchmarking, Update Latency
© 2013 Mellanox Technologies 9
Hitting the media capabilities
HBase Benchmarking, Read Latency
© 2013 Mellanox Technologies 10
4 Region servers, 1 Master, 3 Zookeeper quorum servers 64GB RAM
• 8 x 8GB DDR3 1333MHz 2 x E5-2670
• 8 Cores per socket 5 x Seagate® Constellation® ES SATA 6Gb/s 2TB Hard Drive
• 7200 RPM NIC: Mellanox Technologies MT27500 Family [ConnectX-3]
• 10Gb Ethernet• FW_VER=2.11.500
Switch SX1036 OS: RH 6.3
• MLNX_OFED_LINUX-1.5.3 Apache Hbase 0.94.9, Zookeeper 3.4.5, Apache Hadoop 1.1.2
System Used for HBase Benchmarks
© 2013 Mellanox Technologies 11
EMC 1000-Node Analytic Platform Accelerates Industry's Hadoop Development 24 PetaByte of physical storage Mellanox VPI Solutions
Test Drive Your Big Data
2X Faster Hadoop Job Run-TimeHadoopAcceleration
High Throughput, Low Latency, RDMA Critical for ROI
© 2013 Mellanox Technologies 12
The Great Things in Hadoop Distributed File System
• HDFS is a block storage solution• Block size can be modified to provide efficient solutions for very large files• Inherent reliability, no need for high end storage solution to make sure data is there!• Tuned for Hadoop work loads, write one and read many
© 2013 Mellanox Technologies 13
The Less Great Things in HDFS
It’s hard to manage the different settingto get the right nodes into the right capabilities.
Ingress and extraction of data requires additional tools.
Small files or latency sensitiveDefault 3x ReplicationMetadata Server Failure
© 2013 Mellanox Technologies 14
Local Disks – The Common Practice
© 2013 Mellanox Technologies 15
Other Distributed Storage Solution for Hadoop, Really?!
© 2013 Mellanox Technologies 16
OrangeFS as Hadoop Storage Solution
© 2013 Mellanox Technologies 17
Lustre as Hadoop Storage Solution
Source: Map/Reduce on Lustre, Hadoop Performance in HPC Environments, Nathan Rutman, Senior Architect, Networked Storage Solutions, Xyratex
© 2013 Mellanox Technologies 18
CEPH as Hadoop Storage Solution
Generating lot of Interest since the Ceph kernel client was pulled into Linux kernel 2.6.34• Object-based parallel file system• Scalable metadata server• Each file can specify it’s own striping strategy and object size• Automatic rebalancing of data with minimal data movement• Hadoop module for integrating Ceph has been in development since 0.12 release
Benchmarks on Ceph is still WIP• We are currently working on using running benchmarks on Ceph – Stay tuned!!
© 2013 Mellanox Technologies 19
Thank You