HBase @ Hadoop Day Seattle

HBaseAmandeep Khurana

University of California, Santa CruzTwitter: @amansk

[email protected]

Tuesday, August 17, 2010

mailto:[email protected]

mailto:[email protected]

How did it start?

• At Google

• Lots of semi structured data

• Commodity hardware

• Horizontal scalability

• Tight integration with MapReduce

2


Why NoSQL?

• RDBMS don’t scale

• Typically large monolithic systems

• Hard to shard

• Specialized hardware.. expensive!

• Buzzword!

3


Google BigTable

• Distributed multi level map

• Fault tolerant, persistent

• Scalable

• Runs on commodity hardware

• Self managing

• Large number of read/write ops

• Fast scans

4


HBase

• Open source BigTable

• HDFS as underlying DFS

• ZooKeeper as lock service

• Tight integration with Hadoop MapReduce

5


HBase

• Data model

• Architecture, implementation

• Regions, Region Servers etc

• API

• Current status and future direction

• Use cases

• How to think HBase (or NoSQL)?

6


Data Model

• Sparse, multi dimensional map(row, column, timestamp) cell

• Column = Column Family:Column Qualifier

v1

Fam1:Qual1

t1AK

Rows

Columns

Timestamps

7


Data Model

• Sparse, multi dimensional map(row, column, timestamp) cell

• Column = Column Family:Column Qualifier

v1

Fam1:Qual1

t1v2

t2>t1

t2AK

Rows

Columns

Timestamps

7


Regions

• Region: Contiguous set of lexicographically sorted rows

• hbase.hregion.max.filesize (default 256MB)

• Regions hosted by Region Servers

8


Regions and Splittingrow1

row256

row257

row600

9



row256

row257

row600

9

Writes



row256

row257

row600

row400

row401

9


System Structure

10

Region Servers Master

ZooKeeperHDFS

Map

Reduce


Master

• Region splitting

• Load balancing

• Metadata operations

• Multiple masters for failover

11


ZooKeeper

• Master election

• Locate -ROOT- region

• Region Server membership

12


Where is my row?

13

ZooKeeper

MyRow

-ROOT-

.META.MyTable

• 3 level hierarchical lookup scheme


Where is my row?

13

ZooKeeper

MyRow

-ROOT-

.META.MyTable


Row per META region


Where is my row?

13

ZooKeeper

MyRow

-ROOT-

.META.MyTable


Row per META region

Row per table region


Region

14

HFile(on HDFS)

HLog(Append only

WAL on HDFS)(Sequence File)(one per RS)

HFile: Immutable sorted map (byte[] byte[])(row, column, timestamp) cell value

Memstore

Region

HFile(on HDFS)


Region

14

HFile(on HDFS)

HLog(Append only



Memstore

Region

HFile(on HDFS)

Write


Region

14

HFile(on HDFS)

HLog(Append only



Memstore

Region

HFile(on HDFS)


Region

14

HFile(on HDFS)

HLog(Append only



Memstore

Region

HFile(on HDFS)

SmallHFile

Flush


Region

14

HFile(on HDFS)

HLog(Append only



Memstore

Region

HFile(on HDFS)

SmallHFile


Region

14

HFile(on HDFS)

HLog(Append only



Memstore

Region

HFile(on HDFS)

SmallHFile

Compaction


Region

14

HLog(Append only



Memstore

Region

Compaction


Region

14

HLog(Append only



Memstore

Region

HFile(on HDFS)


Region

15

HFile(on HDFS)

HLog(Append only


Memstore

Region

HFile(on HDFS)

HFile(on HDFS)


Region

15

HFile(on HDFS)

HLog(Append only


Memstore

Region

HFile(on HDFS)

HFile(on HDFS)

Read


Ways to access• Java

• REST

• Thrift

• Scala

• Jython

• Groovy DSL

• Ruby shell

• Java MR, Cascading, Pig, Hive

16


Java API

• Get

• Put

• Delete

• Scan

• IncrementColumnValue

• TableInputFormat - MapReduce Source

• TableOutputFormat - MapReduce Sink

17


Other Features

• Compression

• In memory column families

• Multiple masters

• Rolling restart

• Bloom filters

• Efficient bulk loads

• Source and sink for Hive, Pig, Cascading

18


Things being worked on

• Master rewrite

• Move more stuff into ZooKeeper

• Column family based access control

• Inter cluster replication (managed by ZK)

• Store Lucene indexes (HBasene)

19


Use Cases


HBase @ SU*

• Backend for su.pr

• Real time serving + MR analytics (separate clusters)

• 50% cascading, 50% java MR

• Prod cluster (~20 nodes) serves 20k requests/sec

• All new features are backed by HBase

• Hardware: 2xi7, 24GB RAM, 4x1TB

21*Source: Personal communication with

J-D Cryans, StumbleUponTuesday, August 17, 2010

HBase @ Mozilla*• Socorro - crash reporting system

• Catch, process and present crash info for Firefox, Thunderbird, Fennec, Camino, Seamonkey

• 1.5m crash reports/day

• Earlier: NFS, PostgreSQL

• 17 node production cluster

• Dual Quad Core + 24GB RAM + 4x1TB

• Some user facing reports still served by PostgreSQL. Being ported to HBase in next Socorro version

22*Source: http://blog.mozilla.com/webdev/2010/07/26/moving-socorro-to-hbase/Tuesday, August 17, 2010

http://blog.mozilla.com/webdev/2010/07/26/moving-socorro-to-hbase/




Data Integration*

• Multiple heterogenous data sources

• Notion of connected data

• Think RDF

• Graph connecting data elements across systems

• Store in HBase, build transitive closures

• Pattern mining

23*Source: ClouDFuse - Scalable data integration in the cloud, MS Project, Amandeep Khurana, UC Santa CruzTuesday, August 17, 2010

HBase @ Trend Micro*

• Store threat information - Smart Protection Network

• Open source cloud computing initiative - TCloud

• Primarily run off EC2

24*Source: https://hbase.s3.amazonaws.com/hbase/HBase-Trend-HUG10.pdfTuesday, August 17, 2010





HBase @ Yahoo*

• Content optimization

• Meta-data about content stored in HBase

• Used for extracting item features

• Used in conjunction with PNUTS, Hadoop

• Process 100s of GB in each run

25*Source: http://www.slideshare.net/ydn/7-online-contentoptimizationhadoopsummit2010Tuesday, August 17, 2010

http://www.slideshare.net/ydn/7-online-contentoptimizationhadoopsummit2010




HBase @ Twitter*

• 7TB/day incoming data, increasing

• Analytics

• People search

• Building new solutions on HBase

• Part of a much larger scheme of things

• Scribe, Crane, Pig, MySQL, Cassandra, Oink, Elephant Bird, Birdbrain, Hadoop

26

*Sources: http://www.slideshare.net/kevinweil/nosql-at-twitter-nosql-eu-2010http://www.slideshare.net/ydn/3-hadoop-pigattwitterhadoopsummit2010Tuesday, August 17, 2010

http://www.slideshare.net/kevinweil/nosql-at-twitter-nosql-eu-2010




http://www.slideshare.net/ydn/3-hadoop-pigattwitterhadoopsummit2010




Others• Facebook

• Flurry

• Adobe

• Runa

• GumGum

• Openplaces

• Meetup.com

• Powerset

• WorldLingo

• Lily

• Drawn To Scale

• RapLeaf

• ...

27


How to think in HBase?


HBase v/s RDBMS

• Neither solves all problems

• It’s really a wrong comparison

• But puts things in context

29


HBase v/s RDBMS

30

HBase RDBMSColumn oriented Row oriented (mostly)

Flexible schema, add columns on the fly

Fixed schema

Good with sparse tables Not optimized for sparse tables

No query language SQL

Wide tables Narrow tables

Joins using MR - not optimizedOptimized for joins (small, fast ones too!)

Tight integration with MR Not really...


HBase v/s RDBMS

31

HBase RDBMSDe-normalize your data Normalize as you can

Horizontal scalability. Just add hardware

Hard to shard and scale

Consistent Consistent

No transactions Transactional

Good for semi structured data as well as structured data

Good for structured data


HBase v/s RDBMS

32


HBase v/s RDBMS

32

Rule: You probably don’t need HBase if your data can easily fit and be processed on a single

RDBMS box.


HBase v/s RDBMS

32

Rule: You probably don’t need HBase if your data can easily fit and be processed on a single

RDBMS box.

But then, you are at Hadoop Day, so it probably can’t!


Q&A


HBase @ Hadoop Day Seattle

Documents

Transcript of HBase @ Hadoop Day Seattle