HBase
description
Transcript of HBase
1
HBaseA column-centered database
2
Overview•An Apache project•Influenced by Google’s BigTable•Built on Hadoop
▫A distributed file system▫Supports Map-Reduce
•Goals▫Scalability▫Versions▫Compression▫In memory tables
3
Architectural issues•Cluster of nodes is the general
architecture•Standalone mode for single machine•There is a Java API accessed with JRuby •There is a JRuby shell
4
Modeling constructs•Table
▫Has a row key▫A series of column families
Each has a column name and a value•Operations
▫Create table▫Insert a row with “Put” command
Only one column at a time▫Query a table with a “Get” command
(uses a table name and a row key)
5
Filters•Scan
▫can get a series of rows based on two key values
▫Can provide a filter for such things as column families, timestamps
▫Filters can be pushed to the server
6
Updating•When a column value is written to the db,
old values are kept and organized by timestamp▫Each such value is a cell
•You can explicitly assign timestamps manually▫Otherwise, current timestamp with insert▫When getting, uses most recent version
•Operations that alter column family structures is expensive
7
Other characteristics•Text compression•Rows are stored in order by key value•A region is some set of rows
▫Each is stored in a single region server▫Regions can be automatically merged and
split•Uses write-ahead logging to prevent loss
of data with node failures▫This is called journaling in Unix file
systems•Supports a master/slave multi-cluster
strategy
8
An HBase clustertaken from: http://www.packtpub.com/article/hbase-basic-performance-tuning
9
Terminology•A region is a subset of the rows of a table
▫These are automatically sharded•A master coordinates the slaves
▫Assigns regions▫Detects region failures▫Administrative functions
•A client reads and writes rows directly to the region servers
•A client finds region server addresses in zookeeper
10
Tasks of components•Zookeeper cluster is a coordination
service for the HBase cluster▫Finds the correct server▫Selects the master
•Master allocates regions & load balancing•Region servers hold the regions•Hadoop supports Map-Reduce
11
Features•Consistency over available•Efficient mapreduce•Range partition queries, not based on
hashing or other random access•Automatically shards•Very sparse column storage
12
Some key concepts•De-normalization•Fast random, key-row retrieval•Use of a multi-component architecture to
leverage existing software tools•Controllable in-memory selection
13
Important HBase Properties•Strongly consistent reads/writes: HBase is
not an "eventually consistent" DataStore. •Automatic sharding: HBase tables are
distributed on the cluster via regions, and regions are automatically split and re-distributed as your data grows.
•MapReduce: HBase supports massively parallelized processing via MapReduce
14
When to use or not use Hbase •Java Client API: HBase supports an easy
to use Java API for programmatic access.•If you have hundreds of millions or
billions of rows, then HBase is a good candidate.
•If you only have a few thousand/million rows, then using a traditional RDBMS might be a better choice due to the fact that all of your data might wind up on a single node (or two) and the rest of the cluster may be sitting idle.
15
More on using or not using HBase•Make sure you can live without all the extra
features that an RDBMS provides (e.g., typed columns, secondary indexes, transactions, advanced query languages, etc.)
•Make sure you have enough hardware. Even HDFS doesn't do well with anything less than 5 DataNodes
•HBase can run quite well stand-alone on a laptop - but this should be considered a development configuration only.
16
The Hbase API•Get a row•Put a row, with a column/value pair•Scan, with a key range and a filter•mapreduce via Hive
17
High level map reduce diagram
18
A more detailed diagram
19
Map reduce example steps1. The system takes input from a file system and splits it up across separate Map nodes 2. The Map function or code is run and generates an output for each Map node—in the word count function, every word is listed and grouped by word per node 3. This output represents a set of intermediate key-value pairs that are moved to Reduce nodes as input 4. The Reduce function or code is run and generates an output for each Reduce node—in the word count example, the reduce function sums the number of times a group of words or keys occurs 5. The system takes the outputs from each node to aggregate a final view
20
Diagram of map reduce