Hadoop Israel - HBase Browser in Hue
37
HBASE BROWSER IN HUE Abraham Elmahrek Hadoop Israel Apr 23, 2014
-
Upload
gethue -
Category
Data & Analytics
-
view
689 -
download
12
description
Learn about the HBase browser in Hue, the UI for Apache Hadoop. Presented by Abraham Elmahrek at Hadoop Israel www.meetup.com/HadoopIsrael/events/161701092/ Find out everything you need about Hue at http://gethue.com
Transcript of Hadoop Israel - HBase Browser in Hue
HBASE BROWSERIN HUE
Abraham ElmahrekHadoop Israel Apr 23, 2014
WHATIS HUE?
WEB INTERFACE FOR MAKING HADOOP EASIER TO USE Suite of apps for each Hadoop component, like Hive, Pig, Impala, Oozie, Solr, Sqoop2, HBase...
VIEW FROM30K FEET
Hadoop Web Server You and eventhat friend
that uses IE9 ;)
ECOSYSTEM
PIGJO
B BROWSER
JOB DESIG
NER
OOZIE
HIVE IMPA
LA
METASTO
RE BROWSERSEARCH
HBASE BROWSER
SQOOP
ZOOKEEPERUSER ADMIN
DB QUERY
SPARK
HOME ...
GUI DESIG
N
FILE BROWSER
USER
USER WORKFL
OWS
USER
YARN JobTracker Oozie
Pig
HDFS
HiveServer2
Hive Metastore
Cloudera Impala
Solr
HBase
Sqoop2
Zookeeper
LDAP SAML
Hue Plugins
APPS
TARGETOF HUE
GETTING STARTED WITH HADOOP BEING PRODUCTIVE EXPLORING DIFFERENT ANGLES OF THE PLATFORM !
LET ANY USER FOCUS ON BIG DATA PROCESSING
OPEN SOURCE
3000+ COMMITS 35 CONTRIBUTORS665 STARS221 FORKS !
github.com/cloudera/hue
THE CORETEAM PLAYERS
team.gethue.com
ABRAHAM ELMAHREK
ROMAIN RIGAUX
ENRICO BERTI
CHANG BEER
TALKS
Meetups and events in NYC, Paris, LA, Tokyo, SF, Stockholm, Vienna, San Jose, Singapore…Coming up in London, West coast
AROUNDTHE WORLD
RETREATS
Nov 13 Koh Chang, Thailand May 14 Curaçao, Netherlands Antilles
FAST PACE
LAST 30 DAYS
41 issues created and 38 resolved. Core team + Community
NEW APPS IN 6 MONTHS
PIGJO
B BROWSER
JOB DESIG
NER
OOZIE
HIVE IMPA
LA
METASTO
RE BROWSERSEARCH
HBASE BROWSER
SQOOP
ZOOKEEPERUSER ADMIN
DB QUERY
SPARK
HOME ...
GUI DESIG
N
FILE BROWSER
USER
USER WORKFL
OWS
USER
TREND: GROWTH
gethue.com
HISTORY
HUE 1
Desktop-like in a browser, did its job but pretty slow, memory leaks and not very IE friendly but definitely advanced for its time (2009-2010).
HISTORY
HUE 2
The first flat structure port, with Twitter Bootstrap all over the place.
HISTORY
HUE 2.5
New apps, improved the UX adding new nice functionalities like autocomplete and drag & drop.
HISTORY
HUE 3
Transition to the new UI, major improvements and new apps.
HISTORY
HUE 3.5+
Where we are now, new UI, several new apps, the most user friendly features to date.
BIG DATA MEANS HARDER SEARCH PROBLEMS.
HBASE APPLICATION DEVELOPMENT WOES.
UNDERSTANDING DATA BETTER.
PROBLEMS
LACK OF ACCESSIBILITY
LACK OF FAMILIARITY
USABILITY CHALLENGES
DESIGN CHALLENGES
PROBLEM
HBase is a key-value store, not a traditional RDB
SOLUTION
Collapse Sparse Data across HBase Tables
FLEXIBLE SEARCH BAR
PROBLEM
Need high-level tool for complex data fetching & analysisSOLUTION
Simple custom query language Supports HBase filter language Supports selection & Copy + Paste, gracefully degrades in IE Autocomplete Help Menu
Row$Key$
Scan$Length$
Prefix$Scan$
Column/Family$Filters$
Thri=$Filterstring$
Searchbar(Syntax(Breakdown(
DIVING DEEPERDOWN LOW
Thrift with custom connection pooling with socket multiplexing to improve thrift calls. Router to streamline front end requests.UP TOP
KnockoutJS, JQuery, DOM caching, etc. for a dynamic front end.
client = thrift_util.get_client(TCLIService.Client, query_server['server_host'], query_server[‘server_port’], …) client.mutateRow(…) client.scannerOpenWithScan(…) client.scannerGetList(…) !
api = HbaseApi() api.putColumn(…) api.deleteColumn(…) api.bulkUpload(…)
SCALEPROBLEM
Performance on Millions of Columns
SOLUTION
Lazy loading & Truncation using Thrift FilterString Stream raw data cached MEM > DOM elements dynamically generated b64encode binary data to preserve during ASCII dump Detect schema on preview reading byte headers Capped, Lazy Loaded & Bound to DOM via MVVM Pattern
IT DOES.
ZOOKEEPER BROWSER
TECHNICAL DETAILS
Communicate with Contrib. REST API View Node Stats View and Edit Node Data And and Remove ZNodes
QUERY HBASEWITH HIVE
HIVE EDITOR
QUERY HBASEWITH PIG
PIG EDITOR
INDEX HBASEAND SEARCH IT
HUE
LILY INDEXER
CLOUDERA SEARCH
DEMO TIME
SDK
ROADMAP
CDH 5 with Hue 3.5+ http://gethue.com/hadoop-tutorial-new-impala-and-hive-editors/
CDH 5.1 with Hue 3.6: CHARTS, Yarn, Spark, SEARCH create index, Load data, Dashboard, UX ... your idea?
Inter-app integration (e.g. schedule daily Hive query in one click), versioning, export/import. Oozie revamp, dashboard, reporting… Come talk to us about your use cases and what you would like to see next in Hue!
Hue 3.5 OSS release http://gethue.com/hue-3-5-and-its-redesign-are-out/
DECEMBER 2013 Q1 2014 Q2 2014
AFTER
LINKS
DEMO
http://demo.gethue.com
@gethue
USER GROUP
hue-user@
WEBSITE
http://gethue.com
LEARN
http://learn.gethue.com
GET HUE
Try in advance the latest and greatest but you’ll have to configure everything on your own.
Get to play with Hue and various Hadoop components in 5 minutes. It’s a self contained CDH environment ready to use.
Newer version than HDP, close to the original 2.5 minus apps like HBase, Impala, Sqoop, Search.
The newest addition, ships Hue 3.0 through the GreenButton products.
Stable and highly tested releases perfectly integrated with the Hadoop ecosystem, automagically configured by Cloudera Manager.
In HDP there’s an old forked version of Hue 2.3.
CLOUDERA’S CDH TARBALL CLOUDERA’S DEMO VM
HORTONWORKS* MAPR* HP CLOUD*
* YOUR MILEAGE MAY VARY.
BIGTOP EMBEDDED/DEMO IN IND. COMPANIES
WHAT ARE YOUR USE CASES?
WHICH COMPONENTS DO YOU USE?
WHAT WOULD YOU LIKE TO SEE IN HUE?
INTERESTED IN CONTRIBUTING? WANNA SAY HELLO? DO YOU WANT A TAILOR
MADE TEAM RETREAT?
TEAM@ GETHUE.COM
HISTORY
HUE 3 ALPHA
Proposed design, didn’t make it.
