Drupal Big Data

Post on 06-May-2015

1.068 views 0 download

description

Big Data Drupal with Cloudera, Hadoop, MapReduce, Nutch and Solr by niccolo http://groups.drupal.org/node/286763

Transcript of Drupal Big Data

Big Data DrupalDEMOCRATIZING BIG DATA PROCESSES

Elements

Bonita

Cloudera

NutchSolr

Drupal

BonitaJAVA/ECLIPSE-BASED COMMERCIAL OPEN-SOURCE BUSINESS PROCESS AUTOMATION & MODELLING

Bonita StudioDesign business process models

Human or Service Tasks

Human Tasks have Forms

Service Tasks have Connectors

Bonita ExperienceWeb-based admin & workflow

Bonita Forms

Shell Script Task

sudo -u hdfs hadoop jar /opt/nutch/basil-apache-nutch-1.6/build/apache-nutch-1.6.job org.apache.nutch.crawl.Crawl/user/nutch/demo-crawl/urls -dir${dir} -depth ${depth} -topN 10 -threads 50

Runs Nutch job for Hadoop

ClouderaBIG DATA COMMERCIAL OPEN SOURCE

ClouderaCloudera Manager 4 (Free Edition)

Hbase

HDFS

Hive

Hue

Impala

Mapreduce

Oozie

Zookeeper

Nutch Job Hadoop job started by Bonita Shell connector

Apache Foundation

Nutch

Solr

Hbase

HDFS

Hive

Impala

Mapreduce

Home to many of these projects

NutchIndustrial strength general purpose web-crawler

http://blog.csdn.net/hadoopstudy/article/details/1501123

Nutch

http://blog.csdn.net/hadoopstudy/article/details/1501123

SolrSearch & indexing

DrupalPHP WEB APPLICATION FRAMEWORK

Aegir BOA

DrupalNutch & Solr modules

Integrate with search & views

Created at IAS

Sponsored by Acquia

Apache SolrModule

Apache SolrExamples Module

http://drupal.org/project/apachesolr_examples

Nutch Mulisite

Drupal SearchNutch crawl

Solr indexed

Drupal search & views

Nutch SolrSandbox

Big Data DrupalDEMOCRATIZING BIG DATA PROCESSES

Big Data DrupalAuthor

Big Data Drupal

Web www.BigDataDrupal.com

Email niccolo.roberts@gmail.com

Contact Nicholas Roberts