Drupal for big data - is it ready?

Janis Janovskis Passive Management Ltd

London, April 16th, 2015

Janis Janovskis

@passivus Tech Entrepreneur

English, Russian, Latvian, PHP, Javascript, Python, Go

> 100 Frameworks Biggest team size 55

Smallest 2 Runer, chef, photographer

http://passivemanagement.net

Drupal - Big?

http://www.metaltoad.com/blog/drupals-big-data-problem

http://www.metaltoad.com/blog/drupals-big-data-problem

https://twitter.com/Dries/status/568512852249612288

Big Data???

Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, curation, search, sharing, storage, transfer, visualisation, and information privacy. The term often refers simply to the use of predictive analytics or other certain advanced methods to extract value from data, and seldom to a particular size of data set.

@Wikipedia

Volume

Velocity

Variety

Hang on…. What is Drupal?

DataBase

Web server

Structure

Big Data???

Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, curation, search, sharing, storage, transfer, visualisation, and information privacy. The term often refers simply to the use of predictive analytics or other certain advanced methods to extract value from data, and seldom to a particular size of data set.

Unstructured data

Can We parse unstructured data

with Drupal???

function my_analysis_external_views_plugins() { $plugin = array(); $plugin['query']

['my_analysis_master_plugin_query'] = array(); ….. }

Views???

Views???

public function query($get_count = FALSE){

}

function execute(&$view){

$data = drupal_http_request(“what so ever”); crazy_learn(&$data);

return drupal_render(what_so_ever($data)); }

Views???

https://www.lullabot.com/blog/article/building-views-query-plugins-part-2

https://www.drupal.org/project/feeds_crawler

Pros

• Great community

• Support

• Reputation

• Myriad of vendors

Cons

• Too robust

• Slow

• No lazy loading

• Single threaded

• Heavy

• Structured

Case Studies

http://www.fisc.md/

http://cloudsense.com

Tracked search results with nodes???

http://cwds.uw.edu/visualizing-big-data-drupal-using-data-visualizations-drive-knowledge-discovery

Visualizing Big Data In Drupal: Using Data Visualizations to Drive Knowledge

Discovery

http://basilseedlibrary.com

http://blog.csdn.net/hadoopstudy/article/details/1501123

https://www.drupal.org/sandbox/cilefen/1858412

Drupal for big data - is it … ?

Acknowledgements

• Nicolas Roberts - niccolo.roberts@gmail.com

• http://www.westridge.com/media/series/out-of-the-box

• Florian Loretan

• My Wife

• Drupal Community

• Organisers

Thank You

@PassiveManage

passivemanagement.net/contact