How we Leverage Cloud, Big Data, Social Data at QMining (Salon BI nov 6 2012)
36
How we leverage the cloud, big- data, social data to do BI? Salon BI 6 November 2012 [email protected] Founder & CEO
-
Upload
francis-pieraut -
Category
Documents
-
view
327 -
download
0
description
BI landscape is changing drastically. Cloud, BIGDATA, Social Data and opensource are helping the democratization process that is required to fulfill the business gap between technology and today\'s business needs. As proof point, the results of several consulting engagements will be used as examples to support this assertion and to explain why QMining was created.QMining builds software for the fragmented online marketing field. QMining has developed a Marketing Management Platform to help businesses measure and optimize the ROI of their online investments. In order to do so, we will present how we are leveraging amazon web services, BIGDATA frameworks, Python language, open-source and Social Data to give business insides to our customers.We will also present the architecture of the QMining platform, the data used, and how the data is processed.The presentation will conclude with some customer stories and real-life cases. http://salonbi.com/en/content/how-we-leverage-cloud-bigdata-social-data-do-bi-qmining
Transcript of How we Leverage Cloud, Big Data, Social Data at QMining (Salon BI nov 6 2012)
How we leverage the cloud, big-data, social data to do BI?
Salon BI 6 November 2012
[email protected] Founder & CEO
● Story● QMining● BI landscape - Issues ● How QMining is leveraging:
○ Open source○ Cloud○ Hadoop for Big data○ Social data
● Innovation○ QMiner○ QMarketing
Hidden Agenda
Story
Products and Consulting
Big Data Knowledge Action
Big Data
Any data includingweb traffic, social data,
e-commerce transactions
Patterns/clusters
Predictive Analysis
Knowledge
Action
BI Landscape - Issues
● Frustrating○ not simple/not fast/costly
● Increasing data & sources● User want more power (bypass IT)
QMining
● Simple/fast/cost effective● Scalable● More power to the users (bypass IT)
Why?
Examples
Context Enterprise QMining
Big Player (content) 18 mois = failure 3 months = success
online marketing 180+ days = not done
11 days = success
online KPI 30+ days investigations
2 days (bots)
IBM SPSS 15 days 1 day
... ...
Where?
Applications
QMiningIT Application Managementex: performance
monitoring & troubleshooting
Online Marketingex: Marketing
Management Platform
paymentE-commerce
ex: merchant retention
IT Service Management
ex: business process automation
How?
Principles:● Open source● Minimize Abstraction● Cloud
Tools:● Python● Hadoop (for Big Data)
Methodology
Open SourceLeverage collaboration and openness to move faster
-71% of clients don't use open source
All abstractions leakAbstract -> Procrastinate!
http://www.aleax.it/pycon_abst.pdf (Alex Martelli : "Abstraction as a Leverage" )
Minimize A Tower of AbstractionSimplify & lower the layer of abstraction
Examples:
● Work on file not BD if possible● Low level linux command lines (cut, grep, sed etc.)● High level languages : python● Minimize # of languages; ex: Hadoop streaming
(map/reduce=python)● Prototype on a laptop (atom N450)
CloudLeverage IAAS & PAAS infrastructure to build new SaaS
Cloud - PaaS vs IaaS
Pros ● framework● deployment● multi-tenant● https● log & data
visualisation
● less limits● Scaling● Pricing
General Score +20 -20
Recommendations:
The best of both world might be heroku a Cloud application platform but we haven't tried itmontreal python slides on a comparison (appengine vs amazon, pris & cons for a startup)
Prototype
Production
QMining in Cloud
Principles:
● Easy deployment of environments ● Scalable by design● Multi-tenant by design (don't manage multiple versions/instances)
● REST API by design
the power of matlab for prod
example coradiant : 100X speedup
What Howpowerof
PaaS (Platform as a Service)
Web framework
Big Data Leverage cloud to process lot of data (hadoop)
We use amazon EMR (hadoop, EC2 & S3)
Innovation
Why enterprises fail at innovation? (blog)
1. Salon BI 2011: Philippe Nieuwbourg M&A2. Boston CXO Forum 24 October : Best Practice on Global
Innovation (IBM, EMC, P&G, Intuit) Exploit vs Explore 3. Brad Feld (Managing Director at Foundry Group) Startup community 29 October Notman house MTL Hierarchy vs network
QMiner
1. Simplify real user management2. Simplify business view management3. Simplify Persona management
QMiner Simplify real user management
10% of your online users are experiencing slowness
● 100% are located in China ● 75% are using mobile device● 90% are using iphone
http://qmining.qminer.com
QMiner & Social Graph Leverage the social graph apis
Facebook Social Graph:http://developers.facebook.com/docs/reference/api/
QMiner Example: http://fr.qmining.com/facebook/
QMinerSimplify business view management
Your are losing $10K/hour:● 100% are Male● 75% are using mobile device● 90% are using iphone
http://qmining.qminer.com/api/clients/qmining/apps/fbtest/ruleset/
QMiner Simplify persona management
None buying customers are:93% male67% single
87% high educated
Persona fit = 25%
QMarketingSimplify online marketing management
Investing in online marketing and not getting the expected
ROI?
You can't find a global solution in this fragmented market?
QMarketing - Marketing Management
Optimize your online Marketing investments
Hadoop for Channel Optimization
Non sales Patterns with social data
Summary
● Story● QMining● BI landscape - Issues (not simple, slow, costly)
● How QMining is leveraging:○ Open source with linux, python, github
○ Cloud with amazon, google appengine
○ Big data with hadoop
○ Social data with facebook api
● Innovation○ QMiner Simplify real user, business view & persona management ○ QMarketing simplify online marketing management
