Create a Scalable Mobile Media Site with Drupal and MongoDB
-
Upload
acquia -
Category
Technology
-
view
2.302 -
download
0
Transcript of Create a Scalable Mobile Media Site with Drupal and MongoDB
![Page 1: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/1.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Mobile Media Site using Drupal & MongoDB
Presented By:Yash Badiani, Big Data Practice Lead, CIGNEX DatamaticsGaurav Khambhala, Technical Lead, CIGNEX Datamatics30th July, 2013
![Page 2: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/2.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
About Acquia
![Page 3: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/3.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Acquia – Enterprise Guide to Drupal
3
Co-founded by the Drupal project’ creator in 2007, Acquia has helped more than 2,000 clients manage
growth and scale of their digital experiences.
Acquia provides wide range of Drupal products and services, including:
Acquia Cloud Enterprise GardensAcquia Network
Drupal Training Professional Services
Drupal Commons
![Page 4: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/4.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
About CIGNEX Datamatics
4
![Page 5: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/5.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
What Does CIGNEX Datamatics Do?
5
Since 2000, CIGNEX Datamatics has implemented over 400 Open Source enterprise
solutions addressing business requirements related to Portals, Content & Big Data
![Page 6: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/6.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Where We Can Help You
6
SOLUTIONS
Managed Cloud Services - Develop, Deploy, Manage
Annual Product Subscription: Liferay, Alfresco, Magento, Hadoop, Selenium
Extended Development Center – Center of Excellence
UI Development Integration Customization Migration Testing Training Support (24*7)
User eXperience Platform
Portals Liferay, Magento,Drupal, Adobe CQ
• Intranet • Extranet• Social
Collaboration
• Mobile Portals
• E-Commerce
Enterprise Content Management
Content Alfresco, Drupal, Magento, Adobe CQ, Moodle, Ephesoft
• WCM• DM• RM• DAM
• E-Commerce• E-learning• ERP• Imaging Solutions
SERVICES
Making Data WorkBig Data Hadoop, MongoDB,
Hbase, Neo4j, Solr
• Analytics• Mobile• Social• Web• Real-time
• DW - BI
• Log Processing and Analysis
• Enterprise Search
Velocity
Complexity
Volume
Variety
![Page 7: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/7.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Part of Datamatics (DGSL)
• Mission– Experts in improving
Enterprise productivity through Process Engineering & Information Management Solutions
• Key Highlights– Founded in 1975
– Publicly listed in India
– Annual consolidated revenue of US$100 Million
– Fortune 500 clients
– 4,400+ employees across 22 offices in 9 countries
Strategic Alliances
7
![Page 8: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/8.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
About the presenters
• Yash Badiani is the Big Data Practice Lead at CIGNEX Datamatics and focuses on Big Data Technologies including MongoDB & Hadoop. He has worked extensively on large Data warehousing & Business Intelligence projects with tools such as Business Objects, Microsoft SQL Server, Microstrategy, IBM Cognos.
• Gaurav Khambhala works at CIGNEX Datamatics as Technical Lead. He is
the senior member of the PHP Practice at CIGNEX Datamatics and is involved on various technology initiatives like Big Data where he focuses on integration of PHP with NoSQL sources like MongoDB. He has a wide industry experience in software development & management in Open Source technologies such as Drupal, Moodle & Wordpress.
8
![Page 9: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/9.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Agenda
• The Mobile Media Use Case• Requirements and Challenges• Solution : Mobile Media site using Drupal & MongoDB• Why Drupal and MongoDB?• Demo and Solution Features• Benefits • Summary
9
![Page 10: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/10.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
The Mobile Explosion!
10
By 2015, at least 60% of information workers will
interact with their content applications via a mobile
device
Employees work on proposals and presentation on
mobile devices while travelling
People use digital assets (videos, images) longer on
Tablets and Mobiles compared to desktops
Based on a report by a leading IT advisory firm
![Page 11: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/11.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Mobile Media Use Case
• Mobile Media site includes the following features:– Store a variety of Images & associated metadata– Massively Scalable to store billions of images– Access through Mobile– Create /Edit Albums– Add Images to the Albums– Add / Edit Metadata of Images– Search Images / Albums by date, metadata, albums, etc– Social Media features – Likes, comments
11
![Page 12: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/12.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Requirements of Mobile Media sites
12
• Fast performance• Large user base• Concurrent CRUD• Access through various channels
• Millions of digital assets• Variety of content• Complexity of data
• Rich UI features• Social features• Mobile access• Fast search
• Elastic scaling• Cost effectiveness• Centralized storage• Ease of Maintenance
• HIGH availability• Automatic failover• User management
Velocity Volume
User experience
Scalability
Security & Availability
• Easy integration• Shorter dev cycle• Faster deployment• Ease of schema design
Flexibility & Agility
![Page 13: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/13.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Standard Three Layered Data Architecture
13
File System
Metadata in RDBMS
Search
Standard Three Layered Storage
Application layer
![Page 14: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/14.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Limitations of RDBMS
• Support limited to terabytes– No support for petabytes to
exabytes
• Manage only structured data– No support for semi-structured and
unstructured data
• RDBMS don't scale inherently– Scale up/Scale out (Load Balancing
& Replication)
• Hard to shard / partition– Large data files
• Both read / write throughput not possible
– Transactional / Analytical databases
• Specialized hardware - expensive
14
RDBMS can’t manage all dimensions of data with speed & at lower cost.
![Page 15: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/15.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
NoSQL is the right solution
• They are schema less• Designed to support huge data volumes
– Facebook 135 billion messages/month; Twitter 7TB data/day
• Scalable replication and distribution mechanism– Thousands of machines distributed around the world
• Massive write performance with asynchronous inserts and updates• Designed to give high query performance• Runs on commodity hardware• Most NoSQL databases are Open Source
15
Not SQLOnly
![Page 16: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/16.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
NoSQL – Data Models
16
Column FamiliesUsage: Read/Write Intensive
Popular databases: Hbase, Cassandra
Document StoreUsage: Working with Occasionally
changing/consistent dataPopular databases: CouchDB,
MongoDB
Graph Database
Usage: Spatial Data storagePopular databases: Neo4j, Bigdata
Key Value / Tulip StoreUsage: Briskly changing data and
high availabilityPopular databases: Riak, Redis,
Azure Table storage
NoSQL Databases
• 4 broad data models• 120+ variants available in the market
![Page 17: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/17.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Requirements of Mobile Media sites - Recap
17
• Fast performance• Large user base• Concurrent CRUD• Access through various channels
• Millions of digital assets• Variety of content• Complexity of data
• Rich UI features• Social features• Mobile access• Fast search
• Elastic scaling• Cost effectiveness• Centralized storage• Ease of maintenance
• HIGH availability• Automatic failover• User management
Velocity Volume
User experience
Scalability
Security & Availability
• Easy integration• Shorter dev cycle• Faster deployment• Ease of schema design
Flexibility & Agility
Mobile Media Site
![Page 18: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/18.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Drupal with MongoDB Solution
18
Themes
CoreModules
Nodes TaxonomyUserRoles
Forms &Menu
CustomModules
Workflow ForumsComments& Ratings
Tagging
PHP
Web Services
3rd party & Internal
Applications
MongoDBDriver
MongosRouting Process
Replica Set
MongoDB MongoDB
Replica Set
MongoDB MongoDB
![Page 19: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/19.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Why Drupal?
19
PluggableArchitecture
Data Abstraction Layer
Easy to Upgrade
Secure
Active CommunityWidely Adopted
Scalable
3rd Party Tools Integration
User Management& Permissions
HTML5 & CSS Support
Drupal Open Source CMS
![Page 20: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/20.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Websites using Drupal
20
Website: Whitehouse.gov Website: Data.gov.uk
Website: mtv.co.uk
Website: research.yahoo.com
Website: pdx.eduWebsite:
EndPoverty2015.org
![Page 21: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/21.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Why MongoDB?
21
Agile and Scalable
Full Index Support
Document Oriented Storage
ReplicationQuerying
Atomic Updates
Data Processing and aggregation
High Availability
Open Source NoSQL Solution
![Page 22: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/22.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Customers using MongoDB
• Centralized data management platform
• 2 billion+ documents
• 20 TB of photo metadata
• TV episodes and series
• Risk solutions auditing data
22
Source: http://www.10gen.com/customers
![Page 23: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/23.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Demo
23
• Media site on mobile simulator• Like & comment on an image on mobile simulator• Mobile site on web browser• Verify ‘Like’ & comment of the same image on web browser• Search images & access control
![Page 24: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/24.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Solution Features
Architecture and Design
24
![Page 25: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/25.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Architecture
25
User MetadataInd
exes Albums
ImageMetadata GridFS
Form API
Drupal API
CustomModule
Browser/MobileTheme
MongoDB PHP Driver
Menu API
UserMobileDevice
ImageMetadata
GridFS
![Page 26: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/26.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Add Album
Flow
26
Add Image
View Album
View Individual Images
Like Image
Comment Image
Add Tags to Images
View Counter
Search ImagesBy Tags
User Metadata Albums GridFS Image Metadata
DBRef
DBRef DBRef
DBRef
MongoDB Collections
Use
r A
ctio
ns
![Page 27: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/27.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
27
Schema Design
User Metadata GridFS
Albums Image Metadata
• User ID• DBRef (Album)• Tags• Thumbnail• Likes• View Counter• Comments• Permission
• FS.Files• FS.Chunks
• User ID• Tags• Title
• Make• Model• Date Time• Aperture• Exposure• DBRef (GridFS)
![Page 28: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/28.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
28
Schema DesignImage Metadata
Albums
User Metadata
![Page 29: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/29.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
29
MongoDB Monitoring Service (MMS)
• DB Storage
• Cursors
• Replica Sets
• Network Connections
• Non Mapped Virtual Memory
• Opcounters
![Page 30: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/30.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Benefits
Drupal MongoDB
Most advanced content management solutions
Scalability – billions of content items, millions of users
Highly customized websites Performance – FAST writes through sharding, reads through indexes
Most search friendly CMS Data safety through replication
Less coding, high on automation Centralized single system for data storage
Powered by 7000+ plugins and extensions
Monitoring through MMS
Active community, real time assistance Enterprise support through 10gen
30
![Page 31: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/31.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Summary & Key Takeaways
• MongoDB provides the RIGHT fit for CMS applications with flexibility, scale & speed
• Drupal’s advanced & automated CMS features and tight integration with MongoDB makes it the right choice for building agile websites
• Both Drupal & MongoDB are feature rich and being Open Source, provide significant cost benefits
31
![Page 32: Create a Scalable Mobile Media Site with Drupal and MongoDB](https://reader036.fdocuments.us/reader036/viewer/2022062418/554f5fbdb4c9058a148b46ad/html5/thumbnails/32.jpg)
CIGNEX Datamatics Confidential www.cignex.com
CIGNEX Datamatics Confidential
Thank you. Questions?
CIGNEX Datamatics makes Open Source work for you!
32
Yash BadianiBig Data Practice Lead
Gaurav KhambhalaTechnical Lead