M/DB and M/DB:X
-
Upload
georgejames -
Category
Technology
-
view
4.637 -
download
2
description
Transcript of M/DB and M/DB:X
GT.M: Rock Solid, Lightning Fast
M/DB & M/DB:X
Open Source NoSQL Projectsbased on GT.M
Rob Tweed: M/Gateway Developments Ltd
GT.M: Rock Solid, Lightning Fast
M/DB & M/DB:X
• What are they?
• Why might you use them?
• What is GT.M?
• Why are they based on GT.M?
• What should you be interested in GT.M?
GT.M: Rock Solid, Lightning Fast
M/DB
– Free Open Source alternative to SimpleDB– API-compatible with SimpleDB
• Behaves identically• Accessed identically• Just a different endpoint URL
– Designed for:• Alternative Public Cloud to Amazon• Private Cloud• Local use
GT.M: Rock Solid, Lightning Fast
Why emulate Amazon SimpleDB?
• Great concept for data that fits its model– “spreadsheet-like” data structure– Simple to use– Simple to manage
• Potential for freedom from SimpleDB’s limits• Make SimpleDB model no longer exclusive to
Amazon cloud• Potential for hybrid environment
GT.M: Rock Solid, Lightning Fast
Why emulate Amazon SimpleDB?
• Great concept for data that fits its model– “spreadsheet-like” data structure– Simple to use– Simple to manage
• Potential for freedom from SimpleDB’s limits• Make SimpleDB model no longer exclusive to
Amazon cloud• Potential for hybrid environment
• Because we can…!
GT.M: Rock Solid, Lightning Fast
SimpleDB Model
• Domains: individual tables or databases• Items: rows in a table• Attributes: columns in a table• Values: cells (note: an attribute can have multiple values)
GT.M: Rock Solid, Lightning Fast
Main SimpleDB Operations
• CreateDomain• PutAttributes
• List Domains• GetAttributes• Select
• DeleteDomain• DeleteAttributes
GT.M: Rock Solid, Lightning Fast
SimpleDB v M/DB LimitsSimpleDB M/DB
No of domains 100 per account Effectively unlimited
Max domain size 10Gb 512Gb – 8Tb(depends on GT.M block-size)
Attributes Text only, 1k max size Dynamic data type,
4k – 32k max(depends on GT.M block-size)
Cost Pay as you go, can get expensive!
Free Open Source
Consistency Eventually Consistent Immediately Consistent
Max query execution time 5 seconds Unlimited
Responses XML over HTTP XML or JSON over HTTP
Web Server(Apache)
SimpleDBClient
m_apache gateway + xinetd
GT.M
M/DBSecurity
M/DBMethods
Ubuntu (or other) Linux
Physical or Virtual Machine
HTTPRequest
XMLResponsecreation
HTTPResponse
M/DBDatabase
m_pythongateway
PythonM/DB APIs
GT.M: Rock Solid, Lightning Fast
M/DB: Availability
• Free Debian Installer at www.mgateway.com– Installs entire architecture
• GT.M• Apache & m_apache• M/DB
• Ubuntu Enterpr– Private cloud
• Eucalyptus used to emulate Amazon APIs
– M/DB now available in the UEC Image Store• Adds SimpleDB API to UEC’s emulation of Amazon cloud
GT.M: Rock Solid, Lightning Fast
M/DB Client Support
• Many SimpleDB Clients already support M/DB:– Bolso– Mindscape's SimpleDB Management Tools and
LightSpeed – Boto (Python client)– Others?
• Just requires ability to change the endpoint URL
• M/DB emulates SimpleDB API precisely
GT.M: Rock Solid, Lightning Fast
M/DB:X
GT.M: Rock Solid, Lightning Fast
M/DB:X
• Simple Native XML Database– Persistent XML DOM model– Multiple XML documents
• API based on SimpleDB– Uses SimpleDB security– W3C XML DOM API, expressed as:
• HTTP request, XML response• JSON over HTTP
– XPath, also expressed as:• HTTP request, XML response• JSON over HTTP
GT.M: Rock Solid, Lightning Fast
M/DB:X
• Creating DOMs:– Parse an XML document
• Non-validating parser
– JSON input• Converted to corresponding XML
– Build programmatically using DOM API methods, eg• createDocument• createElement• createTextNode• appendChild• insertBefore
GT.M: Rock Solid, Lightning Fast
M/DB:X
• Modifying DOMs:– programmatically using DOM API methods, eg
• createElement• createTextNode• appendChild• insertBefore• setAttribute• removeChild• removeAttribute• copyNode
GT.M: Rock Solid, Lightning Fast
M/DB:X
• Searching DOMs:– programmatically using DOM API methods, eg
• getElementById• getElementsByTagName• getAttribute• getFirstChild• getNextSibling
– XPath• Individual DOM• Across multiple DOMs
GT.M: Rock Solid, Lightning Fast
M/DB:X
• Deleting DOMs:– removeDocument– Otherwise a DOM will persist indefinitely
GT.M: Rock Solid, Lightning Fast
M/DB:X
• Identifying DOMs:– Each DOM has a unique:
• DocumentName• DocumentOID
• Identifying Nodes:– Each node in a DOM has:
• nodeType• nodeOID (unique across all DOMs)
GT.M: Rock Solid, Lightning Fast
M/DB:X
• Typical API call (XML)– Request:
db=mdbx &Action=SetAttribute &AttributeName=width &AttributeValue=100px &ElementOID=1-3 &MDBAccessKeyId=[your access key id] &Signature=[computed signature] &SignatureMethod=HmacSHA256 &SignatureVersion=2 &Timestamp=2009-06-14T13%3A19%3A41 &Version=2009-04-15
– Response:<?xml version='1.0'?> <SetAttributeResponse xmlns="http://mdbx.mgateway.com/doc/2009-06-05/" />
GT.M: Rock Solid, Lightning Fast
M/DB:X
• Typical API call (JSON)– Request:
db=mdbx &
Action=SetAttribute &
JSON={AttributeName:”width”,AttributeValue:”100px”, ElementOID:”1-3”} &
MDBAccessKeyId=[your access key id] &
Signature=[computed signature] &
SignatureMethod=HmacSHA256 &
SignatureVersion=2 &
Timestamp=2009-06-14T13%3A19%3A41 &
Version=2009-04-15
– Response:{ok:true}
GT.M: Rock Solid, Lightning Fast
M/DB:X Hybrid JSON/XML Database
• JSON input:{animals:{dog:[{name:"Rufus",breed:"labrador"},{name:"Marty",breed:"whippet"}], cat:
{name:"Matilda"}}}
• XML representation in DOM<animals>
<dog><name>Rufus</name><breed>labrador</breed>
</dog><dog>
<name>Marty</name> <breed>whippet</breed>
</dog><cat>
<name>Matilda</name></cat>
</animals>
• Output as:– XML– JSON
GT.M: Rock Solid, Lightning Fast
M/DB:X: Use cases
• XML meets SimpleDB and CouchDB!• Lightweight XML “scratchpad”• Simple, easy to use XML DOM• Perfect for storage/manipulation of
hierarchical data• If you get XML documents in and send out
XML documents, why convert to/from some database format?– Store as XML
GT.M: Rock Solid, Lightning Fast
M/DB:X Client Support
• None available to my knowledge
• XML API based on SimpleDB model + SimpleDB security– Should be simple to adapt existing SimpleDB
clients
GT.M: Rock Solid, Lightning Fast
M/DB:X availability
• Bundled with M/DB– Install M/DB and you also have M/DB:X
GT.M: Rock Solid, Lightning Fast
M/DB and M/DB:X – Underlying technology
• GT.M
GT.M: Rock Solid, Lightning Fast
What is GT.M?
• Open source database
• Runs on Linux
• Schema-free, key-pair database
• Schema implicit in application
• Persistent dynamic sparse arrays– Alphanumeric subscripts
^employee(“london”,”mgt”,1234)=“Rob Tweed”
GT.M: Rock Solid, Lightning Fast
M/DB and M/DB:X – Why GT.M?
• Why GT.M was the ideal technology:– Flexibility of its database structure:
• Very simple to represent SimpleDB model and its indices
• Very simple to represent XML DOM• No pre-declared schema• Build/modify “on the fly”
– No pre-declared array size– Completely dynamic– Automatic garbage collection
GT.M: Rock Solid, Lightning Fast
M/DB and M/DB:X – Why GT.M?
• Why GT.M was the ideal technology:– Free Open Source– Very high performance database– Highly scalable:
• from netbook to replicated network of high-end servers
– Rock-solid reliability• Its pedigree is in banking & healthcare systems since mid-
1980s• It runs the world’s largest core banking system
– >17 million accounts
GT.M: Rock Solid, Lightning Fast
M/DB and M/DB:X – Why GT.M?
• So GT.M is no toy!– It’s a serious, well tried and tested enterprise-
grade database
• M/DB and M/DB:X can make use of GT.M’s system management facilities:– Backup– Journalling– Replication (multi-tier)
GT.M: Rock Solid, Lightning Fast
Why you should check out GT.M?
• “Swiss Army Knife of Databases”• Underlying persistent sparse array model:
– Can be used to represent many other logical database structures:
• Tables• Hierarchicial• XML• Relational• Object
• Own built-in language, directly linked to database• Or can use most other languages, eg:
– Python– Java– Ruby, etc
GT.M: Rock Solid, Lightning Fast
Why you should check out GT.M?
• Speed of development:– M/DB took 1 week to write and release– M/DB:X about the same
• Documentation was the most time-consuming task!
• Open source– Can redistribute freely on Linux– No limitations imposed– Full version, full functionality
GT.M: Rock Solid, Lightning Fast
Find out more
• M/DB & M/DB:X:– http://www.mgateway.com
• GT.M: – http://www.fis-gtm.com