NoSQL - "simple" web monitoring
12
“Easy” web monitoring NoSQL NoSQL
-
Upload
samir-siqueira -
Category
Technology
-
view
563 -
download
1
Transcript of NoSQL - "simple" web monitoring
“Easy” web monitoring
NoSQLNoSQL
HEP Computing environment
• Big datacentersBig datacenters• Existent monitoringExistent monitoring
o Nagios/Cacti/similars monitoringNagios/Cacti/similars monitoringo Site Availability Monitoring/HammercloudSite Availability Monitoring/Hammercloudo High level tests (Grid services)High level tests (Grid services)
o General Monitoring of grid servicesGeneral Monitoring of grid services• Missing/incomplete monitoringMissing/incomplete monitoring
o Specific/detailed information about specific Grid servicesSpecific/detailed information about specific Grid serviceso Transfer Management tool (PhEDEx for CMS)Transfer Management tool (PhEDEx for CMS)
o Batch system (Condor,PBS,LSF,Torque, etc)Batch system (Condor,PBS,LSF,Torque, etc)
o Quota report based on Experiment's characteristic Quota report based on Experiment's characteristic organization (Physics groups, T2 federation, etc)organization (Physics groups, T2 federation, etc)
Options
• SGDB's & SQL - Might be too mucho More costs for deploymento Less flexibilityo Increase application's complexity/time spento Black sheep - SQLite - however, scaling problems
• Scripting - Might be too bado Database is too much, query the info from CLI,
format and print (text file, HTML)o Parse logs, format and print (text file, HTML)
• Bottom line is evaluate – Effort x Result x Quality you need
NoSQL
CouchDB,MongoDB, etc...
It doesn't mean SQL is a bad idea =)
JSON My apologies to the ones that know it... Format that represents associative arrays (key-
value pairs){
"Name": "Smith",
"age": 25,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021"
},
}
For the curious... MapReduce Trivial parallelization
First Map (filter, get documents subset)
Then reduce (count, sum, etc)
Overview Server
Databases
Documents Unique ID, revision All the rest is free Key/Value – datatype for many languages
Views Static sets of what you want to show View code scans all documents in the database Caching
For Your Information – “CouchApps”
Documents{
"_id": "[email protected]#1146503.0#1335286041",
"_rev": "10-12a26ecf9bc0fecf5ed9073c374a620c",
"data": {
"NumJobStarts": {
"i": "1"
},
"EnteredCurrentStatus": {
"i": "1335415262"
},
"CurrentHosts": {
"i": "0"
}
}
}
http://<MYSERVER>/<MYDATABASE>/<DOCUMENT_ID>
Views
Extract from the documents, only what you want to see
Only documents you want to see
function(doc) {
if (doc.Type == "customer") {
emit(doc._id, {LastName: doc.LastName, FirstName: doc.FirstName, Address: doc.Address});
}
}
http://<MYSERVER>/<MYDATABASE>/_design/<MYDESIGN>/
_view/<MYVIEW>
CouchApps
Webserver CouchDB hosts your HTML(5...)+Javascript
application
Upload to the given “DB”, has access to it's views, and docs.
DB is replicated, everything goes together
http://<MYSERVER>/<MYDATABASE>/_design/<MYCOUCHAPPAME>/index.htmlhttp://<MYSERVER>/<MYDATABASE>/_design/<MYCOUCHAPPAME>/index.html
GlobalMonitor
