Supporting Autonomic Management of Clouds: Service-Level-Agreement, Cloud Monitoring and Similarity...
-
Upload
rafael-uriarte -
Category
Science
-
view
41 -
download
1
Transcript of Supporting Autonomic Management of Clouds: Service-Level-Agreement, Cloud Monitoring and Similarity...
Supporting Autonomic Management of
Clouds: Service-Level-Agreement, Cloud
Monitoring and Similarity Learning
byRafael Brundo [email protected]
Under the Supervision of:Prof. Rocco De Nicola and Prof. Francesco Tiezzi
Doctoral Thesis Defense - March 30th, 2015 - Lucca, Italy
Contents
1 Introduction
2 SLA for Clouds
3 Cloud Monitoring
4 Similarity Learning
5 Polus Framework
6 Conclusions
Rafael Brundo Uriarte 1/51
Introduction
Introduction Rafael Brundo Uriarte 2/51
Cloud Computing
Introduction Rafael Brundo Uriarte 3/51
Cloud Characteristics
I Services
I Heterogeneity
I Virtualization
I Large-Scale
I Complexity
Introduction Rafael Brundo Uriarte 4/51
Autonomic Computing
Introduction Rafael Brundo Uriarte 5/51
Autonomic Computing
Introduction Rafael Brundo Uriarte 6/51
Knowledge for the Self-Management
I Policies
I Service Definition and Objectives
I Status of the Cloud and Services
I Specific Knowledge
Introduction Rafael Brundo Uriarte 7/51
Challenge and Scope
Introduction Rafael Brundo Uriarte 8/51
Research Questions
Research Question 1How to describe services and their objectives in the cloud
domain?
Research Question 2What is data, information, knowledge and wisdom in the
autonomic cloud domain?
Research Question 3How to collect and transform operational data into useful
knowledge without overloading the autonomic cloud?
Introduction Rafael Brundo Uriarte 9/51
Research Questions
Research Question 4How to produce a robust measure of similarity for services in
the domain and how can this knowledge be used?
Research Question 5How to integrate different sources of knowledge and feed the
autonomic managers?
Introduction Rafael Brundo Uriarte 10/51
SLA for Clouds
SLA for Clouds Rafael Brundo Uriarte 11/51
Service-Level-Agreement (SLA)
I Contract
I Service Description
I Quality-of-Service
I Formalism
I Guarantees!
SLA for Clouds Rafael Brundo Uriarte 12/51
SLA for Cloud Computing - SLAC
I Domain Specific
I Multi-Party
I Deployment Models
I Formalism
I Ease-of-Use
SLA for Clouds Rafael Brundo Uriarte 13/51
Yet Another SLA Definition Language?
Features WSOL WSLA SLAng WSA SLA* SLAC
General Deployment Models � � � � � �
Broker Support - - - - - �
Business
Pricing Schemes � � - � � �
Formal Semantics - - � - - �
Verification - - � - - �
“�” feature covered“�” feature partially covered
“-” no support
SLA for Clouds Rafael Brundo Uriarte 14/51
Main Concepts
I Predefined Metrics - Involved Parties and Unit
I Intervals for Metrics - Template and Variations
I Groups - Multiple Service, Community Cloud
I Constraint Solving Problem
SLA for Clouds Rafael Brundo Uriarte 15/51
Example
SLA for Clouds Rafael Brundo Uriarte 16/51
Business Aspects
I Business Actions
I Flat and Variable Models
I Pricing Schemes - Exchange, Auction, Tender,Bilateral, Fixed, Posted
SLA for Clouds Rafael Brundo Uriarte 17/51
Implementation
I Editor for SLAs (Ecplise-based using Xtext)
I SLA Evaluator (Z3 Solver)
I Integration with the Monitoring System
SLA for Clouds Rafael Brundo Uriarte 18/51
Cloud Monitoring
Cloud Monitoring Rafael Brundo Uriarte 19/51
DIKW in the Domain
I Data
I Information
I Knowledge
I Wisdom
Cloud Monitoring Rafael Brundo Uriarte 20/51
Cloud Monitoring
The Role of the Monitoring System in Clouds:
I Collect data and Provide Information andKnowledge
I No Wisdom - Related to Decision-Making
I Sensor of MAPE-K Loop
Cloud Monitoring Rafael Brundo Uriarte 21/51
Related Works
Property PCMONS Monalytics Lattice Wang
Cloud � - - -
Autonomic Integration - - - -
Scalability - � � �
Adaptability - � - �
Resilience - - - -
Timeliness - � - �
Extensibility � � � �
“�” feature covered“�” feature partially covered
“-” no support
Cloud Monitoring Rafael Brundo Uriarte 22/51
Panoptes
I Multi-agent system
I Monitoring in different levels
I Monitoring Modules - What needs to bemonitored and how to process the data
Cloud Monitoring Rafael Brundo Uriarte 23/51
Architecture
Cloud Monitoring Rafael Brundo Uriarte 24/51
Architecture
Communication:
I Publish/Subscribe
I Private Message
Adaptativeness:
I Priority for Modules
I Change of Roles
Cloud Monitoring Rafael Brundo Uriarte 25/51
Architecture: Autonomic Integration
I Urgency Mechanism
I Decentralised Architecture
I On-the-Fly Configuration
I Multiple Abstractions
Cloud Monitoring Rafael Brundo Uriarte 26/51
Experiments
I Self-Protection System
I Urgency Mechanism
I Scalability
Cloud Monitoring Rafael Brundo Uriarte 27/51
Similarity Learning
Similarity Learning Rafael Brundo Uriarte 28/51
Specific Knowledge
Generation of Knowledge for a Specific Purpose, i.e.not applicable in all clouds. For example, similarity.
But what is similarity?
I How much an object (service) resembles other
Similarity Learning Rafael Brundo Uriarte 29/51
Applications of Similarity
Cluster Services:
I Group Similar Services
I Different Algorithms(K-Means, PAM, EM)
Applications in the Domain:
I Anomalous Behaviour Detections
I Service Scheduling
I Application Profiling
I SLA Risk Assessment
Similarity Learning Rafael Brundo Uriarte 30/51
Domain Requirements
I Categorical Characteristics of Services
I On-line Prediction
I Large Number of Characteristics
I Fast Prediction
Similarity Learning Rafael Brundo Uriarte 31/51
Random Forest
Clustering with Random Forest
I Originally Developed for Classification
I Calculate the Similarity
I Clustering Algorithm (PAM)
Similarity Learning Rafael Brundo Uriarte 32/51
Similarity Using RF: Criteria
Similarity Learning Rafael Brundo Uriarte 33/51
Problems
I Similarity Matrix (Big Memory Footprint)
I Re-cluster on Every New Observation
I Cannot be Used in the Domain
Similarity Learning Rafael Brundo Uriarte 34/51
Solution: RF+PAM
Similarity Learning Rafael Brundo Uriarte 35/51
Solution: RF+PAM
Similarity Learning Rafael Brundo Uriarte 36/51
Experiments
I Compared the performance of our algorithm toother 2 methodologies
I Compared the performance of RF+PAM withthe standard off-line similarity learning
I Use Case:
I Scheduler deploys together the mostdissimilar services
I Similarity based on their SLAs
Similarity Learning Rafael Brundo Uriarte 37/51
Polus Framework
Polus Framework Rafael Brundo Uriarte 38/51
Polus Framework
Polus Framework Rafael Brundo Uriarte 39/51
Use Case
Polus Framework Rafael Brundo Uriarte 40/51
Use Case
Polus Framework Rafael Brundo Uriarte 41/51
Use Case
Polus Framework Rafael Brundo Uriarte 42/51
Conclusions
Conclusions Rafael Brundo Uriarte 43/51
Summary
Conclusions Rafael Brundo Uriarte 44/51
Research Questions
Research Question 1How to describe services and their objectives in the cloud
domain?
SLAC
Research Question 2What is data, information, knowledge and wisdom in the
autonomic cloud domain?
DIKW Hierarchy
Research Question 3How to collect and transform operational data into useful
knowledge without overloading the autonomic cloud?
Panoptes
Conclusions Rafael Brundo Uriarte 45/51
Research Questions
Research Question 4How to produce a robust measure of similarity for services in
the domain and how can this knowledge be used?RF+PAM
Research Question 5How to integrate different sources of knowledge and feed the
autonomic managers?Polus Framework
Conclusions Rafael Brundo Uriarte 46/51
Limitations
I Intelligence of Autonomic Managers
I Wide Range of Specific Knowledge
I Off-line Training of RF+PAM
Conclusions Rafael Brundo Uriarte 47/51
Contributions
I A theoretical and practical framework for thegeneration and provision of knowledge for theautonomic management of clouds (PolusFramework):
I SLAC - SLA Definition and EvaluationI Panoptes - MonitoringI RF+PAM - Similarity Learning
Conclusions Rafael Brundo Uriarte 48/51
Publications
1. R. B. Uriarte, S. Tsaftaris and F. Tiezzi. Service Clustering forAutonomic Clouds Using Random Forest. In Proc. of the 15thIEEE/ACM CCGrid [In Press], 2015.
2. R.B. Uriarte, F. Tiezzi, R. De Nicola, SLAC: A FormalService-Level-Agreement Language for Cloud Computing. In IEEE/ACM7th International Conference on Utility and Cloud Computing (UCC),2014.
3. R.B. Uriarte, C.B. Westphall, Panoptes: A monitoring architecture andframework for supporting autonomic Clouds, In Proc. of the 16thIEEE/IFIP Network Operations and Management Symposium (NOMS),2014.
4. R.B. Uriarte, S.A. Chaves, C.B. Westphall, Towards an Architecture forMonitoring Private Clouds. In IEEE Communications Magazine, 49,pages 130-137, 2011.
Conclusions Rafael Brundo Uriarte 49/51
Future Works
I Dynamic SLAs
I Negotiation of SLAs
I Cloud Case Study
Conclusions Rafael Brundo Uriarte 50/51
SLAC - Expressivity
I Core Language
I Extensions - Business Aspects
I Formal Definition for Extensions
Conclusions Rafael Brundo Uriarte 51/51
SLAC - Implementation
Compatibility only with OpenNebula
I Toy Implementation
I Easily adapted
Conclusions Rafael Brundo Uriarte 51/51
SLAC - Cloud Metrics
DTMF Cloud Computing Service MetricsDescription
I Recent Document (Still a Draft)
I Creation of a Model for the Definition of Metrics
I The SLAC Metrics can be Adapted for thisModel
Conclusions Rafael Brundo Uriarte 51/51
SLAC Violation
I Violation and Penalty are Separated Concepts
I “Violation” Concept Flexible
I Easy to Understand
Conclusions Rafael Brundo Uriarte 51/51
Panoptes - Scalability
I Designed to be scalable
I Adapt itself
I Experiments suggest it is scalable
I More experiments for future works
Conclusions Rafael Brundo Uriarte 51/51
Panoptes - Analysis of Apache Broklyn
I Not Focused on Monitoring
I Does Not Process the Data
Conclusions Rafael Brundo Uriarte 51/51
Panoptes - Analysis with CSPARQL
I Data is not Decorated (e.g. RDF)
I Impact of Decorated Monitoring Data(Scalability)
I Very Interesting Option
Conclusions Rafael Brundo Uriarte 51/51