Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram...
-
Upload
philomena-butler -
Category
Documents
-
view
216 -
download
1
Transcript of Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram...
![Page 1: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/1.jpg)
Elastic Computing – Towards Integration of IoT, People, and Clouds
WKO 5-5-2015, Wien
Schahram Dustdar
Distributed Systems GroupTU Vienna
dsg.tuwien.ac.at
![Page 2: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/2.jpg)
Acknowledgements
NOTE: The content includes some ongoing work
Includes some joint work with Hong-Linh Truong, Muhammad Z.C. Candra, Georgiana Copil, Duc-Hung Le, Daniel Moldovan, Stefan Nastic, Mirela Riveri, Sanjin Sehic, Ognjen Scekic
![Page 3: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/3.jpg)
Smart City Example
3
Air Sensor
Fire Sensor
Hear Sens.
SmartContainers
WaterSensor
Humidity Sensor
TrashSensor
Access Control
SmartMeter
DashboardCamera
Loca-lisation
Parking Space Sens.
VideoSurveillance
TrafficDensity
Things
TrainingAssistant
Goods Tracking
Water Management
WateringService
GarbageCollection
AutomatedParking
ProductLocalisation
CrowdManagement
Traffic Control
PersonDetection
Smart Facility Management
Desaster Management
Software
Training History
ContainerUtilization
WaterConsumption
Watering Needs
RecyclingRates
Parking Utilization
ProductInformation
CrowdMovement
Traffic Density
Facility Statistics
Desaster Information
Data
CrimeSurveillance
People
Everything-as-a-Service (EaaS)
![Page 4: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/4.jpg)
eHealth & Smart Healthnetworks
Game Machine
Telephone
PC
DVD
Audio
TVSTBDVC
SmartHomes
Smart eGovernments & eAdministrationsSmart Energy
Networks
Smart Evolution – People, Services,Things
Elastic Systems
Smart TransportNetworks
![Page 5: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/5.jpg)
1. “Resources” provided as services
2. Illusion of infinite resources
3. Usage-based pricing model -> New and connected business models
Cloud Computing
![Page 6: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/6.jpg)
Marine Ecosystem: http://www.xbordercurrents.co.uk/wildlife/marine-ecosystem-2
Think Ecosystems: People, Services, Things
Diverse users with complex networked dependencies and intrinsic adaptive behavior – has:
1. Robustness mechanisms: achieving stability in the presence of disruption
2. Measures of health: diversity, population trends, other key indicators
![Page 7: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/7.jpg)
IoT &
Services Delivery Ecosystem
![Page 8: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/8.jpg)
Smart City DubaiPacific Controls
Command Control Center
![Page 9: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/9.jpg)
HVAC (Heating, Ventilation, Air Conditioning) Ecosystem
![Page 10: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/10.jpg)
Water Ecosystem
![Page 11: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/11.jpg)
Air Ecosystem
![Page 12: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/12.jpg)
Monitoring
![Page 13: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/13.jpg)
Chiller Plant Analysis Tool
![Page 14: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/14.jpg)
Some 50 billion devices and sensors exist for M2M applications
IoT and Cloud Computing enable smart services ecosystem and collaboration opportunities
Managed services• Portfolio
management• Event management• Analytics
Provisioning• Services• SIM profile
configuration• Network
configuration
Controls• Activation• Deactivation• Privacy• Security
Transaction Mgmt.• Visibility• Billing• Reporting
Integration framework
Algorithm engineChart
builder
Predictive
modeling
Incidents manager
Expert rule engine
FDD Service Mgmt
Storage policies
Database
mangerOperations manager
PortfolioMgmt Analyic
s engine
Blackbox
module
Location awarenes
s
GUI builde
r
Event mgmt
Data mining
Resource mgmt.
Regression engine
Open integration platform
Resource manager
Point metering framewor
kNumerous Forms Of Smart Services…
Access control
Environment Compliance
Street Light Management
Food Transfer Process
Public
Safety
Industrial process
parameters
ParkingControl
WasteManagement
FacilitiesControl
HealthCare
Power Quality Control
LightingControl
KIOSK Monitoring
CCTVMonitoring
Hospitality Sector Healthcare Sector
Education SectorTransport Sector
Datacenters
Government Sector
Industrial Sector Finance Sector
Utilities and Smart Grid
Airports, ports and
Critical Infrastructure
Ubiquitous Managed Services Solution Across Business Verticals
Smart City Governance Ecosystem
![Page 15: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/15.jpg)
Command Control Center for Managed Services
![Page 16: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/16.jpg)
Approach
Smart City Research & Innovation
- Elastic Computing
People
ThingsSoftware
![Page 17: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/17.jpg)
Connecting machines and peopleEvent Analyzer on
PaaS
Peak Operation
Other stakeholders
...events stream
Normal Operation
Human Analysts
Peak OperationNormal Operation
Machine/HumanEvent Analyzers
Critical situation 1
Experts
SCU
(Big) Data analytics
Wf. A
Wf. B
Criticalsituation 2
Cloud DaaS
Data analytics
M2M PaaS
Cloud IaaS
Operation problem
Maintenance process
Core principles: Human computation capabilities under elastic service units “Programming“ human-based units together with software-based units
![Page 18: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/18.jpg)
Elasticity ≠ Scaleability
Resource elasticity Software / human-basedcomputing elements,multiple clouds
Quality elasticityNon-functional parameters e.g., performance, quality of data,service availability, humantrust
Costs & Benefit elasticityrewards, incentives
Elasticity
![Page 19: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/19.jpg)
The Vienna Elastic Computing Model
Multi-dimensional Elasticity
Service computing models
Cloud provisioning models
Schahram Dustdar, Hong Linh Truong: Virtualizing Software and Humans for Elastic Processes in Multiple Clouds- a Service Management Perspective. IJNGC 3(2) (2012)
Vienna Elastic Computing Modeldsg.tuwien.ac.at/research/viecom
![Page 20: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/20.jpg)
Elasticity in computing – broad view
1. Demand elasticity Elastic demands from consumers
2. Output elasticityMultiple outputs with different price and quality
3. Input elasticity Elastic data inputs, e.g., deal with opportunistic data
4. Elastic pricing and quality models associated resources
![Page 21: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/21.jpg)
Diverse types of elasticity requirements
Application user: “If the cost is greater than 800 Euro, there should be a scale-in action for keeping costs in acceptable limits”
Software provider: “Response time should be less than amount X varying with the number of users.”
Developer: “The result from the data analytics algorithm must reach a certain data accuracy under a cost constraint. I don’t care about how many resources should be used for executing this code.”
Cloud provider: “When availability is higher than 99% for a period of time, and the cost is the same as for availability 80%, the cost should increase with 10%.”
![Page 22: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/22.jpg)
Elasticity Engineering
![Page 23: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/23.jpg)
Specifying and controling elasticity
Elasticitc Control Language Familiy
Data/Compute-intensive services
Software/Human-intensive services
Business/E-scienceHybrid Mixed systems
Workflows/Application Services/Middleware/S
ystems
Basic primitives
Domain-specific/Customized features
Schahram Dustdar, Yike Guo, Rui Han, Benjamin Satzger, Hong Linh Truong: Programming Directives for Elastic Computing. IEEE Internet Computing 16(6): 72-77 (2012)
![Page 24: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/24.jpg)
High Level Description of Elasticity Requirements
SYBL (Simple Yet Beautiful Language) for specifying elasticity requirements
SYBL-supported requirement levelsCloud Service Level
Service Topology Level
Service Unit Level
Relationship Level
Programming/Code Level
#SYBL.CloudServiceLevelCons1: CONSTRAINT responseTime < 5 ms Cons2: CONSTRAINT responseTime < 10 ms WHEN nbOfUsers > 10000Str1: STRATEGY CASE fulfilled(Cons1) OR fulfilled(Cons2): minimize(cost)
#SYBL.ServiceUnitLevelStr2: STRATEGY CASE ioCost < 3 Euro : maximize( dataFreshness )
#SYBL.CodeRegionLevelCons4: CONSTRAINT dataAccuracy>90% AND cost<4 Euro
Georgiana Copil, Daniel Moldovan, Hong-Linh Truong, Schahram Dustdar, "SYBL: an Extensible Language for Controlling Elasticity in Cloud Applications", 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), May 14-16, 2013, Delft, Netherlands
![Page 25: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/25.jpg)
High Level Description of Elasticity Requirements
Current SYBL implementationin Java using Java annotations
@SYBLAnnotation(monitoring=„“,constraints=„“,strategies=„“)
in XML<ProgrammingDirective><Constraints><Constraint
name=c1>...</Constraint></Constraints>...</ProgrammingDirective>
as TOSCA Policies<tosca:ServiceTemplate name="PilotCloudService"> <tosca:Policy name="St1"
policyType="SYBLStrategy"> St1:STRATEGY minimize(Cost) WHEN high(overallQuality) </tosca:Policy>...
Other possibilitiesC# Attributes
[ProgrammingAttribute(monitoring=„“,constraints=„“,strategies=„“)]
Python Decorators@ProgrammingDecorator(monitoring,constraints,strategies)
Georgiana Copil, Daniel Moldovan, Hong-Linh Truong, Schahram Dustdar, "SYBL: an Extensible Language for Controlling Elasticity in Cloud Applications", 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), May 14-16, 2013, Delft, Netherlands
![Page 26: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/26.jpg)
Elasticity Model for Cloud ServicesMoldovan D., G. Copil,Truong H.-L., Dustdar S. (2013). MELA: Monitoring and Analyzing Elasticity of Cloud Service. CloudCom 2013
Elasticity space functions: to determine if a service unit/service is in the “elasticity behavior”
Elasticity Pathway functions: to characterize the elasticity behavior from a general/particular view
Elasticity Space
![Page 27: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/27.jpg)
Multi-Level Elasticity SpaceService requirement
COST<= 0.0034$/client/h
2.5$ monthly subscription for each service client (sensor)
Elasticity Space “Clients/h” Dimension
Elasticity Space “Response Time” Dimension
Determined Elasticity Space Boundaries Clients/h > 148 300ms ≤ ResponseTime ≤ 1100 ms
![Page 28: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/28.jpg)
Multi-Level Elasticity Pathway
Service requirement
COST<= 0.0034$/client/h
2.5$ monthly subscription for each service client (sensor)
Event Processing service unit Elasticity Pathway
Cloud Service Elasticity Pathway
![Page 29: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/29.jpg)
Specifying and controling elasticity of human-based services
What if we need to invoke a human?
#predictive maintanance analyzing chiller measurement#SYBL.ServiceUnitLevelMon1 MONITORING accuracy = Quality.AccuracyCons1 CONSTRAINT accuracy < 0.7 Str1 STRATEGY CASE Violated(Cons1): Notify(Incident.DEFAULT, ServiceUnitType.HBS)
![Page 30: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/30.jpg)
Elastic SCU provisioning atop ICUs
Elastic profileSCU (pre-)runtime/static formation
Cloud APIs
Muhammad Z.C. Candra, Hong-Linh Truong, and Schahram Dustdar, Provisioning Quality-aware Social Compute Units in the Cloud, ICSOC 2013.
Algorithms Ant Colony
Optimization variants
FCFS Greedy
SCU extension/reduction Task reassignment
based on trust, cost, availability
Mirela Riveni, Hong-Linh Truong, and Schahram Dustdar, On the Elasticity of Social Compute Units, CAISE 2014
![Page 31: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/31.jpg)
Conclusions (1) – Smart City means Engineering Elasticity
The evolution of underlying systems and the utilization of different types of resources under different models for elasticity requires
Complex, open hybrid service unit provisioning frameworks
Different strategies for dealing with different types of tasks
Quality issues for software, data, and people in an integrated manner
![Page 32: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/32.jpg)
Conclusions (2) – From Smart City to an Internet of Cities
• Interconnected network of capabilities
• To be able to move freely between clouds
• Reliable exchange/communication/coordination standards that also respect concerns like Privacy, Security, Compliance, and Costs
![Page 33: Elastic Computing – Towards Integration of IoT, People, and Clouds WKO 5-5-2015, Wien Schahram Dustdar Distributed Systems Group TU Vienna dsg.tuwien.ac.at.](https://reader037.fdocuments.us/reader037/viewer/2022110100/56649dd95503460f94ace6d1/html5/thumbnails/33.jpg)
Thanks for your attention!
Prof. Dr. Schahram Dustdar
Distributed Systems GroupTU Wien
dsg.tuwien.ac.at