Engineering human-based services in
elastic systems
Hong-Linh Truong
Distributed Systems Group,
Vienna University of Technology
[email protected]://dsg.tuwien.ac.at/staff/truong
1ASE Summer 2015
Advanced Services Engineering,
Summer 2015 – Lecture 9
Advanced Services Engineering,
Summer 2015 – Lecture 9
Outline
Human service units
Provisoning and employing human sevice units
– frameworks
Human service units in Vienna Elastic
Computing Model (VIECOM)
Evaluating human services
ASE Summer 2015 2
Scenario
3
Offers services for
handling IoT Data
Offers services for
handling IoT DataOffers services for big,
data analytics
Offers services for big,
data analytics
Offers services for
complex problem solving
using human experts
Offers services for
complex problem solving
using human experts
IoT Cloud PlatformData Analytics
Platform
Expert Provisioning
Platform
Sensors
<<send data>>
<<analyze data>> <<notify possible
problem>>
<<control/configure
sensors>>
Predictive maintenance company Predictive maintenance company
<<monitor>>
Chillers
<<predict and solve
problems>>
<<control
services>>
<<control
algorithms>>
ASE Summer 2015
Near-Realtime Data Analytics
Offline Data Analytics CloudLyra
Integrated systems of software, things
and people services
ASE Summer 2015 4
SensorsNoSQL BigData
Sensor data
GatewaysEventHandling Web ServiceLoad
Balancer
MOM
Sensor data
Cloud-based M2M Platform
Control Analysis Configuration
control, analyze,deploy
COMOT – Elasticity ManagementPlatformcritical situation msg
Smart Communication
Collective Provisioning
ExternalService
task
forms
elasticitiy control/management msg
criticalsituation msg
criticalsituationmsg critical situation
msg/task
clasticity control/management msg; task
HDA-CAS
Human Interface(Mobile, Mail, Web)
Human-basedServices
execute/manage
Human-based services for solving
complex problems (2)
5
But how to program human-based services and
software-based services together?
But how to program human-based services and
software-based services together?
ASE Summer 2015
Example: some common tasks in
data analytics
ASE Summer 2015 6
Human activities in Data Analytics
LabelingAnnotating
CleansingEvaluating
data/contentDectecting patterns
Classifying objects
Steering analytics
Human service units in data
analytics -- functions
Evaluating: is the quality of picture good?
Classifying: is it a man‘s or a woman‘s picture?
Decting: any unidentified object in a picture?
Labeling: adding location information of a picture
Cleansing: remove duplicated pictures
Steering: the quality of picture is bad, should we
continue to merge it with others?
ASE Summer 2015 7
How to model such functions for human units ?
E.g., with WSDL or REST?
How to model such functions for human units ?
E.g., with WSDL or REST?
Human service units
ASE Summer 2015 9
Human service unit
Human service unit
Functions
Non-functional
parameters
Service models
Technical interfaces
Provisioning mechanisms
Interaction models
Human acting as a „service unit“Human acting as a „service unit“
Human service units
10
Modeling type of
units (e.g.,
computation, data,
monitor,) and their
dependencies
Consumption,
ownership, provisioning,
price, etc.
ElasticService
Unit
Service model
Unit Dependency
ElasticCapability
Function
The functional
capability of the unit
and interface to
access the function
Capabilities to be elastic
under different
requirements
SoftwareSoftware PeoplePeople
VolunteersProfessionals
ThingThing
Resources
ASE Summer 2015
Forms of human service
Individual Compute Unit
An individual is treated like „a processor“ or “functional
unit“. A service can wrap human capabilities to support
the communication and coordination of tasks
Social Compute Unit
A set of people and software that are initiated and
provisioned as a service for solving tasks
Web services interfaces can be built
Different pricing models and different
quality models
ASE Summer 2015 11
Human service units – provisioning
mechanisms (1)
An infrastructure can be introduced for
accessing many ICUs in a crowd
Allow people to register their service unit capabilities
Facilitate communication, task bidding, retrieval and
result delivery
Act like a marketplace: multiple providers and multiple
consumers
ASE Summer 2015 12
ICU ICU
ICUICU
ICUICU ICU
CrowdCrowd
Human service units – provisioning
mechanisms (2)
An „infrastructure-as-a-service“ for ICUs
Facilitate communication, task retrieval and result
delivery
Single ICUaaS provider and multiple consumers
ASE Summer 2015 13
ICU ICU
ICUICU
ICUICU ICU
Cloud
provisioning
models ICUICU
Human service units – provisioning
mechanisms (3)
An „infrastructure-as-a-service“ for SCUs
Facilitate communication, task retrieval and result
delivery
Single SCUaaS provider and multiple consumers
ASE Summer 2015 14
SCUSCU
SCUSCU
SCUSCU
SCU
Cloud
provisioning
models SCUSCU
Human service units – technical
interfaces (1)
ASE Summer 2015 15
People
Web Servicce
(REST/SOAP)
People
Web page
Analytics
Activity
Analytics
Activity
Human service units – technical
interfaces (2)
ASE Summer 2015 16
Email/SMS/Instant
Messaging
People
Web page
Web Servicce
(REST/SOAP)Communication
Services
People
Analytics
ActivityAnalytics
Activity
Human service units – interaction
model
ASE Summer 2015 17
ICU/SCU
Task queue
Scheduler
ICU/SCU
Task queue
Scheduler
Analytics
Activity
Analytics
Activity
ICU/SCU
Analytics
Activity
Human service units -- NfPs
ASE Summer 2015 18
Which are important
considerations when
interpreting non-functional
properties for human service
units?
Which are important
considerations when
interpreting non-functional
properties for human service
units?
Incorporating human units into
complex processes
How to provision and employ human compute
units?
How to select human units?
Where to place human units in data analytics
and why?
How to monitor and test human units in data
analytics?
ASE Summer 2015 19
Qurk system architecture (1)
ASE Summer 2015 21
Source: Adam Marcus, Eugene Wu, David Karger, Samuel Madden, and Robert Miller. 2011. Human-powered sorts and joins. Proc. VLDB
Endow. 5, 1 (September 2011), 13-24.
Source: Adam Marcus, Eugene Wu, David Karger, Samuel Madden, and Robert Miller. 2011. Human-powered sorts and joins. Proc. VLDB
Endow. 5, 1 (September 2011), 13-24.
Qurk system architecture (2)
ASE Summer 2015 22
Source: Adam Marcus, Eugene Wu, David Karger, Samuel Madden, and Robert Miller. 2011. Human-powered sorts and joins. Proc. VLDB
Endow. 5, 1 (September 2011), 13-24.
Source: Adam Marcus, Eugene Wu, David Karger, Samuel Madden, and Robert Miller. 2011. Human-powered sorts and joins. Proc. VLDB
Endow. 5, 1 (September 2011), 13-24.
Jabberwocky approach (1)
ASE Summer 2015 23
Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: The jabberwocky programming environment for structured
social computing. UIST 2011: 53-64
Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: The jabberwocky programming environment for structured
social computing. UIST 2011: 53-64
Jabberwocky approach (2)
ASE Summer 2015 24
Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: The jabberwocky programming environment for structured
social computing. UIST 2011: 53-64
Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: The jabberwocky programming environment for structured
social computing. UIST 2011: 53-64
Automan approach
ASE Summer 2015 25
Source: Daniel W. Barowy, Charlie Curtsinger, Emery D. Berger, Andrew McGregor: AutoMan: a platform for integrating human-based
and digital computation. OOPSLA 2012: 639-654
Source: Daniel W. Barowy, Charlie Curtsinger, Emery D. Berger, Andrew McGregor: AutoMan: a platform for integrating human-based
and digital computation. OOPSLA 2012: 639-654
SW4H approach (1)
ASE Summer 2015 26
Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm, David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina: Service-based Integration of
Human Users in Workflow-driven Scientific Experiments. In: Proceedings of the 8th IEEE International Conference on eScience (eScience
2012
Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm, David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina: Service-based Integration of
Human Users in Workflow-driven Scientific Experiments. In: Proceedings of the 8th IEEE International Conference on eScience (eScience
2012
SW4H approach (2)
Similar concepts in
collaborative working
environments but
integrated into
workflows
Do not discuss about
where and how to
select human units
ASE Summer 2015 27
Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm,
David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina:
Service-based Integration of Human Users in Workflow-
driven Scientific Experiments. In: Proceedings of the 8th
IEEE International Conference on eScience (eScience
2012
Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm,
David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina:
Service-based Integration of Human Users in Workflow-
driven Scientific Experiments. In: Proceedings of the 8th
IEEE International Conference on eScience (eScience
2012
VieCOM -- incorporate humans into a
programming paradigm (1)
• Abstracting human compute units as program elements
• Extending programming languages to support human compute units
• Data/control flows via extensible APIs
Programming languages
• Shared memory (e.g., human –software – human), message passing (human-to-human), artifact-centric, etc., via APIs working atop the compute unit abstraction layer
Multiple programming
models
• Computing capability /profile management: human computing power, reputation and incentive models
• Monitoring and enforcing incentives/rewards, quality of results, availability
• Communication between human-middleware, among Individual Compute Units (SCU)/Social Compute Units (SCU) for exchanging artifacts and comprehensing l tasks
Execution environment
29 http://dsg.tuwien.ac.at/research/viecomASE Summer 2015
VieCOM-- incorporate humans into a
programming paradigm (2)
VolunteersProfessionalsTeamIndividual
Service-based Middleware
CommunicationCommunicationMonitoringMonitoringCapability/Profile
Management
Capability/Profile
Management
Provisioning/Negotiation/Execution APIProvisioning/Negotiation/Execution API
Abstraction of Human-based Compute Units
SCUSC
U
SC
U SCU SCU
Program languages and programming models
Program
elements
Software
Compute
Units
program human actions
and dependencies
program incentive condition
and rewarding action
program result evaluation
method
Human-to-middleware
interfaces:
•visualization of collective tasks
•embedding of common forms
•mobile app
30ASE Summer 2015
Cloud of hybrid service units
Cloud of HBS: A cloud of HBS includes HBS that can be provisioned, deployed, and utilized on-demand based on different pricing and incentive models.
Cloud of HBS: A cloud of HBS includes HBS that can be provisioned, deployed, and utilized on-demand based on different pricing and incentive models.
Cloud of hybrid services: A cloud of hybrid services includes SBS and HBS that can be provisioned, deployed and utilized on-demand based on different pricing andincentive models.
Cloud of hybrid services: A cloud of hybrid services includes SBS and HBS that can be provisioned, deployed and utilized on-demand based on different pricing andincentive models.
ASE Summer 2015 31
Hybrid compute unit design –
fundamental elements
32ASE Summer 2015
Hong-Linh Truong, Hoa Khanh Dam, Aditya Ghose, Schahram Dustdar "Augmenting Complex Problem Solving with
Hybrid Compute Units",9th International Workshop on Engineering Service-Oriented Application (WESOA's 2013), In
conjunction with ICSOC 2013, Dec 2, 2013, Berlin, Germany, (c)Springer-Verlag
Hong-Linh Truong, Hoa Khanh Dam, Aditya Ghose, Schahram Dustdar "Augmenting Complex Problem Solving with
Hybrid Compute Units",9th International Workshop on Engineering Service-Oriented Application (WESOA's 2013), In
conjunction with ICSOC 2013, Dec 2, 2013, Berlin, Germany, (c)Springer-Verlag
Hybrid compute unit design --
Relationships
33
Relationship Type HBS SBS TBS HCU
Similarity Yes Yes Yes Yes
Composition Yes Yes Yes Yes
Data Dependency Yes Yes Yes Yes
Control Dependency Yes Yes Yes Yes
Location Dependency Yes Yes Yes Yes
Forwarding Yes Yes No Yes
Delegation Yes Yes No Yes
Social Relation Yes No No Yes
Elasticity Yes Yes No Yes
ASE Summer 2015
Hybrid compute units
34
Hybrid compute unit (HCU): a set of service units
includes software-based services, human-based
services and things-based services that can be
provisioned, deployed and utilized as a collective
on-demand based on different quality, pricing and
incentive models.
Hybrid compute unit (HCU): a set of service units
includes software-based services, human-based
services and things-based services that can be
provisioned, deployed and utilized as a collective
on-demand based on different quality, pricing and
incentive models.
ASE Summer 2015
Extensible architecture
Adapters for: email,
Dropbox, REST,
Android
Integrated with
WP4,6,8;
API access for WP5,2
Open source and
documentation:
https://github.com/tuwi
endsg/SmartCom
ASE Summer 2015 35
P. Zeppezauer, O. Scekic, H.-L. Truong and S. Dustdar, "Virtualizing Communication for Hybrid and Diversity-Aware Collective Adaptive Systems,"
10th International Workshop on Engineering Service-Oriented Applications (WESOA'14@ICSOC), Paris, 2014.
Zeppezauer, Virtualizing Communication for Hybrid and Diversity-aware Collective Adaptive Systems, Master thesis, Dec 2014.
P. Zeppezauer, O. Scekic, H.-L. Truong and S. Dustdar, "Virtualizing Communication for Hybrid and Diversity-Aware Collective Adaptive Systems,"
10th International Workshop on Engineering Service-Oriented Applications (WESOA'14@ICSOC), Paris, 2014.
Zeppezauer, Virtualizing Communication for Hybrid and Diversity-aware Collective Adaptive Systems, Master thesis, Dec 2014.
Highlights: Virtualizing Communication
Specifying and controling elasticity
of human-based services
What if we need to
invoke a human?
#predictive maintanance analyzing chiller measurement
#SYBL.ServiceUnitLevel
Mon1 MONITORING accuracy = Quality.Accuracy
Cons1 CONSTRAINT accuracy < 0.7
Str1 STRATEGY CASE Violated(Cons1):
Notify(Incident.DEFAULT, ServiceUnitType.HBS)
#predictive maintanance analyzing chiller measurement
#SYBL.ServiceUnitLevel
Mon1 MONITORING accuracy = Quality.Accuracy
Cons1 CONSTRAINT accuracy < 0.7
Str1 STRATEGY CASE Violated(Cons1):
Notify(Incident.DEFAULT, ServiceUnitType.HBS)
ASE Summer 2015 36
Utilizing hybrid services for
evolving/dependent task graphs
Hong-Linh Truong, Schahram
Dustdar, Kamal Bhattacharya
"Programming Hybrid Services
in the Cloud", 10th International
Conference on Service-oriented
Computing (ICSOC 2012),
November 12-16, 2012, Shanghai,
China. Best Paper Award.
Hong-Linh Truong, Schahram
Dustdar, Kamal Bhattacharya
"Programming Hybrid Services
in the Cloud", 10th International
Conference on Service-oriented
Computing (ICSOC 2012),
November 12-16, 2012, Shanghai,
China. Best Paper Award.
ASE Summer 2015 37
Elastic SCU provisioning atop ICUs
Elastic profileSCU (pre-)runtime/static formation
Cloud APIs
Muhammad Z.C. Candra, Hong-Linh Truong, and Schahram
Dustdar, Provisioning Quality-aware Social Compute Units in
the Cloud, ICSOC 2013.
Muhammad Z.C. Candra, Hong-Linh Truong, and Schahram
Dustdar, Provisioning Quality-aware Social Compute Units in
the Cloud, ICSOC 2013.
Algorithms
Ant Colony
Optimization
variants
FCFS
Greedy
SCU
extension/reduction
Task reassignment
based on trust, cost,
availability
Mirela Riveni, Hong-Linh Truong, and Schahram
Dustdar, On the Elasticity of Social Compute Units,
CAISE 2014
Mirela Riveni, Hong-Linh Truong, and Schahram
Dustdar, On the Elasticity of Social Compute Units,
CAISE 2014
ASE Summer 2015 38
Selecting human units
Do not select at all
Let human units bid the tasks
E.g., in crowdsourcing platforms
Static/fix mapping
E.g., using static information for human-task mapping
Simple selection techniques
Using the requirement of the task to find the suitable
human units based on their capabilities
Complex selection techniques
Utilizing complex dependency graphs to find suitable
human units
ASE Summer 2015 39
Selecting SCU based on task
graphs
40
SCU Formation
Algorithms
Business As
Usual
Corrective
Action
HBS
Constraints
Hong Linh Truong, Schahram Dustdar, Kamal Bhattacharya: Programming Hybrid Services in the Cloud. ICSOC 2012: 96-110Hong Linh Truong, Schahram Dustdar, Kamal Bhattacharya: Programming Hybrid Services in the Cloud. ICSOC 2012: 96-110
ASE Summer 2015
Placement techniques for human
units
Usually at design time the developer/designer decides
Where to put human units
Where some triggers should be put in order to invoke
human units if needed
At runtime
Find suitable human units
Invoke human units
Placement of human units
Application-specific
Needs automatic algorithms and supporting tools
ASE Summer 2015 41
Configuring iSCU
Establish „connectedness“ based on compliance
constraints and network topology
Addional cost might occur!
Program SBS and HBS for the iSCU to have a
complete working environment.
Different connectedness
E.g., ring-based, star-based, and master-slave
topologies
ASE Summer 2015 42
Towards programming framework
for HCU
43
Coordination and
Composition Models
Coordination and
Composition Models
[ICSOC12]
ASE Summer 2015
Turkalytics
Develop „Interaction
Model“ for human
activities
Monitor and analyze
metrics, e.g.,
performance and
location
ASE Summer 2015 45
Paul Heymann and Hector Garcia-Molina. 2011. Turkalytics: analytics for human computation. In Proceedings of the 20th international
conference on World wide web (WWW '11). ACM, New York, NY, USA, 477-486. DOI=10.1145/1963405.1963473
http://doi.acm.org/10.1145/1963405.1963473
Paul Heymann and Hector Garcia-Molina. 2011. Turkalytics: analytics for human computation. In Proceedings of the 20th international
conference on World wide web (WWW '11). ACM, New York, NY, USA, 477-486. DOI=10.1145/1963405.1963473
http://doi.acm.org/10.1145/1963405.1963473
Turkalytics
Track client
detail at client
side (in browser)
Store tracked
information in a
log server
Analyze events
ASE Summer 2015 46
Paul Heymann and Hector Garcia-Molina. 2011. Turkalytics: analytics for human computation. In Proceedings of the 20th international
conference on World wide web (WWW '11). ACM, New York, NY, USA, 477-486. DOI=10.1145/1963405.1963473
http://doi.acm.org/10.1145/1963405.1963473
Paul Heymann and Hector Garcia-Molina. 2011. Turkalytics: analytics for human computation. In Proceedings of the 20th international
conference on World wide web (WWW '11). ACM, New York, NY, USA, 477-486. DOI=10.1145/1963405.1963473
http://doi.acm.org/10.1145/1963405.1963473
VieCOM
SCU execution model and lifecycle
management
Metrics for ICUs and SCUs
Mirela Riveni, Hong-Linh Truong, and Schahram Dustdar, On the Elasticity of Social Compute
Units, CAISE 2014
Mirela Riveni, Hong-Linh Truong, and Schahram Dustdar, On the Elasticity of Social Compute
Units, CAISE 2014
ASE Summer 2015 47
Exercises
Read mentioned papers
Analyze pros and cons of existing frameworks
for data analytics
Survey existing algorithms for matching human
units to data analytics tasks
Examine requirements for locating places for
human units and implement some algorithms
Examine monitoring techniques for cloud of
human compute units
ASE Summer 2015 48
49
Thanks for your attention
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
http://dsg.tuwien.ac.at/staff/truong
ASE Summer 2015
Top Related