Self-healing Multi-Cloud Application ModellingErkuden Rios

Tecnalia Research amp InnovationC Geldo 700

Derio Spain 48160erkudenriostecnaliacom

Eider IturbeTecnalia Research amp Innovation

C Geldo 700Derio Spain 48160

eideriturbetecnaliacom

Maria Carmen PalaciosTecnalia Research amp Innovation

C Geldo 700Derio Spain 48160

maricarmenpalaciostecnaliacom

ABSTRACTCloud computing market forecasts and technology trends conrmthat Cloud is an IT disrupting phenomena and that the numberof companies with multi-cloud strategy is continuously growingCost optimization and increased competitiveness of companies thatexploit multi-cloud will only be possible when they are able to lever-age multiple cloud oerings while mastering both the complexityof multiple cloud provider management and the protection againstthe higher exposure to aacks that multi-cloud brings

is paper presents the MUSA Security modelling language formulti-cloud applications which is based on the Cloud ApplicationModelling and Execution Language (CAMEL) to overcome the lackof expressiveness of state-of-the-art modelling languages towardseasing a) the automation of distributed deployment b) the compu-tation of composite Service Level Agreements (SLAs) that includesecurity and privacy aspects and c) the risk analysis and servicematch-making taking into account not only functionality and busi-ness aspects of the cloud services but also security aspects epaper includes the description of the MUSA Modeller as the Webtool supporting the modelling with the MUSA modelling languagee paper introduces also the MUSA SecDevOps framework inwhich the MUSA Modeller is integrated and with which the MUSAModeller will be validated

CCS CONCEPTSbullSecurity and privacy rarrSoware security engineering Ac-cess control bullComputing methodologies rarrModeling method-ologies bullComputer systems organization rarrCloud comput-ing

KEYWORDSCloud Multi-cloud security modelling deployment

ACM Reference formatErkuden Rios Eider Iturbe and Maria Carmen Palacios 2017 Self-healingMulti-Cloud Application Modelling In Proceedings of ARES rsquo17 ReggioCalabria Italy August 29-September 01 2017 9 pagesDOI 10114530989543104059

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor prot or commercial advantage and that copies bear this notice and the full citationon the rst page Copyrights for components of this work owned by others than theauthor(s) must be honored Abstracting with credit is permied To copy otherwise orrepublish to post on servers or to redistribute to lists requires prior specic permissionandor a fee Request permissions from permissionsacmorgARES rsquo17 Reggio Calabria Italycopy 2017 Copyright held by the ownerauthor(s) Publication rights licensed to ACM978-1-4503-5257-41708 $1500DOI 10114530989543104059

1 INTRODUCTIONCloud computing forecasts and market estimates of the last yearsconrm that Cloud is an IT disrupting phenomena with a tremen-dous growth potential [5] According to Gartner [7] by 2020 thersquoCloud Shirsquo (shiing from traditional IT oerings to cloud services)will impact more than $1 Trillion in IT spending

In the last years the percentage of companies that have a strategyto use multiple clouds for running applications or for experimen-tation is continuously growing [25] In such landscape cost opti-mization and increased competitiveness of companies that exploitmulti-cloud will only be possible when they are able to leveragemultiple cloud providers oerings while mastering both the com-plexity of multiple cloud provider management and the protectionagainst the higher exposure to aacks that multi-cloud brings

Multi-cloud applications are those that take most out of the cloudby combining multiple cloud oerings ie those which componentsuse or are distributed in heterogeneous cloud resources thus facinghighly complex challenges with regards to both distributed deploy-ment automation as well as component and overall applicationsecurity assurance

As opposite to cloud federations in multi-cloud paradigm thecloud service providers which services are combined by the user donot need necessarily to have previously reached to an agreementabout the way or model in which the services will be oered

erefore a number of problems arise on how to model anddecide application components distribution in the clouds as wellas issues on how to specify security properties oered and required(from the cloud providers) by the multi-cloud application individualcomponents and the overall application

In order to solve these problems we present a novel approachfor addressing security in the modelling of multi-cloud applica-tions e formalisms and supporting tool presented herein havebeen developed in the context of the EU-funded MUSA project[18] e solution relies in the extension of the Cloud ApplicationModelling and Execution Language (CAMEL) [20] to address richerdeployment requirements specication as well as fully-edged se-curity behaviour specication addressing cases when applicationcomponents require and provide security capabilities

e current state-of-the-art modelling languages lack expressive-ness that eases a) the automation of distributed deployment b) thecomputation of composite Service Level Agreements (SLAs) thatinclude security and privacy aspects and c) the risk analysis andservice match-making taking into account not only functionalityand business aspects of the cloud services but also security aspectsand this paper addresses such gaps

e rest of the paper is organized as follows Section 2 explainsstate-of-the-art and challenges of multi-cloud application security

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

modelling In Section 3 we introduce the MUSA SecDevOps frame-work and workow for multi-cloud applications which gives thecontext of the work developed Section 4 details the MUSA SecurityDomain Specic Language and in Section 5 we describe its support-ing tool named MUSA Modeller included in the MUSA frameworkSection 6 describes the validation scenarios of the work presentedFinally Section 7 presents the conclusions and future work

2 MULTI-CLOUD APPLICATION SECURITYMODELLING

In the last years the cloud based infrastructure and platform servicesoering has increased notably as well as the number of providersOne of the main problems when architecting and operating (multi-)cloud applications is that cloud providers support dierent inter-faces for dierent sets of services Furthermore in order to exploitmulti-cloud potential dierent architectural models can be adoptedto enlarge security [1]

bull replication of applications deploying the same system inmore than one provider so as malicious aacks can beeasily discovered comparing operation results

bull partition of application system into tiers which allowsseparating logic from data and thus minimizing risks ofaacks or incidents in both parts at a time

bull partition of application logic into fragments in order toobfuscate the overall application logic to providers

bull partition of application data into fragments to prevent asingle provider reconstructing data safeguarding conden-tiality

Exploiting multi-cloud for higher availability and security meansthat DevOps teams in charge of developing and operating (multi-)cloud applications need to learn systematise and automate theprocess behind the provisioning of services when deploying theapplication components Among other tasks they need to learnhow to create Virtual Machines (VMs) and how to choose the rightVM sizes for each application service or component

To address these challenges current research stakeholders pro-pose to take advantage of the well-known Model Driven Engi-neering (MDE) techniques to congure the deployment of cloudapplications According to the work in [22] application models canbe specied using general-purpose languages like the Unied Mod-elling Language (UML) [8] However to fully unfold the potentialof MDE models are frequently specied using domain-specic lan-guages (DSLs) which are tailored to a specic domain of concern

e MDE techniques become really interesting for multi-cloudapplication specication when the model captures multi-concerninformation expressed at high level rst and detailed at low levelapplication platform aerwards and when the model is enacted atruntime is allows for a seamless alignment of design decisionswith actual deployment and application execution In this linework CloudML [28] and CAMEL [22] (which includes CloudML asDeployment model for expressing deployment needs) languagesCAMEL was developed as part of the research work in PaaSage[19] and CloudSocket [17] EU-funded projects

Another great exponent in the literature on cloud-based applica-tions modelling is the Topology and Orchestration Specication forCloud Applications (TOSCA) language developed by OASIS [15]

which provides a language for specifying the components com-prising the topology of cloud-based applications along with theprocesses for their orchestration Another example is the CloudApplication Modelling Language (CAML) dened in the ARTISTproject [16] which realised CloudML as a UML internal DSL basedon extensions to the deployment meta-model in terms of a libraryand proles capturing domain knowledge

Following a similar approach of that of PaaSage Security DSL[14] in MUSA project [4] that contextualises the work presentedin this paper CAMEL has been adopted in order to cope with themodelling of multi-cloud applications

CAMEL includes two main security-oriented meta-models [22]the Security meta-model to support the specication of securityrequirements posed by users and capabilities of cloud providers (inform of security controls and service level objectives) and the Or-ganisation meta-model that captures security-oriented informationabout organisations including organisation security policies usersand roles

e application requirements in CAMEL are mainly capturedby the Requirements meta-model us both the Security and theRequirements meta-models can complementarily capture securityrequirements CAMEL oers support to the following tasks [22](i) matching in deployment phase security capabilities and require-ments of the application to the security controls oered by thecloud providers (ii) monitoring and assessing security service levelobjectives (SLOs) which can be mapped to adaptation rules in orderto adapt the structure or behaviour of an application to exhibit thesecurity level required

Although both PaaSage and MUSA projects oer support to bothtasks they dier in the modelling aspects mainly because PaaSagerelies on CAMEL model enactment and MUSA follows a dierentapproach as explained in Section 3 e rationale for selectingCAMEL in our approach on top of other versions of CloudML andTOSCA is described in Section 41

3 THE MUSA SECDEVOPS FRAMEWORK FORMULTI-CLOUD APPLICATIONS

31 e MUSA frameworkMulti-cloud solutions pose new challenges when trying to add valueto overall cloud client experience [29]

e MUSA framework [26] introduced in this paper aims atensuring that the desired security and privacy levels are reached inall types of multi-cloud environments including those that combinemultiple architectural scenarios as described in Section 2

To this aim the MUSA framework combines preventive securitywith reactive security For prevention the MUSA framework sup-ports Security by Design practices in the development as well asembedding in the application the required security mechanismsthe so called MUSA Enforcement Agents which will enable the self-healing of the application in operation For reaction the MUSAframework includes monitoring of the application at runtime todetect notify and early mitigate security incidents so multi-cloudapplication providers can be informed and promptly react to secu-rity problems or aacks

In order to ensure the preventive security mechanisms to beembedded in the application components and aligned with reactive

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

Figure 1 MUSA approach and framework tools

security measures MUSA holistic framework oers a number ofmechanisms and supporting tools that seamlessly work togetheralong all phases of the application lifecycle as shown in Fig 1

32 e MUSA workowe MUSA framework is intended to provide integrated support toaddressing security aspects of multi-cloud applications in all thesteps of the engineering lifecycle as illustrated in the process ofFig 2

During the design phase the DevOps team rst models the CloudProvider Independent Model (CPIM) of the multi-cloud applicationusing the MUSA Modeller e CPIM is a MUSA extended CAMELmodel specication of the multi-cloud application in a level ofabstraction independent from specic information about the CloudService Providers (CSP) the application components will use or bedeployed in

Once the application CPIM is specied the DevOps team obtainsthe security requirements in the risk assessment step In this steptogether with the security requirements the DevOps team cananalyse other criteria as well such as business criteria and cansearch for the cloud services that best match all the criteria byrelying on the use of the MUSA Decision Support Tool (DST) [4]

Modelling risk analysis and cloud services selection are madefollowing an iterative process that allows identifying which securityrequirements (if any) were not possible to address with securitycontrols oered by the cloud services under study and thereforethe DevOps team can update the CPIM at modelling step to specifythe use of MUSA security agents that oer the pending securitycontrols (if available)

Having selected the combination of cloud services that bestmatch the requirements of the multi-cloud application and havingpreviously dened the security requirements the DevOps teamcan generate the Security SLA templates for the components ofthe multi-cloud application ese Security SLA templates will bestored in the SLA Repository and will be retrieved by the MUSADeployer so it can generate the Implementation plan for the multi-cloud application Once the Implementation plan is generated theMUSA Deployer shares it with the SLA Generator [2] so this tool

Figure 2 MUSA overall process

can generate the Composite Security SLA for the whole multi-cloudapplication

Aerwards at deployment phase the MUSA Deployer is invokedby the DevOps team in order to deploy (by following the Imple-mentation plan) both the multi-cloud application components andthe corresponding MUSA agents declared in the CPIM

Finally at runtime or operation phase the MUSA Security Assur-ance platform [27] starts monitoring the multi-cloud applicationbased on the nal SLAs and the Implementation plan

If the MUSA Security Assurance Platform detects any alert eventor violation of the SLAs in place it noties to the DevOps teamand the appropriate reactive measure is triggered e reaction tosecurity incidents in MUSA relies on dierent mechanisms depend-ing on the cause of the incident Reactive measures include there-deployment of multi-cloud application component(s) or even theapplication re-design In the re-design phase the need of includingMUSA Enforcement Agents can be evaluated again in order to tryto address the security incident detected Adaptation of multi-cloudapplications at execution is supported in MUSA by the enforcementservices in the MUSA Security Assurance Platform that remotelycontrol the conguration activation and deactivation of some ofthe MUSA Enforcement Agents We refer to future publications ofMUSA for detailed explanation of the enforcement services oered

4 MUSA SECURITY DSLe MUSA Security Domain Specic Language (DSL) builds on topof the CAMEL language and the main contributions are related toits expressiveness to dene and congure multi-cloud applicationsat design-time Such a powerful denition can be used later on byother MUSA tools to perform the risk analysis and to generate theindividual componentsrsquo security SLAs and the Composite SecuritySLA Furthermore this allows deploying and monitoring the secu-rity properties of multi-cloud applications and its components atrun-time In brief the main innovations achieved in MUSA are theimprovements to the CAMEL language the accompanying modelsyntax and semantics verication rules and the development of aWeb-based modelling tool (named MUSA Modeller) as explained inthe following e MUSA Security DSL is shown in Fig 3

41 Rationale for CAMEL language selectione main reasons for selecting CAMEL language as the basis of theMUSA security DSL are explained in the following

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

Figure 3 MUSA Security DSL

First the enactment of the multi-cloud application model atruntime is not needed in MUSA is is because in MUSA we donot rely on a single model of the multi-cloud application to captureall the aspects but on a number of models In MUSA we use theCAMEL model of the application to express its architecture securityand deployment requirements but in addition we use SecuritySLA templates of the components to capture the required securityfeatures over the cloud providers and the Composite Security SLAto express the guarantees of the overall application e CompositeSecurity SLA serves in MUSA to control the application behaviourat runtime Multi-cloud applications re-designs will be driven byactual measurements taken of the Service Level Objectives andmetrics stated in the Composite Security SLA In MUSA we also usethe Decision Support Tool data model for expressing business andsecurity requirements over the cloud providers and the deploymentImplementation plan as the cloud platform dependent deploymentmodel

Second CAMEL includes CloudML as Deployment meta-modeltherefore adopting it implies adopting CloudML too BesidesCAMEL includes also other meta-models such as Security and Re-quirements meta-models that are valuable when placing the focuson security CAMEL follows the same approach that CloudML in re-lation to the provision of a single set of abstractions and APIs so thatdevelopers can dene declaratively (i) the application architecturemade of components (ii) their usehost relationships so that theycan be properly congured and deployment orders automaticallyderived (iii) constraints on the characteristics of the required types

of VMs and (iv) the execution commands to provision applicationcomponents

At this point it is important to note that MUSA project wasborn with the objective of supporting cloud adoption by addressingcurrent security open issues since companies are reluctant to adoptcloud computing because of the diculty in evaluating the trade-o between cloud benets and the additional security risks andprivacy issues it may bring ey have to deal with the security ofthe individual components as well as with the overall applicationsecurity including the communications and the data ow betweenthe components erefore it is required to adopt a language thatallows end-users to friendly and easily create and deploy security-aware components balancing security with performance properties

As opposite to PaaSage approach where the low level security in-formation is captured in the CAMEL based Cloud Provider SpecicModel (CPSM) in MUSA it is captured and managed by two otherspecialised languages respectively the MUSA Security SLA model(extended from that of SPECS project [23] to address multi-cloudscenarios) and the MUSA Decision Support Toolrsquos Cloud ServiceProvider data model that allows dening full avoured businessand security requirements and capabilities for multi-factor servicematch-making

ird opposite to TOSCA MUSA end-users had already experi-enced with CAMEL and liked the language expressiveness becauseit was the only language that already provided rich Security modeland Requirements model (for deployment) CAMEL and TOSCA

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

basically share the same purpose but CAMEL results in being lesscomplex and easy to use

Furthermore TOSCA community did not oer advanced toolsupport at the time when MUSA solution was dened

Last but not least the feedback received from the MUSA projectend-users about previous experiences with multi-cloud modellinglanguages and tools was considered as fundamental One of theend-user teams had experienced with CAMEL and liked the ex-pressiveness of the language for multi-cloud application descrip-tion although security aspects were not considered by them previ-ously About the modelling tools they had previously used PaaSageCAMEL Textual Editor [21] and other graphical editors (such asModelio [3]) and preferred the non-graphical editor that includedfriendly functionality such as identication of required vs optionalaributes auto completion capabilities and model validation amongothers us these are exactly the features we aim to maintain inthe MUSA Modeller As conceptually both graphical and textualeditors allow for the same degree of detail and completeness wedecided to develop our MUSA Modeller as a text editor

42 Innovations for security behaviourspecications

e MUSA innovations to CAMEL language include the enhancedcomponent security behaviour characterization which addressesconcepts required to support both composition of componentsrsquosecurity SLAs and risk analysis

421 Classification of components by their nature which allowsdescribing what the component does is information is requiredby the MUSA SLA Generator to create the Composite Security SLAof the multi-cloud application from individual componentsrsquo SLAsand by the MUSA Risk Analysis tool in order to identify the secu-rity threats and risks at component level Our CAMEL extensionallows specifying the type of the component in fact the compo-nent can be classied by two features WHAT and HOW WhileWHAT indicates the type of the functionality delivered by the com-ponent HOW indicates the way the component is delivering suchfunctionality Currently three types of HOW have been dened

bull COTS which refers to Commercial o-the-shelf sowarethat the application uses

bull SERVICE meaning that the component is not a commercialpackage but developed by the DevOps team responsiblefor the multi-cloud application engineering

bull AGENT ie a MUSA Agent component provided by theMUSA framework and available in the MUSA SecurityAgent Catalogue

e types of WHAT include

bull in case the component is COTS or SERVICE the possibleWHAT values are Web Storage IDM or Firewall Webrefers to any functionality provided through a Web inter-face Storage refers to data storage solutions (eg MySQL)IDM stays for Identity Management and Firewall for anysoware solution that protects resources from unautho-rised access

bull in case the component is AGENT the possible values ofWHAT come from the list of the agents in the MUSA Secu-rity Agent Catalogue

422 Security Controls information that properly supports Se-curity Control Framework families For example name aribute hasbeen updated to ltFamilygt-ltNumbergt(Number) format In addi-tion the subdomain aribute in the Security Control entity now isoptional instead of compulsory

e CAMEL extension developed in MUSA allows specifyingwhich security capabilities are required and provided by each multi-cloud application component e security capabilities are denedin the model by selecting and grouping the security controls partof the capability e security controls of a Security Control Fam-ily are predened and the list is oered to the user by the MUSAModeller to ease the selection of the ones included in the capabilitycurrently the security controls from the NIST SP 800-53 rev4 [6]are supported In the following example two security capabilitiesCAP1 and CAP2 are dened the rst with three security controlsand the second with only two

security model SEC security capability CAP1 controls [MUSASECAC-11(1) MUSASECAU-13(2)MUSASECAC-17(6)]security capability CAP2 controls [MUSASECAC-11(1) MUSASECAC-17(6)]

Once the security capabilities are dened in the CAMEL (inthe security model part of the model) the user can specify whatsecurity capabilities the components require andor provide Inthe following example Comp1Cap is a provided capability andComp1CapReq a requested one

provided security capability Comp1Cap security capability SECCAP2required security capability Comp1CapReq

When a component requires a specic security capability fromanother component (in the example Comp1CapReq) then the match-ing of the capability needs to be modelled as follows

capability match Comp1ToComp2 from Comp1Comp1CapReq to Comp2Comp2Cap

43 Innovations for multi-cloud deploymentOther MUSA innovations to CAMEL language address improve-ments for enhancing the expressiveness of the deployment require-ments as follows

431 Explicit characterization of the nature of the IP addressassociated with virtual machines ponents will be deployed At de-ployment phase when acquiring new cloud resources such as VMs

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

the system operator needs to indicate whether a public IP address isrequired e CAMEL extension allows specifying whether the IPaddress required for a VM should be public or not by the IP publicaribute on each component e possible values for this aributeare true or false

432 Specification of the deployment order of the applicationcomponents Dealing with multi-cloud environments it is criticalto identify the order in which each component should be deployedand congured since there are inter-dependencies among the com-ponents that are part of the same application For example the startup of a component may require that previously another componentis already up and running e CAMEL extension allows specifyingthe order in which the components are required to be deployedis can be done by using the rdquoorderrdquo aribute for each componente expected value for the order aribute is an integer number

433 Explicit definition of data exchange protocols In the MUSAextended CAMEL users can model the communications betweenthe components (eg by seing the IP addresses and ports in theconguration of the components) and specify the need to use aspecic data exchange protocol (eg MySQL OAuth Other)

434 Modelling of dynamic configurations of communicationsbetween components In CAMEL users model the communicationsbetween the components in a static way (ie through specic portnumbers and operating system conguration variables) Howeverin MUSA dynamic characteristics have been introduced such ascontext paths (instead of IP addresses) and dynamic port rangesSuch new capabilities are useful for instance to congure explicitlyinbound trac when users deploy components in Docker contain-ers

435 Modelling of deployment handlers In CAMEL the usercan model components and associate deployment instructions forinstalling conguring starting and stopping the components onvirtual machines However such deployment instructions are re-stricted to scripted commands and CAMEL lacks support to thespecication of conguration management tools such as Cloudify[12] Puppet [24] or Chef [9] erefore in MUSA this gap betweenmulti-cloud application models and these advanced frameworkshas been faced via the new Conguration entity and its associatedconcepts (eg cookbooks and recipes in case of Chef)

436 Modelling of PaaS layer elements CAMEL lacks supportto the description of architectures where the application compo-nents are not directly deployed in Virtual Machines (VMs) but incontainers Our extension allows specifying the container type thatwill be used in deployment and dening the component allocationstrategy it should follow even in cases when the container usesVM pools e new elements in our extension include

bull pool is a cluster of VMs is cluster can be used bya container or directly by a component for example adatabase solution that is capable of managing a cluster

bull manager the VM in a pool that will act as the manager vsthe rest which will be workers

bull container ndash type the container solution to use for example Docker

Swarm [10]

ndash allocationStrategy denes the allocation strategy ofthe containers on top of the acquired VMs for resourceoptimization (eg automatically scheduling containerworkloads) It supports the following four values

lowast spread balance containers across the VMs in apool based on the available CPU and RAM ofthe VMs

lowast binpak shedule containers to fully use each VMcapacity Once the full VM capacity has beenused the container moves on to the next one inthe pool

lowast random choose a VM randomlylowast custom the user denes the specic VMs in

which the containers should run

437 Refinement of security aspects in Organisation User Cre-dentials and Role entities A number of enhancements to CAMELhave been made in order to manage the authorisation of dierentroles in the DevOps team to multi-cloud application deploymentexecution For instance in MUSA the types of credentials availableto authenticate a user have been extended Among other changesit has been added expiration date to Credentials and additionalparameter properties to User entity

44 Innovations for self-healing capability ofmulti-cloud applications

Considering self-healing as the capability of a multi-cloud applica-tion of being able to self-control or modify its security behaviourat runtime so as security incidents or aacks are corrected or miti-gated the self-healing is enabled in MUSA by the MUSA Enforce-ment Agents

As their name suggests the MUSA Enforcement Agents enforcemulti-cloud application security properties at runtime such as ac-cess control security vulnerability scanning or Denial of servicemitigation mechanisms For these mechanisms to work they needto be deployed at the same time as the application componentsare deployed Some of these mechanisms require to be deployedtogether with (in the same VM) the component they will enforcethe property on erefore the MUSA extended CAMEL allowsthe denition of MUSA Security Enforcement Agents as InternalComponents of the application similarly to application compo-nents themselves so as they can be included in the deploymentplan Such agents are already pre-dened in the MUSA SecurityAgent Catalogue so users are able to re-use and congure them in afriendly way Some of these agents are always on and some will bemanaged through the enforcement services in the MUSA SecurityAssurance Platform

5 MUSA MODELLER51 MUSA Modeller Architecturee MUSA Modeller is a web editor tool that allows the creationand maintenance of (MUSA extended) CAMEL models roughthese models it is possible to dene a complete specication of therequirements needed by an application to be deployed in a securemulti-cloud environment MUSA Modeller has leveraged one ofthe new capabilities of the Xtext technology [11] the Xtext Web

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

editor support erefore multi-cloud application models can beedited and updated remotely by end-users while they are storedin shared repositories All in all without any program installationand using any web browser since its JavaScript-based API allowsadding language-specic features such as auto completion and livevalidation

As the initial step of the design phase the DevOps team createsthe Cloud Provider Independent Model (CPIM) of the multi-cloudapplication using the MUSA Modeller tool e generated CPIMwill be used as input to the risk assessment the SLA generationand the multi-cloud application deployment processes

At high level the MUSA Modeller is structured as followsbull A Web component the GUI from which the end-users ac-

cess the MUSA Modeller tool and exploit all the internalweb-services oered It also includes the web Xtext li-braries that oer syntactic and sematic services for remotemanagement of syntax validation and auto completion

bull A Server component which is the core of the MUSA Mod-eller since it implements all the business functionalitiesMoreover it oers a series of web services that are con-sumed by the Web component

bull A Database component which manages all the data storedin the central database It uses the Hibernate framework[13] that allows the abstraction of relational database en-gines

52 MUSA Modeller APIAs explained in the previous chapter the Server component of theMUSA Modeller oers a series of REST API interfaces that supportthe following functionality

bull Creation of a new multi-cloud application model in (MUSAextended) CAMEL language End-users can instantiate pre-viously dened templates for particular component typesor applications

bull Denition and storage of multi-cloud application compo-nent templates for reusing CAMEL models of the compo-nents

bull Edition and storage of multi-cloud application modelswhere application models can be dened by multiple end-users

bull Model checking for syntax and semantic correctness andintegrity of the created models e tool provides messagesof warnings and errors whenever non-conformities areidentied in the model

bull the selection of Security Controls previously identiedand stored in the MUSA Security Service Level AgreementCatalogue

bull the selection of MUSA Security Enforcement Agents pre-viously identied and stored in the MUSA Security AgentCatalogue See Fig 4

6 VALIDATION IN USE CASESDuring the rst period of the MUSA project the evaluation of theMUSA solution has been performed using generic usage scenariosalong with two specic use cases one led by Luhansa Systems(LHS) and the other by Tampere University of Technology (TUT)

Figure 4 MUSA Modeller support for Security Agent selec-tion

both partners of the project ese case studies have been designedto ensure that MUSA framework evaluation takes place in dierentcontexts of multi-cloud deployment as well as diverse security andprivacy requirements

e rst use case is related to the ight scheduling applicationprototype by Luhansa Systems that is intended to be used bymultiple airlines around the world to plan airplane ight sched-ules Todayrsquos airlines need to permanently revise their scheduleplans in response to competitor actions or to follow updated salesand marketing plans while constantly maintaining operational in-tegrity is makes schedule management a very complex processe MUSA Modeller enabled LHS to specify the breakdown of themulti-cloud application into its components as well as componentsrsquosecurity capabilities is includes also the integration of the MUSAsecurity agents into the application and the design of the provi-sioning and deployment LHS has validated the MUSA Modeller tohelp architects and developers of the multi-cloud application com-ponents in the security design to gain more focus on the complexeld of security requirements analysis and enforcing the securityby design principles even for the less experienced colleagues

In the second use case the Smart Mobility application developedby Tampere University of Technology provides Tampere citizenscontext-aware energy ecient smart mobility services and recom-mendations for transportation means is case study representsthe scenario of SMEs which create applications that exploit serviceshosted in a number of clouds As many SMEs the workgroup thatdevelops this case study does not have a specialized division forcyber security Because of this the use of MUSA framework andits tools has allowed easily addressing security aspects at dierentstages of the application engineering project design deploymentand runtime

Specically in this case study the MUSA Modeller tool has beenevaluated to model the multi-cloud application components andMUSA Enforcement Agents to integrate security controls for highavailability condentiality and integrity

e added value of the MUSA modelling language and tool hasbeen already identied in the initial evaluation performed in bothcase studies e usefulness and acceptance of the approach was

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

conrmed not only in the usage of the MUSA Modeller as a stand-alone tool but more signicantly in its integration with the restof the tools in the MUSA SecDevOps framework particularly incollaboration with Risk Analysis SLA Generator and Deployertools It is expected that the nal evaluation of the MUSA solutionwill enable the validation of all the enhancements dened in theMUSA Security DSL

7 CONCLUSIONS AND FUTUREWORKAs the number of cloud oerings grows and cloud environmentsbecome more and more complex cloud consumers will need to facemulti-cloud challenges related to multiple cloud service combina-tion and holistic protection of application components deployed indistributed heterogeneous clouds

is paper presents a novel modelling language and support-ing tool that address the special needs of multi-cloud applicationsmodelling e solution overcomes the lack of expressiveness ofstate-of-the-art modelling by easing both a) the computation ofComposite Security SLAs that include security and privacy aspectsand b) the risk analysis and service match-making taking into ac-count not only functionality and business aspects of the cloudservices but also security aspects

e language and tool presented were developed in the contextof the MUSA EU-funded project on the basis of enhancements tothe CAMEL language which already provided rich RequirementsDeployment Scalability and Security meta-models that cover manyof the requirements for multi-cloud applications specication Nev-ertheless additional requirements were identied in MUSA in orderto address richer deployment and security specication risk analy-sis and Security SLA composition for which extensions to CAMELhave been developed e modelling tool supporting this extendedCAMEL meta-model the MUSA Modeller is already integratedwith the MUSA framework and available in the MUSA website atwwwmusa-projecteu

Consequently the contributions of this paper pave the way todevelop security and privacy-aware multi-cloud applications bymastering the expressiveness of security aspects in the CPIM so asthey enable integrated SecDevOps support

In the future we have identied the need to research on how toleverage the Scalability meta-model of CAMEL to dene scalabilityrules as pre-dened conguration of high availability enforcementagents aimed at scaling up (and down) internal components ofmulti-cloud applications when needed (similar to HA Proxy [30])

Another important aspect is the research on how to supportcomposability of CAMEL models required to easily combine andrefer to CAMEL models of individual components in the creation ofmulti-cloud applicationsrsquo models Furthermore this would largelyincrease the models readability and dramatically improve the toolusability

ACKNOWLEDGMENTSe MUSA project leading to this paper has received funding fromthe European Unionrsquos Horizon 2020 research and innovation pro-gramme under grant agreement No 644429 We would also like toacknowledge all the members of the MUSA Consortium for theirvaluable help

REFERENCES[1] Jens-Mahias Bohli Nils Gruschka Meiko Jensen Luigi Lo Iacono and Ninja

Marnau 2013 Security and privacy-enhancing multicloud architectures IEEETransactions on Dependable and Secure Computing 10 4 (2013) 212ndash224

[2] Valentina Casola Alessandra De Benedictis Massimiliano Rak and ErkudenRios 2016 Security-by-design in clouds a security-SLA driven methodology tobuild secure cloud applications Procedia Computer Science 97 (2016) 53ndash62

[3] Inc Eclipse Foundation 2011 Modelio e open source modelling environment(2011) Retrieved June 26 2017 from hpswwwmodelioorg

[4] Jaume Ferrarons Smrati Gupta Victor Muntes-Mulero Josep-Lluis Larriba-Peyand Peter Mahews 2016 Scoring cloud services through digital ecosystemcommunity analysis In International Conference on Electronic Commerce and WebTechnologies Springer 142ndash153

[5] Forbes 2016 Roundup Of Cloud Computing ForecastsAnd Market Estimates (2016) Retrieved June 26 2017from hpswwwforbescomsiteslouiscolumbus20160313roundup-of-cloud-computing-forecasts-and-market-estimates-20167971c3de2187

[6] JOINT TASK FORCE and TRANSFORMATION INITIATIVE 2013 Security andprivacy controls for federal information systems and organizations NIST SpecialPublication 800 53 (2013) 8ndash13

[7] Gartner 2016 Gartner Says by 2020 rdquoCloud Shirdquo Will Aect More an $1Trillion in IT Spending (2016) Retrieved June 26 2017 from hpwwwgartnercomnewsroomid3384720

[8] Object Management Group 2015 Unied Modeling Language Specication v25(2015) Retrieved June 26 2017 from hpwwwomgorgspecUML25

[9] Chef Soware Inc 2017 Chef technology (2017) Retrieved June 26 2017 fromhpswwwchefiochef

[10] Docker Inc 2017 Docker Swarm (2017) Retrieved June 26 2017 from hpsdocsdockercomswarm

[11] Eclipse Foundation Inc 2017 Xtext framework (2017) Retrieved June 26 2017from hpwwweclipseorgXtextdocumentation330 web supporthtml

[12] GigaSpaces Technologies Inc 2017 Cloudify (2017) Retrieved June 26 2017from hpcloudifyco

[13] Red Hat Inc 2017 Hibernate framework (2017) Retrieved June 26 2017 fromhphibernateorg

[14] Kyriakos Kritikos and Philippe Massonet 2016 An integrated meta-model forcloud application security modelling Procedia Computer Science 97 (2016) 84ndash93

[15] OASIS 2013 TOSCA 10 (Topology and Orchestration Specication for CloudApplications) Version 10 (2013) Retrieved June 26 2017 from hpdocsoasis-openorgtoscaTOSCAv10osTOSCA-v10-oshtml

[16] ARTIST EU project consortium 2013 Advanced soware-based service provi-sioning and migration of legacy soware (2013) Retrieved June 26 2017 fromhpwwwartist-projecteu

[17] CloudSocket EU project consortium 2015 Business and IT-Cloud Alignmentusing a Smart Socket (2015) Retrieved June 26 2017 from hpssitecloudsocketeu

[18] MUSA EU project consortium 2015 Multi-cloud Secure Applications (2015)Retrieved June 26 2017 from hpwwwmusa-projecteu

[19] PaaSage EU project consortium 2013 A Model-based Cross-Cloud developmentand deployment platform (2013) Retrieved June 26 2017 from hpwwwpaasageeu

[20] PaaSage EU project consortium 2014 Deliverable D212 CloudML Implementa-tion Documentation (2014) Retrieved June 26 2017 from hpwwwpaasageeuimagesdocumentspaasage d212 nalpdf

[21] PaaSage EU project consortium 2015 Deliverable D213 CAMEL Documentation(2015) Retrieved June 26 2017 from hpswwwpaasageeuimagesdocumentsdocsD213 CAMEL Documentationpdf

[22] PaaSage EU project consortium 2016 CAMEL Documentation v20159 (2016)Retrieved June 26 2017 from hpcamel-dslorgdocumentation

[23] SPECS EU project consortium 2015 Secure Provisioning of Cloud Servicesbased on SLA Management (2015) Retrieved June 26 2017 from hpwwwspecs-projecteu

[24] Puppet 2017 Puppet documentation (2017) Retrieved June 26 2017 fromhpsdocspuppetcompuppet

[25] Rightscale 2017 Cloud Computing Trends 2017 State of the CloudSurvey (2017) hpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2017-state-cloud-survey

[26] Erkuden Rios Eider Iturbe Leire Orue-Echevarria Massimiliano Rak ValentinaCasola and others 2015 Towards Self-Protective Multi-Cloud ApplicationsMUSAndasha Holistic Framework to Support the Security-Intelligent Lifecycle Man-agement of Multi-Cloud Applications (2015)

[27] Erkuden Rios Wissam Mallouli Massimiliano Rak Valentina Casola and Anto-nio M Ortiz 2016 SLA-driven monitoring of multi-cloud application componentsusing the MUSA framework In Distributed Computing Systems Workshops (ICD-CSW) 2016 IEEE 36th International Conference on IEEE 55ndash60

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

modelling In Section 3 we introduce the MUSA SecDevOps frame-work and workow for multi-cloud applications which gives thecontext of the work developed Section 4 details the MUSA SecurityDomain Specic Language and in Section 5 we describe its support-ing tool named MUSA Modeller included in the MUSA frameworkSection 6 describes the validation scenarios of the work presentedFinally Section 7 presents the conclusions and future work

2 MULTI-CLOUD APPLICATION SECURITYMODELLING

In the last years the cloud based infrastructure and platform servicesoering has increased notably as well as the number of providersOne of the main problems when architecting and operating (multi-)cloud applications is that cloud providers support dierent inter-faces for dierent sets of services Furthermore in order to exploitmulti-cloud potential dierent architectural models can be adoptedto enlarge security [1]

bull replication of applications deploying the same system inmore than one provider so as malicious aacks can beeasily discovered comparing operation results

bull partition of application system into tiers which allowsseparating logic from data and thus minimizing risks ofaacks or incidents in both parts at a time

bull partition of application logic into fragments in order toobfuscate the overall application logic to providers

bull partition of application data into fragments to prevent asingle provider reconstructing data safeguarding conden-tiality

Exploiting multi-cloud for higher availability and security meansthat DevOps teams in charge of developing and operating (multi-)cloud applications need to learn systematise and automate theprocess behind the provisioning of services when deploying theapplication components Among other tasks they need to learnhow to create Virtual Machines (VMs) and how to choose the rightVM sizes for each application service or component

To address these challenges current research stakeholders pro-pose to take advantage of the well-known Model Driven Engi-neering (MDE) techniques to congure the deployment of cloudapplications According to the work in [22] application models canbe specied using general-purpose languages like the Unied Mod-elling Language (UML) [8] However to fully unfold the potentialof MDE models are frequently specied using domain-specic lan-guages (DSLs) which are tailored to a specic domain of concern

e MDE techniques become really interesting for multi-cloudapplication specication when the model captures multi-concerninformation expressed at high level rst and detailed at low levelapplication platform aerwards and when the model is enacted atruntime is allows for a seamless alignment of design decisionswith actual deployment and application execution In this linework CloudML [28] and CAMEL [22] (which includes CloudML asDeployment model for expressing deployment needs) languagesCAMEL was developed as part of the research work in PaaSage[19] and CloudSocket [17] EU-funded projects

Another great exponent in the literature on cloud-based applica-tions modelling is the Topology and Orchestration Specication forCloud Applications (TOSCA) language developed by OASIS [15]

which provides a language for specifying the components com-prising the topology of cloud-based applications along with theprocesses for their orchestration Another example is the CloudApplication Modelling Language (CAML) dened in the ARTISTproject [16] which realised CloudML as a UML internal DSL basedon extensions to the deployment meta-model in terms of a libraryand proles capturing domain knowledge

Following a similar approach of that of PaaSage Security DSL[14] in MUSA project [4] that contextualises the work presentedin this paper CAMEL has been adopted in order to cope with themodelling of multi-cloud applications

CAMEL includes two main security-oriented meta-models [22]the Security meta-model to support the specication of securityrequirements posed by users and capabilities of cloud providers (inform of security controls and service level objectives) and the Or-ganisation meta-model that captures security-oriented informationabout organisations including organisation security policies usersand roles

e application requirements in CAMEL are mainly capturedby the Requirements meta-model us both the Security and theRequirements meta-models can complementarily capture securityrequirements CAMEL oers support to the following tasks [22](i) matching in deployment phase security capabilities and require-ments of the application to the security controls oered by thecloud providers (ii) monitoring and assessing security service levelobjectives (SLOs) which can be mapped to adaptation rules in orderto adapt the structure or behaviour of an application to exhibit thesecurity level required

Although both PaaSage and MUSA projects oer support to bothtasks they dier in the modelling aspects mainly because PaaSagerelies on CAMEL model enactment and MUSA follows a dierentapproach as explained in Section 3 e rationale for selectingCAMEL in our approach on top of other versions of CloudML andTOSCA is described in Section 41

3 THE MUSA SECDEVOPS FRAMEWORK FORMULTI-CLOUD APPLICATIONS

31 e MUSA frameworkMulti-cloud solutions pose new challenges when trying to add valueto overall cloud client experience [29]

e MUSA framework [26] introduced in this paper aims atensuring that the desired security and privacy levels are reached inall types of multi-cloud environments including those that combinemultiple architectural scenarios as described in Section 2

To this aim the MUSA framework combines preventive securitywith reactive security For prevention the MUSA framework sup-ports Security by Design practices in the development as well asembedding in the application the required security mechanismsthe so called MUSA Enforcement Agents which will enable the self-healing of the application in operation For reaction the MUSAframework includes monitoring of the application at runtime todetect notify and early mitigate security incidents so multi-cloudapplication providers can be informed and promptly react to secu-rity problems or aacks

In order to ensure the preventive security mechanisms to beembedded in the application components and aligned with reactive

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

Figure 1 MUSA approach and framework tools

security measures MUSA holistic framework oers a number ofmechanisms and supporting tools that seamlessly work togetheralong all phases of the application lifecycle as shown in Fig 1

32 e MUSA workowe MUSA framework is intended to provide integrated support toaddressing security aspects of multi-cloud applications in all thesteps of the engineering lifecycle as illustrated in the process ofFig 2

During the design phase the DevOps team rst models the CloudProvider Independent Model (CPIM) of the multi-cloud applicationusing the MUSA Modeller e CPIM is a MUSA extended CAMELmodel specication of the multi-cloud application in a level ofabstraction independent from specic information about the CloudService Providers (CSP) the application components will use or bedeployed in

Once the application CPIM is specied the DevOps team obtainsthe security requirements in the risk assessment step In this steptogether with the security requirements the DevOps team cananalyse other criteria as well such as business criteria and cansearch for the cloud services that best match all the criteria byrelying on the use of the MUSA Decision Support Tool (DST) [4]

Modelling risk analysis and cloud services selection are madefollowing an iterative process that allows identifying which securityrequirements (if any) were not possible to address with securitycontrols oered by the cloud services under study and thereforethe DevOps team can update the CPIM at modelling step to specifythe use of MUSA security agents that oer the pending securitycontrols (if available)

Having selected the combination of cloud services that bestmatch the requirements of the multi-cloud application and havingpreviously dened the security requirements the DevOps teamcan generate the Security SLA templates for the components ofthe multi-cloud application ese Security SLA templates will bestored in the SLA Repository and will be retrieved by the MUSADeployer so it can generate the Implementation plan for the multi-cloud application Once the Implementation plan is generated theMUSA Deployer shares it with the SLA Generator [2] so this tool

Figure 2 MUSA overall process

can generate the Composite Security SLA for the whole multi-cloudapplication

Aerwards at deployment phase the MUSA Deployer is invokedby the DevOps team in order to deploy (by following the Imple-mentation plan) both the multi-cloud application components andthe corresponding MUSA agents declared in the CPIM

Finally at runtime or operation phase the MUSA Security Assur-ance platform [27] starts monitoring the multi-cloud applicationbased on the nal SLAs and the Implementation plan

If the MUSA Security Assurance Platform detects any alert eventor violation of the SLAs in place it noties to the DevOps teamand the appropriate reactive measure is triggered e reaction tosecurity incidents in MUSA relies on dierent mechanisms depend-ing on the cause of the incident Reactive measures include there-deployment of multi-cloud application component(s) or even theapplication re-design In the re-design phase the need of includingMUSA Enforcement Agents can be evaluated again in order to tryto address the security incident detected Adaptation of multi-cloudapplications at execution is supported in MUSA by the enforcementservices in the MUSA Security Assurance Platform that remotelycontrol the conguration activation and deactivation of some ofthe MUSA Enforcement Agents We refer to future publications ofMUSA for detailed explanation of the enforcement services oered

4 MUSA SECURITY DSLe MUSA Security Domain Specic Language (DSL) builds on topof the CAMEL language and the main contributions are related toits expressiveness to dene and congure multi-cloud applicationsat design-time Such a powerful denition can be used later on byother MUSA tools to perform the risk analysis and to generate theindividual componentsrsquo security SLAs and the Composite SecuritySLA Furthermore this allows deploying and monitoring the secu-rity properties of multi-cloud applications and its components atrun-time In brief the main innovations achieved in MUSA are theimprovements to the CAMEL language the accompanying modelsyntax and semantics verication rules and the development of aWeb-based modelling tool (named MUSA Modeller) as explained inthe following e MUSA Security DSL is shown in Fig 3

41 Rationale for CAMEL language selectione main reasons for selecting CAMEL language as the basis of theMUSA security DSL are explained in the following

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

Figure 3 MUSA Security DSL

First the enactment of the multi-cloud application model atruntime is not needed in MUSA is is because in MUSA we donot rely on a single model of the multi-cloud application to captureall the aspects but on a number of models In MUSA we use theCAMEL model of the application to express its architecture securityand deployment requirements but in addition we use SecuritySLA templates of the components to capture the required securityfeatures over the cloud providers and the Composite Security SLAto express the guarantees of the overall application e CompositeSecurity SLA serves in MUSA to control the application behaviourat runtime Multi-cloud applications re-designs will be driven byactual measurements taken of the Service Level Objectives andmetrics stated in the Composite Security SLA In MUSA we also usethe Decision Support Tool data model for expressing business andsecurity requirements over the cloud providers and the deploymentImplementation plan as the cloud platform dependent deploymentmodel

Second CAMEL includes CloudML as Deployment meta-modeltherefore adopting it implies adopting CloudML too BesidesCAMEL includes also other meta-models such as Security and Re-quirements meta-models that are valuable when placing the focuson security CAMEL follows the same approach that CloudML in re-lation to the provision of a single set of abstractions and APIs so thatdevelopers can dene declaratively (i) the application architecturemade of components (ii) their usehost relationships so that theycan be properly congured and deployment orders automaticallyderived (iii) constraints on the characteristics of the required types

of VMs and (iv) the execution commands to provision applicationcomponents

At this point it is important to note that MUSA project wasborn with the objective of supporting cloud adoption by addressingcurrent security open issues since companies are reluctant to adoptcloud computing because of the diculty in evaluating the trade-o between cloud benets and the additional security risks andprivacy issues it may bring ey have to deal with the security ofthe individual components as well as with the overall applicationsecurity including the communications and the data ow betweenthe components erefore it is required to adopt a language thatallows end-users to friendly and easily create and deploy security-aware components balancing security with performance properties

As opposite to PaaSage approach where the low level security in-formation is captured in the CAMEL based Cloud Provider SpecicModel (CPSM) in MUSA it is captured and managed by two otherspecialised languages respectively the MUSA Security SLA model(extended from that of SPECS project [23] to address multi-cloudscenarios) and the MUSA Decision Support Toolrsquos Cloud ServiceProvider data model that allows dening full avoured businessand security requirements and capabilities for multi-factor servicematch-making

ird opposite to TOSCA MUSA end-users had already experi-enced with CAMEL and liked the language expressiveness becauseit was the only language that already provided rich Security modeland Requirements model (for deployment) CAMEL and TOSCA

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

basically share the same purpose but CAMEL results in being lesscomplex and easy to use

Furthermore TOSCA community did not oer advanced toolsupport at the time when MUSA solution was dened

Last but not least the feedback received from the MUSA projectend-users about previous experiences with multi-cloud modellinglanguages and tools was considered as fundamental One of theend-user teams had experienced with CAMEL and liked the ex-pressiveness of the language for multi-cloud application descrip-tion although security aspects were not considered by them previ-ously About the modelling tools they had previously used PaaSageCAMEL Textual Editor [21] and other graphical editors (such asModelio [3]) and preferred the non-graphical editor that includedfriendly functionality such as identication of required vs optionalaributes auto completion capabilities and model validation amongothers us these are exactly the features we aim to maintain inthe MUSA Modeller As conceptually both graphical and textualeditors allow for the same degree of detail and completeness wedecided to develop our MUSA Modeller as a text editor

42 Innovations for security behaviourspecications

e MUSA innovations to CAMEL language include the enhancedcomponent security behaviour characterization which addressesconcepts required to support both composition of componentsrsquosecurity SLAs and risk analysis

421 Classification of components by their nature which allowsdescribing what the component does is information is requiredby the MUSA SLA Generator to create the Composite Security SLAof the multi-cloud application from individual componentsrsquo SLAsand by the MUSA Risk Analysis tool in order to identify the secu-rity threats and risks at component level Our CAMEL extensionallows specifying the type of the component in fact the compo-nent can be classied by two features WHAT and HOW WhileWHAT indicates the type of the functionality delivered by the com-ponent HOW indicates the way the component is delivering suchfunctionality Currently three types of HOW have been dened

bull COTS which refers to Commercial o-the-shelf sowarethat the application uses

bull SERVICE meaning that the component is not a commercialpackage but developed by the DevOps team responsiblefor the multi-cloud application engineering

bull AGENT ie a MUSA Agent component provided by theMUSA framework and available in the MUSA SecurityAgent Catalogue

e types of WHAT include

bull in case the component is COTS or SERVICE the possibleWHAT values are Web Storage IDM or Firewall Webrefers to any functionality provided through a Web inter-face Storage refers to data storage solutions (eg MySQL)IDM stays for Identity Management and Firewall for anysoware solution that protects resources from unautho-rised access

bull in case the component is AGENT the possible values ofWHAT come from the list of the agents in the MUSA Secu-rity Agent Catalogue

422 Security Controls information that properly supports Se-curity Control Framework families For example name aribute hasbeen updated to ltFamilygt-ltNumbergt(Number) format In addi-tion the subdomain aribute in the Security Control entity now isoptional instead of compulsory

e CAMEL extension developed in MUSA allows specifyingwhich security capabilities are required and provided by each multi-cloud application component e security capabilities are denedin the model by selecting and grouping the security controls partof the capability e security controls of a Security Control Fam-ily are predened and the list is oered to the user by the MUSAModeller to ease the selection of the ones included in the capabilitycurrently the security controls from the NIST SP 800-53 rev4 [6]are supported In the following example two security capabilitiesCAP1 and CAP2 are dened the rst with three security controlsand the second with only two

security model SEC security capability CAP1 controls [MUSASECAC-11(1) MUSASECAU-13(2)MUSASECAC-17(6)]security capability CAP2 controls [MUSASECAC-11(1) MUSASECAC-17(6)]

Once the security capabilities are dened in the CAMEL (inthe security model part of the model) the user can specify whatsecurity capabilities the components require andor provide Inthe following example Comp1Cap is a provided capability andComp1CapReq a requested one

provided security capability Comp1Cap security capability SECCAP2required security capability Comp1CapReq

When a component requires a specic security capability fromanother component (in the example Comp1CapReq) then the match-ing of the capability needs to be modelled as follows

capability match Comp1ToComp2 from Comp1Comp1CapReq to Comp2Comp2Cap

43 Innovations for multi-cloud deploymentOther MUSA innovations to CAMEL language address improve-ments for enhancing the expressiveness of the deployment require-ments as follows

431 Explicit characterization of the nature of the IP addressassociated with virtual machines ponents will be deployed At de-ployment phase when acquiring new cloud resources such as VMs

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

the system operator needs to indicate whether a public IP address isrequired e CAMEL extension allows specifying whether the IPaddress required for a VM should be public or not by the IP publicaribute on each component e possible values for this aributeare true or false

432 Specification of the deployment order of the applicationcomponents Dealing with multi-cloud environments it is criticalto identify the order in which each component should be deployedand congured since there are inter-dependencies among the com-ponents that are part of the same application For example the startup of a component may require that previously another componentis already up and running e CAMEL extension allows specifyingthe order in which the components are required to be deployedis can be done by using the rdquoorderrdquo aribute for each componente expected value for the order aribute is an integer number

433 Explicit definition of data exchange protocols In the MUSAextended CAMEL users can model the communications betweenthe components (eg by seing the IP addresses and ports in theconguration of the components) and specify the need to use aspecic data exchange protocol (eg MySQL OAuth Other)

434 Modelling of dynamic configurations of communicationsbetween components In CAMEL users model the communicationsbetween the components in a static way (ie through specic portnumbers and operating system conguration variables) Howeverin MUSA dynamic characteristics have been introduced such ascontext paths (instead of IP addresses) and dynamic port rangesSuch new capabilities are useful for instance to congure explicitlyinbound trac when users deploy components in Docker contain-ers

435 Modelling of deployment handlers In CAMEL the usercan model components and associate deployment instructions forinstalling conguring starting and stopping the components onvirtual machines However such deployment instructions are re-stricted to scripted commands and CAMEL lacks support to thespecication of conguration management tools such as Cloudify[12] Puppet [24] or Chef [9] erefore in MUSA this gap betweenmulti-cloud application models and these advanced frameworkshas been faced via the new Conguration entity and its associatedconcepts (eg cookbooks and recipes in case of Chef)

436 Modelling of PaaS layer elements CAMEL lacks supportto the description of architectures where the application compo-nents are not directly deployed in Virtual Machines (VMs) but incontainers Our extension allows specifying the container type thatwill be used in deployment and dening the component allocationstrategy it should follow even in cases when the container usesVM pools e new elements in our extension include

bull pool is a cluster of VMs is cluster can be used bya container or directly by a component for example adatabase solution that is capable of managing a cluster

bull manager the VM in a pool that will act as the manager vsthe rest which will be workers

bull container ndash type the container solution to use for example Docker

Swarm [10]

ndash allocationStrategy denes the allocation strategy ofthe containers on top of the acquired VMs for resourceoptimization (eg automatically scheduling containerworkloads) It supports the following four values

lowast spread balance containers across the VMs in apool based on the available CPU and RAM ofthe VMs

lowast binpak shedule containers to fully use each VMcapacity Once the full VM capacity has beenused the container moves on to the next one inthe pool

lowast random choose a VM randomlylowast custom the user denes the specic VMs in

which the containers should run

437 Refinement of security aspects in Organisation User Cre-dentials and Role entities A number of enhancements to CAMELhave been made in order to manage the authorisation of dierentroles in the DevOps team to multi-cloud application deploymentexecution For instance in MUSA the types of credentials availableto authenticate a user have been extended Among other changesit has been added expiration date to Credentials and additionalparameter properties to User entity

44 Innovations for self-healing capability ofmulti-cloud applications

Considering self-healing as the capability of a multi-cloud applica-tion of being able to self-control or modify its security behaviourat runtime so as security incidents or aacks are corrected or miti-gated the self-healing is enabled in MUSA by the MUSA Enforce-ment Agents

As their name suggests the MUSA Enforcement Agents enforcemulti-cloud application security properties at runtime such as ac-cess control security vulnerability scanning or Denial of servicemitigation mechanisms For these mechanisms to work they needto be deployed at the same time as the application componentsare deployed Some of these mechanisms require to be deployedtogether with (in the same VM) the component they will enforcethe property on erefore the MUSA extended CAMEL allowsthe denition of MUSA Security Enforcement Agents as InternalComponents of the application similarly to application compo-nents themselves so as they can be included in the deploymentplan Such agents are already pre-dened in the MUSA SecurityAgent Catalogue so users are able to re-use and congure them in afriendly way Some of these agents are always on and some will bemanaged through the enforcement services in the MUSA SecurityAssurance Platform

5 MUSA MODELLER51 MUSA Modeller Architecturee MUSA Modeller is a web editor tool that allows the creationand maintenance of (MUSA extended) CAMEL models roughthese models it is possible to dene a complete specication of therequirements needed by an application to be deployed in a securemulti-cloud environment MUSA Modeller has leveraged one ofthe new capabilities of the Xtext technology [11] the Xtext Web

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

editor support erefore multi-cloud application models can beedited and updated remotely by end-users while they are storedin shared repositories All in all without any program installationand using any web browser since its JavaScript-based API allowsadding language-specic features such as auto completion and livevalidation

As the initial step of the design phase the DevOps team createsthe Cloud Provider Independent Model (CPIM) of the multi-cloudapplication using the MUSA Modeller tool e generated CPIMwill be used as input to the risk assessment the SLA generationand the multi-cloud application deployment processes

At high level the MUSA Modeller is structured as followsbull A Web component the GUI from which the end-users ac-

cess the MUSA Modeller tool and exploit all the internalweb-services oered It also includes the web Xtext li-braries that oer syntactic and sematic services for remotemanagement of syntax validation and auto completion

bull A Server component which is the core of the MUSA Mod-eller since it implements all the business functionalitiesMoreover it oers a series of web services that are con-sumed by the Web component

bull A Database component which manages all the data storedin the central database It uses the Hibernate framework[13] that allows the abstraction of relational database en-gines

52 MUSA Modeller APIAs explained in the previous chapter the Server component of theMUSA Modeller oers a series of REST API interfaces that supportthe following functionality

bull Creation of a new multi-cloud application model in (MUSAextended) CAMEL language End-users can instantiate pre-viously dened templates for particular component typesor applications

bull Denition and storage of multi-cloud application compo-nent templates for reusing CAMEL models of the compo-nents

bull Edition and storage of multi-cloud application modelswhere application models can be dened by multiple end-users

bull Model checking for syntax and semantic correctness andintegrity of the created models e tool provides messagesof warnings and errors whenever non-conformities areidentied in the model

bull the selection of Security Controls previously identiedand stored in the MUSA Security Service Level AgreementCatalogue

bull the selection of MUSA Security Enforcement Agents pre-viously identied and stored in the MUSA Security AgentCatalogue See Fig 4

6 VALIDATION IN USE CASESDuring the rst period of the MUSA project the evaluation of theMUSA solution has been performed using generic usage scenariosalong with two specic use cases one led by Luhansa Systems(LHS) and the other by Tampere University of Technology (TUT)

Figure 4 MUSA Modeller support for Security Agent selec-tion

both partners of the project ese case studies have been designedto ensure that MUSA framework evaluation takes place in dierentcontexts of multi-cloud deployment as well as diverse security andprivacy requirements

e rst use case is related to the ight scheduling applicationprototype by Luhansa Systems that is intended to be used bymultiple airlines around the world to plan airplane ight sched-ules Todayrsquos airlines need to permanently revise their scheduleplans in response to competitor actions or to follow updated salesand marketing plans while constantly maintaining operational in-tegrity is makes schedule management a very complex processe MUSA Modeller enabled LHS to specify the breakdown of themulti-cloud application into its components as well as componentsrsquosecurity capabilities is includes also the integration of the MUSAsecurity agents into the application and the design of the provi-sioning and deployment LHS has validated the MUSA Modeller tohelp architects and developers of the multi-cloud application com-ponents in the security design to gain more focus on the complexeld of security requirements analysis and enforcing the securityby design principles even for the less experienced colleagues

In the second use case the Smart Mobility application developedby Tampere University of Technology provides Tampere citizenscontext-aware energy ecient smart mobility services and recom-mendations for transportation means is case study representsthe scenario of SMEs which create applications that exploit serviceshosted in a number of clouds As many SMEs the workgroup thatdevelops this case study does not have a specialized division forcyber security Because of this the use of MUSA framework andits tools has allowed easily addressing security aspects at dierentstages of the application engineering project design deploymentand runtime

Specically in this case study the MUSA Modeller tool has beenevaluated to model the multi-cloud application components andMUSA Enforcement Agents to integrate security controls for highavailability condentiality and integrity

e added value of the MUSA modelling language and tool hasbeen already identied in the initial evaluation performed in bothcase studies e usefulness and acceptance of the approach was

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

conrmed not only in the usage of the MUSA Modeller as a stand-alone tool but more signicantly in its integration with the restof the tools in the MUSA SecDevOps framework particularly incollaboration with Risk Analysis SLA Generator and Deployertools It is expected that the nal evaluation of the MUSA solutionwill enable the validation of all the enhancements dened in theMUSA Security DSL

7 CONCLUSIONS AND FUTUREWORKAs the number of cloud oerings grows and cloud environmentsbecome more and more complex cloud consumers will need to facemulti-cloud challenges related to multiple cloud service combina-tion and holistic protection of application components deployed indistributed heterogeneous clouds

is paper presents a novel modelling language and support-ing tool that address the special needs of multi-cloud applicationsmodelling e solution overcomes the lack of expressiveness ofstate-of-the-art modelling by easing both a) the computation ofComposite Security SLAs that include security and privacy aspectsand b) the risk analysis and service match-making taking into ac-count not only functionality and business aspects of the cloudservices but also security aspects

e language and tool presented were developed in the contextof the MUSA EU-funded project on the basis of enhancements tothe CAMEL language which already provided rich RequirementsDeployment Scalability and Security meta-models that cover manyof the requirements for multi-cloud applications specication Nev-ertheless additional requirements were identied in MUSA in orderto address richer deployment and security specication risk analy-sis and Security SLA composition for which extensions to CAMELhave been developed e modelling tool supporting this extendedCAMEL meta-model the MUSA Modeller is already integratedwith the MUSA framework and available in the MUSA website atwwwmusa-projecteu

Consequently the contributions of this paper pave the way todevelop security and privacy-aware multi-cloud applications bymastering the expressiveness of security aspects in the CPIM so asthey enable integrated SecDevOps support

In the future we have identied the need to research on how toleverage the Scalability meta-model of CAMEL to dene scalabilityrules as pre-dened conguration of high availability enforcementagents aimed at scaling up (and down) internal components ofmulti-cloud applications when needed (similar to HA Proxy [30])

Another important aspect is the research on how to supportcomposability of CAMEL models required to easily combine andrefer to CAMEL models of individual components in the creation ofmulti-cloud applicationsrsquo models Furthermore this would largelyincrease the models readability and dramatically improve the toolusability

ACKNOWLEDGMENTSe MUSA project leading to this paper has received funding fromthe European Unionrsquos Horizon 2020 research and innovation pro-gramme under grant agreement No 644429 We would also like toacknowledge all the members of the MUSA Consortium for theirvaluable help

REFERENCES[1] Jens-Mahias Bohli Nils Gruschka Meiko Jensen Luigi Lo Iacono and Ninja

Marnau 2013 Security and privacy-enhancing multicloud architectures IEEETransactions on Dependable and Secure Computing 10 4 (2013) 212ndash224

[2] Valentina Casola Alessandra De Benedictis Massimiliano Rak and ErkudenRios 2016 Security-by-design in clouds a security-SLA driven methodology tobuild secure cloud applications Procedia Computer Science 97 (2016) 53ndash62

[3] Inc Eclipse Foundation 2011 Modelio e open source modelling environment(2011) Retrieved June 26 2017 from hpswwwmodelioorg

[4] Jaume Ferrarons Smrati Gupta Victor Muntes-Mulero Josep-Lluis Larriba-Peyand Peter Mahews 2016 Scoring cloud services through digital ecosystemcommunity analysis In International Conference on Electronic Commerce and WebTechnologies Springer 142ndash153

[5] Forbes 2016 Roundup Of Cloud Computing ForecastsAnd Market Estimates (2016) Retrieved June 26 2017from hpswwwforbescomsiteslouiscolumbus20160313roundup-of-cloud-computing-forecasts-and-market-estimates-20167971c3de2187

[6] JOINT TASK FORCE and TRANSFORMATION INITIATIVE 2013 Security andprivacy controls for federal information systems and organizations NIST SpecialPublication 800 53 (2013) 8ndash13

[7] Gartner 2016 Gartner Says by 2020 rdquoCloud Shirdquo Will Aect More an $1Trillion in IT Spending (2016) Retrieved June 26 2017 from hpwwwgartnercomnewsroomid3384720

[8] Object Management Group 2015 Unied Modeling Language Specication v25(2015) Retrieved June 26 2017 from hpwwwomgorgspecUML25

[9] Chef Soware Inc 2017 Chef technology (2017) Retrieved June 26 2017 fromhpswwwchefiochef

[10] Docker Inc 2017 Docker Swarm (2017) Retrieved June 26 2017 from hpsdocsdockercomswarm

[11] Eclipse Foundation Inc 2017 Xtext framework (2017) Retrieved June 26 2017from hpwwweclipseorgXtextdocumentation330 web supporthtml

[12] GigaSpaces Technologies Inc 2017 Cloudify (2017) Retrieved June 26 2017from hpcloudifyco

[13] Red Hat Inc 2017 Hibernate framework (2017) Retrieved June 26 2017 fromhphibernateorg

[14] Kyriakos Kritikos and Philippe Massonet 2016 An integrated meta-model forcloud application security modelling Procedia Computer Science 97 (2016) 84ndash93

[15] OASIS 2013 TOSCA 10 (Topology and Orchestration Specication for CloudApplications) Version 10 (2013) Retrieved June 26 2017 from hpdocsoasis-openorgtoscaTOSCAv10osTOSCA-v10-oshtml

[16] ARTIST EU project consortium 2013 Advanced soware-based service provi-sioning and migration of legacy soware (2013) Retrieved June 26 2017 fromhpwwwartist-projecteu

[17] CloudSocket EU project consortium 2015 Business and IT-Cloud Alignmentusing a Smart Socket (2015) Retrieved June 26 2017 from hpssitecloudsocketeu

[18] MUSA EU project consortium 2015 Multi-cloud Secure Applications (2015)Retrieved June 26 2017 from hpwwwmusa-projecteu

[19] PaaSage EU project consortium 2013 A Model-based Cross-Cloud developmentand deployment platform (2013) Retrieved June 26 2017 from hpwwwpaasageeu

[20] PaaSage EU project consortium 2014 Deliverable D212 CloudML Implementa-tion Documentation (2014) Retrieved June 26 2017 from hpwwwpaasageeuimagesdocumentspaasage d212 nalpdf

[21] PaaSage EU project consortium 2015 Deliverable D213 CAMEL Documentation(2015) Retrieved June 26 2017 from hpswwwpaasageeuimagesdocumentsdocsD213 CAMEL Documentationpdf

[22] PaaSage EU project consortium 2016 CAMEL Documentation v20159 (2016)Retrieved June 26 2017 from hpcamel-dslorgdocumentation

[23] SPECS EU project consortium 2015 Secure Provisioning of Cloud Servicesbased on SLA Management (2015) Retrieved June 26 2017 from hpwwwspecs-projecteu

[24] Puppet 2017 Puppet documentation (2017) Retrieved June 26 2017 fromhpsdocspuppetcompuppet

[25] Rightscale 2017 Cloud Computing Trends 2017 State of the CloudSurvey (2017) hpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2017-state-cloud-survey

[26] Erkuden Rios Eider Iturbe Leire Orue-Echevarria Massimiliano Rak ValentinaCasola and others 2015 Towards Self-Protective Multi-Cloud ApplicationsMUSAndasha Holistic Framework to Support the Security-Intelligent Lifecycle Man-agement of Multi-Cloud Applications (2015)

[27] Erkuden Rios Wissam Mallouli Massimiliano Rak Valentina Casola and Anto-nio M Ortiz 2016 SLA-driven monitoring of multi-cloud application componentsusing the MUSA framework In Distributed Computing Systems Workshops (ICD-CSW) 2016 IEEE 36th International Conference on IEEE 55ndash60

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

Figure 1 MUSA approach and framework tools

security measures MUSA holistic framework oers a number ofmechanisms and supporting tools that seamlessly work togetheralong all phases of the application lifecycle as shown in Fig 1

32 e MUSA workowe MUSA framework is intended to provide integrated support toaddressing security aspects of multi-cloud applications in all thesteps of the engineering lifecycle as illustrated in the process ofFig 2

During the design phase the DevOps team rst models the CloudProvider Independent Model (CPIM) of the multi-cloud applicationusing the MUSA Modeller e CPIM is a MUSA extended CAMELmodel specication of the multi-cloud application in a level ofabstraction independent from specic information about the CloudService Providers (CSP) the application components will use or bedeployed in

Once the application CPIM is specied the DevOps team obtainsthe security requirements in the risk assessment step In this steptogether with the security requirements the DevOps team cananalyse other criteria as well such as business criteria and cansearch for the cloud services that best match all the criteria byrelying on the use of the MUSA Decision Support Tool (DST) [4]

Modelling risk analysis and cloud services selection are madefollowing an iterative process that allows identifying which securityrequirements (if any) were not possible to address with securitycontrols oered by the cloud services under study and thereforethe DevOps team can update the CPIM at modelling step to specifythe use of MUSA security agents that oer the pending securitycontrols (if available)

Having selected the combination of cloud services that bestmatch the requirements of the multi-cloud application and havingpreviously dened the security requirements the DevOps teamcan generate the Security SLA templates for the components ofthe multi-cloud application ese Security SLA templates will bestored in the SLA Repository and will be retrieved by the MUSADeployer so it can generate the Implementation plan for the multi-cloud application Once the Implementation plan is generated theMUSA Deployer shares it with the SLA Generator [2] so this tool

Figure 2 MUSA overall process

can generate the Composite Security SLA for the whole multi-cloudapplication

Aerwards at deployment phase the MUSA Deployer is invokedby the DevOps team in order to deploy (by following the Imple-mentation plan) both the multi-cloud application components andthe corresponding MUSA agents declared in the CPIM

Finally at runtime or operation phase the MUSA Security Assur-ance platform [27] starts monitoring the multi-cloud applicationbased on the nal SLAs and the Implementation plan

If the MUSA Security Assurance Platform detects any alert eventor violation of the SLAs in place it noties to the DevOps teamand the appropriate reactive measure is triggered e reaction tosecurity incidents in MUSA relies on dierent mechanisms depend-ing on the cause of the incident Reactive measures include there-deployment of multi-cloud application component(s) or even theapplication re-design In the re-design phase the need of includingMUSA Enforcement Agents can be evaluated again in order to tryto address the security incident detected Adaptation of multi-cloudapplications at execution is supported in MUSA by the enforcementservices in the MUSA Security Assurance Platform that remotelycontrol the conguration activation and deactivation of some ofthe MUSA Enforcement Agents We refer to future publications ofMUSA for detailed explanation of the enforcement services oered

4 MUSA SECURITY DSLe MUSA Security Domain Specic Language (DSL) builds on topof the CAMEL language and the main contributions are related toits expressiveness to dene and congure multi-cloud applicationsat design-time Such a powerful denition can be used later on byother MUSA tools to perform the risk analysis and to generate theindividual componentsrsquo security SLAs and the Composite SecuritySLA Furthermore this allows deploying and monitoring the secu-rity properties of multi-cloud applications and its components atrun-time In brief the main innovations achieved in MUSA are theimprovements to the CAMEL language the accompanying modelsyntax and semantics verication rules and the development of aWeb-based modelling tool (named MUSA Modeller) as explained inthe following e MUSA Security DSL is shown in Fig 3

41 Rationale for CAMEL language selectione main reasons for selecting CAMEL language as the basis of theMUSA security DSL are explained in the following

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

Figure 3 MUSA Security DSL

First the enactment of the multi-cloud application model atruntime is not needed in MUSA is is because in MUSA we donot rely on a single model of the multi-cloud application to captureall the aspects but on a number of models In MUSA we use theCAMEL model of the application to express its architecture securityand deployment requirements but in addition we use SecuritySLA templates of the components to capture the required securityfeatures over the cloud providers and the Composite Security SLAto express the guarantees of the overall application e CompositeSecurity SLA serves in MUSA to control the application behaviourat runtime Multi-cloud applications re-designs will be driven byactual measurements taken of the Service Level Objectives andmetrics stated in the Composite Security SLA In MUSA we also usethe Decision Support Tool data model for expressing business andsecurity requirements over the cloud providers and the deploymentImplementation plan as the cloud platform dependent deploymentmodel

Second CAMEL includes CloudML as Deployment meta-modeltherefore adopting it implies adopting CloudML too BesidesCAMEL includes also other meta-models such as Security and Re-quirements meta-models that are valuable when placing the focuson security CAMEL follows the same approach that CloudML in re-lation to the provision of a single set of abstractions and APIs so thatdevelopers can dene declaratively (i) the application architecturemade of components (ii) their usehost relationships so that theycan be properly congured and deployment orders automaticallyderived (iii) constraints on the characteristics of the required types

of VMs and (iv) the execution commands to provision applicationcomponents

At this point it is important to note that MUSA project wasborn with the objective of supporting cloud adoption by addressingcurrent security open issues since companies are reluctant to adoptcloud computing because of the diculty in evaluating the trade-o between cloud benets and the additional security risks andprivacy issues it may bring ey have to deal with the security ofthe individual components as well as with the overall applicationsecurity including the communications and the data ow betweenthe components erefore it is required to adopt a language thatallows end-users to friendly and easily create and deploy security-aware components balancing security with performance properties

As opposite to PaaSage approach where the low level security in-formation is captured in the CAMEL based Cloud Provider SpecicModel (CPSM) in MUSA it is captured and managed by two otherspecialised languages respectively the MUSA Security SLA model(extended from that of SPECS project [23] to address multi-cloudscenarios) and the MUSA Decision Support Toolrsquos Cloud ServiceProvider data model that allows dening full avoured businessand security requirements and capabilities for multi-factor servicematch-making

ird opposite to TOSCA MUSA end-users had already experi-enced with CAMEL and liked the language expressiveness becauseit was the only language that already provided rich Security modeland Requirements model (for deployment) CAMEL and TOSCA

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

basically share the same purpose but CAMEL results in being lesscomplex and easy to use

Furthermore TOSCA community did not oer advanced toolsupport at the time when MUSA solution was dened

Last but not least the feedback received from the MUSA projectend-users about previous experiences with multi-cloud modellinglanguages and tools was considered as fundamental One of theend-user teams had experienced with CAMEL and liked the ex-pressiveness of the language for multi-cloud application descrip-tion although security aspects were not considered by them previ-ously About the modelling tools they had previously used PaaSageCAMEL Textual Editor [21] and other graphical editors (such asModelio [3]) and preferred the non-graphical editor that includedfriendly functionality such as identication of required vs optionalaributes auto completion capabilities and model validation amongothers us these are exactly the features we aim to maintain inthe MUSA Modeller As conceptually both graphical and textualeditors allow for the same degree of detail and completeness wedecided to develop our MUSA Modeller as a text editor

42 Innovations for security behaviourspecications

e MUSA innovations to CAMEL language include the enhancedcomponent security behaviour characterization which addressesconcepts required to support both composition of componentsrsquosecurity SLAs and risk analysis

421 Classification of components by their nature which allowsdescribing what the component does is information is requiredby the MUSA SLA Generator to create the Composite Security SLAof the multi-cloud application from individual componentsrsquo SLAsand by the MUSA Risk Analysis tool in order to identify the secu-rity threats and risks at component level Our CAMEL extensionallows specifying the type of the component in fact the compo-nent can be classied by two features WHAT and HOW WhileWHAT indicates the type of the functionality delivered by the com-ponent HOW indicates the way the component is delivering suchfunctionality Currently three types of HOW have been dened

bull COTS which refers to Commercial o-the-shelf sowarethat the application uses

bull SERVICE meaning that the component is not a commercialpackage but developed by the DevOps team responsiblefor the multi-cloud application engineering

bull AGENT ie a MUSA Agent component provided by theMUSA framework and available in the MUSA SecurityAgent Catalogue

e types of WHAT include

bull in case the component is COTS or SERVICE the possibleWHAT values are Web Storage IDM or Firewall Webrefers to any functionality provided through a Web inter-face Storage refers to data storage solutions (eg MySQL)IDM stays for Identity Management and Firewall for anysoware solution that protects resources from unautho-rised access

bull in case the component is AGENT the possible values ofWHAT come from the list of the agents in the MUSA Secu-rity Agent Catalogue

422 Security Controls information that properly supports Se-curity Control Framework families For example name aribute hasbeen updated to ltFamilygt-ltNumbergt(Number) format In addi-tion the subdomain aribute in the Security Control entity now isoptional instead of compulsory

e CAMEL extension developed in MUSA allows specifyingwhich security capabilities are required and provided by each multi-cloud application component e security capabilities are denedin the model by selecting and grouping the security controls partof the capability e security controls of a Security Control Fam-ily are predened and the list is oered to the user by the MUSAModeller to ease the selection of the ones included in the capabilitycurrently the security controls from the NIST SP 800-53 rev4 [6]are supported In the following example two security capabilitiesCAP1 and CAP2 are dened the rst with three security controlsand the second with only two

security model SEC security capability CAP1 controls [MUSASECAC-11(1) MUSASECAU-13(2)MUSASECAC-17(6)]security capability CAP2 controls [MUSASECAC-11(1) MUSASECAC-17(6)]

Once the security capabilities are dened in the CAMEL (inthe security model part of the model) the user can specify whatsecurity capabilities the components require andor provide Inthe following example Comp1Cap is a provided capability andComp1CapReq a requested one

provided security capability Comp1Cap security capability SECCAP2required security capability Comp1CapReq

When a component requires a specic security capability fromanother component (in the example Comp1CapReq) then the match-ing of the capability needs to be modelled as follows

capability match Comp1ToComp2 from Comp1Comp1CapReq to Comp2Comp2Cap

43 Innovations for multi-cloud deploymentOther MUSA innovations to CAMEL language address improve-ments for enhancing the expressiveness of the deployment require-ments as follows

431 Explicit characterization of the nature of the IP addressassociated with virtual machines ponents will be deployed At de-ployment phase when acquiring new cloud resources such as VMs

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

the system operator needs to indicate whether a public IP address isrequired e CAMEL extension allows specifying whether the IPaddress required for a VM should be public or not by the IP publicaribute on each component e possible values for this aributeare true or false

432 Specification of the deployment order of the applicationcomponents Dealing with multi-cloud environments it is criticalto identify the order in which each component should be deployedand congured since there are inter-dependencies among the com-ponents that are part of the same application For example the startup of a component may require that previously another componentis already up and running e CAMEL extension allows specifyingthe order in which the components are required to be deployedis can be done by using the rdquoorderrdquo aribute for each componente expected value for the order aribute is an integer number

433 Explicit definition of data exchange protocols In the MUSAextended CAMEL users can model the communications betweenthe components (eg by seing the IP addresses and ports in theconguration of the components) and specify the need to use aspecic data exchange protocol (eg MySQL OAuth Other)

434 Modelling of dynamic configurations of communicationsbetween components In CAMEL users model the communicationsbetween the components in a static way (ie through specic portnumbers and operating system conguration variables) Howeverin MUSA dynamic characteristics have been introduced such ascontext paths (instead of IP addresses) and dynamic port rangesSuch new capabilities are useful for instance to congure explicitlyinbound trac when users deploy components in Docker contain-ers

435 Modelling of deployment handlers In CAMEL the usercan model components and associate deployment instructions forinstalling conguring starting and stopping the components onvirtual machines However such deployment instructions are re-stricted to scripted commands and CAMEL lacks support to thespecication of conguration management tools such as Cloudify[12] Puppet [24] or Chef [9] erefore in MUSA this gap betweenmulti-cloud application models and these advanced frameworkshas been faced via the new Conguration entity and its associatedconcepts (eg cookbooks and recipes in case of Chef)

436 Modelling of PaaS layer elements CAMEL lacks supportto the description of architectures where the application compo-nents are not directly deployed in Virtual Machines (VMs) but incontainers Our extension allows specifying the container type thatwill be used in deployment and dening the component allocationstrategy it should follow even in cases when the container usesVM pools e new elements in our extension include

bull pool is a cluster of VMs is cluster can be used bya container or directly by a component for example adatabase solution that is capable of managing a cluster

bull manager the VM in a pool that will act as the manager vsthe rest which will be workers

bull container ndash type the container solution to use for example Docker

Swarm [10]

ndash allocationStrategy denes the allocation strategy ofthe containers on top of the acquired VMs for resourceoptimization (eg automatically scheduling containerworkloads) It supports the following four values

lowast spread balance containers across the VMs in apool based on the available CPU and RAM ofthe VMs

lowast binpak shedule containers to fully use each VMcapacity Once the full VM capacity has beenused the container moves on to the next one inthe pool

lowast random choose a VM randomlylowast custom the user denes the specic VMs in

which the containers should run

437 Refinement of security aspects in Organisation User Cre-dentials and Role entities A number of enhancements to CAMELhave been made in order to manage the authorisation of dierentroles in the DevOps team to multi-cloud application deploymentexecution For instance in MUSA the types of credentials availableto authenticate a user have been extended Among other changesit has been added expiration date to Credentials and additionalparameter properties to User entity

44 Innovations for self-healing capability ofmulti-cloud applications

Considering self-healing as the capability of a multi-cloud applica-tion of being able to self-control or modify its security behaviourat runtime so as security incidents or aacks are corrected or miti-gated the self-healing is enabled in MUSA by the MUSA Enforce-ment Agents

As their name suggests the MUSA Enforcement Agents enforcemulti-cloud application security properties at runtime such as ac-cess control security vulnerability scanning or Denial of servicemitigation mechanisms For these mechanisms to work they needto be deployed at the same time as the application componentsare deployed Some of these mechanisms require to be deployedtogether with (in the same VM) the component they will enforcethe property on erefore the MUSA extended CAMEL allowsthe denition of MUSA Security Enforcement Agents as InternalComponents of the application similarly to application compo-nents themselves so as they can be included in the deploymentplan Such agents are already pre-dened in the MUSA SecurityAgent Catalogue so users are able to re-use and congure them in afriendly way Some of these agents are always on and some will bemanaged through the enforcement services in the MUSA SecurityAssurance Platform

5 MUSA MODELLER51 MUSA Modeller Architecturee MUSA Modeller is a web editor tool that allows the creationand maintenance of (MUSA extended) CAMEL models roughthese models it is possible to dene a complete specication of therequirements needed by an application to be deployed in a securemulti-cloud environment MUSA Modeller has leveraged one ofthe new capabilities of the Xtext technology [11] the Xtext Web

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

editor support erefore multi-cloud application models can beedited and updated remotely by end-users while they are storedin shared repositories All in all without any program installationand using any web browser since its JavaScript-based API allowsadding language-specic features such as auto completion and livevalidation

As the initial step of the design phase the DevOps team createsthe Cloud Provider Independent Model (CPIM) of the multi-cloudapplication using the MUSA Modeller tool e generated CPIMwill be used as input to the risk assessment the SLA generationand the multi-cloud application deployment processes

At high level the MUSA Modeller is structured as followsbull A Web component the GUI from which the end-users ac-

cess the MUSA Modeller tool and exploit all the internalweb-services oered It also includes the web Xtext li-braries that oer syntactic and sematic services for remotemanagement of syntax validation and auto completion

bull A Server component which is the core of the MUSA Mod-eller since it implements all the business functionalitiesMoreover it oers a series of web services that are con-sumed by the Web component

bull A Database component which manages all the data storedin the central database It uses the Hibernate framework[13] that allows the abstraction of relational database en-gines

52 MUSA Modeller APIAs explained in the previous chapter the Server component of theMUSA Modeller oers a series of REST API interfaces that supportthe following functionality

bull Creation of a new multi-cloud application model in (MUSAextended) CAMEL language End-users can instantiate pre-viously dened templates for particular component typesor applications

bull Denition and storage of multi-cloud application compo-nent templates for reusing CAMEL models of the compo-nents

bull Edition and storage of multi-cloud application modelswhere application models can be dened by multiple end-users

bull Model checking for syntax and semantic correctness andintegrity of the created models e tool provides messagesof warnings and errors whenever non-conformities areidentied in the model

bull the selection of Security Controls previously identiedand stored in the MUSA Security Service Level AgreementCatalogue

bull the selection of MUSA Security Enforcement Agents pre-viously identied and stored in the MUSA Security AgentCatalogue See Fig 4

6 VALIDATION IN USE CASESDuring the rst period of the MUSA project the evaluation of theMUSA solution has been performed using generic usage scenariosalong with two specic use cases one led by Luhansa Systems(LHS) and the other by Tampere University of Technology (TUT)

Figure 4 MUSA Modeller support for Security Agent selec-tion

both partners of the project ese case studies have been designedto ensure that MUSA framework evaluation takes place in dierentcontexts of multi-cloud deployment as well as diverse security andprivacy requirements

e rst use case is related to the ight scheduling applicationprototype by Luhansa Systems that is intended to be used bymultiple airlines around the world to plan airplane ight sched-ules Todayrsquos airlines need to permanently revise their scheduleplans in response to competitor actions or to follow updated salesand marketing plans while constantly maintaining operational in-tegrity is makes schedule management a very complex processe MUSA Modeller enabled LHS to specify the breakdown of themulti-cloud application into its components as well as componentsrsquosecurity capabilities is includes also the integration of the MUSAsecurity agents into the application and the design of the provi-sioning and deployment LHS has validated the MUSA Modeller tohelp architects and developers of the multi-cloud application com-ponents in the security design to gain more focus on the complexeld of security requirements analysis and enforcing the securityby design principles even for the less experienced colleagues

In the second use case the Smart Mobility application developedby Tampere University of Technology provides Tampere citizenscontext-aware energy ecient smart mobility services and recom-mendations for transportation means is case study representsthe scenario of SMEs which create applications that exploit serviceshosted in a number of clouds As many SMEs the workgroup thatdevelops this case study does not have a specialized division forcyber security Because of this the use of MUSA framework andits tools has allowed easily addressing security aspects at dierentstages of the application engineering project design deploymentand runtime

Specically in this case study the MUSA Modeller tool has beenevaluated to model the multi-cloud application components andMUSA Enforcement Agents to integrate security controls for highavailability condentiality and integrity

e added value of the MUSA modelling language and tool hasbeen already identied in the initial evaluation performed in bothcase studies e usefulness and acceptance of the approach was

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

conrmed not only in the usage of the MUSA Modeller as a stand-alone tool but more signicantly in its integration with the restof the tools in the MUSA SecDevOps framework particularly incollaboration with Risk Analysis SLA Generator and Deployertools It is expected that the nal evaluation of the MUSA solutionwill enable the validation of all the enhancements dened in theMUSA Security DSL

7 CONCLUSIONS AND FUTUREWORKAs the number of cloud oerings grows and cloud environmentsbecome more and more complex cloud consumers will need to facemulti-cloud challenges related to multiple cloud service combina-tion and holistic protection of application components deployed indistributed heterogeneous clouds

is paper presents a novel modelling language and support-ing tool that address the special needs of multi-cloud applicationsmodelling e solution overcomes the lack of expressiveness ofstate-of-the-art modelling by easing both a) the computation ofComposite Security SLAs that include security and privacy aspectsand b) the risk analysis and service match-making taking into ac-count not only functionality and business aspects of the cloudservices but also security aspects

e language and tool presented were developed in the contextof the MUSA EU-funded project on the basis of enhancements tothe CAMEL language which already provided rich RequirementsDeployment Scalability and Security meta-models that cover manyof the requirements for multi-cloud applications specication Nev-ertheless additional requirements were identied in MUSA in orderto address richer deployment and security specication risk analy-sis and Security SLA composition for which extensions to CAMELhave been developed e modelling tool supporting this extendedCAMEL meta-model the MUSA Modeller is already integratedwith the MUSA framework and available in the MUSA website atwwwmusa-projecteu

Consequently the contributions of this paper pave the way todevelop security and privacy-aware multi-cloud applications bymastering the expressiveness of security aspects in the CPIM so asthey enable integrated SecDevOps support

In the future we have identied the need to research on how toleverage the Scalability meta-model of CAMEL to dene scalabilityrules as pre-dened conguration of high availability enforcementagents aimed at scaling up (and down) internal components ofmulti-cloud applications when needed (similar to HA Proxy [30])

Another important aspect is the research on how to supportcomposability of CAMEL models required to easily combine andrefer to CAMEL models of individual components in the creation ofmulti-cloud applicationsrsquo models Furthermore this would largelyincrease the models readability and dramatically improve the toolusability

ACKNOWLEDGMENTSe MUSA project leading to this paper has received funding fromthe European Unionrsquos Horizon 2020 research and innovation pro-gramme under grant agreement No 644429 We would also like toacknowledge all the members of the MUSA Consortium for theirvaluable help

REFERENCES[1] Jens-Mahias Bohli Nils Gruschka Meiko Jensen Luigi Lo Iacono and Ninja

Marnau 2013 Security and privacy-enhancing multicloud architectures IEEETransactions on Dependable and Secure Computing 10 4 (2013) 212ndash224

[2] Valentina Casola Alessandra De Benedictis Massimiliano Rak and ErkudenRios 2016 Security-by-design in clouds a security-SLA driven methodology tobuild secure cloud applications Procedia Computer Science 97 (2016) 53ndash62

[3] Inc Eclipse Foundation 2011 Modelio e open source modelling environment(2011) Retrieved June 26 2017 from hpswwwmodelioorg

[4] Jaume Ferrarons Smrati Gupta Victor Muntes-Mulero Josep-Lluis Larriba-Peyand Peter Mahews 2016 Scoring cloud services through digital ecosystemcommunity analysis In International Conference on Electronic Commerce and WebTechnologies Springer 142ndash153

[5] Forbes 2016 Roundup Of Cloud Computing ForecastsAnd Market Estimates (2016) Retrieved June 26 2017from hpswwwforbescomsiteslouiscolumbus20160313roundup-of-cloud-computing-forecasts-and-market-estimates-20167971c3de2187

[6] JOINT TASK FORCE and TRANSFORMATION INITIATIVE 2013 Security andprivacy controls for federal information systems and organizations NIST SpecialPublication 800 53 (2013) 8ndash13

[7] Gartner 2016 Gartner Says by 2020 rdquoCloud Shirdquo Will Aect More an $1Trillion in IT Spending (2016) Retrieved June 26 2017 from hpwwwgartnercomnewsroomid3384720

[8] Object Management Group 2015 Unied Modeling Language Specication v25(2015) Retrieved June 26 2017 from hpwwwomgorgspecUML25

[9] Chef Soware Inc 2017 Chef technology (2017) Retrieved June 26 2017 fromhpswwwchefiochef

[10] Docker Inc 2017 Docker Swarm (2017) Retrieved June 26 2017 from hpsdocsdockercomswarm

[11] Eclipse Foundation Inc 2017 Xtext framework (2017) Retrieved June 26 2017from hpwwweclipseorgXtextdocumentation330 web supporthtml

[12] GigaSpaces Technologies Inc 2017 Cloudify (2017) Retrieved June 26 2017from hpcloudifyco

[13] Red Hat Inc 2017 Hibernate framework (2017) Retrieved June 26 2017 fromhphibernateorg

[14] Kyriakos Kritikos and Philippe Massonet 2016 An integrated meta-model forcloud application security modelling Procedia Computer Science 97 (2016) 84ndash93

[15] OASIS 2013 TOSCA 10 (Topology and Orchestration Specication for CloudApplications) Version 10 (2013) Retrieved June 26 2017 from hpdocsoasis-openorgtoscaTOSCAv10osTOSCA-v10-oshtml

[16] ARTIST EU project consortium 2013 Advanced soware-based service provi-sioning and migration of legacy soware (2013) Retrieved June 26 2017 fromhpwwwartist-projecteu

[17] CloudSocket EU project consortium 2015 Business and IT-Cloud Alignmentusing a Smart Socket (2015) Retrieved June 26 2017 from hpssitecloudsocketeu

[18] MUSA EU project consortium 2015 Multi-cloud Secure Applications (2015)Retrieved June 26 2017 from hpwwwmusa-projecteu

[19] PaaSage EU project consortium 2013 A Model-based Cross-Cloud developmentand deployment platform (2013) Retrieved June 26 2017 from hpwwwpaasageeu

[20] PaaSage EU project consortium 2014 Deliverable D212 CloudML Implementa-tion Documentation (2014) Retrieved June 26 2017 from hpwwwpaasageeuimagesdocumentspaasage d212 nalpdf

[21] PaaSage EU project consortium 2015 Deliverable D213 CAMEL Documentation(2015) Retrieved June 26 2017 from hpswwwpaasageeuimagesdocumentsdocsD213 CAMEL Documentationpdf

[22] PaaSage EU project consortium 2016 CAMEL Documentation v20159 (2016)Retrieved June 26 2017 from hpcamel-dslorgdocumentation

[23] SPECS EU project consortium 2015 Secure Provisioning of Cloud Servicesbased on SLA Management (2015) Retrieved June 26 2017 from hpwwwspecs-projecteu

[24] Puppet 2017 Puppet documentation (2017) Retrieved June 26 2017 fromhpsdocspuppetcompuppet

[25] Rightscale 2017 Cloud Computing Trends 2017 State of the CloudSurvey (2017) hpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2017-state-cloud-survey

[26] Erkuden Rios Eider Iturbe Leire Orue-Echevarria Massimiliano Rak ValentinaCasola and others 2015 Towards Self-Protective Multi-Cloud ApplicationsMUSAndasha Holistic Framework to Support the Security-Intelligent Lifecycle Man-agement of Multi-Cloud Applications (2015)

[27] Erkuden Rios Wissam Mallouli Massimiliano Rak Valentina Casola and Anto-nio M Ortiz 2016 SLA-driven monitoring of multi-cloud application componentsusing the MUSA framework In Distributed Computing Systems Workshops (ICD-CSW) 2016 IEEE 36th International Conference on IEEE 55ndash60

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

Figure 3 MUSA Security DSL

First the enactment of the multi-cloud application model atruntime is not needed in MUSA is is because in MUSA we donot rely on a single model of the multi-cloud application to captureall the aspects but on a number of models In MUSA we use theCAMEL model of the application to express its architecture securityand deployment requirements but in addition we use SecuritySLA templates of the components to capture the required securityfeatures over the cloud providers and the Composite Security SLAto express the guarantees of the overall application e CompositeSecurity SLA serves in MUSA to control the application behaviourat runtime Multi-cloud applications re-designs will be driven byactual measurements taken of the Service Level Objectives andmetrics stated in the Composite Security SLA In MUSA we also usethe Decision Support Tool data model for expressing business andsecurity requirements over the cloud providers and the deploymentImplementation plan as the cloud platform dependent deploymentmodel

Second CAMEL includes CloudML as Deployment meta-modeltherefore adopting it implies adopting CloudML too BesidesCAMEL includes also other meta-models such as Security and Re-quirements meta-models that are valuable when placing the focuson security CAMEL follows the same approach that CloudML in re-lation to the provision of a single set of abstractions and APIs so thatdevelopers can dene declaratively (i) the application architecturemade of components (ii) their usehost relationships so that theycan be properly congured and deployment orders automaticallyderived (iii) constraints on the characteristics of the required types

of VMs and (iv) the execution commands to provision applicationcomponents

At this point it is important to note that MUSA project wasborn with the objective of supporting cloud adoption by addressingcurrent security open issues since companies are reluctant to adoptcloud computing because of the diculty in evaluating the trade-o between cloud benets and the additional security risks andprivacy issues it may bring ey have to deal with the security ofthe individual components as well as with the overall applicationsecurity including the communications and the data ow betweenthe components erefore it is required to adopt a language thatallows end-users to friendly and easily create and deploy security-aware components balancing security with performance properties

As opposite to PaaSage approach where the low level security in-formation is captured in the CAMEL based Cloud Provider SpecicModel (CPSM) in MUSA it is captured and managed by two otherspecialised languages respectively the MUSA Security SLA model(extended from that of SPECS project [23] to address multi-cloudscenarios) and the MUSA Decision Support Toolrsquos Cloud ServiceProvider data model that allows dening full avoured businessand security requirements and capabilities for multi-factor servicematch-making

ird opposite to TOSCA MUSA end-users had already experi-enced with CAMEL and liked the language expressiveness becauseit was the only language that already provided rich Security modeland Requirements model (for deployment) CAMEL and TOSCA

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

basically share the same purpose but CAMEL results in being lesscomplex and easy to use

Furthermore TOSCA community did not oer advanced toolsupport at the time when MUSA solution was dened

Last but not least the feedback received from the MUSA projectend-users about previous experiences with multi-cloud modellinglanguages and tools was considered as fundamental One of theend-user teams had experienced with CAMEL and liked the ex-pressiveness of the language for multi-cloud application descrip-tion although security aspects were not considered by them previ-ously About the modelling tools they had previously used PaaSageCAMEL Textual Editor [21] and other graphical editors (such asModelio [3]) and preferred the non-graphical editor that includedfriendly functionality such as identication of required vs optionalaributes auto completion capabilities and model validation amongothers us these are exactly the features we aim to maintain inthe MUSA Modeller As conceptually both graphical and textualeditors allow for the same degree of detail and completeness wedecided to develop our MUSA Modeller as a text editor

42 Innovations for security behaviourspecications

e MUSA innovations to CAMEL language include the enhancedcomponent security behaviour characterization which addressesconcepts required to support both composition of componentsrsquosecurity SLAs and risk analysis

421 Classification of components by their nature which allowsdescribing what the component does is information is requiredby the MUSA SLA Generator to create the Composite Security SLAof the multi-cloud application from individual componentsrsquo SLAsand by the MUSA Risk Analysis tool in order to identify the secu-rity threats and risks at component level Our CAMEL extensionallows specifying the type of the component in fact the compo-nent can be classied by two features WHAT and HOW WhileWHAT indicates the type of the functionality delivered by the com-ponent HOW indicates the way the component is delivering suchfunctionality Currently three types of HOW have been dened

bull COTS which refers to Commercial o-the-shelf sowarethat the application uses

bull SERVICE meaning that the component is not a commercialpackage but developed by the DevOps team responsiblefor the multi-cloud application engineering

bull AGENT ie a MUSA Agent component provided by theMUSA framework and available in the MUSA SecurityAgent Catalogue

e types of WHAT include

bull in case the component is COTS or SERVICE the possibleWHAT values are Web Storage IDM or Firewall Webrefers to any functionality provided through a Web inter-face Storage refers to data storage solutions (eg MySQL)IDM stays for Identity Management and Firewall for anysoware solution that protects resources from unautho-rised access

bull in case the component is AGENT the possible values ofWHAT come from the list of the agents in the MUSA Secu-rity Agent Catalogue

422 Security Controls information that properly supports Se-curity Control Framework families For example name aribute hasbeen updated to ltFamilygt-ltNumbergt(Number) format In addi-tion the subdomain aribute in the Security Control entity now isoptional instead of compulsory

e CAMEL extension developed in MUSA allows specifyingwhich security capabilities are required and provided by each multi-cloud application component e security capabilities are denedin the model by selecting and grouping the security controls partof the capability e security controls of a Security Control Fam-ily are predened and the list is oered to the user by the MUSAModeller to ease the selection of the ones included in the capabilitycurrently the security controls from the NIST SP 800-53 rev4 [6]are supported In the following example two security capabilitiesCAP1 and CAP2 are dened the rst with three security controlsand the second with only two

security model SEC security capability CAP1 controls [MUSASECAC-11(1) MUSASECAU-13(2)MUSASECAC-17(6)]security capability CAP2 controls [MUSASECAC-11(1) MUSASECAC-17(6)]

Once the security capabilities are dened in the CAMEL (inthe security model part of the model) the user can specify whatsecurity capabilities the components require andor provide Inthe following example Comp1Cap is a provided capability andComp1CapReq a requested one

provided security capability Comp1Cap security capability SECCAP2required security capability Comp1CapReq

When a component requires a specic security capability fromanother component (in the example Comp1CapReq) then the match-ing of the capability needs to be modelled as follows

capability match Comp1ToComp2 from Comp1Comp1CapReq to Comp2Comp2Cap

43 Innovations for multi-cloud deploymentOther MUSA innovations to CAMEL language address improve-ments for enhancing the expressiveness of the deployment require-ments as follows

431 Explicit characterization of the nature of the IP addressassociated with virtual machines ponents will be deployed At de-ployment phase when acquiring new cloud resources such as VMs

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

the system operator needs to indicate whether a public IP address isrequired e CAMEL extension allows specifying whether the IPaddress required for a VM should be public or not by the IP publicaribute on each component e possible values for this aributeare true or false

432 Specification of the deployment order of the applicationcomponents Dealing with multi-cloud environments it is criticalto identify the order in which each component should be deployedand congured since there are inter-dependencies among the com-ponents that are part of the same application For example the startup of a component may require that previously another componentis already up and running e CAMEL extension allows specifyingthe order in which the components are required to be deployedis can be done by using the rdquoorderrdquo aribute for each componente expected value for the order aribute is an integer number

433 Explicit definition of data exchange protocols In the MUSAextended CAMEL users can model the communications betweenthe components (eg by seing the IP addresses and ports in theconguration of the components) and specify the need to use aspecic data exchange protocol (eg MySQL OAuth Other)

434 Modelling of dynamic configurations of communicationsbetween components In CAMEL users model the communicationsbetween the components in a static way (ie through specic portnumbers and operating system conguration variables) Howeverin MUSA dynamic characteristics have been introduced such ascontext paths (instead of IP addresses) and dynamic port rangesSuch new capabilities are useful for instance to congure explicitlyinbound trac when users deploy components in Docker contain-ers

435 Modelling of deployment handlers In CAMEL the usercan model components and associate deployment instructions forinstalling conguring starting and stopping the components onvirtual machines However such deployment instructions are re-stricted to scripted commands and CAMEL lacks support to thespecication of conguration management tools such as Cloudify[12] Puppet [24] or Chef [9] erefore in MUSA this gap betweenmulti-cloud application models and these advanced frameworkshas been faced via the new Conguration entity and its associatedconcepts (eg cookbooks and recipes in case of Chef)

436 Modelling of PaaS layer elements CAMEL lacks supportto the description of architectures where the application compo-nents are not directly deployed in Virtual Machines (VMs) but incontainers Our extension allows specifying the container type thatwill be used in deployment and dening the component allocationstrategy it should follow even in cases when the container usesVM pools e new elements in our extension include

bull pool is a cluster of VMs is cluster can be used bya container or directly by a component for example adatabase solution that is capable of managing a cluster

bull manager the VM in a pool that will act as the manager vsthe rest which will be workers

bull container ndash type the container solution to use for example Docker

Swarm [10]

ndash allocationStrategy denes the allocation strategy ofthe containers on top of the acquired VMs for resourceoptimization (eg automatically scheduling containerworkloads) It supports the following four values

lowast spread balance containers across the VMs in apool based on the available CPU and RAM ofthe VMs

lowast binpak shedule containers to fully use each VMcapacity Once the full VM capacity has beenused the container moves on to the next one inthe pool

lowast random choose a VM randomlylowast custom the user denes the specic VMs in

which the containers should run

437 Refinement of security aspects in Organisation User Cre-dentials and Role entities A number of enhancements to CAMELhave been made in order to manage the authorisation of dierentroles in the DevOps team to multi-cloud application deploymentexecution For instance in MUSA the types of credentials availableto authenticate a user have been extended Among other changesit has been added expiration date to Credentials and additionalparameter properties to User entity

44 Innovations for self-healing capability ofmulti-cloud applications

Considering self-healing as the capability of a multi-cloud applica-tion of being able to self-control or modify its security behaviourat runtime so as security incidents or aacks are corrected or miti-gated the self-healing is enabled in MUSA by the MUSA Enforce-ment Agents

As their name suggests the MUSA Enforcement Agents enforcemulti-cloud application security properties at runtime such as ac-cess control security vulnerability scanning or Denial of servicemitigation mechanisms For these mechanisms to work they needto be deployed at the same time as the application componentsare deployed Some of these mechanisms require to be deployedtogether with (in the same VM) the component they will enforcethe property on erefore the MUSA extended CAMEL allowsthe denition of MUSA Security Enforcement Agents as InternalComponents of the application similarly to application compo-nents themselves so as they can be included in the deploymentplan Such agents are already pre-dened in the MUSA SecurityAgent Catalogue so users are able to re-use and congure them in afriendly way Some of these agents are always on and some will bemanaged through the enforcement services in the MUSA SecurityAssurance Platform

5 MUSA MODELLER51 MUSA Modeller Architecturee MUSA Modeller is a web editor tool that allows the creationand maintenance of (MUSA extended) CAMEL models roughthese models it is possible to dene a complete specication of therequirements needed by an application to be deployed in a securemulti-cloud environment MUSA Modeller has leveraged one ofthe new capabilities of the Xtext technology [11] the Xtext Web

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

editor support erefore multi-cloud application models can beedited and updated remotely by end-users while they are storedin shared repositories All in all without any program installationand using any web browser since its JavaScript-based API allowsadding language-specic features such as auto completion and livevalidation

As the initial step of the design phase the DevOps team createsthe Cloud Provider Independent Model (CPIM) of the multi-cloudapplication using the MUSA Modeller tool e generated CPIMwill be used as input to the risk assessment the SLA generationand the multi-cloud application deployment processes

At high level the MUSA Modeller is structured as followsbull A Web component the GUI from which the end-users ac-

cess the MUSA Modeller tool and exploit all the internalweb-services oered It also includes the web Xtext li-braries that oer syntactic and sematic services for remotemanagement of syntax validation and auto completion

bull A Server component which is the core of the MUSA Mod-eller since it implements all the business functionalitiesMoreover it oers a series of web services that are con-sumed by the Web component

bull A Database component which manages all the data storedin the central database It uses the Hibernate framework[13] that allows the abstraction of relational database en-gines

52 MUSA Modeller APIAs explained in the previous chapter the Server component of theMUSA Modeller oers a series of REST API interfaces that supportthe following functionality

bull Creation of a new multi-cloud application model in (MUSAextended) CAMEL language End-users can instantiate pre-viously dened templates for particular component typesor applications

bull Denition and storage of multi-cloud application compo-nent templates for reusing CAMEL models of the compo-nents

bull Edition and storage of multi-cloud application modelswhere application models can be dened by multiple end-users

bull Model checking for syntax and semantic correctness andintegrity of the created models e tool provides messagesof warnings and errors whenever non-conformities areidentied in the model

bull the selection of Security Controls previously identiedand stored in the MUSA Security Service Level AgreementCatalogue

bull the selection of MUSA Security Enforcement Agents pre-viously identied and stored in the MUSA Security AgentCatalogue See Fig 4

6 VALIDATION IN USE CASESDuring the rst period of the MUSA project the evaluation of theMUSA solution has been performed using generic usage scenariosalong with two specic use cases one led by Luhansa Systems(LHS) and the other by Tampere University of Technology (TUT)

Figure 4 MUSA Modeller support for Security Agent selec-tion

both partners of the project ese case studies have been designedto ensure that MUSA framework evaluation takes place in dierentcontexts of multi-cloud deployment as well as diverse security andprivacy requirements

e rst use case is related to the ight scheduling applicationprototype by Luhansa Systems that is intended to be used bymultiple airlines around the world to plan airplane ight sched-ules Todayrsquos airlines need to permanently revise their scheduleplans in response to competitor actions or to follow updated salesand marketing plans while constantly maintaining operational in-tegrity is makes schedule management a very complex processe MUSA Modeller enabled LHS to specify the breakdown of themulti-cloud application into its components as well as componentsrsquosecurity capabilities is includes also the integration of the MUSAsecurity agents into the application and the design of the provi-sioning and deployment LHS has validated the MUSA Modeller tohelp architects and developers of the multi-cloud application com-ponents in the security design to gain more focus on the complexeld of security requirements analysis and enforcing the securityby design principles even for the less experienced colleagues

In the second use case the Smart Mobility application developedby Tampere University of Technology provides Tampere citizenscontext-aware energy ecient smart mobility services and recom-mendations for transportation means is case study representsthe scenario of SMEs which create applications that exploit serviceshosted in a number of clouds As many SMEs the workgroup thatdevelops this case study does not have a specialized division forcyber security Because of this the use of MUSA framework andits tools has allowed easily addressing security aspects at dierentstages of the application engineering project design deploymentand runtime

Specically in this case study the MUSA Modeller tool has beenevaluated to model the multi-cloud application components andMUSA Enforcement Agents to integrate security controls for highavailability condentiality and integrity

e added value of the MUSA modelling language and tool hasbeen already identied in the initial evaluation performed in bothcase studies e usefulness and acceptance of the approach was

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

conrmed not only in the usage of the MUSA Modeller as a stand-alone tool but more signicantly in its integration with the restof the tools in the MUSA SecDevOps framework particularly incollaboration with Risk Analysis SLA Generator and Deployertools It is expected that the nal evaluation of the MUSA solutionwill enable the validation of all the enhancements dened in theMUSA Security DSL

7 CONCLUSIONS AND FUTUREWORKAs the number of cloud oerings grows and cloud environmentsbecome more and more complex cloud consumers will need to facemulti-cloud challenges related to multiple cloud service combina-tion and holistic protection of application components deployed indistributed heterogeneous clouds

is paper presents a novel modelling language and support-ing tool that address the special needs of multi-cloud applicationsmodelling e solution overcomes the lack of expressiveness ofstate-of-the-art modelling by easing both a) the computation ofComposite Security SLAs that include security and privacy aspectsand b) the risk analysis and service match-making taking into ac-count not only functionality and business aspects of the cloudservices but also security aspects

e language and tool presented were developed in the contextof the MUSA EU-funded project on the basis of enhancements tothe CAMEL language which already provided rich RequirementsDeployment Scalability and Security meta-models that cover manyof the requirements for multi-cloud applications specication Nev-ertheless additional requirements were identied in MUSA in orderto address richer deployment and security specication risk analy-sis and Security SLA composition for which extensions to CAMELhave been developed e modelling tool supporting this extendedCAMEL meta-model the MUSA Modeller is already integratedwith the MUSA framework and available in the MUSA website atwwwmusa-projecteu

Consequently the contributions of this paper pave the way todevelop security and privacy-aware multi-cloud applications bymastering the expressiveness of security aspects in the CPIM so asthey enable integrated SecDevOps support

In the future we have identied the need to research on how toleverage the Scalability meta-model of CAMEL to dene scalabilityrules as pre-dened conguration of high availability enforcementagents aimed at scaling up (and down) internal components ofmulti-cloud applications when needed (similar to HA Proxy [30])

Another important aspect is the research on how to supportcomposability of CAMEL models required to easily combine andrefer to CAMEL models of individual components in the creation ofmulti-cloud applicationsrsquo models Furthermore this would largelyincrease the models readability and dramatically improve the toolusability

ACKNOWLEDGMENTSe MUSA project leading to this paper has received funding fromthe European Unionrsquos Horizon 2020 research and innovation pro-gramme under grant agreement No 644429 We would also like toacknowledge all the members of the MUSA Consortium for theirvaluable help

REFERENCES[1] Jens-Mahias Bohli Nils Gruschka Meiko Jensen Luigi Lo Iacono and Ninja

Marnau 2013 Security and privacy-enhancing multicloud architectures IEEETransactions on Dependable and Secure Computing 10 4 (2013) 212ndash224

[2] Valentina Casola Alessandra De Benedictis Massimiliano Rak and ErkudenRios 2016 Security-by-design in clouds a security-SLA driven methodology tobuild secure cloud applications Procedia Computer Science 97 (2016) 53ndash62

[3] Inc Eclipse Foundation 2011 Modelio e open source modelling environment(2011) Retrieved June 26 2017 from hpswwwmodelioorg

[4] Jaume Ferrarons Smrati Gupta Victor Muntes-Mulero Josep-Lluis Larriba-Peyand Peter Mahews 2016 Scoring cloud services through digital ecosystemcommunity analysis In International Conference on Electronic Commerce and WebTechnologies Springer 142ndash153

[5] Forbes 2016 Roundup Of Cloud Computing ForecastsAnd Market Estimates (2016) Retrieved June 26 2017from hpswwwforbescomsiteslouiscolumbus20160313roundup-of-cloud-computing-forecasts-and-market-estimates-20167971c3de2187

[6] JOINT TASK FORCE and TRANSFORMATION INITIATIVE 2013 Security andprivacy controls for federal information systems and organizations NIST SpecialPublication 800 53 (2013) 8ndash13

[7] Gartner 2016 Gartner Says by 2020 rdquoCloud Shirdquo Will Aect More an $1Trillion in IT Spending (2016) Retrieved June 26 2017 from hpwwwgartnercomnewsroomid3384720

[8] Object Management Group 2015 Unied Modeling Language Specication v25(2015) Retrieved June 26 2017 from hpwwwomgorgspecUML25

[9] Chef Soware Inc 2017 Chef technology (2017) Retrieved June 26 2017 fromhpswwwchefiochef

[10] Docker Inc 2017 Docker Swarm (2017) Retrieved June 26 2017 from hpsdocsdockercomswarm

[11] Eclipse Foundation Inc 2017 Xtext framework (2017) Retrieved June 26 2017from hpwwweclipseorgXtextdocumentation330 web supporthtml

[12] GigaSpaces Technologies Inc 2017 Cloudify (2017) Retrieved June 26 2017from hpcloudifyco

[13] Red Hat Inc 2017 Hibernate framework (2017) Retrieved June 26 2017 fromhphibernateorg

[14] Kyriakos Kritikos and Philippe Massonet 2016 An integrated meta-model forcloud application security modelling Procedia Computer Science 97 (2016) 84ndash93

[15] OASIS 2013 TOSCA 10 (Topology and Orchestration Specication for CloudApplications) Version 10 (2013) Retrieved June 26 2017 from hpdocsoasis-openorgtoscaTOSCAv10osTOSCA-v10-oshtml

[16] ARTIST EU project consortium 2013 Advanced soware-based service provi-sioning and migration of legacy soware (2013) Retrieved June 26 2017 fromhpwwwartist-projecteu

[17] CloudSocket EU project consortium 2015 Business and IT-Cloud Alignmentusing a Smart Socket (2015) Retrieved June 26 2017 from hpssitecloudsocketeu

[18] MUSA EU project consortium 2015 Multi-cloud Secure Applications (2015)Retrieved June 26 2017 from hpwwwmusa-projecteu

[19] PaaSage EU project consortium 2013 A Model-based Cross-Cloud developmentand deployment platform (2013) Retrieved June 26 2017 from hpwwwpaasageeu

[20] PaaSage EU project consortium 2014 Deliverable D212 CloudML Implementa-tion Documentation (2014) Retrieved June 26 2017 from hpwwwpaasageeuimagesdocumentspaasage d212 nalpdf

[21] PaaSage EU project consortium 2015 Deliverable D213 CAMEL Documentation(2015) Retrieved June 26 2017 from hpswwwpaasageeuimagesdocumentsdocsD213 CAMEL Documentationpdf

[22] PaaSage EU project consortium 2016 CAMEL Documentation v20159 (2016)Retrieved June 26 2017 from hpcamel-dslorgdocumentation

[23] SPECS EU project consortium 2015 Secure Provisioning of Cloud Servicesbased on SLA Management (2015) Retrieved June 26 2017 from hpwwwspecs-projecteu

[24] Puppet 2017 Puppet documentation (2017) Retrieved June 26 2017 fromhpsdocspuppetcompuppet

[25] Rightscale 2017 Cloud Computing Trends 2017 State of the CloudSurvey (2017) hpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2017-state-cloud-survey

[26] Erkuden Rios Eider Iturbe Leire Orue-Echevarria Massimiliano Rak ValentinaCasola and others 2015 Towards Self-Protective Multi-Cloud ApplicationsMUSAndasha Holistic Framework to Support the Security-Intelligent Lifecycle Man-agement of Multi-Cloud Applications (2015)

[27] Erkuden Rios Wissam Mallouli Massimiliano Rak Valentina Casola and Anto-nio M Ortiz 2016 SLA-driven monitoring of multi-cloud application componentsusing the MUSA framework In Distributed Computing Systems Workshops (ICD-CSW) 2016 IEEE 36th International Conference on IEEE 55ndash60

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

basically share the same purpose but CAMEL results in being lesscomplex and easy to use

Furthermore TOSCA community did not oer advanced toolsupport at the time when MUSA solution was dened

Last but not least the feedback received from the MUSA projectend-users about previous experiences with multi-cloud modellinglanguages and tools was considered as fundamental One of theend-user teams had experienced with CAMEL and liked the ex-pressiveness of the language for multi-cloud application descrip-tion although security aspects were not considered by them previ-ously About the modelling tools they had previously used PaaSageCAMEL Textual Editor [21] and other graphical editors (such asModelio [3]) and preferred the non-graphical editor that includedfriendly functionality such as identication of required vs optionalaributes auto completion capabilities and model validation amongothers us these are exactly the features we aim to maintain inthe MUSA Modeller As conceptually both graphical and textualeditors allow for the same degree of detail and completeness wedecided to develop our MUSA Modeller as a text editor

42 Innovations for security behaviourspecications

e MUSA innovations to CAMEL language include the enhancedcomponent security behaviour characterization which addressesconcepts required to support both composition of componentsrsquosecurity SLAs and risk analysis

421 Classification of components by their nature which allowsdescribing what the component does is information is requiredby the MUSA SLA Generator to create the Composite Security SLAof the multi-cloud application from individual componentsrsquo SLAsand by the MUSA Risk Analysis tool in order to identify the secu-rity threats and risks at component level Our CAMEL extensionallows specifying the type of the component in fact the compo-nent can be classied by two features WHAT and HOW WhileWHAT indicates the type of the functionality delivered by the com-ponent HOW indicates the way the component is delivering suchfunctionality Currently three types of HOW have been dened

bull COTS which refers to Commercial o-the-shelf sowarethat the application uses

bull SERVICE meaning that the component is not a commercialpackage but developed by the DevOps team responsiblefor the multi-cloud application engineering

bull AGENT ie a MUSA Agent component provided by theMUSA framework and available in the MUSA SecurityAgent Catalogue

e types of WHAT include

bull in case the component is COTS or SERVICE the possibleWHAT values are Web Storage IDM or Firewall Webrefers to any functionality provided through a Web inter-face Storage refers to data storage solutions (eg MySQL)IDM stays for Identity Management and Firewall for anysoware solution that protects resources from unautho-rised access

bull in case the component is AGENT the possible values ofWHAT come from the list of the agents in the MUSA Secu-rity Agent Catalogue

422 Security Controls information that properly supports Se-curity Control Framework families For example name aribute hasbeen updated to ltFamilygt-ltNumbergt(Number) format In addi-tion the subdomain aribute in the Security Control entity now isoptional instead of compulsory

e CAMEL extension developed in MUSA allows specifyingwhich security capabilities are required and provided by each multi-cloud application component e security capabilities are denedin the model by selecting and grouping the security controls partof the capability e security controls of a Security Control Fam-ily are predened and the list is oered to the user by the MUSAModeller to ease the selection of the ones included in the capabilitycurrently the security controls from the NIST SP 800-53 rev4 [6]are supported In the following example two security capabilitiesCAP1 and CAP2 are dened the rst with three security controlsand the second with only two

security model SEC security capability CAP1 controls [MUSASECAC-11(1) MUSASECAU-13(2)MUSASECAC-17(6)]security capability CAP2 controls [MUSASECAC-11(1) MUSASECAC-17(6)]

Once the security capabilities are dened in the CAMEL (inthe security model part of the model) the user can specify whatsecurity capabilities the components require andor provide Inthe following example Comp1Cap is a provided capability andComp1CapReq a requested one

provided security capability Comp1Cap security capability SECCAP2required security capability Comp1CapReq

When a component requires a specic security capability fromanother component (in the example Comp1CapReq) then the match-ing of the capability needs to be modelled as follows

capability match Comp1ToComp2 from Comp1Comp1CapReq to Comp2Comp2Cap

43 Innovations for multi-cloud deploymentOther MUSA innovations to CAMEL language address improve-ments for enhancing the expressiveness of the deployment require-ments as follows

431 Explicit characterization of the nature of the IP addressassociated with virtual machines ponents will be deployed At de-ployment phase when acquiring new cloud resources such as VMs

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

the system operator needs to indicate whether a public IP address isrequired e CAMEL extension allows specifying whether the IPaddress required for a VM should be public or not by the IP publicaribute on each component e possible values for this aributeare true or false

432 Specification of the deployment order of the applicationcomponents Dealing with multi-cloud environments it is criticalto identify the order in which each component should be deployedand congured since there are inter-dependencies among the com-ponents that are part of the same application For example the startup of a component may require that previously another componentis already up and running e CAMEL extension allows specifyingthe order in which the components are required to be deployedis can be done by using the rdquoorderrdquo aribute for each componente expected value for the order aribute is an integer number

433 Explicit definition of data exchange protocols In the MUSAextended CAMEL users can model the communications betweenthe components (eg by seing the IP addresses and ports in theconguration of the components) and specify the need to use aspecic data exchange protocol (eg MySQL OAuth Other)

434 Modelling of dynamic configurations of communicationsbetween components In CAMEL users model the communicationsbetween the components in a static way (ie through specic portnumbers and operating system conguration variables) Howeverin MUSA dynamic characteristics have been introduced such ascontext paths (instead of IP addresses) and dynamic port rangesSuch new capabilities are useful for instance to congure explicitlyinbound trac when users deploy components in Docker contain-ers

435 Modelling of deployment handlers In CAMEL the usercan model components and associate deployment instructions forinstalling conguring starting and stopping the components onvirtual machines However such deployment instructions are re-stricted to scripted commands and CAMEL lacks support to thespecication of conguration management tools such as Cloudify[12] Puppet [24] or Chef [9] erefore in MUSA this gap betweenmulti-cloud application models and these advanced frameworkshas been faced via the new Conguration entity and its associatedconcepts (eg cookbooks and recipes in case of Chef)

436 Modelling of PaaS layer elements CAMEL lacks supportto the description of architectures where the application compo-nents are not directly deployed in Virtual Machines (VMs) but incontainers Our extension allows specifying the container type thatwill be used in deployment and dening the component allocationstrategy it should follow even in cases when the container usesVM pools e new elements in our extension include

bull pool is a cluster of VMs is cluster can be used bya container or directly by a component for example adatabase solution that is capable of managing a cluster

bull manager the VM in a pool that will act as the manager vsthe rest which will be workers

bull container ndash type the container solution to use for example Docker

Swarm [10]

ndash allocationStrategy denes the allocation strategy ofthe containers on top of the acquired VMs for resourceoptimization (eg automatically scheduling containerworkloads) It supports the following four values

lowast spread balance containers across the VMs in apool based on the available CPU and RAM ofthe VMs

lowast binpak shedule containers to fully use each VMcapacity Once the full VM capacity has beenused the container moves on to the next one inthe pool

lowast random choose a VM randomlylowast custom the user denes the specic VMs in

which the containers should run

437 Refinement of security aspects in Organisation User Cre-dentials and Role entities A number of enhancements to CAMELhave been made in order to manage the authorisation of dierentroles in the DevOps team to multi-cloud application deploymentexecution For instance in MUSA the types of credentials availableto authenticate a user have been extended Among other changesit has been added expiration date to Credentials and additionalparameter properties to User entity

44 Innovations for self-healing capability ofmulti-cloud applications

Considering self-healing as the capability of a multi-cloud applica-tion of being able to self-control or modify its security behaviourat runtime so as security incidents or aacks are corrected or miti-gated the self-healing is enabled in MUSA by the MUSA Enforce-ment Agents

As their name suggests the MUSA Enforcement Agents enforcemulti-cloud application security properties at runtime such as ac-cess control security vulnerability scanning or Denial of servicemitigation mechanisms For these mechanisms to work they needto be deployed at the same time as the application componentsare deployed Some of these mechanisms require to be deployedtogether with (in the same VM) the component they will enforcethe property on erefore the MUSA extended CAMEL allowsthe denition of MUSA Security Enforcement Agents as InternalComponents of the application similarly to application compo-nents themselves so as they can be included in the deploymentplan Such agents are already pre-dened in the MUSA SecurityAgent Catalogue so users are able to re-use and congure them in afriendly way Some of these agents are always on and some will bemanaged through the enforcement services in the MUSA SecurityAssurance Platform

5 MUSA MODELLER51 MUSA Modeller Architecturee MUSA Modeller is a web editor tool that allows the creationand maintenance of (MUSA extended) CAMEL models roughthese models it is possible to dene a complete specication of therequirements needed by an application to be deployed in a securemulti-cloud environment MUSA Modeller has leveraged one ofthe new capabilities of the Xtext technology [11] the Xtext Web

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

editor support erefore multi-cloud application models can beedited and updated remotely by end-users while they are storedin shared repositories All in all without any program installationand using any web browser since its JavaScript-based API allowsadding language-specic features such as auto completion and livevalidation

As the initial step of the design phase the DevOps team createsthe Cloud Provider Independent Model (CPIM) of the multi-cloudapplication using the MUSA Modeller tool e generated CPIMwill be used as input to the risk assessment the SLA generationand the multi-cloud application deployment processes

At high level the MUSA Modeller is structured as followsbull A Web component the GUI from which the end-users ac-

cess the MUSA Modeller tool and exploit all the internalweb-services oered It also includes the web Xtext li-braries that oer syntactic and sematic services for remotemanagement of syntax validation and auto completion

bull A Server component which is the core of the MUSA Mod-eller since it implements all the business functionalitiesMoreover it oers a series of web services that are con-sumed by the Web component

bull A Database component which manages all the data storedin the central database It uses the Hibernate framework[13] that allows the abstraction of relational database en-gines

52 MUSA Modeller APIAs explained in the previous chapter the Server component of theMUSA Modeller oers a series of REST API interfaces that supportthe following functionality

bull Creation of a new multi-cloud application model in (MUSAextended) CAMEL language End-users can instantiate pre-viously dened templates for particular component typesor applications

bull Denition and storage of multi-cloud application compo-nent templates for reusing CAMEL models of the compo-nents

bull Edition and storage of multi-cloud application modelswhere application models can be dened by multiple end-users

bull Model checking for syntax and semantic correctness andintegrity of the created models e tool provides messagesof warnings and errors whenever non-conformities areidentied in the model

bull the selection of Security Controls previously identiedand stored in the MUSA Security Service Level AgreementCatalogue

bull the selection of MUSA Security Enforcement Agents pre-viously identied and stored in the MUSA Security AgentCatalogue See Fig 4

6 VALIDATION IN USE CASESDuring the rst period of the MUSA project the evaluation of theMUSA solution has been performed using generic usage scenariosalong with two specic use cases one led by Luhansa Systems(LHS) and the other by Tampere University of Technology (TUT)

Figure 4 MUSA Modeller support for Security Agent selec-tion

both partners of the project ese case studies have been designedto ensure that MUSA framework evaluation takes place in dierentcontexts of multi-cloud deployment as well as diverse security andprivacy requirements

e rst use case is related to the ight scheduling applicationprototype by Luhansa Systems that is intended to be used bymultiple airlines around the world to plan airplane ight sched-ules Todayrsquos airlines need to permanently revise their scheduleplans in response to competitor actions or to follow updated salesand marketing plans while constantly maintaining operational in-tegrity is makes schedule management a very complex processe MUSA Modeller enabled LHS to specify the breakdown of themulti-cloud application into its components as well as componentsrsquosecurity capabilities is includes also the integration of the MUSAsecurity agents into the application and the design of the provi-sioning and deployment LHS has validated the MUSA Modeller tohelp architects and developers of the multi-cloud application com-ponents in the security design to gain more focus on the complexeld of security requirements analysis and enforcing the securityby design principles even for the less experienced colleagues

In the second use case the Smart Mobility application developedby Tampere University of Technology provides Tampere citizenscontext-aware energy ecient smart mobility services and recom-mendations for transportation means is case study representsthe scenario of SMEs which create applications that exploit serviceshosted in a number of clouds As many SMEs the workgroup thatdevelops this case study does not have a specialized division forcyber security Because of this the use of MUSA framework andits tools has allowed easily addressing security aspects at dierentstages of the application engineering project design deploymentand runtime

Specically in this case study the MUSA Modeller tool has beenevaluated to model the multi-cloud application components andMUSA Enforcement Agents to integrate security controls for highavailability condentiality and integrity

e added value of the MUSA modelling language and tool hasbeen already identied in the initial evaluation performed in bothcase studies e usefulness and acceptance of the approach was

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

conrmed not only in the usage of the MUSA Modeller as a stand-alone tool but more signicantly in its integration with the restof the tools in the MUSA SecDevOps framework particularly incollaboration with Risk Analysis SLA Generator and Deployertools It is expected that the nal evaluation of the MUSA solutionwill enable the validation of all the enhancements dened in theMUSA Security DSL

7 CONCLUSIONS AND FUTUREWORKAs the number of cloud oerings grows and cloud environmentsbecome more and more complex cloud consumers will need to facemulti-cloud challenges related to multiple cloud service combina-tion and holistic protection of application components deployed indistributed heterogeneous clouds

is paper presents a novel modelling language and support-ing tool that address the special needs of multi-cloud applicationsmodelling e solution overcomes the lack of expressiveness ofstate-of-the-art modelling by easing both a) the computation ofComposite Security SLAs that include security and privacy aspectsand b) the risk analysis and service match-making taking into ac-count not only functionality and business aspects of the cloudservices but also security aspects

e language and tool presented were developed in the contextof the MUSA EU-funded project on the basis of enhancements tothe CAMEL language which already provided rich RequirementsDeployment Scalability and Security meta-models that cover manyof the requirements for multi-cloud applications specication Nev-ertheless additional requirements were identied in MUSA in orderto address richer deployment and security specication risk analy-sis and Security SLA composition for which extensions to CAMELhave been developed e modelling tool supporting this extendedCAMEL meta-model the MUSA Modeller is already integratedwith the MUSA framework and available in the MUSA website atwwwmusa-projecteu

Consequently the contributions of this paper pave the way todevelop security and privacy-aware multi-cloud applications bymastering the expressiveness of security aspects in the CPIM so asthey enable integrated SecDevOps support

In the future we have identied the need to research on how toleverage the Scalability meta-model of CAMEL to dene scalabilityrules as pre-dened conguration of high availability enforcementagents aimed at scaling up (and down) internal components ofmulti-cloud applications when needed (similar to HA Proxy [30])

Another important aspect is the research on how to supportcomposability of CAMEL models required to easily combine andrefer to CAMEL models of individual components in the creation ofmulti-cloud applicationsrsquo models Furthermore this would largelyincrease the models readability and dramatically improve the toolusability

ACKNOWLEDGMENTSe MUSA project leading to this paper has received funding fromthe European Unionrsquos Horizon 2020 research and innovation pro-gramme under grant agreement No 644429 We would also like toacknowledge all the members of the MUSA Consortium for theirvaluable help

REFERENCES[1] Jens-Mahias Bohli Nils Gruschka Meiko Jensen Luigi Lo Iacono and Ninja

Marnau 2013 Security and privacy-enhancing multicloud architectures IEEETransactions on Dependable and Secure Computing 10 4 (2013) 212ndash224

[2] Valentina Casola Alessandra De Benedictis Massimiliano Rak and ErkudenRios 2016 Security-by-design in clouds a security-SLA driven methodology tobuild secure cloud applications Procedia Computer Science 97 (2016) 53ndash62

[3] Inc Eclipse Foundation 2011 Modelio e open source modelling environment(2011) Retrieved June 26 2017 from hpswwwmodelioorg

[4] Jaume Ferrarons Smrati Gupta Victor Muntes-Mulero Josep-Lluis Larriba-Peyand Peter Mahews 2016 Scoring cloud services through digital ecosystemcommunity analysis In International Conference on Electronic Commerce and WebTechnologies Springer 142ndash153

[5] Forbes 2016 Roundup Of Cloud Computing ForecastsAnd Market Estimates (2016) Retrieved June 26 2017from hpswwwforbescomsiteslouiscolumbus20160313roundup-of-cloud-computing-forecasts-and-market-estimates-20167971c3de2187

[6] JOINT TASK FORCE and TRANSFORMATION INITIATIVE 2013 Security andprivacy controls for federal information systems and organizations NIST SpecialPublication 800 53 (2013) 8ndash13

[7] Gartner 2016 Gartner Says by 2020 rdquoCloud Shirdquo Will Aect More an $1Trillion in IT Spending (2016) Retrieved June 26 2017 from hpwwwgartnercomnewsroomid3384720

[8] Object Management Group 2015 Unied Modeling Language Specication v25(2015) Retrieved June 26 2017 from hpwwwomgorgspecUML25

[9] Chef Soware Inc 2017 Chef technology (2017) Retrieved June 26 2017 fromhpswwwchefiochef

[10] Docker Inc 2017 Docker Swarm (2017) Retrieved June 26 2017 from hpsdocsdockercomswarm

[11] Eclipse Foundation Inc 2017 Xtext framework (2017) Retrieved June 26 2017from hpwwweclipseorgXtextdocumentation330 web supporthtml

[12] GigaSpaces Technologies Inc 2017 Cloudify (2017) Retrieved June 26 2017from hpcloudifyco

[13] Red Hat Inc 2017 Hibernate framework (2017) Retrieved June 26 2017 fromhphibernateorg

[14] Kyriakos Kritikos and Philippe Massonet 2016 An integrated meta-model forcloud application security modelling Procedia Computer Science 97 (2016) 84ndash93

[15] OASIS 2013 TOSCA 10 (Topology and Orchestration Specication for CloudApplications) Version 10 (2013) Retrieved June 26 2017 from hpdocsoasis-openorgtoscaTOSCAv10osTOSCA-v10-oshtml

[16] ARTIST EU project consortium 2013 Advanced soware-based service provi-sioning and migration of legacy soware (2013) Retrieved June 26 2017 fromhpwwwartist-projecteu

[17] CloudSocket EU project consortium 2015 Business and IT-Cloud Alignmentusing a Smart Socket (2015) Retrieved June 26 2017 from hpssitecloudsocketeu

[18] MUSA EU project consortium 2015 Multi-cloud Secure Applications (2015)Retrieved June 26 2017 from hpwwwmusa-projecteu

[19] PaaSage EU project consortium 2013 A Model-based Cross-Cloud developmentand deployment platform (2013) Retrieved June 26 2017 from hpwwwpaasageeu

[20] PaaSage EU project consortium 2014 Deliverable D212 CloudML Implementa-tion Documentation (2014) Retrieved June 26 2017 from hpwwwpaasageeuimagesdocumentspaasage d212 nalpdf

[21] PaaSage EU project consortium 2015 Deliverable D213 CAMEL Documentation(2015) Retrieved June 26 2017 from hpswwwpaasageeuimagesdocumentsdocsD213 CAMEL Documentationpdf

[22] PaaSage EU project consortium 2016 CAMEL Documentation v20159 (2016)Retrieved June 26 2017 from hpcamel-dslorgdocumentation

[23] SPECS EU project consortium 2015 Secure Provisioning of Cloud Servicesbased on SLA Management (2015) Retrieved June 26 2017 from hpwwwspecs-projecteu

[24] Puppet 2017 Puppet documentation (2017) Retrieved June 26 2017 fromhpsdocspuppetcompuppet

[25] Rightscale 2017 Cloud Computing Trends 2017 State of the CloudSurvey (2017) hpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2017-state-cloud-survey

[26] Erkuden Rios Eider Iturbe Leire Orue-Echevarria Massimiliano Rak ValentinaCasola and others 2015 Towards Self-Protective Multi-Cloud ApplicationsMUSAndasha Holistic Framework to Support the Security-Intelligent Lifecycle Man-agement of Multi-Cloud Applications (2015)

[27] Erkuden Rios Wissam Mallouli Massimiliano Rak Valentina Casola and Anto-nio M Ortiz 2016 SLA-driven monitoring of multi-cloud application componentsusing the MUSA framework In Distributed Computing Systems Workshops (ICD-CSW) 2016 IEEE 36th International Conference on IEEE 55ndash60

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

the system operator needs to indicate whether a public IP address isrequired e CAMEL extension allows specifying whether the IPaddress required for a VM should be public or not by the IP publicaribute on each component e possible values for this aributeare true or false

432 Specification of the deployment order of the applicationcomponents Dealing with multi-cloud environments it is criticalto identify the order in which each component should be deployedand congured since there are inter-dependencies among the com-ponents that are part of the same application For example the startup of a component may require that previously another componentis already up and running e CAMEL extension allows specifyingthe order in which the components are required to be deployedis can be done by using the rdquoorderrdquo aribute for each componente expected value for the order aribute is an integer number

433 Explicit definition of data exchange protocols In the MUSAextended CAMEL users can model the communications betweenthe components (eg by seing the IP addresses and ports in theconguration of the components) and specify the need to use aspecic data exchange protocol (eg MySQL OAuth Other)

434 Modelling of dynamic configurations of communicationsbetween components In CAMEL users model the communicationsbetween the components in a static way (ie through specic portnumbers and operating system conguration variables) Howeverin MUSA dynamic characteristics have been introduced such ascontext paths (instead of IP addresses) and dynamic port rangesSuch new capabilities are useful for instance to congure explicitlyinbound trac when users deploy components in Docker contain-ers

435 Modelling of deployment handlers In CAMEL the usercan model components and associate deployment instructions forinstalling conguring starting and stopping the components onvirtual machines However such deployment instructions are re-stricted to scripted commands and CAMEL lacks support to thespecication of conguration management tools such as Cloudify[12] Puppet [24] or Chef [9] erefore in MUSA this gap betweenmulti-cloud application models and these advanced frameworkshas been faced via the new Conguration entity and its associatedconcepts (eg cookbooks and recipes in case of Chef)

436 Modelling of PaaS layer elements CAMEL lacks supportto the description of architectures where the application compo-nents are not directly deployed in Virtual Machines (VMs) but incontainers Our extension allows specifying the container type thatwill be used in deployment and dening the component allocationstrategy it should follow even in cases when the container usesVM pools e new elements in our extension include

bull pool is a cluster of VMs is cluster can be used bya container or directly by a component for example adatabase solution that is capable of managing a cluster

bull manager the VM in a pool that will act as the manager vsthe rest which will be workers

bull container ndash type the container solution to use for example Docker

Swarm [10]

ndash allocationStrategy denes the allocation strategy ofthe containers on top of the acquired VMs for resourceoptimization (eg automatically scheduling containerworkloads) It supports the following four values

lowast spread balance containers across the VMs in apool based on the available CPU and RAM ofthe VMs

lowast binpak shedule containers to fully use each VMcapacity Once the full VM capacity has beenused the container moves on to the next one inthe pool

lowast random choose a VM randomlylowast custom the user denes the specic VMs in

which the containers should run

437 Refinement of security aspects in Organisation User Cre-dentials and Role entities A number of enhancements to CAMELhave been made in order to manage the authorisation of dierentroles in the DevOps team to multi-cloud application deploymentexecution For instance in MUSA the types of credentials availableto authenticate a user have been extended Among other changesit has been added expiration date to Credentials and additionalparameter properties to User entity

44 Innovations for self-healing capability ofmulti-cloud applications

Considering self-healing as the capability of a multi-cloud applica-tion of being able to self-control or modify its security behaviourat runtime so as security incidents or aacks are corrected or miti-gated the self-healing is enabled in MUSA by the MUSA Enforce-ment Agents

As their name suggests the MUSA Enforcement Agents enforcemulti-cloud application security properties at runtime such as ac-cess control security vulnerability scanning or Denial of servicemitigation mechanisms For these mechanisms to work they needto be deployed at the same time as the application componentsare deployed Some of these mechanisms require to be deployedtogether with (in the same VM) the component they will enforcethe property on erefore the MUSA extended CAMEL allowsthe denition of MUSA Security Enforcement Agents as InternalComponents of the application similarly to application compo-nents themselves so as they can be included in the deploymentplan Such agents are already pre-dened in the MUSA SecurityAgent Catalogue so users are able to re-use and congure them in afriendly way Some of these agents are always on and some will bemanaged through the enforcement services in the MUSA SecurityAssurance Platform

5 MUSA MODELLER51 MUSA Modeller Architecturee MUSA Modeller is a web editor tool that allows the creationand maintenance of (MUSA extended) CAMEL models roughthese models it is possible to dene a complete specication of therequirements needed by an application to be deployed in a securemulti-cloud environment MUSA Modeller has leveraged one ofthe new capabilities of the Xtext technology [11] the Xtext Web

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

editor support erefore multi-cloud application models can beedited and updated remotely by end-users while they are storedin shared repositories All in all without any program installationand using any web browser since its JavaScript-based API allowsadding language-specic features such as auto completion and livevalidation

As the initial step of the design phase the DevOps team createsthe Cloud Provider Independent Model (CPIM) of the multi-cloudapplication using the MUSA Modeller tool e generated CPIMwill be used as input to the risk assessment the SLA generationand the multi-cloud application deployment processes

At high level the MUSA Modeller is structured as followsbull A Web component the GUI from which the end-users ac-

cess the MUSA Modeller tool and exploit all the internalweb-services oered It also includes the web Xtext li-braries that oer syntactic and sematic services for remotemanagement of syntax validation and auto completion

bull A Server component which is the core of the MUSA Mod-eller since it implements all the business functionalitiesMoreover it oers a series of web services that are con-sumed by the Web component

bull A Database component which manages all the data storedin the central database It uses the Hibernate framework[13] that allows the abstraction of relational database en-gines

52 MUSA Modeller APIAs explained in the previous chapter the Server component of theMUSA Modeller oers a series of REST API interfaces that supportthe following functionality

bull Creation of a new multi-cloud application model in (MUSAextended) CAMEL language End-users can instantiate pre-viously dened templates for particular component typesor applications

bull Denition and storage of multi-cloud application compo-nent templates for reusing CAMEL models of the compo-nents

bull Edition and storage of multi-cloud application modelswhere application models can be dened by multiple end-users

bull Model checking for syntax and semantic correctness andintegrity of the created models e tool provides messagesof warnings and errors whenever non-conformities areidentied in the model

bull the selection of Security Controls previously identiedand stored in the MUSA Security Service Level AgreementCatalogue

bull the selection of MUSA Security Enforcement Agents pre-viously identied and stored in the MUSA Security AgentCatalogue See Fig 4

6 VALIDATION IN USE CASESDuring the rst period of the MUSA project the evaluation of theMUSA solution has been performed using generic usage scenariosalong with two specic use cases one led by Luhansa Systems(LHS) and the other by Tampere University of Technology (TUT)

Figure 4 MUSA Modeller support for Security Agent selec-tion

both partners of the project ese case studies have been designedto ensure that MUSA framework evaluation takes place in dierentcontexts of multi-cloud deployment as well as diverse security andprivacy requirements

e rst use case is related to the ight scheduling applicationprototype by Luhansa Systems that is intended to be used bymultiple airlines around the world to plan airplane ight sched-ules Todayrsquos airlines need to permanently revise their scheduleplans in response to competitor actions or to follow updated salesand marketing plans while constantly maintaining operational in-tegrity is makes schedule management a very complex processe MUSA Modeller enabled LHS to specify the breakdown of themulti-cloud application into its components as well as componentsrsquosecurity capabilities is includes also the integration of the MUSAsecurity agents into the application and the design of the provi-sioning and deployment LHS has validated the MUSA Modeller tohelp architects and developers of the multi-cloud application com-ponents in the security design to gain more focus on the complexeld of security requirements analysis and enforcing the securityby design principles even for the less experienced colleagues

In the second use case the Smart Mobility application developedby Tampere University of Technology provides Tampere citizenscontext-aware energy ecient smart mobility services and recom-mendations for transportation means is case study representsthe scenario of SMEs which create applications that exploit serviceshosted in a number of clouds As many SMEs the workgroup thatdevelops this case study does not have a specialized division forcyber security Because of this the use of MUSA framework andits tools has allowed easily addressing security aspects at dierentstages of the application engineering project design deploymentand runtime

Specically in this case study the MUSA Modeller tool has beenevaluated to model the multi-cloud application components andMUSA Enforcement Agents to integrate security controls for highavailability condentiality and integrity

e added value of the MUSA modelling language and tool hasbeen already identied in the initial evaluation performed in bothcase studies e usefulness and acceptance of the approach was

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

conrmed not only in the usage of the MUSA Modeller as a stand-alone tool but more signicantly in its integration with the restof the tools in the MUSA SecDevOps framework particularly incollaboration with Risk Analysis SLA Generator and Deployertools It is expected that the nal evaluation of the MUSA solutionwill enable the validation of all the enhancements dened in theMUSA Security DSL

7 CONCLUSIONS AND FUTUREWORKAs the number of cloud oerings grows and cloud environmentsbecome more and more complex cloud consumers will need to facemulti-cloud challenges related to multiple cloud service combina-tion and holistic protection of application components deployed indistributed heterogeneous clouds

is paper presents a novel modelling language and support-ing tool that address the special needs of multi-cloud applicationsmodelling e solution overcomes the lack of expressiveness ofstate-of-the-art modelling by easing both a) the computation ofComposite Security SLAs that include security and privacy aspectsand b) the risk analysis and service match-making taking into ac-count not only functionality and business aspects of the cloudservices but also security aspects

e language and tool presented were developed in the contextof the MUSA EU-funded project on the basis of enhancements tothe CAMEL language which already provided rich RequirementsDeployment Scalability and Security meta-models that cover manyof the requirements for multi-cloud applications specication Nev-ertheless additional requirements were identied in MUSA in orderto address richer deployment and security specication risk analy-sis and Security SLA composition for which extensions to CAMELhave been developed e modelling tool supporting this extendedCAMEL meta-model the MUSA Modeller is already integratedwith the MUSA framework and available in the MUSA website atwwwmusa-projecteu

Consequently the contributions of this paper pave the way todevelop security and privacy-aware multi-cloud applications bymastering the expressiveness of security aspects in the CPIM so asthey enable integrated SecDevOps support

In the future we have identied the need to research on how toleverage the Scalability meta-model of CAMEL to dene scalabilityrules as pre-dened conguration of high availability enforcementagents aimed at scaling up (and down) internal components ofmulti-cloud applications when needed (similar to HA Proxy [30])

Another important aspect is the research on how to supportcomposability of CAMEL models required to easily combine andrefer to CAMEL models of individual components in the creation ofmulti-cloud applicationsrsquo models Furthermore this would largelyincrease the models readability and dramatically improve the toolusability

ACKNOWLEDGMENTSe MUSA project leading to this paper has received funding fromthe European Unionrsquos Horizon 2020 research and innovation pro-gramme under grant agreement No 644429 We would also like toacknowledge all the members of the MUSA Consortium for theirvaluable help

REFERENCES[1] Jens-Mahias Bohli Nils Gruschka Meiko Jensen Luigi Lo Iacono and Ninja

Marnau 2013 Security and privacy-enhancing multicloud architectures IEEETransactions on Dependable and Secure Computing 10 4 (2013) 212ndash224

[2] Valentina Casola Alessandra De Benedictis Massimiliano Rak and ErkudenRios 2016 Security-by-design in clouds a security-SLA driven methodology tobuild secure cloud applications Procedia Computer Science 97 (2016) 53ndash62

[3] Inc Eclipse Foundation 2011 Modelio e open source modelling environment(2011) Retrieved June 26 2017 from hpswwwmodelioorg

[4] Jaume Ferrarons Smrati Gupta Victor Muntes-Mulero Josep-Lluis Larriba-Peyand Peter Mahews 2016 Scoring cloud services through digital ecosystemcommunity analysis In International Conference on Electronic Commerce and WebTechnologies Springer 142ndash153

[5] Forbes 2016 Roundup Of Cloud Computing ForecastsAnd Market Estimates (2016) Retrieved June 26 2017from hpswwwforbescomsiteslouiscolumbus20160313roundup-of-cloud-computing-forecasts-and-market-estimates-20167971c3de2187

[6] JOINT TASK FORCE and TRANSFORMATION INITIATIVE 2013 Security andprivacy controls for federal information systems and organizations NIST SpecialPublication 800 53 (2013) 8ndash13

[7] Gartner 2016 Gartner Says by 2020 rdquoCloud Shirdquo Will Aect More an $1Trillion in IT Spending (2016) Retrieved June 26 2017 from hpwwwgartnercomnewsroomid3384720

[8] Object Management Group 2015 Unied Modeling Language Specication v25(2015) Retrieved June 26 2017 from hpwwwomgorgspecUML25

[9] Chef Soware Inc 2017 Chef technology (2017) Retrieved June 26 2017 fromhpswwwchefiochef

[10] Docker Inc 2017 Docker Swarm (2017) Retrieved June 26 2017 from hpsdocsdockercomswarm

[11] Eclipse Foundation Inc 2017 Xtext framework (2017) Retrieved June 26 2017from hpwwweclipseorgXtextdocumentation330 web supporthtml

[12] GigaSpaces Technologies Inc 2017 Cloudify (2017) Retrieved June 26 2017from hpcloudifyco

[13] Red Hat Inc 2017 Hibernate framework (2017) Retrieved June 26 2017 fromhphibernateorg

[14] Kyriakos Kritikos and Philippe Massonet 2016 An integrated meta-model forcloud application security modelling Procedia Computer Science 97 (2016) 84ndash93

[15] OASIS 2013 TOSCA 10 (Topology and Orchestration Specication for CloudApplications) Version 10 (2013) Retrieved June 26 2017 from hpdocsoasis-openorgtoscaTOSCAv10osTOSCA-v10-oshtml

[16] ARTIST EU project consortium 2013 Advanced soware-based service provi-sioning and migration of legacy soware (2013) Retrieved June 26 2017 fromhpwwwartist-projecteu

[17] CloudSocket EU project consortium 2015 Business and IT-Cloud Alignmentusing a Smart Socket (2015) Retrieved June 26 2017 from hpssitecloudsocketeu

[18] MUSA EU project consortium 2015 Multi-cloud Secure Applications (2015)Retrieved June 26 2017 from hpwwwmusa-projecteu

[19] PaaSage EU project consortium 2013 A Model-based Cross-Cloud developmentand deployment platform (2013) Retrieved June 26 2017 from hpwwwpaasageeu

[20] PaaSage EU project consortium 2014 Deliverable D212 CloudML Implementa-tion Documentation (2014) Retrieved June 26 2017 from hpwwwpaasageeuimagesdocumentspaasage d212 nalpdf

[21] PaaSage EU project consortium 2015 Deliverable D213 CAMEL Documentation(2015) Retrieved June 26 2017 from hpswwwpaasageeuimagesdocumentsdocsD213 CAMEL Documentationpdf

[22] PaaSage EU project consortium 2016 CAMEL Documentation v20159 (2016)Retrieved June 26 2017 from hpcamel-dslorgdocumentation

[23] SPECS EU project consortium 2015 Secure Provisioning of Cloud Servicesbased on SLA Management (2015) Retrieved June 26 2017 from hpwwwspecs-projecteu

[24] Puppet 2017 Puppet documentation (2017) Retrieved June 26 2017 fromhpsdocspuppetcompuppet

[25] Rightscale 2017 Cloud Computing Trends 2017 State of the CloudSurvey (2017) hpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2017-state-cloud-survey

[26] Erkuden Rios Eider Iturbe Leire Orue-Echevarria Massimiliano Rak ValentinaCasola and others 2015 Towards Self-Protective Multi-Cloud ApplicationsMUSAndasha Holistic Framework to Support the Security-Intelligent Lifecycle Man-agement of Multi-Cloud Applications (2015)

[27] Erkuden Rios Wissam Mallouli Massimiliano Rak Valentina Casola and Anto-nio M Ortiz 2016 SLA-driven monitoring of multi-cloud application componentsusing the MUSA framework In Distributed Computing Systems Workshops (ICD-CSW) 2016 IEEE 36th International Conference on IEEE 55ndash60

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

editor support erefore multi-cloud application models can beedited and updated remotely by end-users while they are storedin shared repositories All in all without any program installationand using any web browser since its JavaScript-based API allowsadding language-specic features such as auto completion and livevalidation

As the initial step of the design phase the DevOps team createsthe Cloud Provider Independent Model (CPIM) of the multi-cloudapplication using the MUSA Modeller tool e generated CPIMwill be used as input to the risk assessment the SLA generationand the multi-cloud application deployment processes

At high level the MUSA Modeller is structured as followsbull A Web component the GUI from which the end-users ac-

cess the MUSA Modeller tool and exploit all the internalweb-services oered It also includes the web Xtext li-braries that oer syntactic and sematic services for remotemanagement of syntax validation and auto completion

bull A Server component which is the core of the MUSA Mod-eller since it implements all the business functionalitiesMoreover it oers a series of web services that are con-sumed by the Web component

bull A Database component which manages all the data storedin the central database It uses the Hibernate framework[13] that allows the abstraction of relational database en-gines

52 MUSA Modeller APIAs explained in the previous chapter the Server component of theMUSA Modeller oers a series of REST API interfaces that supportthe following functionality

bull Creation of a new multi-cloud application model in (MUSAextended) CAMEL language End-users can instantiate pre-viously dened templates for particular component typesor applications

bull Denition and storage of multi-cloud application compo-nent templates for reusing CAMEL models of the compo-nents

bull Edition and storage of multi-cloud application modelswhere application models can be dened by multiple end-users

bull Model checking for syntax and semantic correctness andintegrity of the created models e tool provides messagesof warnings and errors whenever non-conformities areidentied in the model

bull the selection of Security Controls previously identiedand stored in the MUSA Security Service Level AgreementCatalogue

bull the selection of MUSA Security Enforcement Agents pre-viously identied and stored in the MUSA Security AgentCatalogue See Fig 4

6 VALIDATION IN USE CASESDuring the rst period of the MUSA project the evaluation of theMUSA solution has been performed using generic usage scenariosalong with two specic use cases one led by Luhansa Systems(LHS) and the other by Tampere University of Technology (TUT)

Figure 4 MUSA Modeller support for Security Agent selec-tion

both partners of the project ese case studies have been designedto ensure that MUSA framework evaluation takes place in dierentcontexts of multi-cloud deployment as well as diverse security andprivacy requirements

e rst use case is related to the ight scheduling applicationprototype by Luhansa Systems that is intended to be used bymultiple airlines around the world to plan airplane ight sched-ules Todayrsquos airlines need to permanently revise their scheduleplans in response to competitor actions or to follow updated salesand marketing plans while constantly maintaining operational in-tegrity is makes schedule management a very complex processe MUSA Modeller enabled LHS to specify the breakdown of themulti-cloud application into its components as well as componentsrsquosecurity capabilities is includes also the integration of the MUSAsecurity agents into the application and the design of the provi-sioning and deployment LHS has validated the MUSA Modeller tohelp architects and developers of the multi-cloud application com-ponents in the security design to gain more focus on the complexeld of security requirements analysis and enforcing the securityby design principles even for the less experienced colleagues

In the second use case the Smart Mobility application developedby Tampere University of Technology provides Tampere citizenscontext-aware energy ecient smart mobility services and recom-mendations for transportation means is case study representsthe scenario of SMEs which create applications that exploit serviceshosted in a number of clouds As many SMEs the workgroup thatdevelops this case study does not have a specialized division forcyber security Because of this the use of MUSA framework andits tools has allowed easily addressing security aspects at dierentstages of the application engineering project design deploymentand runtime

Specically in this case study the MUSA Modeller tool has beenevaluated to model the multi-cloud application components andMUSA Enforcement Agents to integrate security controls for highavailability condentiality and integrity

e added value of the MUSA modelling language and tool hasbeen already identied in the initial evaluation performed in bothcase studies e usefulness and acceptance of the approach was

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

conrmed not only in the usage of the MUSA Modeller as a stand-alone tool but more signicantly in its integration with the restof the tools in the MUSA SecDevOps framework particularly incollaboration with Risk Analysis SLA Generator and Deployertools It is expected that the nal evaluation of the MUSA solutionwill enable the validation of all the enhancements dened in theMUSA Security DSL

7 CONCLUSIONS AND FUTUREWORKAs the number of cloud oerings grows and cloud environmentsbecome more and more complex cloud consumers will need to facemulti-cloud challenges related to multiple cloud service combina-tion and holistic protection of application components deployed indistributed heterogeneous clouds

is paper presents a novel modelling language and support-ing tool that address the special needs of multi-cloud applicationsmodelling e solution overcomes the lack of expressiveness ofstate-of-the-art modelling by easing both a) the computation ofComposite Security SLAs that include security and privacy aspectsand b) the risk analysis and service match-making taking into ac-count not only functionality and business aspects of the cloudservices but also security aspects

e language and tool presented were developed in the contextof the MUSA EU-funded project on the basis of enhancements tothe CAMEL language which already provided rich RequirementsDeployment Scalability and Security meta-models that cover manyof the requirements for multi-cloud applications specication Nev-ertheless additional requirements were identied in MUSA in orderto address richer deployment and security specication risk analy-sis and Security SLA composition for which extensions to CAMELhave been developed e modelling tool supporting this extendedCAMEL meta-model the MUSA Modeller is already integratedwith the MUSA framework and available in the MUSA website atwwwmusa-projecteu

Consequently the contributions of this paper pave the way todevelop security and privacy-aware multi-cloud applications bymastering the expressiveness of security aspects in the CPIM so asthey enable integrated SecDevOps support

In the future we have identied the need to research on how toleverage the Scalability meta-model of CAMEL to dene scalabilityrules as pre-dened conguration of high availability enforcementagents aimed at scaling up (and down) internal components ofmulti-cloud applications when needed (similar to HA Proxy [30])

Another important aspect is the research on how to supportcomposability of CAMEL models required to easily combine andrefer to CAMEL models of individual components in the creation ofmulti-cloud applicationsrsquo models Furthermore this would largelyincrease the models readability and dramatically improve the toolusability

ACKNOWLEDGMENTSe MUSA project leading to this paper has received funding fromthe European Unionrsquos Horizon 2020 research and innovation pro-gramme under grant agreement No 644429 We would also like toacknowledge all the members of the MUSA Consortium for theirvaluable help

REFERENCES[1] Jens-Mahias Bohli Nils Gruschka Meiko Jensen Luigi Lo Iacono and Ninja

Marnau 2013 Security and privacy-enhancing multicloud architectures IEEETransactions on Dependable and Secure Computing 10 4 (2013) 212ndash224

[2] Valentina Casola Alessandra De Benedictis Massimiliano Rak and ErkudenRios 2016 Security-by-design in clouds a security-SLA driven methodology tobuild secure cloud applications Procedia Computer Science 97 (2016) 53ndash62

[3] Inc Eclipse Foundation 2011 Modelio e open source modelling environment(2011) Retrieved June 26 2017 from hpswwwmodelioorg

[4] Jaume Ferrarons Smrati Gupta Victor Muntes-Mulero Josep-Lluis Larriba-Peyand Peter Mahews 2016 Scoring cloud services through digital ecosystemcommunity analysis In International Conference on Electronic Commerce and WebTechnologies Springer 142ndash153

[5] Forbes 2016 Roundup Of Cloud Computing ForecastsAnd Market Estimates (2016) Retrieved June 26 2017from hpswwwforbescomsiteslouiscolumbus20160313roundup-of-cloud-computing-forecasts-and-market-estimates-20167971c3de2187

[6] JOINT TASK FORCE and TRANSFORMATION INITIATIVE 2013 Security andprivacy controls for federal information systems and organizations NIST SpecialPublication 800 53 (2013) 8ndash13

[7] Gartner 2016 Gartner Says by 2020 rdquoCloud Shirdquo Will Aect More an $1Trillion in IT Spending (2016) Retrieved June 26 2017 from hpwwwgartnercomnewsroomid3384720

[8] Object Management Group 2015 Unied Modeling Language Specication v25(2015) Retrieved June 26 2017 from hpwwwomgorgspecUML25

[9] Chef Soware Inc 2017 Chef technology (2017) Retrieved June 26 2017 fromhpswwwchefiochef

[10] Docker Inc 2017 Docker Swarm (2017) Retrieved June 26 2017 from hpsdocsdockercomswarm

[11] Eclipse Foundation Inc 2017 Xtext framework (2017) Retrieved June 26 2017from hpwwweclipseorgXtextdocumentation330 web supporthtml

[12] GigaSpaces Technologies Inc 2017 Cloudify (2017) Retrieved June 26 2017from hpcloudifyco

[13] Red Hat Inc 2017 Hibernate framework (2017) Retrieved June 26 2017 fromhphibernateorg

[14] Kyriakos Kritikos and Philippe Massonet 2016 An integrated meta-model forcloud application security modelling Procedia Computer Science 97 (2016) 84ndash93

[15] OASIS 2013 TOSCA 10 (Topology and Orchestration Specication for CloudApplications) Version 10 (2013) Retrieved June 26 2017 from hpdocsoasis-openorgtoscaTOSCAv10osTOSCA-v10-oshtml

[16] ARTIST EU project consortium 2013 Advanced soware-based service provi-sioning and migration of legacy soware (2013) Retrieved June 26 2017 fromhpwwwartist-projecteu

[17] CloudSocket EU project consortium 2015 Business and IT-Cloud Alignmentusing a Smart Socket (2015) Retrieved June 26 2017 from hpssitecloudsocketeu

[18] MUSA EU project consortium 2015 Multi-cloud Secure Applications (2015)Retrieved June 26 2017 from hpwwwmusa-projecteu

[19] PaaSage EU project consortium 2013 A Model-based Cross-Cloud developmentand deployment platform (2013) Retrieved June 26 2017 from hpwwwpaasageeu

[20] PaaSage EU project consortium 2014 Deliverable D212 CloudML Implementa-tion Documentation (2014) Retrieved June 26 2017 from hpwwwpaasageeuimagesdocumentspaasage d212 nalpdf

[21] PaaSage EU project consortium 2015 Deliverable D213 CAMEL Documentation(2015) Retrieved June 26 2017 from hpswwwpaasageeuimagesdocumentsdocsD213 CAMEL Documentationpdf

[22] PaaSage EU project consortium 2016 CAMEL Documentation v20159 (2016)Retrieved June 26 2017 from hpcamel-dslorgdocumentation

[23] SPECS EU project consortium 2015 Secure Provisioning of Cloud Servicesbased on SLA Management (2015) Retrieved June 26 2017 from hpwwwspecs-projecteu

[24] Puppet 2017 Puppet documentation (2017) Retrieved June 26 2017 fromhpsdocspuppetcompuppet

[25] Rightscale 2017 Cloud Computing Trends 2017 State of the CloudSurvey (2017) hpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2017-state-cloud-survey

[26] Erkuden Rios Eider Iturbe Leire Orue-Echevarria Massimiliano Rak ValentinaCasola and others 2015 Towards Self-Protective Multi-Cloud ApplicationsMUSAndasha Holistic Framework to Support the Security-Intelligent Lifecycle Man-agement of Multi-Cloud Applications (2015)

[27] Erkuden Rios Wissam Mallouli Massimiliano Rak Valentina Casola and Anto-nio M Ortiz 2016 SLA-driven monitoring of multi-cloud application componentsusing the MUSA framework In Distributed Computing Systems Workshops (ICD-CSW) 2016 IEEE 36th International Conference on IEEE 55ndash60

ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy ERios et al

conrmed not only in the usage of the MUSA Modeller as a stand-alone tool but more signicantly in its integration with the restof the tools in the MUSA SecDevOps framework particularly incollaboration with Risk Analysis SLA Generator and Deployertools It is expected that the nal evaluation of the MUSA solutionwill enable the validation of all the enhancements dened in theMUSA Security DSL

7 CONCLUSIONS AND FUTUREWORKAs the number of cloud oerings grows and cloud environmentsbecome more and more complex cloud consumers will need to facemulti-cloud challenges related to multiple cloud service combina-tion and holistic protection of application components deployed indistributed heterogeneous clouds

is paper presents a novel modelling language and support-ing tool that address the special needs of multi-cloud applicationsmodelling e solution overcomes the lack of expressiveness ofstate-of-the-art modelling by easing both a) the computation ofComposite Security SLAs that include security and privacy aspectsand b) the risk analysis and service match-making taking into ac-count not only functionality and business aspects of the cloudservices but also security aspects

e language and tool presented were developed in the contextof the MUSA EU-funded project on the basis of enhancements tothe CAMEL language which already provided rich RequirementsDeployment Scalability and Security meta-models that cover manyof the requirements for multi-cloud applications specication Nev-ertheless additional requirements were identied in MUSA in orderto address richer deployment and security specication risk analy-sis and Security SLA composition for which extensions to CAMELhave been developed e modelling tool supporting this extendedCAMEL meta-model the MUSA Modeller is already integratedwith the MUSA framework and available in the MUSA website atwwwmusa-projecteu

Consequently the contributions of this paper pave the way todevelop security and privacy-aware multi-cloud applications bymastering the expressiveness of security aspects in the CPIM so asthey enable integrated SecDevOps support

In the future we have identied the need to research on how toleverage the Scalability meta-model of CAMEL to dene scalabilityrules as pre-dened conguration of high availability enforcementagents aimed at scaling up (and down) internal components ofmulti-cloud applications when needed (similar to HA Proxy [30])

Another important aspect is the research on how to supportcomposability of CAMEL models required to easily combine andrefer to CAMEL models of individual components in the creation ofmulti-cloud applicationsrsquo models Furthermore this would largelyincrease the models readability and dramatically improve the toolusability

ACKNOWLEDGMENTSe MUSA project leading to this paper has received funding fromthe European Unionrsquos Horizon 2020 research and innovation pro-gramme under grant agreement No 644429 We would also like toacknowledge all the members of the MUSA Consortium for theirvaluable help

REFERENCES[1] Jens-Mahias Bohli Nils Gruschka Meiko Jensen Luigi Lo Iacono and Ninja

Marnau 2013 Security and privacy-enhancing multicloud architectures IEEETransactions on Dependable and Secure Computing 10 4 (2013) 212ndash224

[2] Valentina Casola Alessandra De Benedictis Massimiliano Rak and ErkudenRios 2016 Security-by-design in clouds a security-SLA driven methodology tobuild secure cloud applications Procedia Computer Science 97 (2016) 53ndash62

[3] Inc Eclipse Foundation 2011 Modelio e open source modelling environment(2011) Retrieved June 26 2017 from hpswwwmodelioorg

[4] Jaume Ferrarons Smrati Gupta Victor Muntes-Mulero Josep-Lluis Larriba-Peyand Peter Mahews 2016 Scoring cloud services through digital ecosystemcommunity analysis In International Conference on Electronic Commerce and WebTechnologies Springer 142ndash153

[5] Forbes 2016 Roundup Of Cloud Computing ForecastsAnd Market Estimates (2016) Retrieved June 26 2017from hpswwwforbescomsiteslouiscolumbus20160313roundup-of-cloud-computing-forecasts-and-market-estimates-20167971c3de2187

[6] JOINT TASK FORCE and TRANSFORMATION INITIATIVE 2013 Security andprivacy controls for federal information systems and organizations NIST SpecialPublication 800 53 (2013) 8ndash13

[7] Gartner 2016 Gartner Says by 2020 rdquoCloud Shirdquo Will Aect More an $1Trillion in IT Spending (2016) Retrieved June 26 2017 from hpwwwgartnercomnewsroomid3384720

[8] Object Management Group 2015 Unied Modeling Language Specication v25(2015) Retrieved June 26 2017 from hpwwwomgorgspecUML25

[9] Chef Soware Inc 2017 Chef technology (2017) Retrieved June 26 2017 fromhpswwwchefiochef

[10] Docker Inc 2017 Docker Swarm (2017) Retrieved June 26 2017 from hpsdocsdockercomswarm

[11] Eclipse Foundation Inc 2017 Xtext framework (2017) Retrieved June 26 2017from hpwwweclipseorgXtextdocumentation330 web supporthtml

[12] GigaSpaces Technologies Inc 2017 Cloudify (2017) Retrieved June 26 2017from hpcloudifyco

[13] Red Hat Inc 2017 Hibernate framework (2017) Retrieved June 26 2017 fromhphibernateorg

[14] Kyriakos Kritikos and Philippe Massonet 2016 An integrated meta-model forcloud application security modelling Procedia Computer Science 97 (2016) 84ndash93

[15] OASIS 2013 TOSCA 10 (Topology and Orchestration Specication for CloudApplications) Version 10 (2013) Retrieved June 26 2017 from hpdocsoasis-openorgtoscaTOSCAv10osTOSCA-v10-oshtml

[16] ARTIST EU project consortium 2013 Advanced soware-based service provi-sioning and migration of legacy soware (2013) Retrieved June 26 2017 fromhpwwwartist-projecteu

[17] CloudSocket EU project consortium 2015 Business and IT-Cloud Alignmentusing a Smart Socket (2015) Retrieved June 26 2017 from hpssitecloudsocketeu

[18] MUSA EU project consortium 2015 Multi-cloud Secure Applications (2015)Retrieved June 26 2017 from hpwwwmusa-projecteu

[19] PaaSage EU project consortium 2013 A Model-based Cross-Cloud developmentand deployment platform (2013) Retrieved June 26 2017 from hpwwwpaasageeu

[20] PaaSage EU project consortium 2014 Deliverable D212 CloudML Implementa-tion Documentation (2014) Retrieved June 26 2017 from hpwwwpaasageeuimagesdocumentspaasage d212 nalpdf

[21] PaaSage EU project consortium 2015 Deliverable D213 CAMEL Documentation(2015) Retrieved June 26 2017 from hpswwwpaasageeuimagesdocumentsdocsD213 CAMEL Documentationpdf

[22] PaaSage EU project consortium 2016 CAMEL Documentation v20159 (2016)Retrieved June 26 2017 from hpcamel-dslorgdocumentation

[23] SPECS EU project consortium 2015 Secure Provisioning of Cloud Servicesbased on SLA Management (2015) Retrieved June 26 2017 from hpwwwspecs-projecteu

[24] Puppet 2017 Puppet documentation (2017) Retrieved June 26 2017 fromhpsdocspuppetcompuppet

[25] Rightscale 2017 Cloud Computing Trends 2017 State of the CloudSurvey (2017) hpwwwrightscalecomblogcloud-industry-insightscloud-computing-trends-2017-state-cloud-survey

[26] Erkuden Rios Eider Iturbe Leire Orue-Echevarria Massimiliano Rak ValentinaCasola and others 2015 Towards Self-Protective Multi-Cloud ApplicationsMUSAndasha Holistic Framework to Support the Security-Intelligent Lifecycle Man-agement of Multi-Cloud Applications (2015)

[27] Erkuden Rios Wissam Mallouli Massimiliano Rak Valentina Casola and Anto-nio M Ortiz 2016 SLA-driven monitoring of multi-cloud application componentsusing the MUSA framework In Distributed Computing Systems Workshops (ICD-CSW) 2016 IEEE 36th International Conference on IEEE 55ndash60

Self-healing Multi-Cloud Application Modelling ARES rsquo17 August 29-September 01 2017 Reggio Calabria Italy

[28] SINTEF 2013 Model-based provisioning and deployment of cloud-based systems(2013) Retrieved June 26 2017 from hpcloudmlorg

[29] Marko Vukolic 2010 e Byzantine empire in the intercloud ACM SIGACTNews 41 3 (2010) 105ndash111

[30] Q HAProxy Wu 2017 e Reliable High Performance TCPHTTP Load Balancer(2017) Retrieved June 26 2017 from hpwwwhaproxyorg