Cloud Futurology

Pre-print version

Blesson Varghese, Philipp Leitner, Suprio Ray, Kyle Chard, Adam Barker,Yehia Elkhatib, Herry Herry, Cheol-Ho Hong, Jeremy Singer, Fung Po Tso,

Eiko Yoneki, Mohamed-Faten Zhani

Abstract

The Cloud has become integral to most Internet-based applications and user gadgets. This articleprovides a brief history of the Cloud and presents a researcher’s view of the prospects for innovating at theinfrastructure, middleware, and application and delivery levels of the already crowded Cloud computingstack.

Introduction

The global Cloud computing market exceeds $100 billion and research in this area has rapidly ma-tured over the last decade. During this time many buzz words have come and gone. Consequently,traditional concepts and conventional definitions related to the Cloud are almost obsolete [1].The current Cloud landscape looks very different from what may have been envisioned duringits inception. It may seem that the area is saturated and there is little innovative research anddevelopment to be done in the Cloud, which naturally raises the question - ‘What is the future ofthe Cloud?’

This article first examines the multiple generations of innovation that the Cloud has undergonein the last decade, it then presents a researcher’s view of the prospects and opportunities forinnovation in this area across the entire Cloud stack - highlighting the infrastructure, middleware,and application and delivery levels.

The Cloud Landscape

The first known suggestion of Cloud-like computing was by Professor John McCarthy atMIT’s centennial celebration in 1961 “Computing may someday be organized as a public utilityjust as the telephone system is a public utility ... Each subscriber needs to pay only for the capacityhe actually uses, but he has access to all programming languages characteristic of a very large system... Certain subscribers might offer service to other subscribers ... The computer utility could becomethe basis of a new and important industry”

Two fundamental technologies required for developing the Cloud, namely virtualization andnetworking, were first developed in the 60’s. In 1967, IBM virtualized operating systems to allowmultiple users to share the same computer and in 1969, the US Department of Defense launched

1

Figu

re1:

Atim

elin

eof

the

Clo

udco

mpu

ting

land

scap

e.Ea

rly

men

tions

ofth

eC

loud

inlit

erat

ure

are

desc

ribe

din

the

gree

nbl

ock.

The

peri

odof

ince

ptio

nis

high

light

edin

turq

uois

e.Th

isis

follo

wed

bytw

oge

nera

tions

ofde

velo

pmen

tson

the

Clo

ud-t

heor

ange

bloc

khi

ghlig

hts

first

gene

ratio

nan

dth

epu

rple

colo

urpr

esen

tsse

cond

gene

ratio

nde

velo

pmen

ts.U

pcom

ing

deve

lopm

ents

and

tren

dsar

epr

esen

ted

asne

xtge

nera

tion

deve

lopm

ents

inth

eye

llow

bloc

k.

2

the Advanced Research Projects Agency Network (ARPANET), which defined network protocolsthat led to the development of the Internet. Although the earliest mentions of Cloud computingin literature appear in the 90âAZs as shown in Figure 1, it was Grid computing [2] that laid thefoundation for offering computing resources as a service to users in the 90’s and early 21st century.The inception of the Cloud as a utility service was realized when Amazon launched its commercialpublic Cloud in 2002.

Significant advances over the last decade can be divided into two generations as seen inFigure 1. The first generation focuses on development at the infrastructure level, for examplecreating data centres, which are centralized infrastructure that host significant processing andstorage resources across different geographic regions. Across other layers of the Cloud stack arange of user-facing services emerged, some of which were available only in specific geographicregions. Software developed by OpenNebula (http://www.opennebula.org) and OpenStack(https://www.openstack.org) allowed organizations to own private Clouds and set up their owndata centres.

As big data started to gain popularity in 2005, the Cloud was a natural first choice to tackle bigdata challenges. This led to the popularity of storage services such as Dropbox that relied on theCloud. Relational databases hosted in the Cloud to support enterprise applications emerged.

Since inception in early 2000 and despite significant research and development efforts span-ning over half a decade, a reference architecture for the Cloud was not defined until 2011(https://ws680.nist.gov/publication/get_pdf.cfm?pub_id=909505). Since the Cloud was anew technology it may have taken a few years before definitions were articulated, circulated, andwidely accepted.

Second generation developments focused on enriching the variety of services and their quality.In particular, management services and modular applications emerged. Monitoring services ofcompute, network and storage resources offering aggregate and fine metrics became available toapplication owners, allowing them to maximize performance. More flexible pricing strategies andservice level agreements (SLAs), in addition to the posted price, pay-as-you-go model, such asspot bidding and preemptible virtual machine instances emerged in 2010.

Furthermore, the move from immutable virtual machines to smaller, loosely coupled executionunits in the form of microservices and containers was a game changer for decomposing applicationswithin and across different data centres. Combining public and private (on-premise) Cloudsof different scale (a.k.a cross-cloud or hybrid Cloud computing [3]) gained prominence in order toalleviate concerns related to privacy and vendor lock-in.

An important step in the evolution of the Cloud was the development of Content DeliveryNetworks (CDNs). Compute and storage resources were geographically distributed for improvingthe overall quality of a variety of services, including streaming and caching. CDNs are the basis ofupcoming trends in decentralizing Cloud resources towards the edge of the network, which willbe considered in this article. Amazon launched their CDN in 2008.

Containers are namespaces with ring-fenced resources. For example, Docker [4] is apopular container technology for creating and managing self-sufficient execution units.Containers offer the prospect of seamless application development, testing, and deliveryover heterogeneous environments. The microservice architectural style focuses on howthe application logic is implemented rather than how it is hosted by dividing services intoatomic functions in order to tame operational complexity. The emphasis is on developingsmall, replaceable service units instead of maintaining monolith services [5].

3

Innovation for the Next Generation

Tangential innovations in the first and second generation developments have made way for anothergeneration of Cloud development that will focus on decentralization of resources. Althoughthere has been a decade long explosion of Cloud research and development, there is significantinnovation yet to come in the infrastructure, middleware, and application and delivery areas.

As shown in the ‘Next Generation Development’ block of Figure 1 in the next five years,computing as a utility will be miniaturized and available outside large data centres. Referred to as’Cloud-in-a-Box’, Sun Microsystems first demonstrated these ideas in 2006 and paved the way forFog/Edge computing. The use of hardware accelerators, such as Graphics Processing Units (GPUs),in the Cloud began in 2010 is now leading to inclusion of even more specialized accelerators thatare, for example, customized for modern machine learning or artificial intelligence workloads.Google provides Tensor Processing Units (TPUs) that are customized for such workloads withthe aim to deliver new hardware and software stacks that extend machine learning and artificialintelligence capabilities both within and outside the cloud.

Infrastructure

A range of hardware accelerators, such as Graphics Processing Units (GPUs), Field-ProgrammableGate Arrays (FPGAs), and more specialized Tensor Processing Units (TPUs) are now availablefor improving the performance of applications on the Cloud. Typically, these accelerators are notshared between applications and therefore result in an expensive data centre setup given the largeamount of underutilized hardware.

Accelerator virtualization is the underlying technology that allows multiple applications toshare the same hardware accelerator [6]. All existing virtualization solutions have performancelimitations and are bespoke to each type of accelerator. Given that this is a relatively new areaof study, we are yet to see a robust production-level solution that can easily incorporate andvirtualize different types of accelerators with minimal overheads.

Currently, applications leverage Cloud resources from geographically distant data centres.Consequently, a user device, such as a smartphone must transfer data to a remote Cloud forprocessing data. However, this will simply not be possible when billions of devices are connectedto the Internet, as frequent communication and communication latencies will affect the overallservice quality and experience of the user. These latencies can be reduced by bringing computeresources to the network edge in a model often referred to as Fog/Edge computing [7, 8].

Edge computing is challenging - Edge resources need to be publicly and securely available.However, the risks, security concerns, vulnerabilities and pricing models are not articulated, oreven fully known. Implementations of standardized Edge architectures and a unified marketplaceis likely to emerge. Furthermore, there are no robust solutions to deploy an application acrossthe Cloud and the Edge. Toolkits for deploying and managing applications on the Edge willmaterialize given the community led efforts by the European Telecommunications StandardsInstitute, the OpenFog consortium, and OpenStack.

Fog/Edge computing [7, 8] refers to the use of resources at the Cloud to the networkedge continuum. Data from user devices can be processed at the edge instead of remotedata centers. Gartner and Forester define Edge computing as an enabling and strategictechnology for realizing IoT and for building future Cloud applications. The Edge com-puting market is estimated to be worth US $6.72 billion within the next five years with acompound annual growth rate of over 35%.

The definition of Cloud data centres is also changing - what was conventionally ‘dozens of

4

data centres with millions of cores,’ is now moving towards ‘millions of data centres with dozensof cores.’ These micro data centres are novelty architectures and miniatures of commercial cloudinfrastructure (see Figure 2). Such deployments have become possible by networking severalsingle board computers, such as the popular Raspberry Pi [9]. Consequently, the overall capitalcost, physical footprint and power consumption is only a small fraction when compared to thecommercial macro data centres [10].

Figure 2: A micro data centre comprising a cluster of Raspberry Pis that was assembled at the University of Glasgow,UK. These data centres in contrast to large Cloud data centres are low cost and low power consuming.

Micro data centres are a compelling infrastructure to support education, where students canbe exposed to near-real data centre scenarios in a sandbox. They are widely used in educa-tional contexts and more convincing use-cases will emerge. Global community engagementwill facilitate the further adoption of micro data centres. International competitions andleague tables for micro data centre deployment would catalyze this process.

The challenges to be addressed in making micro data centres operational include reducingoverheads of the software infrastructure. For example, machine learning libraries must beoptimized for use on micro data centre hardware. Additionally, management tools must runon devices that have only limited network access, perhaps due to the presence of firewalls, orintermittent connectivity as commonly found on edge networks. This is different from traditionaldata centres, which operate on the assumption that managed nodes are directly and permanentlyreachable.

Data centers are now one of the largest consumers of energy (http://www.climatechangenews.com/2017/12/11/tsunami-data-consume-one-fifth-global-electricity-2025/). This is inpart due to the end of Moore’s law and the fact that increasing processor speed no longer offsetsenergy consumption. It is also due to the rapid growth of Internet-of-Things (IoT) - with estimatesof up to 80 billion devices online by 2025 - and increasing use in developing countries. As such,Cloud providers are facing both economic and legislative pressures to reduce energy consumption.

5

It is unlikely that new power plants will be sufficient to meet these growing energy needs andthus there will be widespread use of renewable energy sources and ‘stranded power’ [11].

Middleware

Cloud customers are spoiled for choice, to the extent that it has become overwhelming for many.This is because of the incredible rate at which the Cloud resource and service portfolio hasexpanded. As such, there is now a need not only for middleware technologies that abstractdifferences between Clouds and services, but also for decision support systems to aid customerdeployment of applications. For example, to help guide selection of the best Cloud providers,resources and services, and configuration.

Cross-cloud challenges, such as identifying optimal deployment resources from across the vastarray of options from different providers, seamlessly moving workloads between providers, andbuilding systems that work equally well across the services of different providers will need to besurmounted for establishing a viable Cloud federation.

Such systems, naturally, cannot be one-size-fits-all, but they must be tailored to the needs ofthe customer and to follow any changes in the Cloud provisioning market. Resource brokers arelikely to become necessary to fill this void and provide a means of exploiting differences betweenCloud providers, and identifying the real performance profiles of different Cloud services beforematching them to the customer needs. Currently however, no practical brokering solutions areavailable.

Brokers can also be used to automatically configure the parameters of applications for a set ofselected resources so as to maximize the performance of applications. Typically, the configurationparameters are manually tuned, which is cumbersome given the plethora of resources. A genericauto-tuner that operates in near real-time and measures performance cost-effectively is ideal.However, measuring performance in the Cloud is time consuming and therefore expensive [12, 13].In a transient environment, such as the Cloud, the performance metrics will be obsolete if theycannot be gathered in real time. The complexity increases when resources from multiple providersare considered. A starting point may be to develop bespoke auto-tuners based on domainspecific knowledge that a system engineer may possess using a combination of machine learningtechniques, such as deep neural networks and reinforcement learning [14].

While resource brokerage allows customers to select and combine services to their liking, Clouddata centre operators can also choose a variety of Network Functions (NFs) to create in-networkservices and safeguard networks to improve an application’s performance. The process of orderlycombining different combination or subsets of NFs, such as firewalls, network monitors, and loadbalancers, is referred to as service chaining [15].

Service chaining will be particularly useful for Edge computing to, for instance, improvedata privacy. Recent proposals of intent-driven networking [16] allow operators and end-usersto define what they want from the network, not how. This enables the on demand compositionof bespoke network logic, allowing much more refined application control and dynamicity. Aresearch challenge here is to manage services across Cloud and Edge networks and resources thathave different ownership and operating objectives.

An interesting starting point for implementing service chaining will be creating personal-ized network services across Cloud and Edge environments. This will provide, for example,a user with a personalized security profile that is adaptive to different environments, suchas home or office. Network functions will need to be miniaturized for Edge resources tofacilitate chaining.

6

Application and Delivery

The Cloud will continue to be an attractive proposition for big data applications to meet thevolume, velocity, and variety challenges [17]. Apache Spark, Hadoop MapReduce and its variantshave been extensively used to process volumes of data in the last five years. However, formany users these frameworks remain inaccessible due to their steep learning curve and lack ofinteractivity.

Further, while frameworks such as Storm and Kafka address some of the challenges associatedwith data velocity (e.g., as seen in real-time streaming required by social media, IoT and highfrequency trading applications), scaling resources on the Cloud to meet strict response timerequirements is still an active research area. This requires complex stream processing with lowlatency, scalability with self-load balancing capabilities and high availability. Many applicationsthat stream data may have intermittent connectivity to Cloud back-end services for data processingand it would be impossible to process all data at the edge of the network. Thus, efficient streamprocessing frameworks that can replicate stream operators over multiple nodes and dynamicallyroute stream data to increase potential paths for data exchange and communication will bedesirable.

Innovation will be seen in taming traditional data challenges. For example, emergence of toolsthat alleviate the burden on the user, that support elasticity - dynamic provisioning and fair loadbalancing, and that cleverly move data. As data are increasingly large and distributed, the costof moving data can now exceed the cost of processing it. Thus, there will be increasing interestin moving computation to data and constructing federated registries to manage data across theCloud.

A new class of distributed data management systems, called NewSQL [18], are emerging.These systems aim to offer similar scalable performance to NoSQL while supporting Atomicity,Consistency, Isolation, and Durability (ACID) properties for transactional workloads that aretypical of traditional relational databases. However, providing ACID guarantees across databaseinstances that are distributed across multiple data centers is challenging. This is simply because itis not easy to keep data replicas consistent in multiple data locations. More mature approachesthat will allow for data consistency will emerge to support future modularization of applicationson the Cloud.

A variety of existing and upcoming applications, such as smart homes, autonomous tradingalgorithms and self-driving cars, are expected to generate and process vast amounts of time-series data. These applications make decisions based on inputs that change rapidly over time.Conventional database solutions are not designed for dealing with scale and easy-use of time-seriesdata. Therefore, time-series databases are expected to become more common.

Gartner predicts that by 2022 nearly 50% of enterprise-generated data will be processedoutside the traditional Cloud data centre. Processing data outside the Cloud is the premiseof Edge computing. It is likely that novel methods and tools that perform complex eventprocessing across the Cloud and Edge will emerge.

A new class of applications are starting to emerge on the Cloud, namely serverless applications.They are exemplified by the Function-as-a-Service (FaaS) Cloud, such as AWS Lambda or AzureCloud Functions. FaaS Clouds are implemented on top of upcoming containerization technology,but provide convenient developer abstractions on top of, for instance, Docker. Contrary totraditional Cloud applications that are billed for the complete hour or the minute, serverlessapplications are billed by the millisecond [19]. FaaS has not yet seen widespread adoption forbusiness-critical Cloud applications. This is because FaaS services and tooling are still immatureand sometimes unreliable Furthermore, since FaaS Clouds rely on containers significant overheads

7

are incurred for on-demand boot up. This may be problematic for an end-user facing use cases.FaaS Clouds have limited support for reuse, abstraction and modularization, which are usuallytaken for granted when using distributed programming models. Another practical challenge isthat current-day FaaS services are entirely stateless. All application state needs to be handled inexternal storage services, such as Redis. FaaS providers are currently working towards statefulstorage solutions, but it is as of yet unclear what these solutions will look like in practice.Consequently, significant developer effort is currently required to take advantage of FaaS services.

FaaS Clouds have obvious advantages and we will witness innovation at the virtualizationfront either to reduce the overheads of containers or in the development of entirely newlight-weight virtualization technologies. More powerful programming abstractions forcomposing and reusing Cloud functions will emerge.

Delivery of cloud services via economic models is transforming computing into a commodity,inline with McCarthy’s vision of computing utility. These economic models, and benefits fromeconomies of scale, have underpinned much of the success of the Cloud. The Cloud now utilizesa range of economic models, from posted price models, through to dynamic, spot markets fordelivering infrastructure resources and a suite of higher-level platform and software services [20].AWS even allows users to directly exchange reserved resources with one another via a resellermarket. As the Cloud moves towards further decoupled resources (e.g., as seen in serverlesscomputing) new economic models are needed to address granular resource bundles, short-duration leases, and flexible markets for balancing supply and demand. Furthermore, granular,differentiated service levels are likely to become common, enabling greater user flexibility withrespect to price and service quality. This increasingly diverse range of economic models willfurther enable flexibility; however, it will require greater expertise to understand trade-offs andeffectively participate in the market. Cloud automation tools will emerge to alleviate this burdenby enabling users to directly quantify and manage inherent trade-offs such as cost, execution time,and solution accuracy.

Current Cloud providers operate as independent silos, with little to no ability for usersto move resources between providers. From a technology standpoint, creating federatedClouds via standardized abstractions is a solution, but general markets in which Cloudofferings can be compared and delivered are expected to appear soon. Users may bepotentially offered many interchangeable alternatives, and therefore new economic modelswill need to cater for competition between providers rather than consumers.

Opportunities and Outlook

This article highlights a researcher’s view of the prospects at the infrastructure, middleware, andapplication and delivery level of the Cloud computing landscape.

Although there are ongoing efforts to tackle research challenges at the infrastructure levelin four avenues, namely accelerator virtualization, Fog/Edge computing, micro data centres,and power and energy-aware solutions, the timeline to mass adoption will vary. rCUDA(http://rcuda.net/) and gVirtus (https://github.com/RapidProjectH2020/GVirtuS) are exem-plars of accelerator virtualization solutions that have undergone a decade of innovation, but are yetto become available on production level Cloud systems. With the increasing number of acceleratorsused in the Cloud it is foreseeable that accelerator virtualization technologies are adopted withinthe next decade. Similarly, there are limited Fog/Edge computing test-beds and real deployments

8

of such systems are yet to be seen. While the mechanisms required to adopt Fog/Edge systemsare currently unknown, the growing emphasis on 5G communication will accelerate interest in thearea in the short term. Most micro data centres are still prototypes1 and will become widespreadas more compelling use-cases, for example in Fog/Edge computing emerge over the next fiveyears.

At the middleware level Cloud federation, resource brokers and auto-tuners, and servicechaining were considered. Cloud federation in its strictest sense, i.e. reaching operationalarrangement between independent providers, has for long been discussed but never in factemerged on the horizon. Nevertheless, efforts into tools to bridge services of different providersare expected to continue. Current resource brokers and auto-tuners focus on the challengeof abstraction, but very few tackle this along with delegation, i.e., fully adaptive life cyclemanagement. With increasing use of machine learning, the mechanisms of enabling such smartbrokerage and auto-tuning are becoming available. The related challenges of expressing andinterpreting customer requirements is also becoming a significant trend towards a vision wherethe customer needs to know far less about the infrastructure than current DevOps. In a similarvein, capturing and satisfying what an application needs from the network as seen in servicechaining is a prominent research challenge. However, network operators are not as advanced inthe pay-as-you-go model as cloud providers are. We expect this to change over the next five years,opening up a wide range of resources for supporting applications across an end-to-end network.

The four key areas identified for innovation at the applications and delivery level are complexstream processing, big data applications and databases, serverless applications and economicmodels. The rapid increase in connected devices, IoT, and streaming data is driving the immediatedevelopment of new real-time stream processing capabilities. In the next five years, we expectto see a variety of new stream processing techniques, including those designed to leverage edgedevices. Big data applications and databases have been the focus of recent research and innovation,and thus much immediate focus is on the adoption and use of these efforts. In the next 2-5years, there will be increasing effort focused on time-series databases (following on from therelease of Amazon Timestream and Azure Time Series Insights). Serverless computing is still inits infancy and is likely to be the most disruptive of the changes at the applications and deliverylevel. Over the next ten years, serverless technologies will likely permeate IT infrastructure,driving enhancements in terms of performance, flexibility (for example, support for more generalapplication types), and economic models. Economic models will evolve over the next 2-5 yearsdue to maturing serverless and decoupled computing infrastructure. The underlying modelswill account for energy policies and the desire for more flexible SLAs, and will leverage newopportunities for intercloud federations.

We recommend research focus in designing and developing the following important areas:

• Novel lightweight virtualization technologies for workload specific accelerators, such asFPGAs and TPUs, that will proliferate the Cloud to Edge continuum and facilitate lowoverhead serverless computing.

• System software of micro data centres for remote management of the system stack andworkload orchestration.

• New power-aware strategies at the middleware and application levels for reducing energyconsumption of data centres.

• A unified marketplace for Fog/Edge computing that will cater for competition betweenproviders rather than consumers, and techniques for adopting Fog/Edge computing ratherthan simply making devices and resource Fog/Edge enabled.

1https://news.microsoft.com/features/under-the-sea-microsoft-tests-a-datacenter-thats-quick-to-deploy-could-provide-internet-connectivity-for-years/

9

• Common standards for Cloud federation to allow users to freely move between providersand avoid vendor lock-in. This includes, developing interoperable systems that enforce userpolicies, manage the lifecycle of workloads, and negotiate Service Level Agreements (SLAs).

• Mechanisms for vertically chaining network functions across different Cloud and Edgenetworks, multiple operators and heterogeneous hardware resources.

• Novel techniques for spatio-temporal compression to reduce data sizes, adaptive indexing tomanaging indexes while processing queries, and developing ‘NewSQL’ systems to supportboth transactional and analytical workloads on the Cloud.

Although Cloud computing research has matured over the last decade, there are numerousopportunities to pursue meaningful and impactful research in this area.

References

[1] B. Varghese and R. Buyya, “Next Generation Cloud Computing: New Trends and ResearchDirections,” Future Generation Computer Systems, vol. 79, pp. 849 – 861, 2018.

[2] I. Foster and C. Kesselman, Eds., The Grid: Blueprint for a New Computing Infrastructure. SanFrancisco, CA, USA: Morgan Kaufmann Publishers Inc., 1999.

[3] Y. Elkhatib, “Mapping Cross-Cloud Systems: Challenges and Opportunities,” in USENIXConference on Hot Topics in Cloud Computing. USENIX Association, Jun. 2016, pp. 77–83.

[4] D. Merkel, “Docker: Lightweight Linux containers for consistent development and deploy-ment,” Linux Journal, vol. 2014, no. 239, p. 2, 2014.

[5] I. Nadareishvili, R. Mitra, M. McLarty, and M. Amundsen, Microservice Architecture: aligningprinciples, practices, and culture. O’Reilly Media, Inc., 2016.

[6] C.-H. Hong, I. Spence, and D. S. Nikolopoulos, “GPU Virtualization and Scheduling Methods:A Comprehensive Survey,” ACM Computing Surveys, vol. 50, no. 3, p. 35, 2017.

[7] M. Satyanarayanan, “The Emergence of Edge Computing,” Computer, vol. 50, no. 1, pp. 30–39,2017.

[8] B. Varghese, N. Wang, S. Barbhuiya, P. Kilpatrick, and D. S. Nikolopoulos, “Challenges andOpportunities in Edge Computing,” in IEEE International Conference on Smart Cloud, 2016, pp.20–26.

[9] S. J. Johnston, P. J. Basford, C. S. Perkins, H. Herry, F. P. Tso, D. Pezaros, R. D. Mullins,E. Yoneki, S. J. Cox, and J. Singer, “Commodity Single Board Computer Clusters and TheirApplications,” Future Generation Computer Systems, June 2018.

[10] Y. Elkhatib, B. F. Porter, H. B. Ribeiro, M. F. Zhani, J. Qadir, and E. Rivière, “On UsingMicro-Clouds to Deliver the Fog,” Internet Computing, vol. 21, no. 2, pp. 8–15, 2017.

[11] F. Yang and A. A. Chien, “ZCCloud: Exploring Wasted Green Power for High-PerformanceComputing,” in IEEE International Parallel and Distributed Processing Symposium, 2016, pp.1051–1060.

[12] J. Scheuner, P. Leitner, J. Cito, and H. Gall, “Cloud Work Bench - Infrastructure-as-Code BasedCloud Benchmarking,” in IEEE International Conference on Cloud Computing Technology andScience, 2014, pp. 246–253.

10

[13] B. Varghese, O. Akgun, I. Miguel, L. Thai, and A. Barker, “Cloud Benchmarking For Maximis-ing Performance of Scientific Applications,” IEEE Transactions on Cloud Computing, 2018.

[14] V. Dalibard, M. Schaarschmidt, and E. Yoneki, “BOAT: Building Auto-Tuners with StructuredBayesian Optimization,” in Proceedings of the 26th International Conference on World Wide Web,2017, pp. 479–488.

[15] L. Cui, F. P. Tso, and W. Jia, “Enforcing Network Policy in Heterogeneous Network FunctionBox Environment,” Computer Networks, vol. 138, pp. 108 – 118, 2018.

[16] A. Elhabbash, G. Blair, G. Tyson, and Y. Elkhatib, “Adaptive Service Deployment usingIn-Network Mediation,” in International Conference on Network and Service Management, 2018.

[17] M. D. AssunÃgÃco, R. N. Calheiros, S. Bianchi, M. A. Netto, and R. Buyya, “Big DataComputing and Clouds: Trends and Future Directions,” Journal of Parallel and DistributedComputing, vol. 79-80, pp. 3 – 15, 2015.

[18] A. Pavlo and M. Aslett, “What’s Really New with NewSQL?” SIGMOD Record, vol. 45, no. 2,pp. 45–55, Sep. 2016.

[19] S. Hendrickson, S. Sturdevant, T. Harter, V. Venkataramani, A. C. Arpaci-Dusseau, and R. H.Arpaci-Dusseau, “Serverless Computation with OpenLambda,” in 8th USENIX Workshop onHot Topics in Cloud Computing, 2016.

[20] I. A. Kash and P. B. Key, “Pricing the Cloud,” IEEE Internet Computing, vol. 20, no. 1, pp.36–43, 2016.

11