Rapid Cloud Application Development with LimeDS Wannes Kerckhove, Thomas Dupont, Bruno Volckaert 

Context The Cloud revolutionized the IT industry and led to the emergence of a rich ecosystem that                               offers countless services and new opportunities to a wide array of different actors. Startup                           companies and small businesses can benefit greatly from the availability of countless reliable                         and affordable Infrastructure as a Service (IaaS) providers, allowing instant scaling of resources                         without the need for upfront investments in expensive hardware. Enabled by operational tools                         such as Docker and Juju, which facilitate the configuration, provisioning and monitoring of cloud                           deployments, Platform as a Service (PaaS) offerings provide dependable and easily                     manageable runtime environments for all kinds of software platforms. By leveraging PaaS                       solutions, operational overhead can be reduced and development cycles can be accelerated.                       Finally, Software as a Service (SaaS) has become one of the prime ways in which people                               interact with software (next to mobile apps) and often provide accessible Application                       Programming Interfaces (APIs) that enable interesting integration­possibilities with other cloud                   applications and services.  The quality and scope of an ecosystem is an important factor, but at the core of its success are                                     the applications (cfr. the landslide in the mobile industry after the release of the first iPhone and                                 later Android). However, developing quality cloud applications & services is not a trivial task: the                             inherent distributed nature of the cloud and the additional effort required for testing are some of                               the obstacles faced in this regard. At the same time, engineering teams all over the world are                                 still holding on to development practices and technologies that were designed before the                         emergence of cloud technologies: large enterprise applications stacks that are hard to configure                         and take a long time to boot are not compatible with the Cloud philosophy of rapidly deployable                                 instances. Middleware platforms that are difficult to extend with new communication protocols or                         are having trouble accepting dynamic data due to the use of some rigorously defined format.                             Libraries on the other hand are not tuned to running in a fully distributed environment and can’t                                 take full advantage of the power of the infrastructure.   The rise of Open Data initiatives along with the APIs provided by Cloud services such as                               Google Maps, Twitter, OpenWeatherMap make it now easier than ever to create meaningful                         applications that can influence the way we live our lives. Unfortunately, these sources cannot                           always be relied upon because of service interruptions (Open Data is mostly provided on a                             voluntarily basis, no SLAs) or because of the limits imposed by the APIs (sometimes as a                               means of revenue, but often as a way to bound system load). 

LimeDS: Wiring Applications in The Cloud LimeDS (LIghtweight Modular Environment for Data­oriented Services) is a toolkit for rapidly                       building JVM­based Cloud applications with a strong focus on developer productivity and                       interoperability with other services and devices. Development started at the Internet Based                       Communication Networks and Services (IBCN) research group of Ghent University with the goal                         of creating a platform that can provide a fast and qualitative solution to the recurring R&D                               challenges IBCN faces in collaboration with the industry.  In the last two years, LimeDS has been deployed in a public transport research project (with a                                 focus on providing a robust service in a harsh railway environment) and is currently being used                               in several Internet of Things and Mechatronics projects as an integration and control flow engine                             between the edge­devices and the Cloud. This successful track record has allowed us to                           expand upon the platform and we are now looking to take on new applications domains.  

  Conceptually, LimeDS relies on a simple abstraction: functional components with JSON                     formatted input and output are used to model modular pieces of logic ranging from basic filter or                                 

processing steps, to storage operations or network service invocations. These components form                       the building blocks of a LimeDS application. Specific services and functionality can then be                           created by combining these building blocks in what we call Data Flows. A visual editor (see the                                 screenshot above) enables users to design Data Flows in a very intuitive way by drawing                             connections between components and by adding small snippets of Javascript code, which is                         well­suited to perform transformations on the JSON message format. Power users can utilize                         the LimeDS Java API to create Data Flows programmatically or to extend the modular platform                             with e.g. additional authentication algorithms or new communication protocols.  The power of this abstraction lies in the uniformity of the components. While the Data Flow                               paradigm can be used to create complex applications with extensive logical flows, the individual                           components always adhere to a simple interface with JSON flowing in and out. This allows the                               framework to provide generic but configurable capabilities to each of the components without                         needing to know the details of its operation. Examples of such capabilities are component                           caching, I/O validation, security, failure detection, load balancing, etc. Additionally, each                     component can be configured to be triggered by HTTP requests, incoming messages or                         scheduling events.  LimeDS is implemented on top of OSGi, a mature Java middleware framework for creating                           modular software. OSGi embraces the dynamic and uncertain nature of software (Is Service X                           available? Has the system loaded a compatible version of library Y?) and provides developers                           with the necessary hooks to react to at­runtime changes in a production environment. Powered                           by OSGi, LimeDS will dynamically wire and rewire links between the components while the                           software is running. If a component is malfunctioning or a connection with a remote system can                               no longer be established, LimeDS is able to discover new paths in the Data Flow, allowing the                                 application to recover from otherwise system­breaking failures.  The headless version of LimeDS (without the visual editor) has a footprint of under 10 MB, can                                 be deployed on any platform with a Java 8 compatible JVM, boots in a few seconds, can run                                   validated, high­performance RESTful Web services and integrate with external Web APIs                     out­of­the­box.      

In order to highlight some strengths of the LimeDS platform, we show a concise example of a                                 Data Flow: 

  The above Java code uses the LimeDS annotation API to construct a component that                           periodically fetches car park information from another component (InfoGetter), filters the                     information for available spots and then uses the LimeDS EventBus to publish this new                           information to enable other components to react (e.g. by notifying a user). The component that                             fetches the car park information is defined as follows:  

 

Note that this component has a HTTP assertion configured that will enable the framework to                             monitor the availability of a remote API. If the API goes down, LimeDS will react by deactivating                                 the component and all components that depend on it. For the example presented here, this                             implies that when the remote API fails, the FreeSpotPublisher component will no longer be                           scheduled to run until the API becomes available again. Image all the boilerplate code that                             would be required to implement this kind of behaviour in a standard Java application.                           Additionally, the component is configured to cache its responses, reducing the load on the                           remote API.  This functionality can of course also be visually constructed using the visual flow builder, which                             will result in the same behaviour running on top of the LimeDS instance.  

 The screenshot above shows a preview of how this flow would look like in the visual builder.                                 This time Javascript was chosen as the implementation language of choice for the                         FreeSpotPublisher component.  To conclude we would like to state that LimeDS is not a silver bullet and there are certainly                                   aspects of the framework where other technologies have a definite edge. However, by                         combining a (visual) building block approach to development with a dynamic runtime that                         facilitates application robustness and with productivity features aimed at reducing boilerplate                     code, all in a lightweight package, we are confident that LimeDS can be an interesting addition                               to the toolkit of any developer working in the Cloud ecosystem. 

What’s next? We are planning to release LimeDS under an Open Source License as we want it to be used by                                     both corporations and private individuals. Our ambition is to build a community around the                           platform to promote the sharing and reuse of components and to ensure the quality, robustness                             and scalability of the platform by incorporating community feedback.  To this end, we are developing a LimeDS repository system that will facilitate the sharing of                               components through means of a Web interface and will provide the necessary mechanisms to                           enable the provisioning and update management of running LimeDS instances. This system will                         also be made Open Source in order to provide teams with a way to setup their own private                                   repositories.  To show our commitment to the platform, we will use the repository system to share a number of                                   components that will allow LimeDS users to easily integrate with industry­standard technologies                       and APIs.  In the long run, we would like to expose the LimeDS platform as a Cloud Service (PaaS) with a                                     wide range of management and debugging tools, to enable users to create production­ready                         and publicly accessible services in a matter of minutes. 

Contact For more information about LimeDS, visit our website at http://limeds.intec.ugent.be. Feel free to use the contact form for further enquiries.