Integration Microservices

Bridging Microservices, Integration, and APIs

SF Microservices - June Meetup 2017 Kasun IndrasiriDirector-Integration Architecture at WSO2

About Me

● Director - Integration Architecture at WSO2● Author ‘Beginning WSO2 ESB’, several microservices articles ● Committer/PMC Apache Synapse

WSO2

● Open source middleware company ● API Management, Integration, IAM and IOT platforms.

Microservices Architecture in a nutshell

Monolithic Applications

Microservices Architecture in a nutshell

SOA

Microservices Architecture in a nutshell

SOA with APIs

Microservices Architecture in a nutshell

Microservices

Why Microservices Compositions?

● Too many fine-grained microservices. ● Business functionalities often require to interact with more than one

(micro)service. ● Microservices typically don't publish their APIs for direct invocation.● Brown-field Reality requires ‘Integration’

○ Green fields enterprises don’t exists. ○ Hybrid usage of Microservices and monolithic architecture is quite common. ○ Need to integrate with existing legacy systems, proprietary systems, databases and

SaaS.

Microservices and ESB - Facts and Myths

● How to Integrate Microservices/How to create microservice compositions? ● From ESB to ‘Smart endpoints and dumb pipes’

○ Routing logic or business logic that resides at the central ESB layer has to be segregated and distributed among the (smart) clients and (micro) services.

(source : YOW! 2016 — Microservices by Martin Fowler)

Microservices and ESB - Myths and Facts

Point to Point Interactions

● Strict, dogmatic adherence to ‘smart-endpoint and dumb pipes’ results Point-to-point(P2P) connectivity between your services and consumers.

● P2P approach is not scalable and exponentially increases the complexity of the service interactions.

 ‘Smart-endpoint and dumb pipes’ in practice (source : Microservices in Practice)

Glimpse of existing Microservice Implementations

● How do the existing microservice implementations handle the service

compositions/service orchestration?

Microservice - Compositions

Netflix

● The Netflix API is the “front door” to the Netflix ecosystem of microservices.

● API provides the logic of composing calls to all services that are required to construct a response.

● The Netflix API is an orchestration service that exposes coarse grained APIs by composing fined grained functionality provided by the microservices.

● Orchestration logic is built with Java, RxJava

Microservice - Compositions

Uber

● ‘Edge Services’ which are exposed to the external client/mobile applications and the service orchestration logic is burnt into the edge service.

● Edge services are primarily implemented on top of Node.js

Microservice - Compositions

Paypal

● The API façade layer exposes Paypal business functionalities to various internal and external client applications

● The orchestration logic resides in the API façade layer and its implemented using Groovy.

Miniservices

“A miniservice is a coarse-grained, independently deployable and independently scalable application component.”

● “Innovation Insight for Miniservices” : Published: 21 February 2017

○ https://www.gartner.com/doc/3615120/innovation-insight-miniservices

Pragmatic Microservices Architecture

● Microservices types with different granularities

Integration Microservices

● Composing microservices and create a new service● Similar to the ‘Miniservice’ concept wrt to the granularity ● Integrate web APIs/SaaS, legacy systems and microservices● API services/Edge service is also an integration microservice with some API

gateway capabilities.

Technologies for building Integration Microservices

● Frameworks based on general purpose lang. : Java, Node.js, Groovy - e.g.: SpringBoot, Dropwizard ○ Not designed with suitable abstractions for integration/network interactions○ Compositions cannot be illustrated graphically.

● Apache Camel and other ESBs. ○ Bulky traditional ESB architecture -> Not fully compatible with MSA principles. ○ High level DSLs with a lot of tweaks to make it a powerful programming lang. (e.g. expression

languages, properties)

● Ballerina - ballerinalang.org.

Ballerina (ballerinalang.org)

● Ballerina is a new parallel programming language that is designed and optimized for integration.

● Graphical and textual syntax which is built on top of the sequence diagram metaphor.

● Designed for network interactions with JSON/XML/SQL/MIME and HTTP/JMS/File/WebSockets

● Lightweight, high-performance and container native Integration runtime ● 100% open source.

Hello World - Service

Hello World - Main

Ballerina - Key Concepts

Components Overview

Ballerina - Key Concepts

Service and Resource

● A service comprises of a homogeneous collection of network accessible entry points; ‘resources’.

● A ‘resource’ contains the set of statements which are executed sequentially.

Ballerina - Key Concepts

Workers

● Worker is a thread of execution that can be programmed independently. ● A resource or a function contains the default worker that where you

sequentially configure the ballerina programming logic.

Ballerina - Key Concepts

Server Connectors

● Services are protocol agnostic. ● You can bind a ‘Server Connector’ to any service to receive messages from

respective the network protocol.● Are defined as platform extensions – not written in Ballerina● May offer of utility functions to access info in the message● Can be configured via annotations that bind a service to a server connector.

Ballerina - Key Concepts

Connectors and Actions

● Ballerina is all about integrating with other systems using connectors● “Other systems” are represented by a (client) “connector” that defines a set of

“actions” that you can do with that system● The client connectors represent an external system that is accessible through

the network such as HTTP service, Database or any SaaS application.● Graphically, the connector is represented as an actor in the sequence diagram

(i.e. a vertical line) and actions are boxes that go into a worker with lines going to the actor.

Ballerina - Key Concepts

Functions

● Represent the reusable integration scenarios/code.

Ballerina Demo

Code is located at :

https://github.com/kasun04/rnd/tree/master/ballerina_use_cases

Ballerina knows main() and services

● Run as a command line program● Run as a network service.

Ballerina knows JSON, XML and SQL data

● All part of Ballerina type system● Deeply unified and integrated both syntactically and semantically● Data transformation across all of these graphically and textually● Transactional data management

Ballerina knows network protocols

● HTTP, WebSockets, JMS, FTP, Files, ... ● Popular APIs: Facebook, Twitter, Gmail, LinkedIn, …● Extensible authentication / policy support

Ballerina knows Swagger

● Ballerina services’ interface is Swagger and Ballerina syntax, Swagger YAML syntax and graphical syntaxes are interchangeable

● Reduces pains of interface-first design

Ballerina is naturally parallel

● Sequence diagram approach makes parallelism innate● Think parallel first, not sequential first

Ballerina supports safety & resiliency

● Highly structured error and exception handling● Taint tracking of network delivered data● Stability Patterns : Supports Timeout, Circuit Breaker ● Failover, Load balancing, Reliable delivery.

High Performance and Container-native runtime

● Performance ○ Extremely lightweight and high performant runtime. ○ Minimal Startup time(< 1s), low resource consumption.

● Container-native ○ Built in ability to create Docker image with executable Ballerina program package○ Run on any container management platform

Ballerina is Open Source!

● Implementation released under Apache License v2.0○ Fork me on GitHub: https://github.com/ballerinalang/

● Community○ Developers: ballerina-dev@googlegroups.com○ Slack: #ballerinalang○ Twitter: @ballerinalang○ StackOverflow: #ballerinalang

Ballerina - Future

● Why yet another language?○ Make it easier to consume and produce networked services and applications.○ Ballerina has been influenced by Java, Go, C, Node, Javascript, Maven, Tomcat, and a variety of

other awesome tools.

● When should I use Ballerina?○ Write integration microservices/miniservices: 80-20 rule

■ If 80% of your service is about integrating with other services, data and APIs then use Ballerina.

■ If just 20% is integration then use Java / Node / Go / PHP / XYZ○ Re-compose existing services to be API backends○ Write integration scripts

■ Replacement for shell scripts that use curl a lot

Ballerina - Try it!

● DOWNLOAD v0.87 and twirl away:○ http://ballerinalang.org/

● Source Code : https://github.com/ballerinalang/● NOTE: Still work-in-progress. New releases every two weeks with major

features still coming along. Missing docs. A (few) bugs. Etc.!

● Integration Microservices plays a key part of MSA. ● Understanding service granularities. ● Technologies for building integration microservices.

Summary

Thank You!

● kasun@wso2.com● https://twitter.com/kasunindrasiri