Implementing Enterprise API ManagementIn the Oracle CloudUKOUGBirmingham | December 4-7, 2016
Luis [email protected]
uk.linkedin.com/in/lweir@luisw19
soa4u.co.uk/
2Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Table of Contents
§ Introduction
§Context
§API Management and API Value Chain
§Enterprise API Taxonomy, Capability Model and Oracle PaaS Mapping
§Use cases§Wrap-up
3Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Resume
I am very passionate about technology. I have be the lead authored of two books (Oracle SOA Governance 11g Implementation and Oracle API Management 12cImplementation), I am a regular blogger and speaker in major conferences and events. A well-known industry expert especially when it comes to Oracle middlewaretechnologies I am also an OTN certified SOA black belt.
Luis Weir Oracle Ace Director – Cloud Principal at Capgemini UKI am an Oracle Ace Director, Cloud Principal and a Thought Leader specialised in Oracle Fusion Middleware & Oracle PaaS. With more than 15years of experience implementing IT solutions across the globe, I have been exposed to a wide wide variety of business problems many of whichI’ve helped solved by adopting SOA architectural styles such as traditional SOA, API management and now Microservices. My current focus is inassisting organisations define and implement solutions and strategies that can help them realise the benefits that such technologies have to offer.
2nd Place1st OTN Cloud
HackathonJune, 2016
CloudContribution Award
SOA CommunityMarch, 2016
Latest Media:§ Oracle Magazine May/June 2016
(http://bit.ly/1RTCAU3)§ Systematic Approach for Migrating to Oracle
Cloud SaaS (http://bit.ly/1Xr6acs)§ Oracle Magazine Jan/Feb 2016 (http://ora.cl/Vhh)§ API Management Implementation
(http://ora.cl/Gcw)§ A Word About Microservices and SOA
(http://bit.ly/25Dk5go)
4Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Table of Contents
§ Introduction
§Context
§API Management and API Value Chain
§Enterprise API Taxonomy, Capability Model and Oracle PaaS Mapping
§Use cases§Wrap-up
5Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
API growth is exponential
§ API growth in the enterprise exponential§ API accelerated growth will continue:
• Partner integration APIs• B2C APIs• Enterprise mobility APIs• IoT APIs
Growth In [Public] Web APIs Since 2005 Programmable Web
1 186 299 438 593 865 1263 15462026
24183422
5018
7182
9011
10302
0
1500
3000
4500
6000
7500
9000
10500
12000
API
Cou
nt
Month
Fastest Growing Web API (%)Categories -6 months Programmable Web
Financial, 70
Enterprise, 66
Backend, 52
Messaging, 43
Advertising, 43
Government, 38
Mapping, 35
Science, 31Social, 28
6Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Ok get it, a lot of APIs, so what?
……But also a lot of ad-hoc mess
7Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
API Management
Planning
Design
Implementation
Publication
Operation
Consumption
Maintenance
Retirement
API
8Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Vertical vs. Horizontal Integration
SYSTEMS OF ENGAGEMENT
Mobile Apps Response web Applications Devices Customer
ServiceBusiness Partners
Horizontal Integration
Asynchronous in nature. Near-real time or batch. Typical integration styles: as pub/sub, data replications, file transfers
SYSTEMS OF RECORDS
Financials EPM HCM Order Management CRM Data Hubs Legacy
Sync
hron
ous/
Rea
l tim
e
Vert
ical
Inte
grat
ion
Main scope for API Management
9Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Vertical vs. Horizontal Integration – Characteristics
Vertical
§ Human behind the trigger§ Information requested on-demand (real-time) § Synchronous in nature. A request expects a response§ Objective is to deliver functionality and/or information in
support of a user journey§ Directly impacts the user experience (regardless of the channel)§ Best realised with API management
Horizontal
§ System behind the trigger§ Initiated by a system scheduled or a system event§ Asynchronous in nature. No immediate response expected§ Objective is to deliver data or messages from a source
system to a target(s) system§ No immediate impact to the user (unless a malfunction occurs)§ Can be realized in a number of ways
Validate,Enrich,
Transform
.... .. .
.. .. . .
....... .
.... .. .
.. .. . .
....... .
.... .. .
.. .. . .
....... .
.... .. .
.. .. . .
....... .
Route,Operate,
Load
Extract,Capture
Expe
rienc
e D
eliv
ery
Systems of EngagementCoworkers Customers
Rapid access, TransformEnforce, Aggregate, Route
Tailor, Deliver
UX
10Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Table of Contents
§ Introduction
§Context
§API Management and API Value Chain
§Enterprise API Taxonomy, Capability Model and Oracle PaaS Mapping
§Use cases§Wrap-up
11Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
The API Value Chain
3 Differentiation
2Strategic
1Tactical
APIM Maturity
Time
Market Edge
Survival
Business value
Public APIsAPIs for revenue generation
APIs for partner collaboration
§ B2B via APIs§ Multi-org integration
APIs for multi-channel enablement
§ B2C APIs for:§ Web, mobile app, social,
direct, etc
APIs for enterprise mobility
§ Multi-device APIs for employee productivity:
§ Q2C, P2P, R2R, H2R, etc
APIs for systems connectivity
§ Cloud/On-premise connectivity APIs:
§ ERP, CRM, HCM, PPM, Legacy, etc
12Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
From Generation Zero to 3rd Generation API Management
Timeline
HTTPReverse Proxy’s
Generation ZeroThe ESB
§ All about ESB’s§ SOA governance in its infancy§ Service gateways as thin layer § Reverse HTTP proxies for
external access§ Very early adoption of cloud
(mainly by SMBs)
ESB
DMZ
SSL
WS-Security
Service Gateways
SSL
2002-2005
XMLXML
XML
DMZ
2014-2017
3rd GenerationAPIs everywhere
{API}{API}
{API} {API}{API} {API}
{API}{API}
Micro GatewayMicro Gateway
Micro Gateway
API Management
Micro Gateway
SOA
Micro Gateways
Microservices
Micro Gateways
Micro Gateway
§ Proliferation of {REST} APIs§ REST/JSON taking over SOAP/XML§ Microservices gain momentum§ Docker containers to package & deploy§ API management changes shape.
The API Micro Gateway is born§ API management as an enterprise discipline§ IoT gaining momentum
{JSON}
{JSON}
{JSON}{JSON}
{JSON}{JSON}
{JSON}
2nd GenerationREST & API Gateways
SSL SSL
API
M
anag
emen
t
2011-2013
WS-Security
SOAWeb Service Management
SCA RulesBPELWS-*ESB BAM Adapters BPMN
WS-S
§ Raise of API management pure-plays§ SaaS adoption starts to gain momentum§ API Management add-ons to 1st Gen§ REST APIs become very popular § API Gateways for SaaS integration§ SOA governance less popular§ Microservices gaining popularity§ IoT early days
{API} {API}{API}
XML
XML
DMZ
{JSON}{JSON}
1st GenerationXML Appliances
{API}
2006-2010
XML Appliances(1st Gen API Gateway)
DMZ
WS-Security
WS-Security
SOAService Gateways
SCA RulesBPELWS-*ESB BAM Adapters BPMN
SOA
G
over
nanc
e§ All about SOA and SOA Governance§ SCA published (OER, UDDI, etc)§ SOA Governance (Enterprise
Repositories, UDDI’s,monitoring and management)
§ XML appliances gain popularity§ Cloud on the radar for large enterprises§ First web (REST) APIs
XMLXML
XML
{JSON}
13Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Table of Contents
§ Introduction
§Context
§API Management and API Value Chain
§Enterprise API Taxonomy, Capability Model and Oracle PaaS Mapping
§Use cases§Wrap-up
14Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Enterprise API Taxonomy
SaaS
API Applications
Finance SCM Legacy, etc CX HCM
[Managed] Business APIs
Single Purpose APIs Utility APIs
Identity
Logging
ErrorHanding
Notifications
Management &
CollaborationDesign & Development
PortalsPolicy
Definition
Lifecycle Management
RuntimeAnalytics
User Management
SYSTEMS OF ENGAGEMENT
Special Purpose APIs Presentation APIs Partner [B2B] APIsPublic [Consumer] APIs
Microservices
SYSTEMS OF RECORDS SYSTEMS OF INNOVATION
SYSTEMS O
F ENA
BLEM
ENT
$
API
System APIs System APIs System APIs System APIs System APIs
Message Pipe
Open Modern Software Architecture (OMESA) | https://community.oracle.com/groups/omesa
15Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
API Management Capability Model
API RegistryAPI Design & Development Portal
API-FirstDesignConsole
ADLProgrammaticValidation
APIApprovalWorkflow
APIDynamicDocumentationAPIDiscovery&Subscriptions
APIApplications&KeysGenerationDeveloperOn-
boardingCommunityCollaboration
ResourceRegistration
ResourceDiscovery
K/VStorage
K/VReplication
ResourceHealthStatus
RegistryAPI
API Management Console
APILifecycleManagement
PolicyDefinition
RuntimeMonitoring
RuntimeAnalytics
APIGatewayManagement
PolicyDefinition
User&RoleManagement
KeysManagement
Delivery
VersionControl Deployment ContinuousTesting ReleaseManagementContinuousIntegration TeamManagement TeamCollaboration IssueTracking SpringBoards
Message Pipe
Message routing Light transformation Reliable Messaging Push Listener & Durable Subscribers Queuing/De-queuing
Single Purpose APIs
FederatedAuthN/AuthZ APIKeyValidation CallAggregation TailoredContracts ThreadProtection
ThreadProtection EmbeddedAPIApplications PushNots Websockets PolyglotConsumer
SDKs
Business APIs
AuthN/AuthZ APIKeyValidation PolicyEnforcement HTTPRouting Redaction
LightScripting In-memoryCache RateLimiting/Throttling Streaming REST/SOAP
Conversions
MicroservicesAPI ApplicationsSystem
AuthN/AuthZConnectivity
AdaptersConnection &
Session Management
Data Transformation
Orchestrations & Logic
Protocol/Transport Conversions
Polyglot Programming
Polyglot Persistency
Single Responsibility
Choreography Stack Independence Auto Scaling
Utility APIs
IdentityFederation
IdentityMappings
ErrorHandling
Logging
Alerts&Nots
ManagementAPIs
RUNTIMEDESIGN TIME & OPS
Open Modern Software Architecture (OMESA) | https://community.oracle.com/groups/omesa
16Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
API Management Oracle PaaS Product Mapping
RuntimeDesign Time & Ops
API RegistryAPI Design & Development Portal
API Management Console
Delivery
Message Pipe
Single Purpose APIs
Business APIs
MicroservicesAPI Applications
Utility APIs
Mobile Cloud API Platform Cloud App Container CloudJava Cloud
API Platform Cloud
SOA Cloud Java Cloud App Container & Container Cloud DB & NoSQL Cloud
Identity Cloud
ManagementCloud
API Platform Cloud
API Platform Cloud
Public SaaSAPI Catalog
Eureka
REGISTRATOR
Developer Cloud
APIPCS OOTB InteroperabilityOracle PaaS Cloud Services
Messaging CloudIntegration Cloud
App Container Cloud
17Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Do I always need those layers? Not Necessarily
“Gather together those things that change for the same reason, and separate those things that change for different reasons” – The single responsibility principle by Robert C. Martin, November 2009, http://bit.ly/1VDgw79
“Domain driven design (DDD) divides up a large system into Bounded Contexts, each of which can have a unified model –essentially a way of structuring Multiple Canonical Models.”
Opportunity
PipelineTerritory
Customer
Product
Customer
Product
Ticket
Defect
ProductVersion
Sales Person
Sales Context Support Context
Use Bounded Context to Separate Concerns
Bounded context by Martin Follower, January 2014http://martinfowler.com/bliki/BoundedContext.html
18Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Bounded Context for Separation of Concerns
Multiple Bounded Context
<<consumer>>
Presentation API
Request(https/json) Response(https/json)
1) Tailored contract, 2) non-standard JSON, 3) API-key/User-token AuthN4) 2 way SSL 5) Embedded API App
Derived from user journey API
Registry
getAPIendpoint(BAPI)
response(endpoint)
1
2
6
Bounded context A
System X<<provider>>
API Application
<<any i.e. lbb>> <<any i.e. lbb>>
DB<<provider>>
API Application
sqlnet sqlnet
Business API1) API-key verification & AuthN 2) Routing, 3) SOAP/REST protocol conversion,
3) Standard JSON format 5) Caching
System API
1) Logic & Transformation 2) Connectivity
1) Logic & Transformation 2) Connectivity
https/soapor
https/json
https/soaporhttps/json
https/soapor
https/json
https/soaporhttps/json
Bounded context B
3 https/jsonhttps/json
4
Single Bounded Context
System A<<consumer>>
Presentation API
Request(https/json) Response(https/json)
1) Tailored contract, 2) non-standard JSON, 3) API-key/User-token AuthN
4) 2 way SSL
Derived from user journey
1
2
5
Bounded context A
3 https/jsonhttps/json
DB<<provider>>
API Application
<<any i.e. lbb>> <<any i.e. lbb>>
1) Logic & Transformation 2) Connectivity
4
19Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Table of Contents
§ Introduction
§Context
§API Management and API Value Chain
§Enterprise API Taxonomy, Capability Model and Oracle PaaS Mapping
§Use cases§Wrap-up
20Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
API {First} Design
APIM Designer Portal
8) Feedback
13) Evaluates14) No changes
7) Evaluates
5) Creates API definition
12) Submits final definition(Github pull request)
9) Updates definition
4) Opens API editor
1) Enters APIM Dev Portal2) Searches API catalogue
3) No match
11) Thumbs up!10) Evaluates
Assertions checks
Assertions checks
15) Set-up continuous test
6) Creates mockup & shares URL
> Dreed, Circle CI
16) Implements API
17) Requests deploy 18) Gets request19) Approves
API Gateway
API GatewayDMZ
API Gateway
ManagementConsole
API Platform Cloud
API Designer
API Developer
API ConsumerDeveloper
Architects
API Developers
API Gateway Admin
DeveloperPortal
API Platform Cloud
21Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Mobile Application accessing System of Records in Oracle SaaS and SFDC
Cloud PaaS
API Gateway
API Platform
Mobile BackendMobile API
Connections
Mobile Cloud
JSON Object TailoringAuth
Business API
Oracle MAF
Validate API-Key
Limits & throttle
User Authn Route Respond
Cloud SaaS
ERP Cloud
Integration Flows
Integration Cloud
Enterprise WSDL
OrchestrateConnect
TransformConnectR
EST
Auth Service
2
36
5
7
8
9
4
1) Update personal info submitted from app. Call to mobile backend API takes place. Authentication would’ve already happened in this example. Mobile API Key is validated2) Backend API code (node.js) transforms object (into enterprise format), injects and calls business API via the REST connector (in theory connector should inject API key and authentication credentials)3) Business API receives the calls and enforces policies as specified, ie. key validation, user authN/authZ, rate limits, possibly custom script and finally routes the request to the backend (system) API (implemented in ICS)4) An integration flow receives the request (in enterprise format). An orchestration is initiated to: 1) update personal info in SFDC, 2) update personal info in ERP cloud. It happens as following:5) The received object is transformed into target system format and included into a request call to SFDC (via enterprise WSDL). ICS takes care of REST/SOAP conversion and also handles authentication and sessions with SFDC6) The received object is transformed into target system format and included into a request call to ERP Cloud (via enterprise WSDL). ICS takes care of REST/SOAP conversion and also handles authentication and sessions with ERP Cloud7) ICS transforms back the object into the enterprise object format and sends back JSON response to the API gateway
8) API gateway sends back the response to the mobile backend
9) The mobile backend API code transforms object to format expected by the mobile app
{json}
{json}
{json} {json} <soap>
<soap>
<soap>
<soap>
1
{json}
{json}{json}
22Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Service Cloud searches on-premises customer master through existing SOAP web service
API Gateway
Oracle SOA Suite
ACS
Mediator DB Adapter
Cloud SaaS
Service Cloud
API P
latfo
rm
Presentation API
Validate API-Key
Limits & throttle
User AuthN
SOAP-REST Respond
Management Console
API Platform
Cloud PaaS
§ Sends stats§ Pulls
deployments
Customer Data Hub
PLSQL
EBS
{json} {json}
<soap><soap>
0) Customer Service Agent conducts a search in Service Cloud to service for a specific customer (ie. Based on first and last name)
1
2
4
71) Service Cloud triggers a call to an API exposed in a DMZ(i.e. https://myorg.com/customers?name=luis&lastname=weir)
2) The API gateway receives the request, validates the API key and user credentials (ie. OAuth 2.0), enforces limit/throttling policies and then converts the payload into SOAP to invoke the business service exposed by SOA Suite internally
Mediator BPEL WS Adapter
6
3
5
sqlnet
DMZ
3) Typically an enterprise business service (EBS) in SOA Suite will just route the request to the relevant application connector service service also in SOA Suite
4) The ACS will transform the request from a canonical model into the application format and via the adapter (ie. Database) will connect to the system of record and conduct the search in any given protocol (ie. SQLNET)
5) The request is converted back into a canonical model and send back to the invoker service
6) A SOAP response in canonical model is send back to the API Gateway
7) A policy converts back the SOAP payload into JSON (most likely removing fields that are not required by the consumer system) and sends back the JSON payload
https
23Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Modern Application in Oracle PaaS
Connectivity Agent
Frontend Application
Application Container
Cloud PaaS
API Gateway
API Platform
Business API
Validate API-Key
Limits & throttleUser Authn Route Respond
Integration Flows`
Integration Cloud
Dequeue ConnectTransform
Customer Data Hub
PLSQL
DB Adapter
On-premises
Microservice
Container Cloud
Node.JS Container
Main Node Application
Connectivity Modules
Microservice Storage
NoSQL Cloud
JSON Objects
https://xxx
1
HTML5/JS
2
[PUT] {json}
9
Integration Flows
Messaging Cloud
REST API Queue
3
4
7
{json}
http 200
{json ack}
{json ack}
{json}
{json}
{json}http 200
56
10
11
12 sqlnet
https§ Registers agent§ Opens connection
https
8
1) User access URL and renders page
2) User performs action in client side (i.e.. Updatepersonal details) which triggers an API [PUT] request
3) A customer business API resource is invoked i.e.. [PUT]/ customers/{person id}. The person update details are passed inthe HTTP body as a JSON. API key and user token are also passed
4) Request is validated (key, user token), policies applied and if successful request PUT request is routed to the relevant customer microservice endpoint
5, 6, 7) The microservice (implemented in Node.JS) executes the business logic which results in updating the customer personal details JSON object in the NoSQL database and also triggering an update event by calling the messaging cloud API. A HTTP 200 response is send back if all goes OK
8, 9) A HTTP 200 response is send back with a small JSON object in the body with an acknowledgment (i.e. { status: “no errors” }
10, 11, 12) Once ICS detects a new message in the topic, it deques the message, transforms it and via the connectivity agent, calls the relevant PLSQL API to update the customer record
Frontend APIsNode Main App
Express ModulesOracle JET Modules
24Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Table of Contents
§ Introduction
§Context
§API Management and API Value Chain
§Enterprise API Taxonomy, Capability Model and Oracle PaaS Mapping
§Use cases§Wrap-up
25Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Oracle Cloud PaaS – Capability Comparison** Only when combined with Developer Cloud
Capability API Platform Mobile Cloud SOA Cloud** Integration Cloud Java Cloud** App. Cont. Cloud**
E2E API lifecycle (design, mock, build, test, publish, manage, monitor)
Hybrid deployment (cloud/on-prem) –native (installed via cloud)
Rich API focused ops and analytics
REST/JSON end to end
API policies definition & enforcement
Authentication & Authorization
Identity federation support (ie. OAuth 2.0)
API keys management and enforcement
Backend (platform) APIs (ie. Push nots, storage, data sync, etc)
WebSockets
HTTP Routing (declarative)
Data transformation (declarative)
Protocol conversion (declarative)
Call aggregation (declarative)
Orchestrations (declarative)
Custom scripting
Connectivity to several sources (excluding pure REST/SOAP)
Polyglot programming
Light footprint
Full Mostly Some or Custom (libs &| imperative) No supportPartly
26Copyright © Capgemini and Sogeti 2016. All Rights Reserved
UKOUG | Birmingham | December 4-7, 2016
Thank you!! … and remember:
“With great APIscomes great responsibility”
The information contained in this presentation is proprietary.Copyright © 2016 Capgemini and Sogeti. All rights reserved.
Rightshore® is a trademark belonging to Capgemini.
www.capgemini.comwww.sogeti.com
About Capgemini and SogetiWith more than 180,000 people in over 40 countries, Capgemini is a global leader inconsulting, technology and outsourcing services. The Group reported 2015 globalrevenues of EUR 11.9 billion. Together with its clients, Capgemini creates and deliversbusiness, technology and digital solutions that fit their needs, enabling them to achieveinnovation and competitiveness. A deeply multicultural organization, Capgemini hasdeveloped its own way of working, the Collaborative Business Experience™, anddraws on Rightshore®, its worldwide delivery model.
Learn more about us at www.capgemini.com.
Sogeti is a leading provider of technology and software testing,specializing in Application, Infrastructure and Engineering Services.Sogeti offers cutting-edge solutions around Testing, BusinessIntelligence & Analytics, Mobile, Cloud and Cyber Security. Sogetibrings together more than 23,000 professionals in 15 countries and hasa strong local presence in over 100 locations in Europe, USA and India.Sogeti is a wholly-owned subsidiary of Cap Gemini S.A., listed on theParis Stock Exchange.
Top Related