Lessons learned from building enterprise APIsGustaf Nyman

Apitek Aktiebolag

gustaf.nyman@apitek.se

as presented at the Nordic APIs 2016 Platform Summit

About Enterprise APIs

Enterprises have:

• In-house developed systems

• Acquired system

• External SaaS

• Clients as mobiles, tablets etc.

Most systems communicate and expose APIs

Enterprises has many internal APIs

Enterprises has legacy* APIs using old technology

You need API management

Single point API gateway:

• Route message to backend service

• Sometimes convert or translate messages

• Necessary if using microservice architecture

Administration:

• Keep track of routing configuration and metadata

Monitoring:

• Does it work? If not why?

• Is it fast enough?

Miscellaneous:

• Authentication, throttling, signing, validation, archiving etc.

API gateway

Internal API client Internal API server

External API serverExternal API client

API gateway

Internet

IntranetCloud vnet

API gateway

Routing of messages to back-end services

Protocol translation - to support legacy APIs

Keep track of all routing info and metadata

• WSDL/XML Schema, Swagger/JSON Schema etc.

Logging, monitoring and analytics:

• Simplifies monitoring and debugging of connected systems

Performance and reliability

Admin tool

The natural choice for developers and others

Real world: Using legacy API

Task 2002: Access legacy API (monolith) using modern tech?

Legacy API:

• Queue based transport

• Text based message format (structure and value)

• 70 message types

Solution legacy API

Use XML, XML Schema, SOAP and WSDL.

Convert between XML and legacy format using XML Schema

Generate WSDL/XML Schema from repository

API client Legacy API server

XML Schema driven converter

XML Schema

{getspeaker\Nordic APIs\Lesson learned\}

<getspeaker><conference>Nordic APIs<conference><presentation>Lesson learned</presentation>

</getspeaker>

XML Schema driven converter

Queue adapter

SOAP/HTTP Text/Queue

API gateway

Internal API client

Legacy API server

External API serverExternal API client

API Gateway

Internet

Intranet

Converter

Real World: Using legacy API

Task 2015: Access legacy API using modern tech?

Legacy API:

• SOAP/HTTP based transport

• XML message format described in WSDL/XML Schema

Move to modern technology, Microservices?

Solution: legacy API

Use JSON and Swagger/JSON Schema.

Convert between XML and JSON using existing XML Schema

Generate Swagger from WSDL using mapping rules

Lesson: Use metadata

Description of data, message exchange and contracts:

• WSDL/XML schema and Swagger/JSON schema

Use of metadata:

• Validate data and message exchange

• Generate stubs for client and server code

• Generate documentation, test data, user interfaces

• Data format conversion – with certain constraints

But remember:

• API/contract needs to be the first thing you design

• Think before generating from code or tools

• Make sure schemas are comprehensible!

Real World: B2B communication

Swedish insurance business 2002:

• Documents on paper communicated through surface mail.

Task: Define B2B communication guidelines, a ”standard”

Business requirements:

• Non repudiation

• Confidentiality

• Simple to implement on any platform/language

Solution: SOAP/WS-Security

Use standards, but define how and simplify:

• SOAP/HTTPS and WSDL/XML Schema

• X509 certificates and WS-Security for signing

• X509 client certificates

Add what’s missing:

• Organization identification header and correlation id

• A few standardized message types

10 years later still the natural B2B choice in the Swedish insurance business.

Lesson: Simplify and select

If you want things to last..

• Do not use ”everything” technologies provide – select the good stuff!

• Read the specifications – do not rely only on ready-made toolkits or frameworks.

• Remember: Simple things are easier to migrate and adapt to new technology

Problem: API failures

All APIs will eventually fail…

Ongoing task: improve analysis and production debugging support

Monitor and log

Log types:

• Processing logs – what happened?

• Data logs – what was the message content at specific point?

• Analytics logs – what is the response times and failure rate?

Logging subsystem:

• Should be always on

• Must be high performance and

• Must handle large amount of logs

• Must correlate and cluster related log data

What to log?

“Logga lagom”, i.e. log just enough information

• Too much and it becomes incomprehensible and unmanageable

• Too little and it is useless

Use experience and an evolutionary approach:

• You debug a difficult problem – improve log to enable easy identification next time

• Always strive to enable less experienced personnel to effectively debug using logs

Real world: Logging subsystem

Always on log system logs to disk – remove after 24 hours

On API failure related log files are stored in database

Web site to:

• Access and view error log data

• Automatic classification of errors using patterns

• Searching and reading logs on disk

• View other logs and system parameters

Take care of all your users

Your platform should be the natural choice for developers

• Cater to their needs.

More than developers:

• Many more are interested in what you are doing

• Many users are not computer savvy – they need simple tools

Typical non-technical users:

• People responsible for the business

• Testers with extensive business skills

• Customer service

Problem: Old technology

If we only redesign our system with…

• RPC

• CORBA/MTS

• SOAP/SOA

• REST/HTTP

• Event driven

• Microservices

• [your choice of the next big thing]

We will solve all problems and everything will be much better…

Everything will be legacy

…It is just a matter of time.

If redesign is not an option, put legacy APIs behind converters, translators and adapters:• Expose the modern API you prefer

Be prepared…• Use standards – avoid vendor specific extensions• Be selective – do not use everything• Simplify

…mitigate the consequences of systems turning legacy

Summary

Use an API gateway

Use metadata

Simplify and select

Monitor and log

Prepare for systems turning legacy

gustaf.nyman@apitek.se

https://www.apitek.se/

Do not miss my talk tomorrow 14.15:

Build, Deploy and Test APIs and Microservices with Apitek Prisma