Web APIs - Infrastructure for the (Intelligent) Programmable Web (R&D Talk)

Web APIs - Infrastructure for the (Intelligent) Programmable WebSteven Willmott, 3scale networks

Web Services will fundamentally change the way the Internet works within the next 5-7 years


(and much of that change will happen in the next 2-3)

Change in what way?

Web applications will be totally different

The way content is generated, manipulated and consumed will be totally different

Revenue streams and business models will be totally different

“The adoption of Amazon's Web services is currently

driving more network activity than everything

Amazon does through their traditional Web sites.” Jeff Bar (Amazon evangelist) / Dion Hinchcliffe (Journalist)

“60% of all listings on eBay are added via their

API’s – and eBay services over 6 billion API calls

per month”

John Musser (Programmable Web)

“The API which has easily 10 times more traffic than

the website, has been really very important to us.”

Biz Stone (co-founder, Twitter)

Social Network Apps

Social Network Apps

Image: Steve O’Hear / ZDNET

http://www.flickr.com/photos/39881906@N04/


Social Network Apps

Facebook

OpenSocial

Platform API + Backend

Rapid growth

Image: Steve O’Hear / ZDNET



Mobile (& Desktop) Widgets


Image: NinjaTech4u / FLICKR




Small Application

Not browsers

Arbitrary Data

Synchronous or Asynchronous

Image: NinjaTech4u / FLICKR



Web Data Streams

Web Data Streams

Example: Xignite

50+ data streams of financial information

All by Web Service / API

How many services?

How many services?

programmableweb.com

1500+ APIS

4000+ Mashups

70-80% REST APIs

Growing Fast

Hugely Diverse

What Changed?

SOAP gave way to REST

APIs became easy to build

APIs became easy(ier) to use

APIs became incredibly useful

What Changed?

SOAP gave way to REST

APIs became easy to build

APIs became easy(ier) to use

APIs became incredibly useful

Image: Peta M / Flickr

What more is changing?

Javascript is accelerating

HTML5 is coming

Widgets are taking off

Partnering is getting serious

Why 5-7 years?

Several generations of technology

Today’s leading Web Services will have had transformative effects

Almost all segment followers will have launched Web Services

Internet experience will have caught up for users

No time for 3 or 4 complex 5 year R&DProjects!

Technology

Web Services (APIS...)

Programmable Web:

SOAP Web Services

REST Web Services

Plain HTML Forms

They can all be manipulated programatically to produce an effect

RESTfull Web Services

Resources

Resource Identifiers (URIs)

Representations

Operators

Response Codes

REST powers the (new) programmable web

Operators

OperatorsGETGet Information about a resource

POSTCreate : create new elements in a resource

Update : modify a resource

PUTCreate : Create a new resource

Update : Modify a resource

DELETERemove a resource

OperatorsGETGet Information about a resource

POSTCreate : create new elements in a resource

Update : modify a resource

PUTCreate : Create a new resource

Update : Modify a resource

DELETERemove a resource

Manipulating Data Objects

REST

70-80% of current Web APIs

Very Wide Range of Applications

SOAP Seems unlikely to catch up any time soon

REST




Great Rest Presentation: Alex Puig / Softonic / Slidesharehttp://www.slideshare.net/apuig75/desarrollando-un-api-con-rest

http://www.slideshare.net/apuig75/desarrollando-un-api-con-rest


REST




Great Rest Presentation: Alex Puig / Softonic / Slidesharehttp://www.slideshare.net/apuig75/desarrollando-un-api-con-rest

An Example:



Example APIs: Twitter

Microblogging Service

45-50 Million users worldwide

10x traffic on their API v’s Web Page

Large numbers of client and partner applications

REST API

cURL

curl http://twitter.com/statuses/public_timeline.rss

curl http://twitter.com/statuses/public_timeline.xml

curl --head -u username:password http://twitter.com/statuses

curl -u username:password -d status="Playing around with the twitter API (sent via the API)" http://twitter.com/statuses/update.json

http://twitter.com/statuses/public_timeline.rss






http://twitter.com/statuses/friends_timeline.json




http://twitter.com/statuses/update.xml%C2%A0




What happened to...

Governance?

Semantics?

WSDL?

Trust??

Intelligence in the Programmable Web

Two trends

Two trends

User

Two trends

UserMore time on Well

Known Properties

Two trends


Known Properties

More new ways to access content

sources

Two trends


Known Properties


sources

APIs to feedwidgets

Two trends


Known Properties


sources

APIs to feedwidgets

APIs to feedplatforms

Two trends


Known Properties


sources

APIs to feedwidgets

APIs to feedplatforms

Web Applications will get inc. interconnected

The rules change

The Web Page is no longer King

Service to Service relationships are Business to Business relationships

People care about wether Web Services work

People care that applications built on Web Services work

Key Concepts

Agency: the components you depend on are not owned/operated by you, they are beyond your control.

Contracts: individual interactions need to be mediated by expectations, agreements and (potentially) consequences.

Norms: relationships are long running and require defaults, frameworks and other constructs to make them efficient.

IST FP6 Project: Contract

Full Credit - Javier Vazquez, UPCand others from http://www.ist-contract.org/

(an experiment in how notions of contracts and autonomy can power new service ecosystems)

26

IST-CONTRACT Project Parameters

IST Framework 6 STREP Project Funded from the 5th Call IST

Focus: Contracts for Distributed

Applications Engineering Contracts as a basis for formal

verification e-business applications

Project ID: FP6-034418

Area: Digital Business Ecosystems

Costs: Total Cost: 2,509,156 Euro Req. Cont: 1,850,000 Euro

Dates: Start: 1st Sept 2006 End: 31st May 2009

27

IST-CONTRACT Project Partners

Universitat Politècnicade Catalunya

Fujitsu EST Gmbh Czech Technical University

of Prague King's College London Imperial College London 3scale Networks S. L. CertiCon A. S. Lostwax Media Ltd. Y‘All B. V.

28

The problem: Engineering applications in Cross Organisational Service Oriented Computing environments

The behaviour of a software application depends upon: Code, Execution Context (environment), Inputs

In a multi-organisational Distributed Business Application application: No-one has access to all the code No-one has access to all the execution context (Possibly) no-one has access to all inputs

Question: How do you predict the potential run-time behaviour of such applications?

29

Project Core Idea

Normal Verification approaches for software will not work without full source code access. In Contract:

Instead of predicting actions w.r.t code, predict actions w.r.t obligations, rights, permissions

in Contracts Impacts:

Short term: application design tool Longer term: formal verification of distributed business

applications

30

Where are the Contracts?

Contracts: Are the explicit, tangible representation of service interdependencies Make explicit the obligations of each of the parties in the transactions Make explicit what each system can expect from another

Bind together: The electronic interaction (web services) with The business obligation with Prediction as to whether the system will function to get the job done

31

What does the Project Deliver?

Contract Framework – formal theoretical framework for distributed business application modelling based on the interchange of (electronic) contracts

Contracting Language – specifications of how the actors should interact electronically and how they should communicate

Contract Execution Environment for Web services – to create and execute contract-mediated business interactions

Verification, Monitoring and Analysis tools – to analyze and inspect deployed systems

32

Overall Contract Framework

Contracting Language Communication Model


Domain Ontology Layer

Contract Layer

Message Content Layer

Message Layer

Interaction Protocol Layer

Context Layer

33



Contract Layer


Message Layer


Context Layer

33



Contract Layer


Message Layer


Context Layer

DomainOntology

ContractualOntology

Request

AgreeProtocolhandling: S1

S2

A contract: “the workshop is obliged torepair the car in 2 days”

Domain terms: car, workshop, repair

Statements / actions related to contracts: cancel(contract C1)

Message envelope + intentionality: from service S1 to service S2 …Request[cancel(contract C1)]

Interactioncontext:

34

Electronic Contracts: components

35

Electronic Contract: example

<ISTContract> ContractName="AftercareContract" StartingDate="2007-01-01T00:00:00+01:00" EndingDate="2008-01-01T00:00:00+01:00" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://www.ist-contract.org/schemas/ISTContract.xsd"> <Contextualization> ... </Contextualization> <Definitions> ... </Definitions> <Clauses> ... </Clauses></ISTContract>

35



<ContractParties> <Agent AgentName="KLM"> < AgentReference>http://www.ist-contract.org:8080/services/KLM </AgentReference> <AgentDescription>Royal Dutch Airlines</AgentDescription> </Agent> …</ContractParties>…<RoleEnactmentList> <RoleEnactmentElement AgentName="KLM" RoleName=“Operator"/> …</RoleEnactmentList>

35



<ContractParties> <Agent AgentName="KLM"> < AgentReference>http://www.ist-contract.org:8080/services/KLM </AgentReference> <AgentDescription>Royal Dutch Airlines</AgentDescription> </Agent> …</ContractParties>…<RoleEnactmentList> <RoleEnactmentElement AgentName="KLM" RoleName=“Operator"/> …</RoleEnactmentList>

<Clause>…<ExplorationCondition> <BooleanExpression> Before(2007-07-1T15:30:30+01:00) </BooleanExpression> </ExplorationCondition> <DeonticStatement> <Modality><OBLIGATION></Modality> <Who> <RoleName>Operator</RoleName> </Who> <What> <ActionExpression> PayForEngine(amount, engine, Operator, EngineManufacturer) </ActionExpression> </What> </DeonticStatement></Clause>

36

Example of deployment

37

Verification Tool

Off-line tool implements verification mechanisms for contract-governed systems.

Capable to verify system behaviours through notions of compliance/violations of intended behaviours.

Based in the formal framework and the contract language semantics.

Ability to check systems with large state spaces Capable to generate counterexamples when dangerous or

conflicting situations are detected User friendly GUI

38

Verification Tool components/process

39

Example: Modular Certification Testing

What does this give you?

System interactions are described in terms of obligations, permissions, prohibitions etc.

Agents can act according to expected behaviour

You never have to verify actual code

Seems like SciFi now - but these concepts willbe fundamental in 5, 10, 20 years.

Beyond Contracts

Still need to figure out how to actually build an systems which fulfill your contracts (!)

Contracts need to be connected to something - a context or an organisation - in order to have force - we might want to model that too

We can even go further...

Key Concepts

Projects:

Contract: http://www.ist-contract.org/

Alive: http://www.ist-alive.eu/

More Challenges:

Individual rational actions?

Autonomy v’s Control?

Interaction with Business / Legal Frameworks?

http://www.ist-contract.org

http://www.ist-contract.org

http://www.ist-alive.eu

http://www.ist-alive.eu

Business Relevance

Sounds business relationships are critical

Quality of Service

Management of Autonomy and Dependencies will be Mandatory

Some Final Thoughts



Not Convinced Yet?

Conclusions

The programmable web is coming - faster than you might think

Huge challenges lie ahead in:

Making it work

Making it intelligent

Techniques from AI, Autonomous Systems, Multi-agent and the Semantic Web will be key to understanding the resulting ecosystems

Web APIs - Infrastructure for the (Intelligent) Programmable Web (R&D Talk)

Technology

Transcript of Web APIs - Infrastructure for the (Intelligent) Programmable Web (R&D Talk)