Dr. Donald Ferguson IBM Fellow, SWG Chief Architect Chair, IBM Software Group Architecture Board
© Donald F. Ferguson, 2014. All rights reserved. Topics in Computer Science: Modern Internet...
-
Upload
osborn-armstrong -
Category
Documents
-
view
213 -
download
0
Transcript of © Donald F. Ferguson, 2014. All rights reserved. Topics in Computer Science: Modern Internet...
© Donald F. Ferguson, 2014. All rights reserved.
Topics in Computer Science:Modern Internet Service Oriented Application DevelopmentLecture 5: Composite Applications
Dr. Donald F. [email protected] (Admin: [email protected])
2 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Contents
3 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Contents• Components and composite applications
– Introduction– Setting the context
– Composite applications
– Types of composition; Styles of integration
– Modeling, visual notation and executable processes
– Structural composition: A conceptual model – Service Component Architecture
– Data flow composition: Message/event/service bus oriented composition (iPaaS)– Basic building blocks
– Message/data flows and processes
– Control flow composition: Workflow and orchestration oriented composition– Basic building blocks
– Processes and workflows
• Next assignment
• Discussion
4 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
ComponentsandCompositeApplications
5 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Introduction
6 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
APIs Galore
7 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Setting the Context – “The World is not Enough.”
The world gives you … You ned to make …
8 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Composite Applications –Some Definitions• “In computing, a composite application is a software application built by combining multiple existing functions
into a new application. … A composite application consists of functionality drawn from several different sources.
The components may be individual selected functions from within other applications, or entire systems whose
outputs have been packaged as business functions, modules, or web services.” (http://
en.wikipedia.org/wiki/Composite_application)
• “Composition refers to a way of delivering enterprise solutions by assembling them from prebuilt components,
instead of building them from scratch. It also includes personalization and customization abilities, so that users
can easily and quickly modify specific functionality in the solution.” (http://
msdn.microsoft.com/en-us/library/bb220803.aspx)
• “A composite application orchestrates independently developed programs, data and devices to deliver a new
solution that none of the previously available applications could deliver on its own. Each resource accessed by a
composite application uses a different data model. In most cases, the composite application supports user
interactions beyond those provided by the leveraged applications. In other cases, a composite application may
act as a service. Some composite applications are built by leveraging other composite applications.” (http://
www.gartner.com/it-glossary/composite-application-2)
9 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Composite Applications –Some Pictures
10 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Setting the Context• You have learned
– How to build a reusable “component”
– That surfaces a well-defined, reusable API
– And how to monitor, trace, manage and secure the API.
• Any moderately sized company or organization, there are– Thousands of internal APIs.
– 100s of published APIs.
– 100s of consumed, public, external APIs
– Tens (or hundreds) of thousands of available, external, public APIs.
• By and large,– Application development involves
– Controlling the ordering, flow and execution (if … then, while … do, etc) using “code”
– Of API calls (JDBC, class libraries, etc.)
– Modern, Internet SOA Application Development involves– New techniques for controlling the ordering, flow and execution
– Of “web” callable APIs.
11 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Composition Diagrams
• The diagrams are always “nodes” and “arcs.”
• Structure: node A <x> node B, where x is Requires/depends/calls/synchronizes with/…
• Control:– Execute A and then execute B, where A and B are tasks.
– There is a shared set of “data objects” the tasks manipulate.
• Data flow: Do A to the message/document/… and then do B to A’s output
Control Flow Structure Data Flow
12 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Simplistic Taxonomy of Integration
Integration
UIInformationIntegration
ETL, MapReduce
Federated DB
Message/Event
CEP/CPEMessage
Flow
RT events
WorkflowOrchestratio
n
Human Centric
Automation
13 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Modeling, Visual Notation, Executable Models
• Development projects tend to be systematic or opportunistic.
• Opportunistic projects and some systematics project team will tell you that they do not use modeling because– We tried it and it did not work.
– Too slow and too much time. We are agile and want to get to the code.
– That’s the way old guys do it.
• I ask the following questions– Do you have meetings and scrums?
– Do you use a whiteboard? Do you draw diagrams?
– Does someone copy down the information and convert to PPT or diagrams?
– Do you take pictures of the whiteboard?
– Do you email the pictures and PPT?
– Do you realize you are doing modeling, just poorly?
• My view– A model and visual notation is like PPT, but there is precise and consistent interpretation of shapes, colors, symbols, …
– Like PPT, there is an underlying file format/schema for the diagram.
– An executable model has a file format that is complete enough (with possible editing) that some engine can interpret the file the
same way that a JVM interprets bytecodes or a CPU interprets instructions.
14 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Three Core Concepts
Editor
Visual Notation Diagram
“Program”
“Engine”
15 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Of Course, maybe that’s just me …
16 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
StructureCompositionDiagrams
17 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Service Component Architecture
18 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Flexible Implementation – Some of which we will Cover
19 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Eclipse SOA SCA Tool
20 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
OSGi Blueprint are Similar
21 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
OSGi Blueprints
22 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Comments
• Structural composition and models is a recurring concept– OSGi– SCA– J2EE dependency references– Camel– UML Module and Component Diagrams– etc.
• The concept waxes and wanes in popularity, and seems to be waning.– The explosion of micro-services and ad hoc applications.– Structure integrated into behavior composition– More dynamic, query based binding
• We will not spend more time on the topic.
23 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
DataFlowComposition
24 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Point-to-Point Connections are BadDoing this … Leads to this … …
25 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Q&ADiscussion
26 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
SettingtheContext
27 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
An“Old Cranky Guy”ExampleSCA
28 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Three Core Composition Models• Structure
– What are the APIs, and possibly components, that I need? For example, a commerce application needs– Content management system for images, manuals, how-to videos, …– Catalog systems for products, sub-products, prices, …– Inventory management– Credit card processing– Customer account and relationship management
– And what gets connected to what, and how.
• Behavior– Data flow
– Updating thing T in component C needs to– Send an update(Q) thin in component B and D– But I need to split the data because B and D only need some of T– And I need to remap the data because B and D have different schema for “T” stuff.
– Control flow: Opening a new credit card requires things to happen in order– Validate the application is correct– Perform a credit check– Call a system to create a risk score– Have an analyst approve the application– Send an email to the customer for approval– Send a message to the place that is going to make the card– … …
• People like diagrams.
• But tend to use similar diagramstypes for all composition.
• And be inconsistent about whatsymbols means
29 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
You Have Already Seen the Pattern
• This is the same pattern as API Mgmt
• But …
• What happens in the “proxy”is a lot more complicated.
• The relationship betweenthe proxy and the APIs
• Is many-to-many
• And has data flowor sequencing.
30 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Integration is PurpleBoomi Example
31 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Enterprise Integration Patterns with WSO2 ESBhttps://docs.wso2.com/display/IntegrationPatterns/Enterprise+Integration+Patterns+with+WSO2+ESB
32 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
More Examples
33 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
More Examples
34 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
How do you Build the Composite
• Take off a palette
• Set properties
• Put on canvass
• Set control values
• Connect flow
35 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Composition• Each “clipart widget” has
– A unique, documented shape (visual notation).– A little implementation piece of code (think POJO) that does the function.– Rules about what other widgets can connect to it/from it.– A set of properties that tailor this instance’s behavior
• For example, – When I drag a routing widget onto the canvass to make a routing instance– I specify
– The valid, input JSON format.– A the routing rule table
– {“Person.iq” <= 50; http://www.Columbia.edu/api/professors}– {“Person.color” == “Yellow”; http://www.pbs.org/api/SessameStreet/birds}
– And connect to other things on the palette
36 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Dell Boomi
37 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Test Drive
38 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Workflow/BPMN
39 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
BPMN
BPMN 2.0 is an international standard for business process modeling.The BPMN specification describes
• how the elements of a process diagram have to look like (notation),
• how these can be combined with each other (meta model / syntax),
• what a diagram means (semantics) and
• how diagrams can be transferred from one tool to another (XML interchange format).
Process models describe sequences of business activities from start to finish, e.g. Order-to-Cash, Account Open
40 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
BPMN and Activiti• Some terms
– BPMN 2.0 is a standard for defining/documenting workflows/orchestrations.
– Activiti (and many others) are implementations of the standard– Runtime engine that can interpret and execute a BPMN process
– A WYSIWYG editor– With prebuilt, configurable shapes palette that extend base BPMN shapes. Huh?
• BPMN has the concepts of a “Task,” which is one “step” in a process.
• But in a process model, this is just “prose” like “Order a Pizza”
• Implementations come with “common” types of tasks, e.g.• Make an SQL call on a database.
• Call a REST API
• Put a document on a worklist
• etc.
– A composition canvass for assembling shapes into processes.
– Debugger, monitoring, reporting, versioning, etc.
• This is– The same concept as enterprise integration patterns and Boomi
– But, – The clipart is different
– And the graph mostly represents “control flow,” not data flow.
41 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Simple BPMN Diagram
42 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
BPMN Symbols
43 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
BPMN Symbols
44 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
BPMN Symbols
45 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
BPMN Symbols
46 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
BPMN Symbols
47 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
BPMN Symbols
Lots more go here.
There are references, e.g. http://www.bpmn-tool.com/en/tutorial/
48 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
BPMN Symbols
49 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
BPMN Symbols
Seriously dude?
There are references, e.g. http://www.bpmn-tool.com/en/tutorial/
50 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
BPMN Symbols
51 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
And You can Define Processes
52 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
How do you Implement?• Notify a person/role.
• Have them do something.
• Have the use anapp that calls theengine to tell youthey are done.
• Send an email.
• POST a document
• etc.
• Decision table
• Call Drools
• etc.
• Insert into a DB
• Call a web service
• Run a Java app.
• etc.
• Run some JavaScript or Java or …
• Right at this point in the workflow/process
• And manipulate the data/documents/control.
53 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
User Task
54 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Script Task
55 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Service Task
56 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
57 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
58 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
59 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
60 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
61 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
62 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
63 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
64 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
CommentsThree Core Aspects of BPMN and Actviti
• Human Centric Workflow– Define an organization model
– Users, Groups– Properties for users and groups
– Define workflow processes that have Human Tasks. For each Human Task– Define the form and data that enables a person to perform the task.– Define “rules” for assigning a task to people or groups.
• Automation/Orchestration– Implement a set of POJOs implementing the Delegate interface. Each POJO implements an automation
action by …– Call a REST service– Sending a message on JMS or SQS– Etc.
– Define a workflow with one or more Service Tasks. For each Service Task– Specify/bind to a Delegate.– Map process data variables to inputs and outputs of the Delegate.
65 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
AWS Simple Workflow
66 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
AmazonSimple Workflow
67 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Application Structure• Role of the Activities Worker: The activities worker performs the
various tasks that the workflow must accomplish. It consists of:– The activities implementation, which includes of a set of activity methods
that perform particular tasks for the workflow.
– An activity worker, which uses HTTP long poll requests to poll Amazon SWF for activity tasks. When a task is available, Amazon SWF responds to the request by sending the information required to perform the task. The activity worker then calls the appropriate activity method, and returns the results to Amazon SWF.
• Role of the Workflow Worker: The workflow worker orchestrates the execution of the various activities, manages data flow, and handles failed activities. It consists of:
– The workflow implementation, which includes the activity orchestration logic, handles failed activities, and so on.
– An activities client, which serves as a proxy for the activities worker and enables the workflow worker to schedule activities to be executed asynchronously.
– A workflow worker object, which uses HTTP long poll requests to poll Amazon SWF for decision tasks. If there are tasks on the workflow task list, Amazon SWF responds to the request by returning the information that is required to perform the task. The framework then executes the workflow to perform the task and returns the results to Amazon SWF.
68 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Assignment 3
69 © Donald F. Ferguson, 2015. All rights reserved.
Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications
Backup