1
Copyright © 2006, ZapThink, LLC 1
The Nuts & Bolts of SOA
Jason BloombergZapThink, LLC
Take Credit Code: NONUTS
Copyright © 2006, ZapThink, LLC 2
The Three Core SOA Infrastructure Challenges
Security
Management
Governance
2
Copyright © 2006, ZapThink, LLC 3
The ZapThink SOA Roadmap:Infrastructure Focus
Copyright © 2006, ZapThink, LLC 4
Juggling SOA Infrastructure
• Successful SOA requires several infrastructure elements– Challenges: – Priorities?– Budgets?– Identifying needs?
• Iterative architectural approach should drive infrastructure decision
Security
Inte
gra
tion
Proc
ess
Man
agem
ent
Gove
rnan
ce
3
Copyright © 2006, ZapThink, LLC 5
The SOA Reliability Conundrum
• Three aspects of reliability– Reliability of runtime
• Will the system work as planned?– Reliability of design
• How can we make sure we have consistency?
– Reliability of people• What if people don’t behave as they
should?
• Traditional view of reliability now insufficient– Reliable components do not guarantee
reliable Services– Loosely coupled composite apps require
new approaches to reliability
• SOA ROI hangs in the balance
Copyright © 2006, ZapThink, LLC 6
XML: The Foundation for SOA
• XML now the lingua franca of distributed messaging
• The XML foundation must perform or the SOA will crumble
http://www.bluestonepartners.com/schemas/StockTrader/RequestQuote
http://schemas.xmlsoap.org/ws/2003/03/addressing/role/anonymous
Message ID and UUID
http://localhost:8080/StockTraderSecure/StockTrader.asmx
Contains Message Creation/Expiration TimeStamps
MSFT
Source: http://www.windowsitlibrary.com/Content/1219/06/1.html
4
Copyright © 2006, ZapThink, LLC 7
The SOA Performance Issue
• SOA requires more metadata than ever…– Web Services-based SOA is mostly
XML-based– XML is text heavy and verbose– More metadata… more XML… more
headaches• The XML Processing challenge
– Per-message parsing, processing– Security steps– Validity steps– Mapping XML to memory models– That darn text format!
• Security, Reliability, Process, and Transaction now on a per-message basis
• High volumes of messages and very largemessages two separate problems
Copyright © 2006, ZapThink, LLC 8
The OSI Stack Revisited
• Content Security, as well as– Semantics– Taxonomies– Policy– Schemas– …
7
6
5
4
3
1
2Traditional perimeterTraditional perimeter--
based approaches wholly based approaches wholly inadequateinadequate
5
Copyright © 2006, ZapThink, LLC 9
The XML Processing Problem
– Receive– Authenticate– Decrypt– Validate– Canonicalize– Transform– Parse– Aggregate– Sign– Encrypt– Transmit
• Steps required to process XML for each message:
Copyright © 2006, ZapThink, LLC 10
parse…send… receive…
understand… store…update… validate…
…parse
The XML Performance Crisis
XML is “Heavy”$FF becomes:
255
Large XML data sets are sent to conform to
bloated XML “industry standard” schema
definitions
Eventually this huge document needs to be fully processed –
which means turning it into usable “objects”
Traditional applications are adapted to services,
sending thousands of messages over the wire
Source: RogueWave
6
Copyright © 2006, ZapThink, LLC 11
What about Binary XML?
• Binary– Efficient, Compact, Proprietary
• “Dumb” approach: compress & decompress
• Adds to processing overhead– Smarter approach: binary
representation of preprocessed XML
• Incremental transmission andprocessing of XML documents
Binary XML of limited use until some standard Binary XML of limited use until some standard becomes widely adoptedbecomes widely adopted
Copyright © 2006, ZapThink, LLC 12
XML traffic on the Network
Network Traffic Utilization
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2002 2003 2004 2005 2006 2007 2008
Other
XMLMail*HTTP*
Network Traffic Utilization
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2002 2003 2004 2005 2006 2007 2008
Other
XMLMail*HTTP*
Makeup of XML Traffic
0%10%20%30%40%50%60%70%80%90%
100%
2004 2005 2006 2007 2008
Device-to-device messages
REST-based messages
"Proprietary" XML formats
Web Services
Makeup of XML Traffic
0%10%20%30%40%50%60%70%80%90%
100%
2004 2005 2006 2007 2008
Device-to-device messages
REST-based messages
"Proprietary" XML formats
Web Services
7
Copyright © 2006, ZapThink, LLC 13
XML in the Networked Enterprise
Source: Tarari
Storage
StorageSwitch
XMLIntegrationSwitch/LoadBalancer
Router
FirewallFirewall XML Security
FirewallVPN
Bandwidth Mgr.
VoIP Server
IDS
Cache
Switch/LoadBalancer
SSL
XML Mgmt.
Switch/LoadBalancer
XML Router
XML Gateway
XML Transformation
Front-endServer
Back-endServer
Storage
Internet
PSTN
Access Control Front-End ServersBack-End Servers & Storage
Purpose:• First line security• Authentication• Access Control• WAN Mgmt
Purpose:• Apply security policies • Control traffic via Services• LAN Management• Switching & Routing
Purpose:• Execute applications• Enable internal server to
server communications• B2B & EAI
Purpose:• Database Access• Transaction Control
AV GatewayPub/SubContent Based Switching/Routing
Application Firewalls
Security & Gateways
Publish & Subscribe
Web Services Management
Web Acceleration/Control
“Raw” XML Acceleration
XML Databases
Web Application Servers
Events Processing
Copyright © 2006, ZapThink, LLC 14
Transformation: A processing challenge
SOA relies on loose coupling, loose coupling relies on transformation
Problem: not all Consumers are the same!
• While 80% of a document format or Service might be appropriate, it’s the 20% differences that cause all the problems– Data format differences– Standards versioning– Protocol differences– Loosely coupling presentation from data
8
Copyright © 2006, ZapThink, LLC 15
Hardware vs. Software Approaches to Improve XML Performance
• Pros:– Dedicated processing
power– Control of variability– Hardened OS– Simple to install &
maintain• Cons:
– Must go in data center– More difficult to upgrade– Limited customizability
• Pros:– Developers in control– Easy to customize &
upgrade• Cons:
– Harder to install & maintain
– Relies upon hardware for performance
– Too much variability in configuration
Hardware Software
Copyright © 2006, ZapThink, LLC 16
Is Moore’s Law the Solution?
• One idea – throw more iron at the problem– Moore’s Law and its corollaries – growing
processor power, storage space,bandwidth…
– Cost per computing is going down…– But cost of management and complexity,
flexibility remains high• Biggest problem – where’s the loose coupling?
– What’s the logic in using a general-purposeapp server for application-specificprocessing?
– Bottlenecks require intelligent solutions to be cost-effective
• SOA impacts ALL endpoints, not just Service Providers!
9
Copyright © 2006, ZapThink, LLC 17
Challenges in Content Processing
• Transformation & content-based routing -two parts of same problem: getting systems to understand what they are transmitting.
• Significant operational problems:– Requires intensive processing, especially at wire speed– How involved is a human in the mapping steps?– Simplifying the management of transformations
You must address BOTH transformation and performance issues without trading off either!
Copyright © 2006, ZapThink, LLC 18
Concept review: XML Performance Concerns
• XML is text heavy & verbose.
• XML processing/parsing requires many steps.
• Human-readable XML introduces security issues.
• The volume of XML network traffic is rapidly increasing – SOA is just one driver.
• It’s important to take a smart approach to dealing with XML performance.
10
Copyright © 2006, ZapThink, LLC 19
First Roadblock: Security
Service-orientedcomputing
Network (Packet) Level
TCP/IP packetinspection
Perimeter-based
Fragmented Control
Islands of securityMaintained as separate
units
Closed SystemsProprietary, closed
APIsTight coupling
Single administratorLocalized users
Application Level
XML-basedContent aware
Application control
Open Systems
Open, accessible APIsLoose coupling
Multiple administratorsDistributed users
Managed Security
Whole network policiesTiered administration
Traditional DistributedComputing
Copyright © 2006, ZapThink, LLC 20
The “20 doors” problem
• Any Web Services security approach must be a part of a comprehensive enterprise security initiative
If your company has 20 doors, If your company has 20 doors, and you lock 19, are you 95% secure?and you lock 19, are you 95% secure?
11
Copyright © 2006, ZapThink, LLC 21
Addressing Security Issues
• Should have security assessment in place
• Identify scope of identity management need
• Determine/develop enterprise ID policies (may be a project in its own right)
• Identify appropriate solution/vendor– Single sign-on– Application security– Web Services security
Copyright © 2006, ZapThink, LLC 22
Message-Level Security
• Digitally signed SOAP request message (simplified)
• Security parts highlighted
12
Copyright © 2006, ZapThink, LLC 23
The Security Context Challenge
rschmelzer
RonSchmelzer
rschm123
Selective
Read / Write
Read Only
Full Read/Write???
???
Copyright © 2006, ZapThink, LLC 24
Solving the Security Context Challenge
NetworkFirewall
SAML
SOAPHTTPSMTPFTP
JMS/MQ
SecurityAppliance(2)
SOAP ProxyWSM
Enforcement(1)
TxMinderXML Agent
NetegrityPolicy Server
Proprietary Legacy
.NET
J2EE
TxMAgentSOAP
User Stores(LDAP, RDBMS, etc.)
WSMPolicies
NetworkFirewall
TxMAgent
WSMEnf.
WSMEnf.
SAML
Source: CA Netegrity
13
Copyright © 2006, ZapThink, LLC 25
Identity Management: Kill Two Birds…
• Many enterprises already dealing with “Single Sign-On”– “Sticky Note” problem: too many
passwords for too many systems– Problems administering users– Too many people with root access– Unknown security holes
• Now: need enterprise ID & access management to prepare for SOA
Copyright © 2006, ZapThink, LLC 26
21st Century Network Security
• The Old Days:– Internal = Trusted– External = Untrusted
• Today:– Internal = Untrusted– External = Hostile!
You MUST address security You MUST address security issues throughout the issues throughout the
networknetwork
14
Copyright © 2006, ZapThink, LLC 27
Federated Security
• Coordinate two separate security domains via a trust relationship
• Can be internal to an enterprise or B2B
• Enables scalability of enterprise security
• Based upon standards:– SAML – Security Assertions Markup Language– WS-Security – Web Services Security– WS-Trust – Web Services Trust
• Similar approach offered by Liberty Alliance (see http://www.projectliberty.org)
Copyright © 2006, ZapThink, LLC 28
Federated Security Illustration
.NET Client
AuthoritativeIdentity
Foundation
Web Services Consumer
AuthoritativeIdentity
Foundation
Web Services Provider
SecurityTokenService
local ID Token
J2EE
Token local ID
SecureSpan Bridge
SecureSpanGateway
Policy Service
SOAP Request SecurityToken
SecurityTokenService
Policy Service
WS-Trust
WS-Trust
Source: Layer 7 Technologies
State ofthe Art!
15
Copyright © 2006, ZapThink, LLC 29
SOA Management: Many Facets
• System management – identifying root causes of technical problems, mitigating or solving them
• Metadata management – storing, managing, and enabling the use of various metadata
• Business Service management – managing business Key Performance Indicators (KPIs) in the context of SOA
Copyright © 2006, ZapThink, LLC 30
The Problem with SOA Management…
• Managing Service interfaces & dependencies
• Application management
• Systems management
• Network management
• Root cause analysis• Etc…
16
Copyright © 2006, ZapThink, LLC 31
The First Rule of SOA Management
• You need management when you offer your first “mission critical” Web Service
• Even one Service requires management
• Don’t wait!
Copyright © 2006, ZapThink, LLC 32
The SOA Management Conundrum
ApplicationInfrastructure
Database
Systems
Network
Business Process
ApplicationJ2EE
IIS WebSphere MQ Series
Oracle9i
Windows UNIX
TCP/IP
Hire New Employee
SAP
Exchange/Domino
zOS
WS & SOA
Source: CA
17
Copyright © 2006, ZapThink, LLC 33
Exception Management & SOA
• “Passive” management– Problem Alert Human intervention– Breaks loose coupling!
• “Active” management– Crosses threshold automated response
problem prevented– Maintains loose coupling– Much more difficult!
State ofthe Art!
Copyright © 2006, ZapThink, LLC 34
SOA Enablement…
• Provide and enforce the SOA layer of abstraction
• Combine fine-grained APIs into coarse-grained business Services
• Mask complexity of underlying technology: message protocols, adapters, APIs, etc.
• Handle quality of service, scalability, etc. “behind the scenes”
18
Copyright © 2006, ZapThink, LLC 35
The State of the Market
• All balls must be in the air at once
• Web Services do not create a permanent, distinct market
• New entrants jockeying for position while incumbents wait/build/acquire
Security
Inte
gra
tion
Proc
ess
Man
agem
ent
Tool
s
Copyright © 2006, ZapThink, LLC 36
The need for SOA Infrastructure
• Service-Oriented Architecture is a set of design principles, methodologies, and best practices for implementing distributed computing using loosely-coupled, composable Services on a heterogeneous network.
• However, SOA doesn’t define exactly how to implement those Services, or what infrastructure to use to make sure the Services can communicate with each other or be made available on the network
• In fact, SOA as a design is not supposed to mandate specific implementation approaches – that’s how we can guarantee loose coupling in the face of heterogeneity.
19
Copyright © 2006, ZapThink, LLC 37
Invocation Mechanisms in SOA
• SOA is more flexible than client/server –supports multiple invocation mechanisms
– Request/Reply
– Publish/Subscribe
– Routed Events
– Reliable Messaging
Serviceconsumer
Serviceprovider
Request channel Request
Reply channelReply
Topic
Data store
Serviceprovider
Serviceconsumer
Serviceconsumer
Message server
Subscribe
Input channel Event
Output channel Event
Output channel Event
Input channelServiceprovider
Serviceconsumer
Serviceconsumer
Output channel
Output channel
Content-basedRouterMessage
Serviceprovider
Serviceconsumer
Message bus
Message Message
Data store Data store
Copyright © 2006, ZapThink, LLC 38
What are Events?
• Ordinary event – something that happens in the real world
• Ordinary business event – a meaningful change in the state of the enterprise or of something relevant to the enterprise– Customer order, the arrival of a shipment,
etc.
• Software event – a record of an ordinary event in software. – Data that describe the ordinary event,
typically in the form of a message
20
Copyright © 2006, ZapThink, LLC 39
Lightweight Events in SOA
• When WSDL is overkill– Only binding information must be
contracted– Ad hoc contract may be sufficient– Situations where some tight coupling is OK
• When HTTP and SSL are sufficient– Most Web Services over HTTP today– SSL provides confidentiality but not content-
based security
Copyright © 2006, ZapThink, LLC 40
Messaging Models
• Store-and-forward– Messages delivered to a one or more coordinating
authorities– These forward to end destination– Post office model– SMTP
• Point-to-point– Real-time connection between systems
21
Copyright © 2006, ZapThink, LLC 41
Message-based Approaches to SOA
• Why SOA Prefers Message-based approaches over RPC-style– Fundamental tenet of loose coupling: not being aware of end point
requirements– Composited (virtualized) Web Services may require greater time for
processing, requiring asynchrony– B2B processes are often asynchronous– Distributed systems can be more reliable when they are asynchronous– Heterogeneous systems, especially those with limited bandwidth
devices, function better asynchronously.– Support human involvement in processes
• Messages, Events, part of the same asynchronous requirements
• ENTER THE Enterprise Service Bus….
Copyright © 2006, ZapThink, LLC 42
The “Enterprise Service Bus”
• What is an ESB? TRICK QUESTION!– MOM++
• Messaging infrastructure or message-based middleware that can natively deal with Service interfaces and composition
– Service Intermediary / Broker• A broker-based intermediary (hub-and-spoke) for intermediating
Service requests on behalf of Service consumers and producers– EAI++
• EAI technology “warmed over” with Service interfaces, but nothing in the infrastructure itself that is Service-Oriented
– Architectural Pattern, not Product• ESB is not a product at all, but an “architectural pattern” – that is, you
can implement an ESB using whatever technology you want, as long as it gives you the design goals of secure, reliable, robust communication using message-based or asynchronous styles as well as synchronous.
• The real issue: – HTTP / Web Services by itself is too immature to support secure, reliable, and
robust long-lived Service interaction, so a stronger infrastructure is needed, hence the need for “something more” than just Web Services…
– People acknowledge that an ESB is a sort of infrastructure for implementing SOA, but there’s NO AGREEMENT on what an ESB should do, nor what it should look like.
SOA is something you do -- not something you buy.
22
Copyright © 2006, ZapThink, LLC 43
Lack of Convergence on ESB Meaning
• Things that ESBs should do:– Facilitate Service exposure– Enable Service composition– Abstract Service runtime heterogeneity– Enable event-driven and asynchronous modes of Service interaction
• Things that ESBs can do:– Business Process Management / Modeling (BPM)– Registry / Repository capability– Provide an actual message bus / message-oriented middleware (not necessary!)
• Things that ESBs shouldn’t do:– EAI with Service interfaces. SOA represents an approach to integration that is
fundamentally opposed to a hub-and-spoke, tightly-coupled approach to integration. Composition favored over explicit mapping
“At this point in time, no single vendor provides a complete SOA infrastructure, and certainly no single product can provide a complete infrastructure.”
• Anne Thomas Manes
Copyright © 2006, ZapThink, LLC 44
Mapping Market Convergence…
SOAImplementation
Framework
SO Mgmt
WSManagement
SystemsManagement
SOAEnablement
SOAI
ESB
IntegrationBrokers
App Server"Platforms"
ApplicationServers
Message-Oriented
Middleware
TransactionMonitors
EAI
SOProcess
BPM
BPI
SOSecurity
WS Security
PKI
IAM
EII
BI Analytics
BAM
SOA Tools
ModelingTools
ApplicationFrameworks
RAD Arch.Tools SODevelopment
WS Tools
IDEs
XMLAppliances
NetworkAppliances
SemanticIntegration
DataIntegration
ETL
Portals
PresentationTools
CMS
OLAP
NXDs
Databases
OperationalData Stores
DataWarehouses
Copyright © 2003 ZapThink LLC
B2Bi
SOII
SO Content
Core SOMarkets
MarketsRemaining
Distinct
TransitionalWS Markets
EstablishedMarkets
23
Copyright © 2006, ZapThink, LLC 45
Next Steps?
• Take iterative approach to reduce risk• Security & management usually come first• Build SOA top-down (architectural plan) and
bottom-up (build Services from existing resources)
Security
Inte
gra
tion
Proc
ess
Man
agem
ent
Tool
s
Copyright © 2006, ZapThink, LLC 46
Thank You!
ZapThink is an advisory, analysis, & influence firm focused exclusively on Service-Oriented Architecture, Web Services, & Enterprise Web 2.0.
Read our new book, Service Orient or Be Doomed! How Service Orientation Will Change Your Business.
Jason Bloomberg
[email protected] Schmelzer
Top Related