Enterprise Service Bus at DOL
description
Transcript of Enterprise Service Bus at DOL
4
DOL SOA IMPLEMENTATION
Location 1p570-SN10EC3CD
Power5+ Max CPUs: 16 Owned: 16 Memory: 48
Location 2p570-SN10EC3BD
Power5+ Max CPUs: 16 Owned: 16 Memory: 48
IBM HTTP Server
WebSphere Process Server
WebSphere Application Server
WebSphere Server registry repository
WebSphere Message Broker
WebSphere Federation Server
LPAR
AIX 5.3 & IBM Http Server 6.1.0.13
AIX 5.3, WAS 6.1.0.13 & WPS 6.1.0.1
AIX 5.3 & WAS 6.1.0.13
AIX 5.3, WAS-ND 6.1.0.13 & WSRR 6.1.0.2
AIX 5.3 & WMB 6.1.0.2, ,
AIX 5.3 & WFS 9.1
AIX 5.3 & IBM Http Server 6.1.0.13
AIX 5.3 & WAS-ND 6.1.0.13
AIX 5.3 WAS 6.1.0.13 & WSRR 6.1.0.2
AIX 5.3 & WMB 6.1.0.2
AIX 5.3 & WFS 9.1
AIX 5.3, WAS-ND 6.1.0.13 & WPS 6.1.0.1
IBM HTTP Server
WebSphere Application Server
WebSphere Server registry repository
WebSphere Message Broker
WebSphere Federation Server
WebSphere Process Server
LPAR
Installed Software Installed SoftwareData Center 1 Data Center 2
Legend:Legend:
= Production= Production
= Staging= Staging
= System= System
AIX 5.3, WAS-ND 6.1.0.13 & IICE 8.4 AIX 5.3, WAS 6.1.0.13 & IICE 8.4
WebSphere IICE Server
IBM HTTP Server
WebSphere Application Server
WebSphere Server registry repository
WebSphere Federation Server
VIOS 1
VIOS 2
AIX 5.3 & IBM Http Server 6.1.0.13
AIX 5.3 & WAS-ND 6.1.0.13
AIX 5.3, WAS-ND 6.1.0.13 & WSRR 6.1.0.2
AIX 5.3 & WFS 9.1
AIX 5.3, IBM Http Server 6.1.0.13
AIX 5.3 & WAS-ND 6.1.0.13
AIX 5.3, WAS-ND 6.1.0.13 & WPS 6.1.0.1
AIX 5.3 & WMB 6.1.0.2
WebSphere IICE Server
IBM HTTP Server
WebSphere Application Server
WebSphere Process Server
WebSphere Message Broker
WebSphere IICE Server
VIOS 1
VIOS 2AIX 5.3, WAS-ND 6.1.0.13 & IICE 8.4
7
DOL SOA IMPLEMENTATION
Location 1p570-SN10EC3CD
Power5+ Max CPUs: 16 Owned: 16 Memory: 48
Location 2p570-SN10EC3BD
Power5+ Max CPUs: 16 Owned: 16 Memory: 48
IBM HTTP Server
WebSphere Process Server
WebSphere Application Server
WebSphere Server registry repository
WebSphere Message Broker
WebSphere Federation Server
LPAR
AIX 5.3 & IBM Http Server 6.1.0.13
AIX 5.3, WAS 6.1.0.13 & WPS 6.1.0.1
AIX 5.3 & WAS 6.1.0.13
AIX 5.3, WAS-ND 6.1.0.13 & WSRR 6.1.0.2
AIX 5.3 & WMB 6.1.0.2, ,
AIX 5.3 & WFS 9.1
AIX 5.3 & IBM Http Server 6.1.0.13
AIX 5.3 & WAS-ND 6.1.0.13
AIX 5.3 WAS 6.1.0.13 & WSRR 6.1.0.2
AIX 5.3 & WMB 6.1.0.2
AIX 5.3 & WFS 9.1
AIX 5.3, WAS-ND 6.1.0.13 & WPS 6.1.0.1
IBM HTTP Server
WebSphere Application Server
WebSphere Server registry repository
WebSphere Message Broker
WebSphere Federation Server
WebSphere Process Server
LPAR
Installed Software Installed SoftwareData Center 1 Data Center 2
Legend:Legend:
= Production= Production
= Staging= Staging
= System= System
AIX 5.3, WAS-ND 6.1.0.13 & IICE 8.4 AIX 5.3, WAS 6.1.0.13 & IICE 8.4
WebSphere IICE Server
IBM HTTP Server
WebSphere Application Server
WebSphere Server registry repository
WebSphere Federation Server
VIOS 1
VIOS 2
AIX 5.3 & IBM Http Server 6.1.0.13
AIX 5.3 & WAS-ND 6.1.0.13
AIX 5.3, WAS-ND 6.1.0.13 & WSRR 6.1.0.2
AIX 5.3 & WFS 9.1
AIX 5.3, IBM Http Server 6.1.0.13
AIX 5.3 & WAS-ND 6.1.0.13
AIX 5.3, WAS-ND 6.1.0.13 & WPS 6.1.0.1
AIX 5.3 & WMB 6.1.0.2
WebSphere IICE Server
IBM HTTP Server
WebSphere Application Server
WebSphere Process Server
WebSphere Message Broker
WebSphere IICE Server
VIOS 1
VIOS 2AIX 5.3, WAS-ND 6.1.0.13 & IICE 8.4
20
DOL SOA IMPLEMENTATION
SOA, ESB and BPEL
NYSDOL enterprise architecture implements SOA on top of an ESB, so that we could have the virtues of SOA, with room to grow.
SOA : Service Oriented Architecture is a business-centric IT architectural approach that supports integrating business as linked, repeatable business tasks, or services.
ESB : Enterprise Service Bus is an architectural pattern to integrate and manage services, not a software product. We can form an ESB with different software products leveraging specific functionality of each product to meet unique requirements.
BPEL : Business Process Execution Language is a standards-based way of orchestrating a business process composed of services.
We have also added BPEL to ESB (Process Server) to easily compose new services out of existing services.
21
DOL SOA IMPLEMENTATION
ESB or BPELESB : Message routing, Message transformation, Protocol mediation, stateless transactions, integration middleware for off-the shelf products, security, logging, auditing, excellent performance, data-centric requirements.
BPEL : Stateful long-running business processes or transactional micro flows, human tasks, business rules, complex logic, process centric requirements .
ESB Pattern : Our ESB pattern is Gateway ESB pattern and provides a controlled point of external access to services. The gateway/ router is currently implemented as a software program in message broker which is separated from the hub where services are exposed.
22
DOL SOA IMPLEMENTATION
NYSDOL Enterprise Architecture for SOA based Applications
Message Broker ESB
Websphere Application Server
Websphere Process Server
Web Server
Internal / External Users Browser
External Agencies, Web service clients
Websphere Service Registry and Repository
HTTP(s)
Mainframe
WS
service requests (SOAP/HTTP)
Lookup endpoint / policy
WSDL-E-WSDL (exposed on WMB)
WSDL
Legacy & other services – Q name / node in WMB
WS-Policies
PeopleSoft Content manager
All External web service requests
WS
WSWS
WS
WS
PS1 PS2
OracleDatabase
MQ
Common Logging Database
Policy server LDAP
1
2
3
4
Common logging service
Port / Routing Gateway
HUBProcess services, Legacy & other services exposed
Rec Req Lookup Route
Generic mediation module to lookup services in a process
PS
WS
MQ MQ
WSWS
Xpressions
MQ
OS
OS
Asynchronous call
Lookup to find endpoints for services / workflow policies
1
2
MQ / http(s)
a
b
d
c
Note: Web services are not exposed with E-WSDL in MB, instead they could either be invoked by process server (by BPEL in choreography) or by Message broker (composite service / simple services).
5
6
23
DOL SOA IMPLEMENTATION
Major Security Considerations in SOASOA introduces new challenges to security as it lowers the barriers between applications (composite services formed with existing / new applications), overcomes technology differences, as interoperability is the key goal of SOA. Some of the new requirements of SOA security are
The identity must be decoupled from the services. All entities in SOA have identities - users, services, and so on, that needs to be properly identified so that appropriate security controls can be applied.
The need to seamlessly connect to other organizations on a real-time, transactional basis.
Each new choreography might require examination of the security policy to ensure it remains valid for this new combination.
The need to manage identity and security across a range of systems and services that are implemented in a diverse mix of new and old technologies
Protection of business data in transit and at rest. Providing end-to-end message security is also a key requirement, because messages can be traversing different transport mechanisms and trust zones. In addition, access must be provided to information (and systems) based on business drivers.
24
DOL SOA IMPLEMENTATION
Security Aspects for SOAFunctional Aspects:
Authentication - Verifying identity of users
Authorization - Deciding whether or not to permit action on a resource.
Data Confidentiality – Protecting secrecy of sensitive data.
Data Integrity – Detecting data tampering and making sure neither the sender nor the receiver can deny the message that they sent or received.
Protections against attacks – Making sure attackers don’t gain control over applications.
Privacy – Making sure the application does not violate the privacy of users.
Audit - Important events need to be logged and available for real-time or later forensic review
Non-Functional Aspects:
Interoperability - This concern is specific to SOA, where different security solutions must not break compatibility of services that are otherwise compatible.
Manageability - As many different services needs to be protected, the security solution must be easily manageable.
Ease of Development – The security solution must be easy enough to adopt and implement.
Availability – The security solution must not impact the availability of the services.
25
DOL SOA IMPLEMENTATION
Layered SOA and Security
The layered SOA requires all of these security elements to be present in each layer across infrastructure, application, business services, and development services.
26
DOL SOA IMPLEMENTATION
New Security Approaches for SOA
Message Level Security : Different parts of a message can be protected differently, to make them usable only by intended parties in the message path.
Security as a service: Security service is central and not part of any application and could evolve in-line with business needs. It offers applications the ability to authenticate, authorize, encrypt/decrypt messages, sign/verify signatures and log messages
Policy-driven Security : Security requirements must not be hard-wired into applications or services themselves. Instead security requirements should be separated from business logic and declared as policies. Policies could be business, architectural, operational.
27
DOL SOA IMPLEMENTATION
Datapower for SOA Security
Provides detailed logging and audit trail.
Datapower SOA security appliances are purpose-built, easy-to-deploy network devices. It provides integrated message-level security (supports WS-Security, WS-Policy, WS- SecurityPolicy, WS-ReliableMessaging, WS-SecureConversation, WS-Trust, SAML, and LDAP) .
Helps in generating dynamic content, content based routing, enables higher performance at wire speed
Provides protection against XML vulnerabilities by acting as an XML proxy and performing XML well-formedness checks, buffer overrun checks, XML schema validation, XML filtering, and XDoS protection .
Provides centralized security functions and acts as an enterprise wide single security- enforcement point for XML and Web services transactions .
It integrates with WSRR and other policy decision points like LDAP and Siteminder Policy server.
Offers robust service level management, policy management, and Web services management support .
28
DOL SOA IMPLEMENTATION
Internal / External Users Browser
NYSDOL Security Architecture for SOA based Applications
External Agencies, Web service clients
Web Services Gateway
Authentication/ authorization (could use X509/Kerberos / digital certificates/ SAML, siteminder etc) XML Threat detectionEncryption (outgoing) / decryption (incoming)Pass all authenticated request to XI50, log unauthenticated requests(optionally notify)
All External web service requests (SOAP/ HTTP req)
Policy server LDAP
Finer level AuthorizationMessage level security XML Validation /XML AccelerationAudits, exception Logging, notification, RoutingGenerate LTPA Tokens / SAML Assertions for communication with service providers / backends
Web Server
1
1a
2
2a
Message Broker ESB
Websphere Application Server
Websphere Process Server
Websphere Service Registry and Repository
(WSDL, XSD, WS-policies)
2b
PeopleSoft Content manager
OracleDatabase
MQ MQ MQ
Xpressions
MQ
Mainframe
3
LTPA
HTTP(s)
1
2
3
4service requests (SOAP/HTTP(s))
HTTP(s)
2a
MQ / http(s)
LTPA
HTTP / SOAP
HTTP(s)
Authenticate & authorize xml requests (http basic auth)
MQ / http(s)
1a
LTPA
30
DOL SOA IMPLEMENTATION
Integration Layer
Thin Client(Web Browser)
Message Broker and Enterprise Service Bus
Employer Profile Service Address Service Employer Tax Rate Service
Servlet ContainerJSF Pages
Java/J2EEBusiness Object
HTTP Request
HTTP Response
JMS / SOAP
Service Mediation
Service Provider
32
DOL SOA IMPLEMENTATION
JNDI Name Propertiesjms/cf/ESB_QM Host: , Port: , Channel: …jms/queue/ESB.ROUTER.REQUEST Qname: ESB.ROUTER.REQUESTjms/queue/ESB.REPLY Qname: ESB.REPLY
Mes
sage
Bro
ker Error and Exception H
andling
WSR
R
ESB
Rou
ter F
low
36
DOL SOA IMPLEMENTATION
WMB
Service ConsumerSOAP/HTTP
Service ConsumerXML/WMQ
Service ProviderSOAP/HTTP
Service ProviderXML/WMQ
WSRR
ESB Router
S
X
S
X
SOAP/HTTPPort
XML/WMQPort
SOAP/HTTPPort
XML/WMQPort
S
X
S-X
X-S
Decouple service consumers and service providers Provide a set of ports associated with specific protocolsRoute to any service providers using any protocol
ESB Router
37
DOL SOA IMPLEMENTATION
WMB
Service ConsumerSOAP/HTTP
Service ProviderSOAP/HTTP
Service ProviderXML/WMQ
WSRR
ESB Router
2
1 SOAP/HTTPPort
SOAP/HTTPPort
XML/WMQPort
3.1
3.2
ESB Router HTTP
1. Service consumer sends a SOAP request message over HTTP to the ESB2. ESB Router looks up in WSRR for requested service provider3. ESB routes the request to the service provider:
1. Listening for SOAP requests over HTTP2. Listening for XML requests over WMQ/JMS
38
DOL SOA IMPLEMENTATION
ESB Router WMQ
1. Service consumer sends a XML message over WMQ to the ESB2. ESB Router look up the WSRR for requested service provider3. ESB route the request to service provider:
1. Listening for SOAP requests over HTTP2. Listening for XML requests over WMQ/JMS
40
DOL SOA IMPLEMENTATION
Business Process
A business process is a sequential flow of execution paths described in WS-BPEL (Web Services Business Process Execution Language), including:
Which services are invoked
In Which order the services are invoked
The transformation of data output from one service as input to another
41
DOL SOA IMPLEMENTATION
JNDI Name Propertiesjms/cf/ESB_QM Host: , Port: , Channel: …jms/queue/ESB.ROUTER.REQUEST Qname: ESB.ROUTER.REQUESTjms/queue/ESB.REPLY Qname: ESB.REPLY
Mes
sage
Bro
ker Error and Exception H
andling
WSR
R
ESB
Rou
ter F
low
42
DOL SOA IMPLEMENTATION
Benefit Charges Adjustment process Assembly
Benefit Charges Adjustment process (Orchestration of composite services) (Service Provider / Service Consumer)
ExperienceRatingAccount service (Service Provider)
UITaxRateCalcRunner service (Service Provider)
Service Provider
Service Provider
Service Consumer
44
DOL SOA IMPLEMENTATION
Mediation flow Assembly for WSRR lookup
WPS_WSRR_TXRateMediation module looks for the services in WSRR using the lookup node.
Lookup node retrieve the service end points & send the request to appropriate service .
47
DOL SOA IMPLEMENTATION
ERROR AND EXCEPTION HANDLING FRAMEWORK
Exceptiondatabase
Check RetryPolicy
Automatic Retry
Enterprise Service Bus
Output queues
Review exceptions
MB Exception Console
Notify - Email, Pager
Re-submit Process1
4
5
2.16
8
Ext. AppListener
Resolve/Compensate
7.1 If the data is correct but the exception was due to systematic causes/conditions (db,application, adapter) then re-submit after the condition was corrected using Error properties.
7.2 If the exception is data related then an application support team member needs toreview the exception and make a decision how will re-issue or compensate the transaction
Support Team
Error Handling Flows
Input queues
ExceptionDetected
Log
Exception
Check Error
RulesException Rules
& Action
7.1
7.2
Error Properties & Action: This file will be used as properties files. It contains retry logic andaction information against any errors.
Retry queue
Proposed forNYDOL
Error queue
2.2
Notific
ation
3
49
DOL SOA IMPLEMENTATION
Publish: add new services that are available and can be managed
•WSDL
•XSD (business objects)
Find: search for services using any metadata associated with that service
•endpoint lookup
•version of the services
Enrich: has the ability to enhance services with useful artifacts
•service availability
•policy enforcement
•notify users of changes
WebSphere Service Registry and Repository
50
DOL SOA IMPLEMENTATION
WebSphere Service Registry and Repository con’t
Manage: manage the lifecycle of services in the registry
•enabling access control
•promote/retire
•change analysis through impact analysis
Govern: provide a central point of overall governance
•WPS, ESB, developer tools (RAD, WID, RSA, WBM)
•Delete, Retrieve, Update, Manage/Govern, Create
51
DOL SOA IMPLEMENTATION
WebSphere Service Registry and Repository (to be)
1. For Service Providers• Manage multiple life cycles in the various stages of development
• Development• Test• Staging• Production
• Register• Define the whole process (for external users of WSRR)
• Provide metadata• Endpoint• Service Name• Port Type• Cost• What???
• What does the service do?• What are the capabilities of the service?
2. Contracts• Private• Production• Public