IBM System z and z/OS Java Batch Solutions...© 2013 IBM Corporation IBM Americas Advanced Technical...
Transcript of IBM System z and z/OS Java Batch Solutions...© 2013 IBM Corporation IBM Americas Advanced Technical...
© 2013 IBM CorporationIBM Advanced Technical Skills
IBM System z and z/OS
Java Batch SolutionsVersion: September 10, 2013
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD2
Version: September 10, 2013
This page intentionally left blank
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD3
Version: September 10, 2013
Agenda / Table of Contents
Material Available Online …
Overview
Java on z/OS
JVM Launcher Solutions
WebSphere Java Batch
Java Batch in CICS, DB2
Batch and Decision Management
Open Standard Java Batch
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD4
Version: September 10, 2013
This Presentation Available Online
Overview …
ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101783
PDF in Chart FormatShowing the charts as you're seeing them here
PDF in Speaker Notes FormatSame charts, but printed so the speaker notes associated with each chart appear.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD5
Version: September 10, 2013
OverviewSetting the stage for the discussion of modern batch solutions
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD6
Version: September 10, 2013
Batch (or Bulk) Processing
Examples of bulk processing …
Many definitions exist ... they all have in common the relative lack of human interaction, and the expectation of results at a future time rather than immediately:
Request
Response
Request
Response
In general:● Interaction is one-for-one ... that
is, request with matching response
● Expectation is for response to follow request in a near-immediate span time frame
Online Processing
Submit
Complete
Start
EndProcess
In general:● Interaction is one-for-many... that is, initial request
results in many results from processing
● Expectation is for results to finish within some determined non-immediate time frame
Bulk Processing
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD7
Version: September 10, 2013
Some Examples of Batch (or Bulk) Processing
What's Behind This? …
Just to set some context for the upcoming discussion
Month-End Tax or Fee Calculation and BillingCustomer records are processed with tax or other calculations processed against activity. This may be part of a larger process of calculating amount owed and formulating billing.
Period-End Statements and ReportsAn example is investment portfolio results and reporting.
Data TransformationRaw data records are processed with goal of transforming some aspect of the data content or layout. Result is a second set of data with the intended formatting.
Data AnalysisData records are analyzed to determine trends or patterns. Data mining to find new potential markets is one example. Analyzing vast quantities of seismic data is another.
Point is these are activities where processing in bulk is a better way to maximize efficiency of data access and computer resources. The
completion time is determined but not immediate.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD8
Version: September 10, 2013
What's Behind This?
Approaches to Batch Modernization …
Every business has different motivators. The common ones we've seen:
Batch Window CompressionThe window in time for batch processing is shrinking. There's a need to better manage online and batch processing concurrently within the same system.
Java Skills and Common ToolingJava skills are more common that traditional programming skills. Further, Java tooling for online work is powerful and capable of being used for batch programming as well.
GP cycles and Offload to Specialty EnginesCost pressures are creating a need to explore ways of offloading processing cycles from GP to specialty engines such as zAAP or zAAP-on-zIIP.
Other motivators may exist. The key is that these motivators are real and they are driving exploration of modern batch.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD9
Version: September 10, 2013
Approaches to "Batch Modernization"
Framework for discussion …
Generally speaking, we see two basic approaches:
Preserve Existing ... Java Batch for NewLeave existing batch processes as they are today, but as new requirements come up then engineer them into the Java batch model
Re-Engineer Some Existing Batch ProcessesIdentify existing non-Java as candidates and then re-engineer them to operate in a Java batch environment
Typical starting approach: identify batch processes with fewer interdependencies and then work out from there
What we don't see is a "rip-and-replace" strategy. That's too costly and too risky.
Always a reasoned incremental approach.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD10
Version: September 10, 2013
Framework for Discussion That Follows
Java on z/OS …
The following diagram illustrates the points we will touch on as we review the modern batch technologies available on System z and z/OS:
Modern Batch Technology
What it isAn explanation of its intended purpose
How it worksAn explanation of its functions and features
DifferentiatorsA discussion of how the technology compares to others discussed, and the particular value this technology offers
How it takes advantage of System zAn explanation of System z and z/OS exploitation that may occur with the technology
How it integrates with existing processesAn exploration of integration and usage patterns when other components in your overall architecture are involved
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD11
Version: September 10, 2013
Java on z/OSA story of taking advantage of the platform
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD12
Version: September 10, 2013
IBM Java on z/OS
Specialty Engines …
It's important to understand that while the Java APIs are industry standards, the implementation below the APIs becomes increasingly platform-aware:
ProgrammingAPIs
High-Level JVM Functions
Low-Level JVM Functions
SDK conforms to the accepted standards● IBM SDK provides all the required APIs according to the
specification at the level being discussed● IBM z/OS SDK provides additional APIs to take advantage
of z/OS platform specific functions (such as SAF security)
JVM Functions common across IBM SDKs● The JVM is entirely IBM's ... first delivered in 2005● Many features: generational GC, shared classes● High-level JVM functions common across IBM Java
System z and z/OS functions● Takes specific advantage of platform, including exploitation
of new CISC instructions available with new System z: z10, z196, EC12
● Big Decimal, Large Page, Out of Order execution, transactional execution, flash paging ... equals performance
● Work with z/OS dispatcher to offload to specialty engines
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD13
Version: September 10, 2013
System z Specialty Engines
Timeline …
Specialty engines provide additional processing capacity with an attractive financial profile: lower acquisition cost, not counted towards software license charges
Java Work1 DDF and DB2 Utilities2 Other Work
z/OSDispatcher
zAAP Processors zIIP Processors General Processors
zAAP - System z Application Assist ProcessorOffload of Java and XML parsing work.
zIIP - System z Integrated Information ProcessorCertain DB2 work and XML parsing services.
Note 2 -- Plus other work, see http://www.ibm.com/systems/z/hardware/features/ziip/
Note 1 -- See http://www.ibm.com/systems/z/hardware/features/zaap/
zAAP-on-zIIPA means of more efficiently using specialty engines by defining only a pool of zIIP processors and allowing eligible zAAP work to run on the zIIPs3.
zAAP-on-zIIP
Note 3 -- EC12 planned to be the last system that supports zAAP; after that, zAAP-on-zIIP will be the offload mechanism
Note: simplified representation of offload mechanism. Various tuning and control parameters exist.
IFL - Integrated Facility for LinuxFor running z/VM and Linux. Does not apply to z/OS, but plays strong role in Linux for System z
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD14
Version: September 10, 2013
IBM System z Java Product Timeline
Performance …
This shows the level of commitment to Java and Java on System z:
SDK 1.1.1● First OS/390 Java product● Followed by 1.1.4 and 1.1.6● 31-bit
SDK 1.1.8● 31-bit
SDK 1.3.1● 31-bit● z/OS ● Linux for System z
SDK 1.4● 31-bit z/OS● 64-bit Linux for System z
SDK 1.4.2● 64-bit z/OS● First use of IBM J9 VM
and JIT Technology
SDK 5● 31-bit and 64-bit● z/OS and Linux for System z● z9 exploitation
SDK 6.0.0● Java SE 6 APIs● 31-bit and 64-bit● z/OS and Linux for System z● z10 exploitation
SDK 6.0.1● Java SE 6 APIs● 31-bit and 64-bit● z/OS and Linux for System z● z196 exploitation● Enhanced JZOS and z/OS security
SDK 7● Java SE 7 APIs● 31-bit and 64-bit● z/OS and Linux for System z● More z196 exploitation● zEC12 exploitation● OpenJDK
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD15
Version: September 10, 2013
IBM z/OS Java Performance
Summary …
The story is one of improvement in hardware and Java software
zEC12 SDK 7 SR3
zEC12 SDK 7 SR1
z196 SDK 7 SR1z196 SDK 6 SR8
z10 SDK 6 SR4
z10 SDK 6 GMz9 SDK 5 SR5
CPU Intensive Multi-threaded Workload on 16-way Machine
Better scaling slope leading up to CPU
over-commit
Around 12X improvement
CPU intensive workload so scaling
past over-commit not expected
Four things going on: (1) faster hardware, (2) exploitation of machine instructions, (3) exploitation of machine features (Flash Express paging with large pages), and (4) increasing efficiency of the JVM function itself
Disclaimer: controlled environment with specific conditions during measured testing. Results may vary. This is not a promise of performance or results.
-Xaggressive and large page
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD16
Version: September 10, 2013
Section Summary
JVM Launchers …
A very broad summary of the Java on z/OS section:
IBM Java is fully compliant with Java specificationsProgramming APIs compliant with the specification
IBM Java on z/OS has programming extensionsProvide you option to code apps to extensions for added functionality
IBM Java has IBM JVM, GC and JIT TechnologyBased on decades of experience in operating systems
IBM Java on z/OS takes advantage of System zWhich manifests as performance improvements over time
IBM continues to invest in Java on z/OSCommitment to the platform is evident
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD17
Version: September 10, 2013
JVM Launcher SolutionsBPXBATCH, JZOS and the Batch Runtime
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD18
Version: September 10, 2013
What is a "JVM Launcher?"
Three JVM Launchers on z/OS …
It's simply a means of initiating a JVM start and the running of Java programs in it:
Four questions arise:
1. What JVM launcher technologies are available on z/OS?
2. What does each provide in addition to the JVM?
3. Why would one be used over another?
4. If not JVM launcher, what else might be considered?
Java Program
Java Virtual Machine
Instantiation JVM tear-downProgram Execution
Launch the JVM
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD19
Version: September 10, 2013
Three JVM Launcher Technologies We'll Explore
BPXBATCH …
For the purposes of this discussion we'll focus on three:
BPXBATCHProvides a UNIX shell environment to JCL. From the UNIX shell you may then issue a "java" command to launch the JVM and run the specified program.For purposes of comparison, consider this equivalent to invoking "java" from a shell script or manually from a command line.
JZOSProvides a more comprehensive JVM launcher utility that overcomes some of the shortcomings of BPXBATCH. Includes a set of class libraries and JNI native to access z/OS functions and services.
z/OS Batch RuntimeProvides a functional subset of the WebSphere Java Batch environment for running Java batch programs. Includes such things as transactional management, integration with COBOL, and transactional integrity between Java, COBOL and DB2.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD20
Version: September 10, 2013
BPXBATCH (and BPXBATSL)
JZOS …
Provides access to a UNIX shell environment from which you may instantiate a JVM and run a program. It works:
http://publib.boulder.ibm.com/infocenter/zos/v1r13/topic/com.ibm.zos.r13.bpxa500/bpxbatr.htm
//BPXBATCH JOB ...//STEP EXEC PGM=BPXBATCH,// PARM='SH java HelloWorld'//STDIN DD DUMMY//STDPARM DD DUMMY//STDERR DD SYSOUT=*//STDOUT DD SYSOUT=*//STDENV DD *CLASSPATH=/u/myclasses/*//*
JVM instantiated from shell with java command, with Java class file specified.
Parameters to Java program may be specified trailing class name in PARM=' ' or in location specified by STDPARM DD.
If STDIN DD specified, it must be a z/OS UNIX file.
STDERR and STDOUT may be JES spool, or UNIX file, or MVS data set
BPXBATSL is like BPXBATCH, but it allows child processes to be fork/exec'd within the same address space. (Restrictions apply, so see InfoCenter article below.)
It provides a JVM with some basic access to z/OS DD statements. It has limitations with respect to other z/OS services. That's why
JZOS was developed and IBM provides that with z/OS.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD21
Version: September 10, 2013
Overview of JZOS
JZOS Toolkit …
JZOS is technology originally developed by Dovetail Technologies and acquired by IBM. It is now supplied as part of z/OS.
http://www.ibm.com/systems/z/os/zos/tools/java/products/jzos/overview.html
JVM Launcher
Toolkit
JZOSLaunch a JVM and run a Java program
● One address space; all accounting data attributes to the job or started task● Mix Java and non-Java steps in same JCL● Pass condition codes between Java steps, or between Java and non-Java● Access MVS data set via DD statements in Java
Toolkit provides Java classes and native libraries● Methods for allocating, deleting, and renaming MVS data sets● Interface for reading and writing sequential data sets in record mode● Methods for issuing single and multi-line WTOs● Interface for invoking DFSORT and directing I/O into/from Java applications.● Java APIs for submitting z/OS batch jobs from Java● Java APIs that provide interface for invoking IDCAMS (VSAM)● Java APIs for many z/OS functions:
Obtaining job/step/user names, process ids, etc. Writing SMF records Reading environment variables
A more flexible JVM launcher mechanism along with a set of class and native libraries which provide Java programs access to z/OS services
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD22
Version: September 10, 2013
JZOS Toolkit Usable Outside JZOS Launcher
JZOS Launcher …
The JZOS JVM launcher and the JZOS toolkit are not tightly coupled. The toolkit services may be used by other Java Batch mechanisms:
WebSphere Java Batch
z/OS Batch Runtime
Toolkit Classpath and Libpath updates
z/OS Services Offered by JZOS Toolkit
We've not yet covered WebSphere Java Batch or z/OS Batch Runtime
Key point is JZOS toolkit and those offerings are not mutually exclusiveAnd as we'll see, the z/OS Batch Runtime in fact uses the JZOS JVM launcher as well
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD23
Version: September 10, 2013
z/OS JZOS JVM Launcher
Multi-Step Jobs …
It comes with the z/OS SDK installation. Sample JCL is provided along with the JZOS load module. Invocation involves customizing sample JCL:
/usr/lpp/java/J7.0/mvstools JVMLDM70 /samples /jcl JVMJCL70 JVMPRC70
Note: JZOS is specific to the Java version and the bit-mode. Here SDK 7 31-bit is shown.
● Load module -- copy out to LIBRARY data set
● Calling JCL -- this JCL calls the proc and passes variables
● Procedure -- this receives variables and invokes JZOS
JCLJCLNames the Java class to invoke
Sets up the environment variables
Provides STDIN DD (optional)
Passes arguments (optional)
PRCPRCReceives Java class and arguments
Calls JZOS load module and specifies the Java class file and any passed-in arguments
Defines standard DD statements
Starts address space, instantiates the JVM,
loads the class, invokes the main()
method and supplies any arguments
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD24
Version: September 10, 2013
JZOS and Multi-Step Batch Jobs
Batch Runtime Overview …
JZOS allows you to mix Java and non-Java as steps in a larger batch job. Condition codes set by each step available to other steps in the job.
//MYBATCH JOB ...//STEP1 EXEC PGM=COBOL1 : (step DD cards)
//STEP2 EXEC PROC=JVMPRC70,// JAVACLS='com.ibm.MyClass1' : (step DD cards)
//STEP3 EXEC PGM=ASSEMBLER : (step DD cards)
//STEP2 EXEC PROC=JVMPRC70,// JAVACLS='com.ibm.MyClass2' : (step DD cards)
One job
Multiple steps
Mixed programs
One address space
Same WLM policyService Class (goals)Reporting Class (statistics)
Accounting data consolidated under one job
COBOLCOBOL
JavaJava
AssemblerAssembler
JavaJava
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD25
Version: September 10, 2013
z/OS Batch Runtime Overview
z/OS 2.1 enhancements …
This is function supplied with z/OS 1.13 and higher. It provides a means of integrating Java and COBOL with DB2 JDBC T2 connection shared and TX coordination:
Java Program
DB2 z/OS
COBOL Program● Start TX● Call DB2● Invoke COBOL● Finish TX
● Business Logic● Call DB2● Business Logic● Return
Batch Runtime Transaction and Execution Services
RRS Shared Attach
z/OS 1.13 Batch Runtime
JCLJCL
Submit
Or initialize with COBOL and have it call Java ... it works in either case
The other JVM Launcher technologies provide nothing in the way of TX services. If your code
has TX requirements, then you write custom code.
With the z/OS Batch Runtime the TX services provided and help coordinate between Java and COBOL when using JDBC Type 2 to local DB2.
z/OS 2.1 adds PL/I
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD26
Version: September 10, 2013
z/OS 2.1 - Enhancements to Batch Runtime
JZOS and z/OS Batch Runtime …
The Batch Runtime with z/OS 2.1 comes with enhancements:
z/OS 1.13 z/OS 2.1
Java main() Programs
Single Step Jobs
COBOL
DB2 and JDBC T2
Java main() Programs
Single Step and Multi-Step Jobs
WebSphere Java Batch programsExceptions:● Java SE runtime, not Java EE● No WAS functionality (EJB, MDB, JCA)● No WLM classification, no SMF 120.9● Parallel Job Manager not available
COBOL
PL/I
DB2 and JDBC T2
Transactional VSAM
More on WebSphere Java Batch coming up
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD27
Version: September 10, 2013
JZOS and z/OS Batch Runtime
Integration with Enterprise Schedulers …
The two are not mutually exclusive. In fact, the Batch Runtime makes use of JZOS to launch the JVM and load the Batch Runtime function:
com.ibm.zos.batch.container.BCDBatchContainer
BCDPROC
BCDBATCH
JZOSLauncher
JZOSLauncher
z/OS Address Space
31-Bit Java Virtual Machine
z/OS Batch Runtime
JavaCOBOL
JZOSToolkit
JZOSToolkit
z/OS Servicesz/OS Services
General Requirements:● IBM 31-bit SDK for z/OS, V6.0.1
Must be 31-bit because COBOL is a 31-bit language
● IBM Enterprise COBOL Version 4.2
● DB2 V9 with PTF UK62190 for JDBC 3.0 specification, or PTF UK62191 for JDBC 4.0 specification
● DB2 V10 with PTF UK62141 for JDBC 3.0 specification, or PTF UK62145 for JDBC 4.0 specification level
Calling JCL / procedure similar to JZOS; Batch Container
supplies samples of its own
The JZOS toolkit may also be used in conjunction with the
Batch Container
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD28
Version: September 10, 2013
Integration with Enterprise Schedulers
JVM Launcher Positioning …
All three JVM launcher technologies discussed may be initiated via submission of JCL. That becomes the integration point with enterprise schedulers:
Enterprise Scheduler Function
Job JCL
Java Program
Java Virtual Machine
JES SpoolOr elsewhere if JCL defines other than spool
Submit
In many ways this is business as usual ... to the Enterprise Scheduler function the submitted JCL job is the batch job. The invocation of a JVM
to run Java is a detail not of concern to the scheduler function
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD29
Version: September 10, 2013
JVM Launcher Positioning
Summary …
The following provides a high-level positioning of the alternatives discussed so far:
JVM Bit-mode
Batch Services
DB2 Access
TX Coordination
Programming Model
BPXBATCH JZOS Batch Runtime
Address Spaces
31-bit or 64-bit 31-bit or 64-bit 31-bit
Multiple(one with BPXBATSL)
One One
NoneProvides a JVM but no
other extra-JVM services
SomeToolkit provides z/OS
services. Batch services such as checkpoint not provided
Yesz/OS 2.1 - WebSphere Java
Batch batch services (checkpoint, data stream, etc.)
JDBC T2 or T4Access code is all
custom to the application
JDBC T2 or T4Access code is all
custom to the application
JDBC T2 onlyTX coordination requires RRSAF connect to DB2
z/OS 2.1 - tVSAM as well
NoneAny TX must be custom
implementation
NoneAny TX must be custom
implementation
YesProvided by Batch
Runtime and coordination with RRS
Java main()Standalone Java
Java main()Standalone Java
WebSphere Java Batchz/OS 2.1 - Based on subset of
WebSphere Java Batch model; runs in J2SE not J2EE, and no
WAS classes available.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD30
Version: September 10, 2013
Section Summary
WebSphere Java Batch …
A very broad summary of the JVM Launcher section:
JVM Launchers instantiate JVMs and run programGenerally an "application per JVM" model
We showed three JVM Launcher ModelsThey range from simple (BPXBATCH) to more functional (Batch Runtime)
Which you use depends on your business needsBPXBATCH may work fine for simple ad hoc functions; JZOS provides a more
flexible launching model with z/OS access services; Batch Runtime addresses the need for Java/COBOL interaction with coordinated transaction support.
Potential downside: cost of JVM start and stopOne potential downside of JVM launchers is the overhead associated with starting and stopping the JVM. May not be an issue if cycles are few; tends to become an
issue when the number of start/stop cycles increases.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD31
Version: September 10, 2013
WebSphere Java BatchJava Batch inside WebSphere Application Server
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD32
Version: September 10, 2013
WebSphere Java Batch Overview
Two Delivery Mechanisms …
The following is a very high-level overview of the WebSphere Java Batch function:
WebSphere AppServer Instance
EJB Container
Web Container
Batch Container
Java Batch Applications
IBM RAD Tooling Support
Dispatcher Function
z/OS Operating System
System z HW Platform
Job Submission Interfaces
xJCLxJCL
Job Properties Declaration File
Key Points:● This JVM stays up for as long as the WAS server stays up
Days or more ... avoids overhead of JVM instantiation in environments where batch invocation is high
● All the services of WAS z/OS are available to the batch applications running thereTransaction, data access, Parallel Sysplex and clustering, WLM, SAF, SMF
● The batch container provides valuable batch services that keep batch logic focused on the business and away from writing custom middleware codeMulti-step jobs, Batch Data Stream Framework, Checkpoint, Skip record, Step retry, Parallel Job Manager
TechDoc: ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101783
● Browser● Command Line● SOAP web services● RMI● MDB
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD33
Version: September 10, 2013
At Present Two Delivery Mechanisms
Container Services …
The "WebSphere Java Batch" function is delivered in two ways:
WebSphere Application Server
Version 7 or Version 8WebSphere Application Server
Version 8.5
Compute Grid V8WebSphere Java Batch Product
Add to WAS ("Augment") WebSphere Java BatchFunctionally equal to CG V8
For those who have a standardized commitment to
WAS V7 or V8 but desire WebSphere Java Batch function
WebSphere Java Batch function provided in WAS V8.5 product
Note how both are functionally identical
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD34
Version: September 10, 2013
Overview of Batch Container Services
Job Submission and the JMC …
The following provides a summary. The WP101783 Techdoc has many more details:
Batch job dispatching and management system● Separates job invocation and management from the
instantiation of the JVM● Provides job control: stop, restart, cancel, suspend● Multiple interfaces: Web, SOAP, RMI, command, MDB
Data record read and write support services● Abstracts low-level stream handling from business logic● Framework passes data to batch logic as Java object
Checkpoint and job restart services● Checkpoint tracking abstracted from batch logic
and performed as function of container● Commit and rollback a function of the container● Job restart from last checkpoint
Job resiliency services● Ability to skip and track records where data read
or write operation throws exception● Ability to retry job steps when unhandled
exception is thrown● Ability to utilize defined call-out points in batch
job execution to invoke custom Listener code
Parallel job management and execution services● Develop and deploy applications to run segments in
parallel based on your designation of data partitioning● Ability to programmatically partition data ranges based on
passed-in parameters● Container manages top-job and all sub-jobs to completion
COBOL Container Support● Invoke COBOL from Java● JNI stub code provided with container● Call stub generator utility to create call stubs and data
parameter code
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD35
Version: September 10, 2013
Job Submission and Monitoring
CI Model …
The Job Management Console (JMC) is the browser interface. It allows you to submit and monitor the jobs:
Job State Flow
Jobs go through a series of states in this model. The interface to the
Dispatcher Function provides you control to submit, view, cancel, stop,
suspend, restart or remove jobs
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD36
Version: September 10, 2013
High-Level: "Compute Intensive" Model
Transactional Batch Model …
This provides a simple execution model suitable for (a) computational work, or (b) a means of wrappering existing Java main() programs to run:
:
public class MyCIWork implements com.ibm.websphere.ci.CIWork {
:
public void run() {
Your batch code goes here
}
}
Batch Controller BeanStateless session bean provided by container framework
Success if no Exceptions
Thrown
Dispatch
Very simple model to develop and run
Very few batch runtime services available● No data read or write patterns to utilize ... roll your own● No checkpoint processing ... roll your own● No commit and rollback ... roll your own
Use tooling to wrapper Java main() into this framework and it will run in WebSphere Java Batch
Useful model for some types of applications.
Transactional Model may be better suited to your needs ...
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD37
Version: September 10, 2013
High-Level: "Transactional Batch" Model
xJCL …
This is what people usually think of when they think of "batch." At the heart of this is the job step code, which encapsulates the data record business logic:
Batch Controller BeanStateless session bean provided by container framework
Dispatch
:public class MyJobStep implements BatchJobStepInterface { : public void createJobStep() { } public int destroyJobStep() { } public int processJobStep() { Step logic for a data record here
if record not null: return BatchConstants.STEP_CONTINUE; if record null: return BatchConstants.STEP_COMPLETE; }}
Controller bean invokes job step in iterative loop until STEP_COMPLETE returned
Your job step logic would process data record
according to business needs
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD38
Version: September 10, 2013
Job Declaration File - xJCL
Checkpoint processing …
Conceptually, xJCL is just like normal // JCL -- it describes the job to be run and the context in which the job is to operate. The difference is xJCL is written in XML:
<?xml version="1.0" encoding="UTF-8" ?><job name="MyJob" ... "> <substitution-props> <prop name="inputDataStream" value="/tmp/input-text.txt" /> <prop name="outputDataStream" value="/tmp/output-text.txt" /> <prop name="checkPoint" value="10" /> </substitution-props>
<job-step name="MyStep1"> <classname>com.ibm.ws.batch.MyStep1</classname> <batch-data-streams> <bds> <logical-name>inputStream</logical-name> <props> <prop name="PATTERN_IMPL_CLASS" value="com.ibm.websphere.batch.EchoReader" /> <prop name="FILENAME" value="${inputDataStream}" /> </props> </bds> </batch-data-streams> </job-step> : <job-step ...> </job-step></job>
Note: this is a trimmed version of actual xJCL
Comparable to the JCL "JOB" card. It sets job-level information along with some substitution properties
Substitution properties may be overridden at submission time.
Comparable to the JCL step. It names the Java class that
implements the step function.
The "Batch Data Stream" implementation is specified. In this
example it defines the class that implements the input stream.
If job consists of more steps they are specified in sequence.
This tells WebSphere Java Batch what to run and how to run it
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD39
Version: September 10, 2013
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Checkpoint Processing
Batch data stream framework …
Checkpoint commit and rollback is a function of the container. It relies on the transaction manager function of WAS itself for this:
Java Batch Application
Batch Container
xJCL where checkpoint declared
WAS Transaction Manager
Commit
Commit
Last good checkpoint value persisted
Checkpoint interval (record or time) specified in the xJCL
This is a function of the batch container, not your application code
As checkpoint intervals are reached, container commits and records the checkpoint attained
In the event of a failure, job may be restarted at the last good checkpoint
Set the checkpoint interval based on your knowledge of balance between recoverability and efficiency
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD40
Version: September 10, 2013
Job Step Class
High-Level: "Batch Data Stream Framework"
Skip Record …
This provides a way of abstracting the data read and write logic away from the batch step code. Several Batch Data Stream patterns are provided:
Batch Controller BeanStateless session bean provided by container framework
Dispatch
Batch Data Stream ClassLogic to read or write a record of
data to the data store
Data passed as Java object to and from the BDS implementation
Version 8.5 InfoCenter: search on cgrid_btchpttrnover
Supplied "patterns" for:● Read and write using JDBC● Read and write JDBC and stored procedures● Read and write byte data from file● Read and write text file● Read and write to z/OS data set● Read and write using JPA
Options:● Take existing pattern and
customize to your needs● Create your own custom pattern● Not use BDS but rather code data
record read and write handling as part of batch job step
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD41
Version: September 10, 2013
Skip Record Processing
Multi-step jobs …
Provides a container-managed way of tolerating data read or write errors so the job itself may continue on. Information about data errors may be logged.
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Data Record
Java Batch Application
Batch Container
Batch Data Stream Framework Code
Exception!
Exception!
xJCL where skip record policy declared
xJCL tells BDSF:
● How many data read or write exceptions to consume
● What exceptions to consider for skiprecord processing
● Alternatively, what exceptions to exclude from skip-record processing
Objective is to provide mechanism to survive the odd data exception rather than stop the whole job
The BDSF implementation code consumes the exception rather than surfacing it up to the batch step code
A "listener" may be called to log information about the data record where exception occurredMore on this coming up
When the skip limit is reached then further exceptions are exposed up to the batch step implementationWhere it might result in job going into a restartable state. Normal checkpoint restart processing would occur.
Technical Flyer at: http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101783
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD42
Version: September 10, 2013
High-Level: Multi-Step Jobs
Retry step …
A job may consist of multiple steps. The job step execution for each is what was shown earlier. Job steps are executed in sequence according to xJCL:
Batch Controller BeanStateless session bean provided by container framework
Dispatch
Job Step Class
Job Step Class
Job Step Class
<job ... <job-step ... > </job-step>
<job-step ... > </job-step> : <job-step ... > </job-step></job>
xJCL
STEP_COMPLETE
STEP_COMPLETE
STEP_COMPLETE
Job may consist of one to many stepsMay be mixed CI, TX, Parallel
Steps are executed in sequential order according to xJCL
Supplied results algorithm allows for passing step results to next step
Conditional step processing supported
The construction of WebSphere Java Batch jobs begins to look very much like traditional batch with "Jobs" and "Steps". This is the one of
the benefits of a container-managed Java batch environment.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD43
Version: September 10, 2013
Retry Step Processing
Batch Listeners …
Provides a way of retrying a job step in the event of an exception thrown. If successful on retry then the job continues and your processing completes.
Job Step● Java class● Input / Output● Other Properties
Job Step● Java class● Input / Output● Other Properties
Batch Job
xJCL tells Container:● How many step retries may be attempted● What exceptions to consider for retry-step
processing● Alternatively, what exceptions to exclude
from retry-step processing● Whether to process a delay before
attempting a retry of the step
If exception, retry step n
times
This is at a higher level from Skip Record. This is at the "invoke batch step" level. This provides a way to retry the step for exceptions.
Batch container falls back to last good checkpoint and restarts from there
A "retry-step listener" may be called so you can perform custom action upon retry-step processingMore on "batch listeners" coming up
When limit reached, then job goes into restartable state. Normal restart from checkpoint processing.
Technical Flyer at: http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101783
Provides a way to survive unexpected transient
exceptions through step retry
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD44
Version: September 10, 2013
Batch "Listeners"
Parallel Job Manager …
These are callout points where your custom "listener" code will be called when key events occur. The callouts are managed by the batch container:
Job Step● Java class● Input / Output● Other Properties
Job Step● Java class● Input / Output● Other Properties
Batch Job
Job Listener
Retry Step Listener
Skip Record Listener
Job Listener● Callouts occur:
Start of the job; Start of each step; End of each step; End of job● Register your code to container with property in xJCL● Use this to perform any special setup or cleanup actions at those
points in the lifecycle of a batch job
Retry-Step Listener● Callouts occur:
When the exception is thrown; When the retry is attempted● Register your listener with code in application createJobStep()
method● Use this to take action at these points, such as logging
information about the exception and the point in the processing where it occurred
Skip-Record Listener● Callouts occur:
On skipped read or skipped write operation● Register your listener with code in application createJobStep()
method● Use this to take action at these points, such as logging
information about the exception and the record skipped
Technical Flyer at: http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101783
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD45
Version: September 10, 2013
Parallel Job Manager
COBOL Container …
This provides a mechanism for partitioning batch activity into parallel jobs based on ranges of data you designate:
Job Dispatching
Function
Batch Endpoint
PJM
Parameterizer
Batch Appl
Batch Endpoint
Batch Appl
Batch Endpoint
Batch Appl
Top-Job xJCL
Sub-Job xJCL
Top-job and job number
Sub-jobs ... top-job number and each sub-job number
State of each job
<job ...<run instances="multiple" jvm="multiple"> <prop name="parallel.jobcount" value="3" /> </props></run> :
Job step seen as PJM eligible based on "multiple" specified in xJCL
PJM function receives "top job" xJCL and invokes Parameterizer to perform your custom data range segmentation
Job Dispatcher then dispatches "sub jobs" to eligible endpoints for execution. Each is passed the data select parameters generated by parameterizer
Job Dispatcher coordinates execution of top job and sub jobs as logical entity
Not all jobs are well suited for PJM, but those that are may benefit greatly
Parallel either in multiple endpoints, on multiple threads, or both.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD46
Version: September 10, 2013
Java Batch Integration with COBOL
Enterprise Schedulers …
This is similar in concept to the z/OS Batch Runtime, but with some differences. The fundamental objectives is the same: low overhead integration Java and COBOL:
Java Batch Application
Batch Container
COBOL Container JNI Code
COBOL Container
COBOL Module
PDSE or USS Directory
WAS z/OS Servant Region Address Space
STEPLIB or LIBPATH
Call Stub Generator
IBM RAD
Call Stubs
COBOL Container manifests as separate LE Enclave inside the WAS z/OS servant address spaceKeeps COBOL/Java separate, so environments stay clean
The JNI code for Java/COBOL exchanges providedKey piece of plumbing provided, which relieves you of writing complex JNI
Call Stub Generator builds Java call stubs and data bindingsBased on what's in COBOL source
Provides a very efficient means of calling COBOL modules as part of a Java Batch solution. The JNI code and Call Stub Generator make using this solution relatively simple.
Share JDBC T2 with TXSimilar to what Batch Runtime provides
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD47
Version: September 10, 2013
Integration with Enterprise Schedulers
WLM Classification …
One of the interfaces on the Job Dispatcher functions is MDB. This allows a client program to interact with Java Batch and integrate that system with Ent. Schedulers:
Dispatcher Function
MDB Interface
Output
Input
Enterprise Scheduler Function
"WSGRID"Utility
WSGRID JCL
JES SpoolOr elsewhere if JCL defines other than spool
Submit
Exact same Java Batch processing as before
● Messaging provider either MQ or SIBus component of WAS● WSGRID submits job request to dispatcher● Job output from WebSphere Java Batch flows back to WSGRID● WSGRID stays active for duration of Java Batch job
Enterprise Scheduler sees "batch job" as the WSGRID invocation. That stays up for life of Java Batch job. Job output from WebSphere Java batch flows back to WSGRID. Return code from Java Batch flows back to WGRID.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD48
Version: September 10, 2013
WLM Classification
SMF 120.9 …
The Dispatcher Function on z/OS has the ability to assign a WLM Transaction Class value to a submitted job. That allows mapping to Service Class and Reporting Class:
Dispatcher Function
xJCL
Control Region
Servant Region(s)
Batch Application
WLM
WLM
Classify Work
DispatchSubmit
If (rule) then TC=tclass● Job class from xJCL● Jobname● Application Name● Submitter ID● other criteria
TCLASS
Assigned Service Class and/or Reporting Class
Two use-cases for this:
Assign separate WLM Service Classes
Have jobs dispatched into separate servant regions where WLM can manage to the goal
Assign separate WLM Report Classes
This allows categories of Java Batch jobs to have system resource information accumulated under the report class for RMF reporting
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD49
Version: September 10, 2013
Job Usage Information -- SMF 120.9 on z/OS
Positioning …
In addition to capturing job statistics in a relational table (all platforms), on z/OS the SMF facility can be used to capture batch job information.
Job Number
Submitter ID
Total CPU used
Job state
Node/Server
Job start time
Last update time
Accounting string
Job Number
Submitter ID
Total CPU used
GP Time used
Job state
Node/Server
Job start time
Last update time
Accounting string
All Platforms z/OS SMF 120.9
The information provided is very similar between "all platforms" and "z/OS SMF 120.9"
SMF 120.9 augments CPU information with GP usage. You can derive offload: Total - GP = Offload
If z/OS SMF is part of other job accounting and chargeback purposes, then SMF 120.9 for WebSphere Java Batch is simply part of that.
An example of taking advantage of the platform services
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD50
Version: September 10, 2013
Positioning WebSphere Java Batch
Summary …
The following provides a high-level positioning of the alternatives discussed so far:
JVM ModelBatch Runtime WebSphere Java Batch
COBOL Invocation Both WaysJava invoke COBOL or COBOL
invoke Java; z/OS 2.1 adds PL/I as wel. You write the JNI glue code
One WayJava invoke COBOL but not
the other way around. JNI glue code supplied with runtime.
Bit-mode 31-bitExpectation is it will be
used with COBOL
31-bit or 64-bit31-bit if COBOL container;
64-bit possible if not
JVM LauncherImplies JVM instantiation and tear-
down for each invocation
Long Running JVMBuilt on WAS server model where
servers stay up for as long as needed
Data Access Full Java EEAny data access protocol supported by Java EE WAS itself (JDBC, JCA,
JMS, WOLA, etc.)
JDBC and tVSAMIn z/OS 2.1 support for JDBC T2
and transactional VSAM.
Batch Services Full Java BatchEverything Batch Runtime has
plus Dispatcher and PJM
Subset of Java BatchIn z/OS 1.13 TX services; in z/OS 2.1 very close to full Java Batch, but no PJM
Programming Model WebSphere Java BatchFull Java Batch programming
model. Ability to wrapper Java main(). Full Java EE runtime.
WebSphere Java BatchIn z/OS 2.1 the support is based on Java
Batch. Difference: no PJM. Also, only JDBC T2 and tVSAM, no other Java EE access.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD51
Version: September 10, 2013
Section Summary
CICS, DB2 …
A very broad summary of the WebSphere Java Batch solution:
WebSphere Java Batch represents a full-function Java Batch framework and runtime model
Much more batch function than BPXBATCH or JZOS
It is built on WebSphere Application ServerWhich means all the functional value statements of WAS accrue to the Java Batch environment. On z/OS, all the "Why WAS z/OS" value statements accrue as well.
Integration with Enterprise SchedulersIt provides a messaging-based interface to the Dispatcher function so Enterprise
Scheduler functions can treat WebSphere Java Batch as any other batch job. This allows WebSphere Java Batch to be part of larger orchestrated batch flows.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD52
Version: September 10, 2013
Java Batch in CICS, DB2Exploring how Java Batch is implemented in those key data systems
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD53
Version: September 10, 2013
WebSphere Java Batch and Access to CICS
Feature Pack for Modern Batch …
Imagine a WebSphere Java Batch application needing access to CICS for data required to complete the batch job. How can that CICS access be accomplished?
WebSphere AppServer Instance
EJB Container
Web Container
Batch Container
Java Batch Application
CICS Program
CICS Region
Dispatcher FunctionxJCLxJCL
Job Properties Declaration File
What options exist for this interaction?
Web ServicesUbiquitous, open-standard protocol that spans platforms
Overhead per invocation somewhat high, particularly if transaction required. Not a good solution if latency per invocation a concern.
CTG JCAProven WAS-to-CICS interaction pattern.
WAS-to-CICS global TX
If EXCI, then 32K limit
If IPIC, then TCP stack code path is invoked
WOLAHighly efficient WAS-to-CICS interaction pattern
WAS-to-CICS global TX
Same LPAR required
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD54
Version: September 10, 2013
CICS TS Feature Pack for Modern Batch V1.0
Programming Model …
This allows a CICS region to become an job endpoint for a WebSphere Java Batch dispatching server. Puts batch logic much closer to the CICS data:
Dispatcher FunctionxJCLxJCL
Job Properties Declaration File
FP Modern Batch 1.0
Java Batch Application CICS
Program
JCICS Interface and Services
CICS Region TS4.2 or TS5.1
XML file in CICS Feature Pack setup lets that batch container communicate with the WAS Dispatcher Function host and port. That lets the Dispatcher know the
endpoint is present and what batch applications are deployed.
The application in the CICS batch container has a programming model very similar to that used by WebSphere Java Batch. Benefit is access to JCICS services and
very close proximity to the CICS program the batch application requires.
The batch job management is maintained in WebSphere Java Batch. The dispatching may take place to either WAS endpoints or CICS endpoints or both.
WAS V8.5 or aboveAny Platform
Communicates availability and applications deployed in container
xJCL submitted with application in CICS container gets dispatched
xJCL submitted with application in WAS container gets dispatched
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD55
Version: September 10, 2013
Programming Model for CICS Modern Batch FP
Positioning …
The programming model is based on the WebSphere Java Batch model. There are a few restrictions plus some added functions:
Essential Programming ModelEverything discussed in the earlier section on WebSphere Java Batch applies to the CICS Modern Batch FP as well, except Java EE functions and those things noted below.
Some Batch Data Stream Patterns Not SupportedThe following patterns of WebSphere Java Batch are not supported in the CICS Modern Batch Feature Pack environment:
JDBCReader and JDBCWriterLocalJDBCReader and LocalJDBCWriter
See below ... Feature Pack provides its own JDBC patterns
Additional Batch Data Stream Patterns ProvidedThe CICS Modern Batch Feature Pack adds the following BDS patterns:
VsamKsdsReaderImpl -- read from a VSAM KSDS as input data to the job.
VsamKsdsUpdaterImpl -- update in a VSAM KSDS file.
CICSJDBCReader -- retrieve data from a database using a Java Database Connectivity (JDBC) connection.
CICSJDBCWriter -- write data to a database using a Java Database Connectivity (JDBC) connection.
In addition, the CICS Modern Batch Feature Pack provides access to the full set of JCICS APIs
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD56
Version: September 10, 2013
Positioning CICS Modern Batch Feature Pack
DB2 Stored Procedures …
The following provides a high-level positioning of the alternatives discussed so far:
WebSphere Java Batch Modern Batch FPJVM Model Long Running JVM
Built on WAS server model where servers stay up for as long as needed
Long Running JVMBuilt on CICS JVM server model; JVM
stays up for as long as needed
Programming Model WebSphere Modern BatchCI or Transactional Batch
WebSphere Modern BatchCI or Transactional Batch
Multi-Step YesMulti-step jobs supported
YesMulti-step jobs supported
Job Dispatching Job Dispatching FunctionJob dispatching is handled
through the WebSphere Java Batch "scheduler" interface
Job Dispatching FunctionJob dispatching is handled
through the WebSphere Java Batch "scheduler" interface
Parallel Job Manager YesSupports WebSphere Java Batch
PJM interfaces and operation
YesSupports WebSphere Java Batch
PJM interfaces and operation
Enterprise Schedulers YesIntegration through WSGRID and
MDB interfaces to dispatcher function of WebSphere Java Batch
YesIntegration through WSGRID and
MDB interfaces to dispatcher function of WebSphere Java Batch
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD57
Version: September 10, 2013
High Level: DB2 Stored Procedures Written in Java
Summary …
Stored procedures allow for compact and efficient SQL calls from Java. Writing them in Java allows the execution of the stored procedure to be offloadable:
DB2 for z/OSV7 or above
WebSphere AppServer Instance
EJB Container
Web Container
Batch Container
Java Batch Applications
z/OS Operating System
System z HW Platform
Java source for DB2 Stored Procedure
Compiled Java classDB2 Stored Procedure
WLMWLM
StoredProcedure
Address spaces established by WLM to run JVM and Stored Procedures
NUMTCB in proc determines how many JVMs per address space; JVMs single threaded and re-used
CREATE PROCEDURESQLAPPL ENVJCL Proc
Reference: http://www.redbooks.ibm.com/abstracts/sg247604.html
Details behind all this, obviously ... key point is DB2 supports Java as a language for Stored Procedures (User Defined Functions as
well) and those may be called by batch processes
This shows WebSphere Java Batch, but any batch environment capable of calling SP in DB2
may exercise Java-based Stored Procedures
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD58
Version: September 10, 2013
Section Summary
Decision Management …
A very broad summary of the CICS and DB2 section:
The CICS Modern Batch Feature Pack extends WebSphere Java Batch management/execution realm
It allows CICS to participate as a WebSphere Java Batch endpoint server.
A CICS endpoint puts batch logic closer to CICSWhat work should run in the CICS endpoint is a function of the batch job and the
nature / intensity of its CICS interactions
DB2 Stored Procedures and JavaNot really "batch" in our normal sense, but it can part of a larger batch process.
Stored procedures can make batch execution more efficient, and Java stored procedures allow for further exploitation of Java on z/OS
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD59
Version: September 10, 2013
Batch and Decision ManagementIntegrating Java Batch with WODM
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD60
Version: September 10, 2013
Batch Step Decision Logic -- In Step or Outside Step?
Operational Decision Manager …
Imagine a batch job that reads data records and acts upon those records based on a set of decisions (or rules). Where would you prefer those rules be?
Job
Step
Read
Process
Write
Rules
Step
End
Decision logic written within the batch step
Job
Step
Read
Process
Write
Step
End
Rules Execution
Server
Call
Result
Rules Designer Tooling
Rules Repository
Rules
All else equal, it provides greater flexibility of design,
change and execution to externalize the rules
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD61
Version: September 10, 2013
IBM Operational Decision Manager
RES Options …
Provides real-time handling of business rule execution and situational awareness via correlation of business events received:
Decision Model:Given a snapshot view of data, determines best course of action at a specific point in a process or application
Main Purpose:To automate a decision based on a combination of factors such as business policies, regulations, and best practices.
Example:If the customer is not primary card holder and age is less than 21 then freeze account and notify primary card holder
Time-Based Pattern Detection:Correlating events as data is in motion
Main Purpose:To determine what of interest is transpiring and coordinate one or more responses by other systems or generate alerts to people
Example:If more than 2 customer withdraws in an ATM are done in the same day and the 2 ATMs are from 2 foreign countries then Investigate possible fraud. Reduce cash redraw max amount to 100$
Business Rules Business Events
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD62
Version: September 10, 2013
High Level: Rules Execution Server Options
JSR 352 …
The Rules Execution Server (RES) has a few means of placement in z/OS architecture:
zRES
zRES
RES on WAS for z
COBOL
Java
COBOL
IMS
COBOL
Batch
CICS Region
WebSphere Application Server for z/OS
Started Task
The Rules Execution Server may run either in a CICS region, as a
standalone started task, or inside the Java EE runtime of WAS z/OS
Our focus is on how a Java batch application in WebSphere Java Batch may access the RES, and it's by running it in WAS as well.
Invocation patterns on WAS include EJB and remote EJB.
Depending on proximity of Java Batch to RES, one of those two
would be optimalOther interfaces: Web Services and JMS/MDB
WO
LA
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD63
Version: September 10, 2013
Open Standard Java BatchA look at the JSR 352 specification
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD64
Version: September 10, 2013
JSR 352 - "Batch Applications for the Java Platform"
Generic Diagram …
The V1.0 final release is dated April 18th of 2013. IBM served as specification lead on this JSR. It is available for download from the web at the following URL:
http://jcp.org/aboutJava/communityprocess/final/jsr352/index.html
The document has a very nice section titled "Domain Language of Batch" which provides
an overview of key concepts and key
terminologyAs well as detailed sections on the specification interfaces and other details of the specification
If interested in this JSR, download and review is encouraged
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD65
Version: September 10, 2013
Generic Diagram for Domain Language of JSR 352
Comparison …
The diagram is generic but useful in that it provides a few key conceptual building blocks on which the rest of the specification relies:
JobOperator Job Step
ItemReader
ItemProcessor
ItemWriter
JobRepository
1 * 1
1
1
1
A job has one to many steps
Retrieval of input, one item at a time
Business processing for an item
Output of the step -- item or chunk
Interface to manage aspects of job processing
Holds information about jobs currently running and jobs that have run in the past
Concepts here are applicable to batch of any flavor -- historic COBOL, WebSphere Java Batch, JSR 352.
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD66
Version: September 10, 2013
JSR 352 Comparison to WebSphere Java Batch
Possible Integration …
A good deal of functional similarity with a few points where each has differential:
From Chris Vignola's IMPACT 2013 Presentation, TAW-1443
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD67
Version: September 10, 2013
Theoretical JSR 352/WebSphere Batch Integration
References …
A topology picture showing both models working cooperatively:
From Chris Vignola's IMPACT 2013 Presentation, TAW-1443
Common Batch Container
WebSphere Java Batch Applications
Dispatcher FunctionxJCLxJCL
WebSphere Java Batch
JSR 352 Java Batch Applications
JSLJSL
JSR 352 JSL
Common Job Repository
Common Job Logging
This implies no promise of future delivery ... that said, the model shown above is well within the possible given the nature of JSR 352 and the
similarities with the WebSphere Java Batch model.
WebSphere Java Batch as well as JSR 352
JobOperator function
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD68
Version: September 10, 2013
ReferencesA guide to reference information for further details
© 2013 IBM CorporationIBM Americas Advanced Technical SkillsGaithersburg, MD69
Version: September 10, 2013
References
http://www.ibm.com/developerworks/systems/library/es-java-batchz.html
JZOS vs. BPXBATCH
http://www.ibmsystemsmag.com/mainframe/trends/whatsnew/java_compiler/
SDK 7 features with EC12
http://www.ibm.com/systems/z/os/zos/tools/java/products/jzos/overview.htmlJZOS
z/OS 2.1 announcement with information on Batch Runtime improvements
http://www.ibm.com/common/ssi/rep_ca/3/897/ENUS213-013/ENUS213-013.PDF
WebSphere Java Batch
http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101783
CICS Transaction Server Feature Pack for Modern Batch V1.0
http://www-01.ibm.com/software/htp/cics/batch/