Toolkit Considerations for Application Bundles in IBM InfoSphere Streams V4.0

© 2015 IBM Corporation

Toolkit Considerations

IBM InfoSphere Streams Version 4.0

James Cancilla

Streams Toolkit Developer

For questions about this presentation contact James Cancilla -

[email protected]

2 © 2015 IBM Corporation

Important Disclaimer

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONALPURPOSES ONLY.

WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THEINFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTYOF ANY KIND, EXPRESS OR IMPLIED.

IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY,WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.

IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OROTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.

NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:

• CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS ORTHEIR SUPPLIERS AND/OR LICENSORS); OR

• ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENTGOVERNING THE USE OF IBM SOFTWARE.

IBM’s statements regarding its plans, directions, and intent are subject to change orwithdrawal without notice at IBM’s sole discretion. Information regarding potentialfuture products is intended to outline our general product direction and it should notbe relied on in making a purchasing decision. The information mentioned regardingpotential future products is not a commitment, promise, or legal obligation to deliverany material, code or functionality. Information about potential future products maynot be incorporated into any contract. The development, release, and timing of anyfuture features or functionality described for our products remains at our solediscretion.

THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.

IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.


Agenda

Summary of Key Points

Impact to Toolkit Developer

General Specialized Toolkit Strategy

Migration Impact of SPL Applications

Summary of Changes in Specialized Toolkits


Summary of Key Points

Application Bundle– All files required for the application to run are built into the application bundle

– This includes any configuration files and toolkit dependencies

Data directory is no longer mandatory– Data directory is no longer created by the compiler at compile-time

– Data directory is not part of the application bundle

Current working directory is no longer the data directory– In previous versions of Streams, the CWD was the data directory

– In Streams v4.0.0, the CWD is the application directory

– The CWD is a temporary directory that is created by the Streams runtime


Impact to Toolkit DeveloperApplication Bundle

Toolkit developers will need to determine what files need to be

included as part of application bundle

Metadata or configuration files are files that are needed by the

application but do not change while the application is running– These are files that should be in the application bundle

– Metadata files may include:• connection.xml documents

• PMML files

• operator initialization scripts

• etc.

Data files are files that may be changed over the life of an application– These files should not be in the application bundle

– Data files may include:• files that are modified by an external app that need to be read by an operator

• files that need to be written to


Impact to Toolkit DeveloperData directory is no longer mandatory

Toolkit operators can no longer assume a data directory is present– In previous releases, the data directory was assumed to be available

• All files were expected to be placed in the data directory

• Relative paths were rooted from the data directory

In Streams v4.0, this assumption is no longer true– The data directory may not exist for your application

– The host running the operator may not contain a data directory

How to migrate:– Review all cases where a relative file path is needed

– Identify cases where you have assumed that relative file paths are rooted in the data

directory

– Determine if the file is a data file or a metadata file, and design your operator to handle

the path accordingly.

– Update operator to gracefully handle the case when data directory is not available


Impact to Toolkit DeveloperCWD is no longer the data directory

The current working directory is now a special directory created by the runtime– The CWD is NOT the data directory

– This change will affect how relative paths are resolved

In previous release, if the following code is specified in SPL:

parameter

file: “myfile.txt” ;

– At runtime, “myfile.txt” would resolve to “<path_to_data_dir>/myfile.txt”

In Streams v4.0, “myfile.txt” will be resolved to “<app_bundle_working_dir>/myfile.txt”– Applications written in previous versions of Streams may no longer work correctly since the file will not

be found in the app bundle working directory

– The operator will get a runtime error/exception stating that the file is not found

To migrate:– Do not assume that the data directory is the current working directory

– Determine where you expect the file to be located based on the file type (metadata vs data)

– Find all cases where relative paths are handled, and build absolute paths from relative paths• For data files, relative paths should be built relative to the data directory

• For metadata files, relative paths should be built relative to the application directory


General Specialized Toolkit Strategy

Updated all file-related parameters in operators to support application

bundle

Handling of Data Files– Data files are recommended to be put in the data directory

• They should not be included as part of the application bundle

– If a relative path is specified, the relative path is relative to the data directory

– Application can also specify data files in SPL as follows:

parameter

file : getDataDirectory() + “/mydata.txt” ;

Handling of Metadata Files:– Metadata files are recommended to be put in the ‘etc’ directory

• These files will be included as part of the application bundle

– If a relative path is specified, the relative path is relative to the application directory

– Application can also specify data files in SPL as follows:

parameter

file : getThisToolkitDir() + “/etc/connection.xml” ;


Migration Impact of SPL Applications

Refer to toolkit documentation for details on how file-related

parameters are handled and have changed

SPL application using any of the affected Specialized Toolkits will

need to be migrated– Make sure that files are placed in the recommended directories

– If relative paths are used in SPL applications, update where relative paths are

defined

Summary of Changes:

http://www-

01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.in

stall.doc/doc/ibminfospherestreams-migrating-applications-

schema.html?lang=en


Summary of Changes in Specialized Toolkits


Add links to migration docs

Migration to Streams 4.0 for SPL Developer– https://developer.ibm.com/streamsdev/docs/migration-streams-4-0-spl-

developer/

Migration to Streams 4.0 for Toolkits Developer– https://developer.ibm.com/streamsdev/docs/migration-streams-4-0-toolkits-

developer/

Migrating streams processing applications to InfoSphere Streams

Version 4.0– http://www-

01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.install

.doc/doc/ibminfospherestreams-migrating-applications-main.html

Removal of an InfoSphere Streams default data directory for

streams processing applications– http://www-

01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.install

.doc/doc/ibminfospherestreams-migrating-applications-data-dir.html?lang=en


Questions?

Toolkit Considerations for Application Bundles in IBM InfoSphere Streams V4.0

Data & Analytics

Transcript of Toolkit Considerations for Application Bundles in IBM InfoSphere Streams V4.0