IBM Platform Application Center...9.1 Restful Web Service client–server separation simplifies...

Issued: May 15, 2013; Revised June

IBM® Platform Application Center

IBM Platform Application

CenterHow to build client to submit and

monitor job via Web Service

June 2014

Platform Application Center


Center V9.1: How to build client to submit and

nitor job via Web Service

George Gao

Technical Product Manager

IBM Systems and Technology Group


How to build client to submit and

nitor job via Web Service

Technical Product Manager

IBM Systems and Technology Group

IBM Platform Application Center

Executive Summary

Introduction ................................

Environment Preparat

Build the client program to submit a job

Build the client program to monitor the job

Build the client program to download the job result and logout

Run the Sample Script

Conclusion ................................

Further reading

Notices ................................

Trademarks ................................

Contacting IBM

Application Center: how to build client to submit and monitor job via Web Service

Executive Summary .............................................................................................

................................................................................................

Environment Preparation ................................................................

Build the client program to submit a job ..........................................................

Build the client program to monitor the job ................................

Build the client program to download the job result and logout

Run the Sample Script .......................................................................................

................................................................................................

Further reading.............................................................................................

................................................................................................

................................................................................................

Contacting IBM ............................................................................................

how to build client to submit and monitor job via Web Service of 19

............................. 3

.......................................... 4

................................................... 5

.......................... 7

................................................... 13

Build the client program to download the job result and logout ............. 15

....................... 16

.......................................... 17

............................. 17

................................................. 18

................................... 19

............................ 19


Executive SummaryIBM Platform Application Center

users to run and manage applications in cloud

program to interact with the system in addition to web por

directly.


which allow user to build client using any language, for example: Java, Python, Perl etc. and

running anywhere.


component implementation, reduces the complexity of connector semantics, improves the

effectiveness of performance tuning, and i


Executive Summary Application Center Version 9.1 is a flexible, application centric interface for cluster

s to run and manage applications in cloud. It provides Restful Web Service to enable client

program to interact with the system in addition to web portal for end user to access the system

Application Center 9.1 Restful Web Service provides the generality of interfaces


running anywhere.

Application Center 9.1 Restful Web Service client–server separation simplifies


effectiveness of performance tuning, and increases the server scalability


a flexible, application centric interface for cluster

provides Restful Web Service to enable client

tal for end user to access the system

9.1 Restful Web Service provides the generality of interfaces


server separation simplifies


ncreases the server scalability .


Introduction

The document serves as a best practice reference

application in cluster/cloud supported by IBM Platform LSF

The document describe

python:

Client program submit a job and monitor the job, when the job is done, download the

result file.

Notes

1. The procedure provided in

written in other language. The logic in the example is generic

2. Refer to administrator guide for details protocol of the APIs.


Introduction

The document serves as a best practice reference to building a client program to run, monitor an

application in cluster/cloud supported by IBM Platform LSF.

The document describes the details steps for administer to implement the following use case in


procedure provided in this document can be easily migrated to

written in other language. The logic in the example is generic.

Refer to administrator guide for details protocol of the APIs.


client program to run, monitor an

details steps for administer to implement the following use case in


an be easily migrated to client program


Environment PreparationThe steps in this document

1. IBM Platform LSF 9.x ha

2. IBM Platform Application Center V9.1 has been installed and running properly

3. HTTPS in Platform Applic

The following steps prepare environment in client and server to be ready to develop and

client program:

1. Identify a host for client program to run. The client program will submit a job and monitor

the job. The host could be

# uname -a

Linux ib05b01 2.6.18

GNU/Linux

2. Install Python 2.6.x in client host if it is not installed. Download from: http://www.python.org/download/releases/

# python –V

Python 2.6

# which python

/usr/local/bin/python

3. Install Google python http library: http://code.google.com/p/httplib2/downloads/list

Note: httplib2 2.0.6 is tested for http and https. If https is enabled, 2.0.7 and above

version may not work.

# tar -zxvf httplib2

httplib2-0.6.0/

httplib2-0.6.0/MANIFEST

httplib2-0.6.0/README

httplib2-0.6.0/python2/

…

httplib2-0.6.0/python3/README

httplib2-0.6.0/python3/httplib2test.py

httplib2-0.6.0/python3/httplib2/

httplib2-0.6.0/python3/httplib2/_

httplib2-0.6.0/python3/httplib2/iri2uri.py

httplib2-0.6.0/setup.py

# cd httplib2-0.6.0


Environment Preparation The steps in this document assume:

IBM Platform LSF 9.x has been installed and running properly

IBM Platform Application Center V9.1 has been installed and running properly

HTTPS in Platform Application Center is off, and the default port 8080 is used.

following steps prepare environment in client and server to be ready to develop and

Identify a host for client program to run. The client program will submit a job and monitor

the job. The host could be Windows, Linux or other OS which allow Python to run.

Linux ib05b01 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64

Install Python 2.6.x in client host if it is not installed. Download from: http://www.python.org/download/releases/

# which python

/usr/local/bin/python

Install Google python http library: httplib2 2.0.6 in client host. Download from:

http://code.google.com/p/httplib2/downloads/list

httplib2 2.0.6 is tested for http and https. If https is enabled, 2.0.7 and above

version may not work.

zxvf httplib2-0.6.0.tar.gz

0.6.0/MANIFEST

0.6.0/README

0.6.0/python2/

0.6.0/python3/README

0.6.0/python3/httplib2test.py

0.6.0/python3/httplib2/

0.6.0/python3/httplib2/__init__.py

0.6.0/python3/httplib2/iri2uri.py

0.6.0/setup.py

0.6.0


.

IBM Platform Application Center V9.1 has been installed and running properly

, and the default port 8080 is used.

following steps prepare environment in client and server to be ready to develop and test

Identify a host for client program to run. The client program will submit a job and monitor

Linux or other OS which allow Python to run.

15:51:48 EDT 2009 x86_64 x86_64 x86_64

Install Python 2.6.x in client host if it is not installed. Download from:

in client host. Download from:

httplib2 2.0.6 is tested for http and https. If https is enabled, 2.0.7 and above


# python setup.py install

running install

running build

…

byte-compiling /usr/lib/python2.6/site

byte-compiling /usr/lib/python2.6/site

running install_egg_info

Writing /usr/lib/python2.6/site

4. Copy over default CLI( pacclient.py and pac_api.py)

directory, verify web Service from the client host

# cp /opt/pac/gui/3.0/bin/pacclient.py .

# cp /opt/pac/gui/3.0/bin/pac_api.py .

#

# ./pacclient.py ping

Web Services are ready on

5. Identify application template to be used for job submission.

In this document

configure “generic” template.

If you want to use other application template

configured and tested from portal by administrator in web portal. Refer the application

integration guide for how to integrate application

following steps are required before start to

• Administrator integrates

• Administrator publish the template in web portal

• Administrator assign View&Control permission to the user who will run the client

program


python setup.py install

compiling /usr/lib/python2.6/site-packages/httplib2/iri2uri.py to iri2uri.pyc

/usr/lib/python2.6/site-packages/httplib2/__init__.py to __init__.pyc

running install_egg_info

Writing /usr/lib/python2.6/site-packages/httplib2-0.6.0-py2.6.egg-

Copy over default CLI( pacclient.py and pac_api.py) from Application Center installation

verify web Service from the client host

cp /opt/pac/gui/3.0/bin/pacclient.py .

# cp /opt/pac/gui/3.0/bin/pac_api.py .

./pacclient.py ping -l http://pac91:8080

Web Services are ready on URL:http://pac91:8080/

Identify application template to be used for job submission.

In this document, the default template “generic” will be used, no extra steps are needed

configure “generic” template.

If you want to use other application template, application template must be properly


integration guide for how to integrate application using an application template.

following steps are required before start to run client program:

Administrator integrates application using a template on web portal.

Administrator publish the template in web portal

Administrator assign View&Control permission to the user who will run the client

program


packages/httplib2/iri2uri.py to iri2uri.pyc

packages/httplib2/__init__.py to __init__.pyc

-info

from Application Center installation

extra steps are needed to

template must be properly


application template. The

application using a template on web portal.

Administrator assign View&Control permission to the user who will run the client


Build the client The client program may include other business logic

calling Application Center Web Service to submit and manage job. This

submission only, other logic could be built o

1. Include proper header for python client program:

#!/usr/bin/env python

import sys

import os

import time

import httplib

import httplib2

from xml.etree import ElementTree as ET

from xml.dom import minidom

from xml.parsers.expat import ExpatError

the first line < #!/usr/bin/env python > is for run python program automatically. Without this

line, user needs to run program like: #python client.py

2. Login and get the security token

Login web service requirement username and

the web server OS username and password. In this document, user “lsfadmin” with password

“lsfadmin” is used as an example.

host name in your environment in: ROOT_URL=’http:

With successfully logon, client will get the token

web service API calls later on.

token can be used for all the coming request

program need to logon agai

however from system effectiveness point of view, it is not

request.

ROOT_URL='http://pac91:8080/'

USERNAME='lsfadmin

PASSWORD='lsfadmin

ACCEPT_TYPE='text/plain,application/xml,text/xml'

ERROR_STR='errMsg'

NOTE_STR='note'

ERROR_TAG='<' + ERROR_STR + '>'

APP_NAME='APPLICATION_NAME'

EXPIRED_MSG='Your

#DESCRIPTION:

# logon service URL: http://pac91:8080/platform/webservice/pacclient/logon

# HTTP Method: POST

# request body format: <User><name>%s</name> <pass>%s</pass> </User>

# token is returned when successfully logon. Return body format:

# <user> <token> XXXXXXXXX</token>

def logon(username, password):


Build the client program to submit a job The client program may include other business logic, for example: create separate console by

calling Application Center Web Service to submit and manage job. This

submission only, other logic could be built on top.

Include proper header for python client program:

#!/usr/bin/env python

from xml.etree import ElementTree as ET

from xml.dom import minidom

from xml.parsers.expat import ExpatError


to run program like: #python client.py

Login and get the security token

Login web service requirement username and password from web server

server OS username and password. In this document, user “lsfadmin” with password

“lsfadmin” is used as an example. You will have to replace “pac91” with your real web server

host name in your environment in: ROOT_URL=’http://pac91:8080’

With successfully logon, client will get the token in the response. Cache and use the token for

web service API calls later on. The generated token has a time-to-live value of 8 hours

for all the coming requests until it is expired. When the token

program need to logon again to get new token. Multiple concurrent login

however from system effectiveness point of view, it is not recommended

ROOT_URL='http://pac91:8080/'

'lsfadmin'

PASSWORD='lsfadmin'

ACCEPT_TYPE='text/plain,application/xml,text/xml'

ERROR_STR='errMsg'

ERROR_TAG='<' + ERROR_STR + '>'

APP_NAME='APPLICATION_NAME'

EXPIRED_MSG='Your logon session has expired. To log on to PAC, run pacclient logon.'

# logon service URL: http://pac91:8080/platform/webservice/pacclient/logon

# HTTP Method: POST


token is returned when successfully logon. Return body format:

<token> XXXXXXXXX</token> </user>

def logon(username, password):


example: create separate console by

calling Application Center Web Service to submit and manage job. This document focus on job


web server host. The user name is

server OS username and password. In this document, user “lsfadmin” with password

You will have to replace “pac91” with your real web server

Cache and use the token for

live value of 8 hours. The

the token is expired, client

logins are supported,

recommended to login for every

logon session has expired. To log on to PAC, run pacclient logon.'

# logon service URL: http://pac91:8080/platform/webservice/pacclient/logonPost



http = httplib2.Http()

url_logon= ROOT_URL + 'platform/webservice/pacclient/logon

headers = {'Content

body='<User><name>%s</name> <pass>%s</pass> </User>' % (username, password)

try:

response, conten

headers=headers)

if response['status'] == '200':

Center as: " + username

else:

except (AttributeError, httplib2.ServerNotFoundError, httplib.InvalidURL,

ExpatError):

print "Cannot connect to t

sys.exit(1)

def main(argv):

token=logon(

print 'TOKEN=' + token

if __name__ == "__main__":

main(sys.argv[1:])

Run the above code to logon:

#./client.py

You have logged on to IBM

lsfadmin"2013-09-

13T03:01:08Z"ZK7sPPVeNRUenOJyfYlWGpeWmzrkj+d/aQp8U2SU5z7bLmQV3/8+Gdmv+pOIvoqV/DpIjWeDyqH7

iYtKJoNIVJJCEBzLpzXFSvjZV6N+ClhFU9jyQnSGBXJzmc7fvUsj"P/tSRJh8mi1dQwJT/lUMXw==

3. Submit a job and get the job

The following code will call web service api to submit a job into “generic” template with

attachment file: demo.data. The job will tar the file into a tar file:

First, check which parameters are mandatory in “generic”

#./pacclient.py app

ID LABEL MANDATORY DEFAULT VALUE

COMMANDTORUN

JOB_NAME Job Name

label1

label2

INPUT_FILE Input file

…

You have to define COMMANDTORUN. In the example, we set:



url_logon= ROOT_URL + 'platform/webservice/pacclient/logon

headers = {'Content-Type': 'application/xml', 'Accept': ACCEPT_TYPE}


response, content = http.request(url_logon, 'POST', body=body,


xdoc=minidom.parseString(content)

tk=xdoc.getElementsByTagName("token")

if len(tk) > 0:

print "You have logged on to IBM Platform Application

Center as: " + username

print tk[0].childNodes[0].nodeValue

return tk[0].childNodes[0].nodeValue

#saveToken(url, tk[0].childNodes[0].nodeValue)

else:

err=xdoc.getElementsByTagName("errMsg")

print err[0].childNodes[0].nodeValue

sys.exit(1)

else:

print "Failed to connect to web service URL: %s" % url_logon

sys.exit(1)


print "Cannot connect to the web service: %s" % url

sys.exit(1)

oken=logon('lsfadmin', 'lsfadmin')


if __name__ == "__main__":

main(sys.argv[1:])

Run the above code to logon:

You have logged on to IBM Platform Application Center as: lsfadmin



Submit a job and get the job ID


attachment file: demo.data. The job will tar the file into a tar file: demo.data.tar.Z.

heck which parameters are mandatory in “generic” template in CLI

./pacclient.py app -p generic


Command to run yes -

JOB_NAME Job Name -

label1 -

label2 -

INPUT_FILE Input file -

to define COMMANDTORUN. In the example, we set:


url_logon= ROOT_URL + 'platform/webservice/pacclient/logonPost/'

Type': 'application/xml', 'Accept': ACCEPT_TYPE}


', body=body,

rint "You have logged on to IBM Platform Application

print tk[0].childNodes[0].nodeValue

return tk[0].childNodes[0].nodeValue

(url, tk[0].childNodes[0].nodeValue)

err=xdoc.getElementsByTagName("errMsg")

print err[0].childNodes[0].nodeValue

print "Failed to connect to web service URL: %s" % url_logon


he web service: %s" % url_logon

Platform Application Center as: lsfadmin




demo.data.tar.Z.

in CLI as the following:



COMMANDTORUN: tar

INPUT_FILE: demo.data

Second: copy any file to demo.data in current directory.

Note: make sure the token

If job is submitted, then return job id for monitoring in the next. Otherwise, return error.

# DESCRIPTION:

# submit service URL:

# Header: Content-

# HTTP Method: POST

# Body: 3 areas--

# toke must be encrypted , and pass in with cookie name "platform_token"

# Response message format:

# <Job> <name>%s</name> <id>%s</id> <status>%s</status>

def submitJob(jobDict, token):

# encrypt the double quotations

ck_token='platform_token=' + token.replace('"', '

if APP_NAME not in jobDict.keys():

return 'error', 'Cannot find the published application: %s. This job

cannot be submitted.' % APP_NAME

boundary='bqJky99mlBWa

if 'PARAMS' in jobDict.keys():

job_params=jobDict['PARAMS']

else:

job_params={}

if 'INPUT_FILES' in jobDict.keys():

job_files=jobDict['INPUT_FILES']

else:

job_files={}

body = encode_body(boundary, jobDict[APP_NAME], job_params, job_files)

if body == None:

return 'error',"The profile Inputfile secti

is wrong\nsee the help."

if "Submit job failed" in body:

return 'error', body

headers = {'Content

url_submit = ROOT_URL + 'platform/webservice/pacclient/submitapp'


try:

response, content = http.request(url_submi

headers=headers)


else:

failed."

except (AttributeError, ExpatError):

return 'error',"Cannot connect to the web service: %s" % url


COMMANDTORUN: tar –zcf demo.data.tar.Z demo.data

INPUT_FILE: demo.data

Second: copy any file to demo.data in current directory.

: make sure the token is encrypted.

then return job id for monitoring in the next. Otherwise, return error.

# submit service URL: http://pac91:8080/platform/webservice/pacclient/submitapp

-Type multipart/mixed

# HTTP Method: POST

-- Application Name area; Parameter Area; File Attachment areas


message format:

<Job> <name>%s</name> <id>%s</id> <status>%s</status> <cmd>%s</cmd> </Job>

obDict, token):


ck_token='platform_token=' + token.replace('"', '#quote#')

if APP_NAME not in jobDict.keys():


not be submitted.' % APP_NAME

boundary='bqJky99mlBWa-ZuqjC53mG6EzbmlxB'

if 'PARAMS' in jobDict.keys():

job_params=jobDict['PARAMS']

job_params={}

if 'INPUT_FILES' in jobDict.keys():

job_files=jobDict['INPUT_FILES']

job_files={}


if body == None:

return 'error',"The profile Inputfile section or inputfile param format

nsee the help."

if "Submit job failed" in body:

return 'error', body

headers = {'Content-Type': 'multipart/mixed; boundary='+boundary,

'Accept': 'text/xml,application/xml;', 'Cookie': ck_token,

'Content-Length': str(len(body))}



response, content = http.request(url_submit, 'POST', body=body,


xdoc=minidom.parseString(content)

if ERROR_TAG not in content:

jobIdTag=xdoc.getElementsByTagName("id")

return 'ok', jobIdTag[0].childNodes[0].nodeValue

else:

err=xdoc.getElementsByTagName(ERROR_STR)

return 'error', err[0].childNodes[0].nodeValue

else:

return 'error', "Failed to connect to web service and submission


return 'error',"Cannot connect to the web service: %s" % url


then return job id for monitoring in the next. Otherwise, return error.

http://pac91:8080/platform/webservice/pacclient/submitapp

le Attachment areas


cmd>%s</cmd> </Job>

')



on or inputfile param format

Type': 'multipart/mixed; boundary='+boundary,

xml;', 'Cookie': ck_token,


t, 'POST', body=body,

yTagName("id")

return 'ok', jobIdTag[0].childNodes[0].nodeValue

err=xdoc.getElementsByTagName(ERROR_STR)

return 'error', err[0].childNodes[0].nodeValue

return 'error', "Failed to connect to web service and submission

return 'error',"Cannot connect to the web service: %s" % url_submit


# DESCRIPTION:

# build HTTP request body with 3 areas

# Application template name area format:

# Content-

# Content-

# "\r\n"

# generic

# Template parameter area format:

# Content-

# Content-

# Content-

# “\r\n”

# --_Part_1_701508.1145579811786

# Content-Disposition: form

# Content-Type: application/xml; charset=US

# Content-Transfer

# “\r\n”

# <AppParam> <id>JOB_NAME<

# --_Part_1_701508.1145579811786


# Content-Type: application/xml; charset=US

# Content-Transfer

# “\r\n”

# <AppParam> <id>COMMANDTORUN</id> <value>tar

demo.data</value><type></type> </AppParam>

# --_Part_1_701508.1145579811786

# Content-

# Content-

# Content-

# “\r\n”

# <AppParam> <id>INPUT_FILE</id> <value> demo.data,upload </value> <type>

file</type> </AppParam>

# --_Part_1_701508.1145579811786

def encode_body(boundary,

boundary2='_Part_1_701508.1145579811786'

def encode_appname():

return ('

'Content

'Content

'', appName)

def encode_paramshead():

return('

'Content

'Content

'C

def encode_param(param_name):

return('

'Content

'Content

'Content

'', '<AppParam><id>%s</id><value>%s</value><type></type></AppParam>'

%(param_name, params[param_name]))

def encode_fileparam(param_name, param_value):

return('

'Content

'Content

'Content

'', '<AppParam><id>%s</id><value>%s</value><

%(param_name, param_value))

def encode_file(filepath, filename):


# build HTTP request body with 3 areas

# Application template name area format:

-Disposition: form-data; name=AppName

-ID: <AppName>

# Template parameter area format:

-Disposition: form-data; name=’data’

-Type: multipart/mixed; boundary=_Part_1_701508.1145579811786

-ID: <data>

_Part_1_701508.1145579811786

Disposition: form-data; name=’c1’

Type: application/xml; charset=US-ASCII

Transfer-Encoding: 8bit

<AppParam> <id>JOB_NAME</id> <value>WS_DEMO</value><type></type> </AppParam>

_Part_1_701508.1145579811786

Disposition: form-data; name=’c2’

Type: application/xml; charset=US-ASCII

Transfer-Encoding: 8bit

<AppParam> <id>COMMANDTORUN</id> <value>tar -zcf demo.data.tar.Z

demo.data</value><type></type> </AppParam>

_Part_1_701508.1145579811786

-Disposition: form-data; name=’c3’

-Type: application/xml; charset=US-ASCII

-Transfer-Encoding: 8bit

<AppParam> <id>INPUT_FILE</id> <value> demo.data,upload </value> <type>

file</type> </AppParam>

_Part_1_701508.1145579811786--

def encode_body(boundary, appName, params, inputFiles):

boundary2='_Part_1_701508.1145579811786'

def encode_appname():

return ('--' + boundary,

'Content-Disposition: form-data; name="AppName"',

'Content-ID: <AppName>',

'', appName)

def encode_paramshead():

return('--' + boundary,

'Content-Disposition: form-data; name="data"',

'Content-Type: multipart/mixed; boundary='+ boundary2,

'Content-ID: <data>', '')

def encode_param(param_name):

return('--' + boundary2,

'Content-Disposition: form-data; name="%s"' % param_name,

'Content-Type: application/xml; charset=US-ASCII',

'Content-Transfer-Encoding: 8bit',


%(param_name, params[param_name]))

def encode_fileparam(param_name, param_value):

return('--' + boundary2,

'Content-Disposition: form-data; name="%s"' % param_name,

'Content-Type: application/xml; charset=US-ASCII',

'Content-Transfer-Encoding: 8bit',

'', '<AppParam><id>%s</id><value>%s</value><type>file</type></AppParam>'

%(param_name, param_value))



Type: multipart/mixed; boundary=_Part_1_701508.1145579811786

/id> <value>WS_DEMO</value><type></type> </AppParam>

zcf demo.data.tar.Z

<AppParam> <id>INPUT_FILE</id> <value> demo.data,upload </value> <type>

Type: multipart/mixed; boundary='+ boundary2,

data; name="%s"' % param_name,

ASCII',


data; name="%s"' % param_name,

ASCII',

type>file</type></AppParam>'


return('

%(filename, filename),

lines = []

upType = ''

upFlag = False

lines.extend(encode_appname())

lines.extend(encode_paramshead())

for name in params:

lines.extend (encode_param(name))

for name in inputFiles:

value=input

if ',' in value:

value.split(',')[0]

'/').split('/').pop()

else:

lines.extend (('

if upFlag == True:


'/').split('/').pop()

filename))

or directory: %s" % filepath

lines.extend (('

return '\r



'Content-Disposition: form-data; name="%s"; filename="%s"'


'Content-Type: application/octet-stream',

'Content-Transfer-Encoding: UTF-8',

'Content-ID: <%s>' % filename,

'', open(filepath, 'rb').read ())

lines = []

upType = ''

upFlag = False

lines.extend(encode_appname())

lines.extend(encode_paramshead())

for name in params:

lines.extend (encode_param(name))


value=inputFiles[name]

if ',' in value:

try:

upType = value.split(',')[1]

if (upType == 'link') | (upType == 'copy'):

lines.extend (encode_fileparam(name, value))

if upType == 'copy':

print "Copying server file: %s " %

value.split(',')[0]

else:

upFlag = True

filename = value.replace('

'/').split('/').pop()

lines.extend (encode_fileparam(name, filename))

except IndexError:

return

else:

return

lines.extend (('--%s--' % boundary2, ''))

if upFlag == True:


value=inputFiles[name]

if ',' in value:

upType = value.split(',')[1]

filepath = value.split(',')[0]

if upType == 'upload':

filename = filepath.replace('

'/').split('/').pop()

try:

lines.extend(encode_file(filepath,

except IOError:

return "Submit job failed, No such file

or directory: %s" % filepath

print "Uploading input file: %s" % filepath

lines.extend (('--%s--' % boundary, ''))

r\n'.join (lines)


data; name="%s"; filename="%s"'

if (upType == 'link') | (upType == 'copy'):

.extend (encode_fileparam(name, value))

print "Copying server file: %s " %

filename = value.replace('\\',

lines.extend (encode_fileparam(name, filename))

filename = filepath.replace('\\',

lines.extend(encode_file(filepath,

return "Submit job failed, No such file

print "Uploading input file: %s" % filepath


#DESCRIPTION:

# build file attachment into HTTP request

# File attachment area format:


# Content-Type: application/octet

# Content-Transfer

# Content-ID: <demo.data>

# “\r\n”

# PGh0bWw+CiAgPGhlYWQ+CiAgPC9oZWFkPgogIDxib2R5PgogICAgPHA+VGhpcyBpcyB0aGUg

def encode_body_upfile(boundary, dir, filelist):

def encode_dir():

return ('


return('


lines = []

lines.extend(encode_dir())

files = filelist.split(',')

for f in files:

filename

try:

except IOError:

lines.extend (('

return 'ok','

#DESCRIPTION:

# main program to define high level work flow.

def main(argv):

token=logon('lsfadmin', 'lsfadmin


JobDict={}

JobDict[APP_NAME]=

JobDict['PARAMS']= {

JobDict['INPUT_FILES']={

status, mess

print 'status =' + status

print 'message =' + message

if (status == 'error'):

logout(token)

sys.exit(2)

jobID = message

logout(token)

if __name__ == "__main__":

main(sys.argv[1:])


# build file attachment into HTTP request

# File attachment area format:

Disposition: form-data; name=’f1’

Type: application/octet-stream

Transfer-Encoding: UTF-8

ID: <demo.data>

PGh0bWw+CiAgPGhlYWQ+CiAgPC9oZWFkPgogIDxib2R5PgogICAgPHA+VGhpcyBpcyB0aGUg

def encode_body_upfile(boundary, dir, filelist):

def encode_dir():

return ('--' + boundary,

'Content-Disposition: form-data; name="DirName"',

'Content-ID: <DirName>',

'', dir)



'Content-Disposition: form-data; name="%s"; filename="%s"'


'Content-Type: application/octet-stream',

'Content-Transfer-Encoding: UTF-8',

'Content-ID: <%s>' % filename,

'', open(filepath, 'rb').read ())

lines = []

lines.extend(encode_dir())

files = filelist.split(',')

for f in files:

filename = f.replace('\\', '/').split('/').pop()

try:

lines.extend(encode_file(f, filename))

except IOError:

return 'error',"Failed to read from file %s" % f

lines.extend (('--%s--' % boundary, ''))

return 'ok','\r\n'.join (lines)


token=logon('lsfadmin', 'lsfadmin')


JobDict={}

JobDict[APP_NAME]='generic'

JobDict['PARAMS']= {'COMMANDTORUN': 'tar -zcf demo.data.tar.Z demo.data'

'JOB_NAME': 'WS_DEMO' }

JobDict['INPUT_FILES']={ 'INPUT_FILE':'demo.data,upload'}

status, message = submitJob(JobDict, token)

print 'status =' + status

print 'message =' + message


logout(token)

sys.exit(2)

jobID = message

logout(token)

"__main__":

main(sys.argv[1:])


PGh0bWw+CiAgPGhlYWQ+CiAgPC9oZWFkPgogIDxib2R5PgogICAgPHA+VGhpcyBpcyB0aGUg

data; name="DirName"',

data; name="%s"; filename="%s"'

return 'error',"Failed to read from file %s" % f

zcf demo.data.tar.Z demo.data',


Build the client After the job is submitted, you want to monitor the job status. Client program will have to deal

with different step based on the job status change. For example: if

program can wait or

then check again. If the job exit, client program simply exit in this case.

The following procedure is to get the job status b

#DESCRIPTION:

# Service URL: http://pac91:8080/platform/webservice/pacclient/jobs?id=<id>

# HTTP Method: GET

# Token is passed in the cookie: platform_token

# Response content format

# SUCCESS: <Jobs> <Job>

<name>%s</name><id>%s</id><stat

# No Job found: <Jobs> <note>%s</note> </Jobs>

# Error: <Jobs> <errMsg> %s </errMsg> </Jobs>

def getJobListInfo(parameter, token):

url_job = ROOT_URL + 'platform/webservice/pacclient/jobs?' + parameter


headers = {'Content

ACCEPT_TYPE}


try:

response, content = ht


else:

url_job


return 'error', "Cannot connect

the following procedure show

#DESCRIPTION:


def main(argv):

# 1. logon web service and get token

token=logon(USERNAME, PASSWORD)


# 2. prepare job input data and submit the job

JobDict={}


JobDict['PARAMS']= {'COMMANDTORUN': 'tar




Build the client program to monitor the jobthe job is submitted, you want to monitor the job status. Client program will have to deal

with different step based on the job status change. For example: if job is still pending

program can wait or do some other tasks. In this example, just simply wait for 2 seconds,

check again. If the job exit, client program simply exit in this case.

following procedure is to get the job status by give a job ID.


# HTTP Method: GET


# Response content format

# SUCCESS: <Jobs> <Job>

<name>%s</name><id>%s</id><status>%s</status><cmd>%s</cmd></Job><Jobs>

# No Job found: <Jobs> <note>%s</note> </Jobs>

# Error: <Jobs> <errMsg> %s </errMsg> </Jobs>

def getJobListInfo(parameter, token):



headers = {'Content-Type': 'application/xml', 'Cookie': ck_token, 'Accept':


response, content = http.request(url_job, 'GET', headers=headers)


xdoc=ET.fromstring(content)

if ERROR_TAG in content:

tree=xdoc.getiterator("Jobs")

err=tree[0].find(ERROR_STR)

return 'error', err.text

elif NOTE_STR in content:

tree=xdoc.getiterator("Jobs")

err=tree[0].find(NOTE_STR)

return 'error', err.text

else:

tree=xdoc.getiterator("Job")

jobstatus=tree[0].find('status')

return 'ok', jobstatus.text

else:

return 'error', "Failed to connect to web service URL: %s" %


return 'error', "Cannot connect to the web service: %s" % url_job

the following procedure shows the logic of monitoring job status by calling above function:


# 1. logon web service and get token

token=logon(USERNAME, PASSWORD)


# 2. prepare job input data and submit the job

JobDict={}


JobDict['PARAMS']= {'COMMANDTORUN': 'tar -zcf demo.data.tar.Z demo

'JOB_NAME': 'WS_DEMO' }




to monitor the job the job is submitted, you want to monitor the job status. Client program will have to deal

job is still pending, the client

simply wait for 2 seconds, and

check again. If the job exit, client program simply exit in this case.


us>%s</status><cmd>%s</cmd></Job><Jobs>



Type': 'application/xml', 'Cookie': ck_token, 'Accept':

tp.request(url_job, 'GET', headers=headers)

return 'error', "Failed to connect to web service URL: %s" %

to the web service: %s" % url_job

monitoring job status by calling above function:

zcf demo.data.tar.Z demo.data',



print(message)

logout(token)

sys.exit(2)

else:

print('Job has been submitted as: %s' %message)

jobID = message

# 3. monitor the job status, if job is not done, wait for 2 seconds, then check

again

for i in range(100):

status, message = getJobListInfo('id='+jobID, token)

if (status == 'ok'):

file.'%(jobID,message))

program agaon')

seconds...'%(jobID,message))

else:

program again')

print('the job is still pending or running, add more time to wait, then run this

program again')



print(message)

logout(token)

sys.exit(2)

print('Job has been submitted as: %s' %message)

jobID = message


range(100):


if (status == 'ok'):

if(message == 'Done' ):

# 4. when job is done successfully, download the result file

print('Job(%s, %s) complete, start to download the result

file.'%(jobID,message))

downloadFile('demo.data.tar.Z',jobID, token)

logout(token)

sys.exit(0)

break

elif (message == 'Exit'):

print('Job failed to run, fix your env, and run this

logout(token)

sys.exit(4)

break

else:

print('Job(%s, %s) does not complete yet, wait for 2

seconds...'%(jobID,message))

time.sleep(2);

else:

if(message == EXPIRED_MSG):

token = logon(USERNAME, PASSWORD)

else:

print('cannot get job status, fix your env, then run this





# 4. when job is done successfully, download the result file

print('Job(%s, %s) complete, start to download the result

downloadFile('demo.data.tar.Z',jobID, token)

print('Job failed to run, fix your env, and run this

print('Job(%s, %s) does not complete yet, wait for 2

x your env, then run this



Build the client

logout After the job is done successfully, you

file name: demo.data.tar.Z.

The following function shows the how to access web service to download a file for a job.

#DESCRIPTION:

# download a file from job's working directory to local host

# Service URL Path: http://pac91:800/platform/webservice/pacclient/file/<jobid>

# HTTP Method: GET


def downloadFile(fName,jobId, token):


ck_token='platform_token=' + tok

url_file = ROOT_URL + 'platform/webservice/pacclient/file/' + jobId

body=fName

headers = {'Content

ACCEPT_TYPE}


try:

response, content = http.request(url_file, 'GET', body=body,

headers=headers)

if len(content) > 0:

%fName)

else:

file does not exist: %s" % fName

except AttributeError:

print "Cannot connect to the web service: %s" % url_file

except IOError:

print "

After everything is done, issue logout request to release the w

#DESCRIPTION:

# Service URL: http://pac91:8080/platform/webservice/pacclient/logout/

# HTTP request Method: GET


def logout(token):


url_logout= ROOT_

ck_token='platform_token='+token

headers = {'Content

ACCEPT_TYPE}

try:

response, content = http.request(url_logout, 'GET', headers=headers)


successfully."


Build the client program to download the job

the job is done successfully, you can download the result file to local directory. The result

file name: demo.data.tar.Z.


# download a file from job's working directory to local host

rvice URL Path: http://pac91:800/platform/webservice/pacclient/file/<jobid>

# HTTP Method: GET


def downloadFile(fName,jobId, token):




body=fName

headers = {'Content-Type': 'text/plain', 'Cookie': ck_token, 'Accept':



if len(content) > 0:

f=open(fName,'wb')

f.write(content)

f.close()

print('File %s has been downloaded into current directory.'

else:

print "Download failed. you have no permission or the specified

file does not exist: %s" % fName

AttributeError:


except IOError:

print " --Permission denied to write the file: %s" %

After everything is done, issue logout request to release the web service session:


# HTTP request Method: GET


def logout(token):


url_logout= ROOT_URL + 'platform/webservice/pacclient/logout/'

ck_token='platform_token='+token

headers = {'Content-Type': 'text/plain', 'Cookie': ck_token, 'Accept':



if content == 'ok':

print "you have logout IBM Platform Application

else:

print content


download the job result and

can download the result file to local directory. The result


rvice URL Path: http://pac91:800/platform/webservice/pacclient/file/<jobid>

en.replace('"', '#quote#')


Type': 'text/plain', 'Cookie': ck_token, 'Accept':


print('File %s has been downloaded into current directory.'

print "Download failed. you have no permission or the specified


Permission denied to write the file: %s" % fName

eb service session:


URL + 'platform/webservice/pacclient/logout/'

Type': 'text/plain', 'Cookie': ck_token, 'Accept':


print "you have logout IBM Platform Application Center


else:


pr

Run the Sample

Above functions are included in a sample script. Run the script as following:

1. Download and copy client.py to current directory

2. Replace “pac91” in ROOT_URL with the your

3. Replace value in USERNAME and PASSWORD with real value in your environment

4. Copy any file to file: dmeo.data in current directory

lsfadmin@pac92-41: ls

-rwxr-xr-x 1 lsfadmin

-rw-r--r-- 1 lsfadmin

lsfadmin@pac92-43: ./client.py

You have logged on to IBM Platfor

TOKEN=lsfadmin"2013

14T03:09:35Z"4WPoaN/Utg0JU39AUpW3VMhUgW+FZwi/SVkjJk8XXew8HmH6Dnk+tpdFWoSmarsfOHm3ROrfEy2s

t5xJBFFvKdQNQxhl34S72q3q3oR/JU87VrUUwLHDeMo6VTOUFHr3"9grzdWtGE3wGOXV0PQfHZw==

Uploading input file: demo.data

Job has been submitted as: 13442

Job(13442, Pending) does not complete yet, wait for 2 seconds...

Job(13442, Pending) does not complete yet, wait f

Job(13442, Done) complete, start to download the result file.

File demo.data.tar.Z has been downloaded into current directory.

you have logout IBM Platform Application Center successfully.

lsfadmin@pac92-45: ls

-rwxr-xr-x 1 lsfadmin




else:

print "Failed to connect to web service URL: %s" % url_logout


print "Cannot connect to the web service: %s" % url_logout

Run the Sample Client


Download and copy client.py to current directory

Replace “pac91” in ROOT_URL with the your real web service hostname

Replace value in USERNAME and PASSWORD with real value in your environment

Copy any file to file: dmeo.data in current directory

41: ls -l

lsfadmin lsf 13825 Sep 13 15:09 client.py*

lsfadmin lsf 1491 Sep 13 15:09 demo.data

43: ./client.py

You have logged on to IBM Platform Application Center as: lsfadmin

"2013-09-


JBFFvKdQNQxhl34S72q3q3oR/JU87VrUUwLHDeMo6VTOUFHr3"9grzdWtGE3wGOXV0PQfHZw==

Uploading input file: demo.data

Job has been submitted as: 13442



Job(13442, Done) complete, start to download the result file.

File demo.data.tar.Z has been downloaded into current directory.

you have logout IBM Platform Application Center successfully.

45: ls -l

lsfadmin lsf 13825 Sep 13 15:09 client.py*

lsfadmin lsf 1491 Sep 13 15:09 demo.data

lsfadmin lsf 509 Sep 13 15:09 demo.data.tar.Z


print "Failed to connect to web service URL: %s" % url_logout

int "Cannot connect to the web service: %s" % url_logout


real web service hostname

Replace value in USERNAME and PASSWORD with real value in your environment

m Application Center as: lsfadmin


JBFFvKdQNQxhl34S72q3q3oR/JU87VrUUwLHDeMo6VTOUFHr3"9grzdWtGE3wGOXV0PQfHZw==


Conclusion IBM Platform Application

management software.

This document has described the steps to

Application Center and LSF, the examp

own client using many other languages: Java, C++, Perl etc. and

based on your own business logic. The most important: your client can run anywhere with IBM

Platform Application Center Web Service interface built

Further reading

• IBM Platform

http://www

03.ibm.com/systems/te

nter.html


Application Center is easy-to-use, application-centric job submission and

software. It is a web portal based bridge for your application and cloud

This document has described the steps to build a client program to interact with IBM Platform

Application Center and LSF, the example is written in python language

own client using many other languages: Java, C++, Perl etc. and you could build your client


ation Center Web Service interface built-in.

eading

IBM Platform Application Center version 9.1

http://www-

03.ibm.com/systems/technicalcomputing/platformcomputing/products/lsf/applicationce


centric job submission and

is a web portal based bridge for your application and cloud.

build a client program to interact with IBM Platform

python language, you could build your

could build your client


chnicalcomputing/platformcomputing/products/lsf/applicationce


Notices This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other count

Consult your local IBM representative for information on the products and services currently available

in your area. Any reference to an IBM product, program, or service is not intended to state or imply

that only that IBM product, program, or servic

program, or service that does not infringe any IBM intellectual property right may be used instead.

However, it is the user's responsibility to evaluate and verify the operation of any non

program, or service.

IBM may have patents or pending patent applications covering subject matter described in this

document. The furnishing of this document does not grant you any license to these patents. You can

send license inquiries, in writing, to:

IBM Director of Licensing

IBM Corporation

North Castle Drive

Armonk, NY 10504

U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such

provisions are inconsistent with local law:

PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,

INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON

OR FITNESS FOR A PARTICULAR P

warranties in certain transactions, therefore, this statement may not apply to you.

Without limiting the above disclaimers, IBM provides no representations or warranties regarding the

accuracy, reliability or serviceability of any information or recommendations provided in this

publication, or with respect to any results that may be obtained by the use of the information or

observance of any recommendations provided herein.

has not been submitted to any formal IBM test and is distributed AS IS.

the implementation of any recommendations or techniques herein is a customer responsibility and

depends on the customer’s

environment. While each item may have been reviewed by IBM for accuracy in a specific situation,

there is no guarantee that the same or similar results will be obtained elsewhere. Anyon

to adapt these techniques to their own environment do

This document and the information contained herein may be used solely in connection with the IBM

products discussed in this document.

This information could include

periodically made to the information herein; these changes will be incorporated in new editions of

the publication. IBM may make improvements and/or changes in the product(s) and/or the

program(s) described in this publication at any time without notice.

Any references in this information to non

in any manner serve as an endorsement of those

part of the materials for this IBM product and use of those

IBM may use or distribute any of the information you supply in any way it believes appropriate

without incurring any obligation to you.

Any performance data contain

the results obtained in other operating environments may vary significantly. Some measurements

may have been made on development

measurement

may have been estimated through extrapolation. Actual results may vary. Users of this document

should verify the applicable data for their specific environment.

Information c

published announcements or other publicly available sources. IBM has not tested those products

and cannot confirm the accuracy of performance, compatibility or any other cl


This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other count



that only that IBM product, program, or service may be used. Any functionally equivalent product,


However, it is the user's responsibility to evaluate and verify the operation of any non

ogram, or service.



send license inquiries, in writing, to:

Director of Licensing

IBM Corporation

North Castle Drive

Armonk, NY 10504-1785


provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION


INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY

OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied





observance of any recommendations provided herein. The information contained in this document

has not been submitted to any formal IBM test and is distributed AS IS. The use of this information or


depends on the customer’s ability to evaluate and integrate them into the customer’s operational


there is no guarantee that the same or similar results will be obtained elsewhere. Anyon

to adapt these techniques to their own environment does so at their own risk.


products discussed in this document.

This information could include technical inaccuracies or typographical errors. Changes are



) described in this publication at any time without notice.

Any references in this information to non-IBM websites are provided for convenience only and do not

in any manner serve as an endorsement of those websites. The materials at those

part of the materials for this IBM product and use of those websites is at your own risk.


without incurring any obligation to you.

Any performance data contained herein was determined in a controlled environment. Therefore,


may have been made on development-level systems and there is no guarantee that these

measurements will be the same on generally available systems. Furthermore, some measurements


should verify the applicable data for their specific environment.

Information concerning non-IBM products was obtained from the suppliers of those products, their


and cannot confirm the accuracy of performance, compatibility or any other cl


This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries.



e may be used. Any functionally equivalent product,


However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product,




INTERNATIONAL BUSINESS MACHINES CORPORATION


INFRINGEMENT, MERCHANTABILITY

URPOSE. Some states do not allow disclaimer of express or implied





ion contained in this document

The use of this information or


ability to evaluate and integrate them into the customer’s operational


there is no guarantee that the same or similar results will be obtained elsewhere. Anyone attempting

so at their own risk.


technical inaccuracies or typographical errors. Changes are



sites are provided for convenience only and do not

sites. The materials at those websites are not

sites is at your own risk.


ed herein was determined in a controlled environment. Therefore,


level systems and there is no guarantee that these

s will be the same on generally available systems. Furthermore, some measurements


IBM products was obtained from the suppliers of those products, their


and cannot confirm the accuracy of performance, compatibility or any other claims related to non-


IBM products. Questions on the capabilities of non

of those products.

All statements regarding IBM's future direction or intent are subject to change or withdrawal without

notice, and re

This information contains examples of data and reports used in daily business operations. To illustrate

them as completely as possible, the examples include the names of individuals, companies, brands,

and products. All o

an actual business enterprise is entirely coincidental.

COPYRIGHT LICENSE:

This information contains sample applicatio

programming techniques on various operating platforms. You may copy, modify, and distribute

these sample programs in any form without payment to IBM, for the purposes of developing, using,

marketing or distri

for the operating platform for which the sample programs are written. These examples have not

been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or i

serviceability, or function of these programs.

Trademarks

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business

Machines Corporation in the United States, other countries, or both. If these and

trademarked terms are marked on their first occurrence in this information with a trademark symbol

(® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the

time this information was published. Such trademarks

trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright

and trademark information” at www.ibm.com/legal/copytrade.shtml

Windows is a trademark

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product,

Contacting IBMTo contact IBM in your country or region, check the IBM Directory of Worldwide Contacts at

http://www.ibm.com/planetwide

To learn more about IBM

http://www-03.ibm.com/systems/technicalcomputing/platformcomputing/products/lsf/


IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers

of those products.


notice, and represent goals and objectives only.



and products. All of these names are fictitious and any similarity to the names and addresses used by

an actual business enterprise is entirely coincidental.

COPYRIGHT LICENSE: © Copyright IBM Corporation 2013. All Rights Reserved.

This information contains sample application programs in source language, which illustrate



marketing or distributing application programs conforming to the application programming interface


been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or i

serviceability, or function of these programs.


Machines Corporation in the United States, other countries, or both. If these and



time this information was published. Such trademarks may also be registered or common law


and trademark information” at www.ibm.com/legal/copytrade.shtml

is a trademark of Microsoft Corporation in the United States, other countries, or both.



Other company, product, or service names may be trademarks or service marks of others.

Contacting IBM To contact IBM in your country or region, check the IBM Directory of Worldwide Contacts at

http://www.ibm.com/planetwide

e about IBM Platform LSF and Application Center, go to

03.ibm.com/systems/technicalcomputing/platformcomputing/products/lsf/


IBM products should be addressed to the suppliers




f these names are fictitious and any similarity to the names and addresses used by

. All Rights Reserved.

n programs in source language, which illustrate



buting application programs conforming to the application programming interface


been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability,


Machines Corporation in the United States, other countries, or both. If these and other IBM



may also be registered or common law


he United States, other countries, or both.



or service names may be trademarks or service marks of others.

To contact IBM in your country or region, check the IBM Directory of Worldwide Contacts at

03.ibm.com/systems/technicalcomputing/platformcomputing/products/lsf/

IBM Platform Application Center...9.1 Restful Web Service client–server separation simplifies...

Documents

Transcript of IBM Platform Application Center...9.1 Restful Web Service client–server separation simplifies...