WS-VLAM Tutorial: Part II: Port a Legacy application to WS-VLAM Adam Belloum.
-
Upload
milton-snow -
Category
Documents
-
view
224 -
download
3
Transcript of WS-VLAM Tutorial: Part II: Port a Legacy application to WS-VLAM Adam Belloum.
WS-VLAM Tutorial: Part II: Port a Legacy application to WS-VLAM
Adam Belloum
Outline
• Part I: Hands on the User Graphical Interface• Part II: Port a Legacy application to WS-VLAM• Part III: Learn how to use WS-VLAM API
Outline (Part II): Porting A Legacy application (LA)
Step 1: Define an Interface for your LA Step 2: Create a description file for your LAStep 3: Define the LA dependencies Step 4: Pack the LA and dependencies libStep 5: Deploy your LA and dependencies libStep 6: Compose a workflow using the LAStep 7: customize the composed workflowStep 8: execute the workflow Step 9: monitor
Who is doing What?
Step 1, Step 2, Step 3, Step 4, Step 5 performed by the application developers, requires– Insight about the application to be ported– Programming knowledge– bit of scripting and OS
Step 6, Step 7, Step 8, Step 9 performed by Scientists, require:– Application domain knowledge
Step 1: Define the component interface
Step 1: Define the component interface
Step 2: Create a description file for your LA
• Using your favorite text editor created a file call config.xml
• config.xml has 4 sections– input
– output
– parameter
– exec
• Note:
– in the omnimatch example there is no parameter section
<?xml version=“1.0” encoding=ISO-8859-1” ?><config>
<output> <port name=“output_file.ccf”> <port name =“output_file.ang”> </output>
<input> <port name=“psf”> <port name =“tomogram”> <port name=“template”> <port name =“mask”> </intput>
<exec command=$BASE_DIR/omnimatch.sh”/>
</config>
Step 3: Define the component dependencies
Step 3: Create a pseudo-module
# ############ NOTE ##################### In the profile.sh export all the environment # variables needed to execute your package## bash environment# # export VARIABLE_NAME=VARIABLE_VALUE## Add your VARIABLES bellow the line######################################
Step 4: package your workflow component
Create a tar file of your component:
$ tar cf COMPONENT_NAME \ COMPONENT_NAME/
Compress the tar file you have created
$ gzip COMPONENT_NAME.tar
Step 5: Deploy your workflow component
• Test mode: you can test your component
– Perform the steps described in the blue box
• Shared mode: you can share your module with other users:
– by moving the tar file you have created in step 4 to a component repository
# On a machine where both GT4 & WS-VLAM# runtime system are installed # Create in your HOME directory called “.wsvlam”
$ mkdir .wsvlam$ cd .wsvlam/$ mkdir modules$ cd modules
#untar the component tar file you have created # in step 4
$ tar –zxvf COMPONENT_NAME.tar.gz
Step 6.1: Compose a workflow using LA
• Create a description of your component: use CLAM workflow component configuration tool
Action: • Click on CLAM
Button in the WS-VLAM menubar
Step 6: Compose a workflow using the LA
Composition panelComponents
Palette
Moduleproperty
panel
monitoring panel
Step 6.2: Compose a workflow using the LA
• CLAM workflow component configuration tool will allow you to define: name of the component, default host, ports, and parameters
Action: • Enter the name of the
component “omnimatch”
• Entre the name of the default host “fs2.das3.science.uva.nl”
Step 6.3: Compose a workflow using the LA
• CLAM workflow component configuration tool allows you to define: name of the component, default host, ports, and parameters
Action: • Click on ports tab
• Click + to add new input or output port
• Type the name of the ports exactly as you did it in config.xml
Step 6.4: Compose a workflow using the LA
• To add a port a pop up window appears “Add input Port” or “Add output port”
Action: • Type the name of the
ports exactly as you did it in config.xml
• Specify the type of the port either a simple data type, a file type, or Complex data type (see documentation for more details)
Step 6.5: Compose a workflow using the LA
• For the omnimatch you need to define 4 input ports and 2 output ports (see omnimatch config.xml)
Action: • Repeat the same
steps 6.3 and 6.4 for each port
Step 6.6: Compose a workflow using the LA
• Create a description of your component: use CLAM workflow component configuration tool
Action: • Click on save to
created the description file “omnimatch.xml”
Step 6.7: Compose a workflow using the LA
• Create a description of your component: use CLAM workflow component configuration tool
Action: • You will see in the
filed file path the full path to the created “ominimathc.xml”
Step 6.8: Compose a workflow using the LA
• Restart the WSVLAM GUI: use CLAM workflow component configuration tool
Action: • Click on the start a
new instance in the WS-VLAM menu bar
Step 6.9: Compose a workflow using the LA
• In the component list window you will see the created omnimatch components as well as other components
Action: • Click on the
omnimatch in the list and drag and drop in the composition panel
Step 6.10: Compose a workflow using LA
• An instance of Omnimatch will be instantiated
Action: • Click on the
omnimatch in the list and drag and drop in the composition panel
Step 6.11: Compose a workflow using LA
• Instantiate the fileReader component: • this component will get the input file needed by
ominimatch from a remote distribute location and stage it in to be used by omnimatch
Step 6.12: Compose a workflow using LA
• Ominimatch requires 4 input files, we need to instantiate 4 fileReaders
Action: • Repeat step 6.11 for
each fileReader
Step 6.13: Compose a workflow using LA
• Instantiate the fileWriter component:• This component will move the file produced by omnimatch
to any remote location the user specifies
Step 6.14: Compose a workflow using LA
• Instantiate the omnimatch workflow component
Action: • Repeat step 6.13 for
each fileWriter
Step 7.1: customize the composed workflow
• In the step you should check if all the default values of the parameters of the workflow components composing your workflow are set correctly
Step 7.1: customize the composed workflow
• For the tutorial all the input file are available from
• Fs2.das2.science.uva.nl: Type in the following URI for parameter in each fileReader
– gsiftp://fs2.das.3science.uva.nl/home2/vleroot/wsvlam/data/psf.em
– gsiftp://fs2.das.3science.uva.nl/home2/vleroot/wsvlam/data/tomogram.em
– gsiftp://fs2.das.3science.uva.nl/home2/vleroot/wsvlam/data/template.em
– gsiftp://fs2.das.3science.uva.nl/home2/vleroot/wsvlam/data/mask.em
Step 7.2: customize the composed workflow
• For the tutorial all the output files can be saved from on fs2.das3.science.uva.nl: Type in the following URI for parameter in each fileWriter
– gsiftp://fs2.das3.science.uva.nl/tmp/output-1
– gsiftp://fs2.das3.science.uva.nl/tmp/output-2
– You can also use your home directory to save the results of das3: gsiftp://fs2.das3.science.uva.nl/home/<user_account>/output1 or 2
– Any other host where ftp server is running and you right to save file on it
Step 8: execute the composed workflow
• Now you are ready to execute the omnimatch workflow, which automatically stage in the data file needed for the omnimatch application, and stage out the result files
Outline (Part III): Learn how to use WS-VLAM API
Step 1: Define an Interface for your componentStep 2: Create a description for your componentStep 3: WS-VLAM APIStep 4: Example of JAVA componentsStep 6: Example of C/C++ componentsStep 5: Example of PYTHON componentsStep 6: Deploy the component/dependencies libStep 7: Compose a workflow using the componentStep 7: execute the workflow / monitor
Some References
1. Science Z Zhao, A Belloum, M Bubak Editorial: Special section on workflow systems and applications in e-Future Generation Computer Systems 25 (5), 525-527
2. A. Wibisono, D. Vasyunin, V. Korkhov, AS.Z. Belloum WS-VLAM: a GT4 based workflow management system , The 2nd International Workshop on Scientific Workflows, In conjunction with ICCS 2007, Beijing, China, 2007
3. A.S.Z. Belloum, V. Korkhov, S. Koulouzis, M. A Inda, and M. Bubak Collaborative e-Science experiments: from scientific workflow to knowledge sharing JULY/AUGUST, IEEE Internet Computing, 2011
4. Ilkay Altintas, Manish Kumar Anand, Daniel Crawl, Shawn Bowers, Adam Belloum, Paolo Missier, Bertram Ludascher, Carole A. Goble, Peter M.A. Sloot, Understanding Collaborative Studies Through Interoperable Workflow Provenance, IPAW2010, Troy, NY, USA
Some References
5. A. Belloum, Z. Zhao, and M. Bubak Workflow systems and applications , Future Generation Comp. Syst. 25 (5): 525-527 (2009)
6. Z. Zhao, A.S.Z. Belloum, et al., Distributed execution of aggregated multi domain workflows using an agent framework The 1st IEEE International Workshop on Scientific Workflows, Salt Lake City, U.SA, 2007
7. Zhiming Zhao, Adam Belloum, Cees De Laat, Pieter Adriaans, Bob Hertzberger Using Jade agent framework to prototype an e-Science workflow bus Authors Cluster Computing and the Grid, 2007. CCGRID 2007