OGCE Overview: Services, Gadgets, and Tags Marlon Pierce, Suresh Marru, Gregor von Laszewski, Mary...
-
Upload
damian-watts -
Category
Documents
-
view
216 -
download
1
Transcript of OGCE Overview: Services, Gadgets, and Tags Marlon Pierce, Suresh Marru, Gregor von Laszewski, Mary...
OGCE Overview: Services, Gadgets, and Tags
Marlon Pierce, Suresh Marru, Gregor von Laszewski, Mary Thomas, Maytal Dahan, Gopi Kandaswamy, Rion Dooley, and Wenjun Wu
Links for More Information Website: http://www.collab-ogce.org
News/RSS/Blog: http://collab-ogce.blogspot.com
Email: [email protected]
SVN Code Repository: http://ogce.svn.sourceforge.net/viewvc/ogce/ And get tar.gz’s
Source Forge: http://sourceforge.net/projects/ogce
See us at SC08: IU booth 1:00-3:00 on Tuesday, November 18.
Project Description Status
Grid Portal Packaged, one step installation of Grid portlets into Tomcat+Gridsphere
Stable, V2.4 for SC08
Axis Services One step install of Resource Discovery and Prediction Services. More services to come.
Dev. Release for SC08.
Workflow Suite Support for scientific workflows, based on LEAD tools. Suresh will describe next time.
Dev. Release for E-Science
GTLAB JSF Tag Libraries for building Grid portlets and Gadgets
Dev. Release for SC08
JavaScript COG(Cyberaide)
JavaScript libraries for building Grid gateways.
Dev. Preview for SC08
Gadget Container
Open Social Inspired container for I Google gadget
Dev. Preview for SC08
Incubator projects
Pylons portals, metascheduling services, bio-gadgets, etc.
SVN if you dare
Services, Workflow, GTLAB, Cyberaide, and Gadget container are part of an emerging stack
OGCE and Gateways We develop and package software for use by
Gateways.
A lot of this comes from active Gateways. GPIR: TeraGrid User Portal Workflow tools: LEAD Resource Discovery Service, File Browser Applet: GridChem
Gateways
OGCESoftwar
e
GPIR for Facebook, iGoogle GPIR: GridPort Information
Repository, stores dynamic and static resource information for VOs
Facebook Mini TeraGrid system monitor and full
size system monitor Mini:
http://apps.facebook.com/mini_tgsysmon/
Full Size: http://apps.facebook.com/tgsysmon/
Google Mini System monitor for iGoogle page To add visit: http://www.google.com/ig/adde?
moduleurl=http://iola.tacc.utexas.edu:22080/gpir-app/minigpir.xml&nocache=1&source=imag
GPIR Facebook App
Resource Discovery Service
Extracted from the GridChem project/gateway Contributed by Rion Dooley
Axis2 Web service
Integrates information from INCA, GPIR services. What machines are in my Grid^H^H^H^H Cloud? What are the service names? Which ones are actually running?
Works with TeraGrid deployed services, but you can use it with your own information service installations.
Resource Prediction Service (RPS)
• Predicts an optimal set of resources for running scientific applications– Predicts a set of resources on which the sum of data
transfer time, queue wait time and compute time for the application will be minimum
– Data transfer time: Uses input data size and NWS info
– Queue wait time: Uses QBETS info– Compute time: Uses application performance info
• Axis2 web service with a MySQL backend
• RPS is a subset of the Fault Tolerance and Recovery service used in LEAD and VGrADS
Packaging ServicesEverything builds with one command
mvn clean install
We use Apache Maven 2 for builds.Structured, modular builds and consistent layouts
Build a single module: mvn clean install –f somemod/pom.xml
We include Apache Tomcat in the download, so all you need is Java and MySQL.
Suitable also for the NMI build and test system.
Want the Javadoc? mvn javadoc:javadoc
User’s BrowserJavaScript
OGCE Gadget Container
iGoogle, Facebook
GTLAB Gadgets
TeraGrid Services Other ServicesRPS, RDS, GPIR
Services
GTLAB’s Relationship to Other Components
MyProxy Example<h:form id="form"> <h:outputText value="Username:"/> <h:inputText value="#{resource.username}" required="true"/>
<h:outputText value="Password: "/> <h:inputSecret value="#{resource.password}" required="true"/>
<h:outputText value="MyProxy Server: "/> <h:inputText value="#{resource.myproxyHostname}" required="true"/>
<o:submit id="submit" value="Submit" binding="#{builder.body}" action="#{builder.junkAction}" >
<o:multitask id="multi" persistent="true" taskname="#{resource.taskname}"> <o:myproxy id="mypr" hostname="#{resource.myproxyHostname}"
lifetime="2" password="#{resource.password}" port="7512"
username="#{resource.username}"/> </o:multitask> </o:submit> </h:form>
Executing Multiple Task<o:submit id="submit" value="Submit" binding="#{builder.body}"
action="#{builder.junkAction}" > <o:multitask id="multi" persistent="true"
taskname="#{resource.taskname}"> <o:jobsubmit id="js" myfaces="true"
arguments="#{resource.arguments}" executable="#{resource.executable}" hostname="#{resource.hostname}" provider="#{resource.provider}" stdout="#{resource.stdout}"/>
<o:filetransfer id="ft" myfaces="true" from="#{resource.from}"
to="#{resource.to}" /> <o:dependency id="dep" task="ft"
dependsOn="js"/> </o:multitask></o:submit>
GTLAB FeaturesExtends Java Server Faces.o Tag components wrap major COG Abstraction Layer
featureso And Web Service clients, GridShib, etc.o Supports tag inter-dependencies.
Allows you to do standalone development.
Use JSF portlet bridge to convert into portlets No new coding, just add/modify XML config files and jars.
Or forget about portlets and o Convert into Google Gadgetso Develop as a Facebook applicationo Etc.
Installing GTLABDownload the release code from www.collab-ogce.org or
check out from SVN:o svn checkout
https://ogce.svn.sourceforge.net/svnroot/ogce/GTLABo SourceForge also supports tar.gz downloadso Tagged release in
http://ogce.svn.sourceforge.net/viewvc/ogce/tags/
Run “mvn clean install” in GTLAB directory.Download includes a web server.Start with “./startup.sh”, go to http://localhost:8080/GTLAB
What’s New with GTLAB?Completely revised dependency graph processing.
Removed dependence on COG graph processing (still use the COG)
We can now support graphs of any tag you care to write. Only support pipelines currently, not full DAGs
Redesigned to simplify writing new tags. Extensive use of factories. You no longer have to change core code to add a tag
Simplified process of creating new tags 4 Java files and 3 XML config files Now can be done in one (mvn) step
Many new tags
Levels of Abstraction in GTLAB code
You can use prepackaged gadgets/portlets.
You can develop new applications using existing tag libraries.
You can make new tags for your services
You can hack or steal the codeNote JSF is compatible with JSP, so you can
mix and match.
And you can embed JavaScriptYUI, Scriptaculous user interface goodies.Google and Facebook APIs.
Tag Description
MyProxy Gets a proxy credential (COG)
JobSubmit Runs remote commands (COG)
FileOperation Create, delete, list remote files and dirs (COG)
FileTransfer GridFTP downloads, 3rd party transfers (COG)
XSubmissionOutput
Format outputs of job and file operations
ResourceDiscovery
Clients for the RDS
ResourcePrediction
Clients for the RPS
MyCommunityProxy
Get and decorate TG Community credentials with Grid Shib
Community Log Log use of TG Community credentials
GTLAB Example Description
MyProxyExample Get a credential and store in OGCE global proxy store.
JobSubmitExamples
Various examples showing how to submit a grid job and get output.
Dashboard Examples
Runs showq on Big Red, Cobalt, and Mercury and formats output.
DirectoryBrowser Displays and downloads files on a remote host via GridFTP
MultipleTaskExample
Shows how to couple file transfers, operations, and
These will work as gadgets, portlets, or standalone applications
GTLAB Gadgets CodeBase
GTLAB PortletsCodeBase
Tomcat Web
Server
Tomcat Web Server
+ Portlet Container
iGoogleClient
Aggregator
Compile and deploy into server
Decorate with bridge and container jars, xml config files.
GTLAB gadgets can also run as portlets with no code changes by using the JSF portlet bridge.
GTLAB Applications as Google Gadgets: MOAB dashboard, remote directory browser, and proxy management.
What’s next?
OpenID and RSS client support
Support for more complicated graphs
JavaScript/AJAX actions as well as HTML FORM actions.
But really, we’re looking for interested developers and users.
Other GadgetsProviders
Tomcat + GTLAB Gadgets
Grid and Web Services
(TeraGrid, OSG, etc)
Other GadgetsProviders
Social Network Services (Orkut,
LinkedIn,etc)
RSS Feed, Cloud, etc
Services
Gadget containers aggregate content from multiple providers. Content is aggregated on the client by the user. Nearly any web application can be a simple gadget (as Iframes)
Tomcat +
Portlets and Container
Grid and Web Services
(TeraGrid, OSG, etc)
Grid and Web Services
(TeraGrid, OSG, etc)
Grid and Web Services
(TeraGrid, OSG, etc)
HTML/HTTP
SOAP/HTTP
Common science gateway architecture. Aggregation is in the portlet container. Users have limited selections of components.
Making GTLAB Google Gadgets
A Gadget is a standalone application that is integrated into your personal iGoogle display.They can run on your Web server.You can use SSL/HTTPS
Google Gadgets have two versionshttp: these use Google’s JavaScript APIsurl: these wrap external applications with IFrames.
Limitations:Not much real estate (but location, location,
location) Cookies manage state on gadgets from the same
server but we need a way to integrate multiple independent servers.
Making GTLAB Gadgets into Portlets
Do all development in jsf_standalone.
When you are ready to convert:cd GTLAB/transitionmvn clean process-resourcescd GTLAB/portletsmvn clean install
You will need to manually edit portlet.xml, group.xml, and layout.xml in GTLAB/portlet/src/main/webapp/WEB-INF/
That’s it.
Various GTLAB applications deployed as portlets: Remote directory browsing, proxy management, and LoadLeveler queues.
Example Gadget Config File <?xml version="1.0" encoding="UTF-8" ?> <Module> <ModulePrefs title="MyProxy Gadget Example"
scrolling="true" height="500"/>
<Content type="url" href="http://…/:8080/GTLAB/examples/
MyProxyExample.jsf"> </Content> </Module>
Save this as MyProxy.xml and place in a Web accessible place.
Under the Hood: Building Components
Portlets can be built with our Apache Velocity bridge.“Template” or “steal this code” approach.
OGCE portlets use the Java COG Abstraction Layer to access Grid services.Provides a buffer over different Grid toolkits and
versions.Provides ability to compose graph-based
workflows.Tutorial in itself
We have also developed Grid Tag Libraries and Beans (GTLAB) to simplify this process.