Transcript of Apache Airavata GSOC 2013. Knowledge and Expertise Computational Resources Scientific Instruments...
- Slide 1
- Apache Airavata GSOC 2013
- Slide 2
- Knowledge and Expertise Computational Resources Scientific
Instruments Algorithms and Models Archived Data and Metadata
Advanced Science Tools Target Community: Science Gateways Enabling
& Democratizing Scientific Research
- Slide 3
- What does Apache Airavata do? Compose, manage, execute, and
monitor distributed, computational workflows. Wrap legacy command
line scientific applications with Web services. Run jobs on
computational resources ranging from local resources to
computational grids and clouds. Manage provenance data.
- Slide 4
- Workflow Interpreter Application Factory Message Box Regist ry
Apache Airavata API Lorem ipsumLorem ipsum insolensinsolens p1p1
m5m5 duo duo x End Users Gateway Developer Scientific Applicati on
Core Developer Computational Resources Apache Airavata
- Slide 5
- Apache Airavata Components ComponentDescription XBayaWorkflow
graphical composition tool. Registry ServiceInsert and access
application, host machine, workflow, and provenance data. Workflow
Interpreter Service Execute the workflow on one or more resources.
Application Factory Service (GFAC) Manages the execution and
management of an application in a workflow Messaging
SystemWS-Notification and WS-Eventing compliant publish/subscribe
messaging system for workflow events Airavata APISingle wrapping
client to provide higher level programming interfaces.
- Slide 6
- Hi, Im Nolram. Im a computational physicist. I run
computational experiments everyday This is how typically I run my
experiments
- Slide 7
- Scientific Application Another Scientific Application First I
collect my observed data And then pass data to my applications
& get the result This is starting to become a very tiring
task
- Slide 8
- How can I make this much simpler? Logically, this is how my
life would be made easier Is it possible to automate this flow
sequence without my guidance?
- Slide 9
- Scientists from many different fields face this problem
everyday. The solution is to use a workflow-powered science gateway
to manage the experiment online. What is a workflow you ask? Well,
you just saw one in our previous animation
- Slide 10
- We introduce Apache Airavata, a system capable of composing,
managing, executing, and monitoring small to large scale
applications and workflows Want to see how it works? A Typical
Workflow
- Slide 11
- Apache Airavata I will handover my data & my experiment
details (the workflow) to the Airavata server The Gateway Airavata
will complete the experiment & return me the results Results
Progress of the experiment and while I wait for results, Airavata
will notify me with progress updates of my experiment
- Slide 12
- Lets look closely how Airavata manages workflows. The Gateway
Results Experiment progress Apache Airavata
- Slide 13
- Lets look closely how Airavata manages workflows. The Gateway
Results Experiment progress
- Slide 14
- Airavata main has 4 components The Gateway 1. Workflow
Interpreter Steer the workflow execution 2. The GFac Steer science
app executions & data transfers Workflow Interpreter GFac
Message Box Registry 3. The Registry Defines the available
applications & records all results of experiments 4. The
Message Box Records the progress of the workflow execution
- Slide 15
- A Stable API for Airavata Apache Airavata End Users Gateway
Developer Scientific Application Computational Resources
- Slide 16
- Application Registration UI Application Registration UI
Application Developer A1 Airavata Service Interface (wraps client
API) Airavata Service Interface (wraps client API) Service Map XML
Airavata Server Service Map to AWSDL Web Based workflow composer
Workflow Developer Get AWSDL Put XWF A2 A3 W1 W2 W3 Web Based
Experiment Builder Experiment Builder Get WIs E1 E2 E3 W4 Shred
Workflow Inputs Launch Workflow Web Based Workflow Monitor Watch
Progress M1 M2 Get Workflow Graph M3 Monitor Workflow
- Slide 17
- Goal of the project Design Web-Based interfaces for Airavata:
Application Registration Workflow Construction Workflow Execution
Workflow Monitoring Provide an opportunity for GSoC to understand
Distributed System in action Scope for Research and Software
Engineering papers
- Slide 18
- Data Model Application Description User describes inputs and
outputs of the application. Currently this information is captured
in Service Map Schema. This schema is stored in Airavata Registry
as XML. Also the schema utility generates a application service
WSDL from this schema using the Airavata WSDL Generator.
- Slide 19
- Application Registration UI Application Registration UI
Application Developer A1 Service Map XML Service Map to AWSDL Web
Based workflow composer Workflow Developer Get AWSDL A2 W1 W2 XML
Airavata Server API Launch & Manage Jobs Notify progress of job
or workflow execution Real-Time Monitoring Messaging Subsystem
Registry Execute & Manage Computations Workflow Interpreter
Application Factory (Gfac) Applicatio n Desc Workflow
- Slide 20
- A peek at one of the cluster Interconnect Nodes
- Slide 21
- Scheduling qsub batch jobs on the cluster worker node SGE
MASTER node Queue-AQueue-BQueue-C A Slot 1A Slot 2B Slot 1C Slot 1C
Slot 2C Slot 3B Slot 1B Slot 2B Slot 3 B Slot 1C Slot 1 C Slot 2A
Slot 1B Slot 1C Slot 1 Queues Policies Priorities Share/Tickets
Resources Users/Projects JOB YJOB ZJOB XJOB UJOB OJOB N
- Slide 22
- System characteristics System status Resources Job policies
Resources Resource Matching JOB User User policies Groups Roles
Departments Projects SelectionScheduling
- Slide 23
- Simplified Gateway Architecture One time Gateway Community
Setup Community Account Grid Certificate username, password Gateway
Interface Gateway Server Compute Servers Gateway Authentication
Fetch Community Credential Grid Proxy Job Submit or File Transfer
request Output Proxy, Job Request Job Status, Output Step 0 Step 1
Step 2,3,,
- Slide 24
- ParamChemBioVLabGridChemDES VLAB UltraScan Apache Airavata 1.0
NSG ParamChemBioVLab CIPRESPOPLAR GridChemDES UltraScan VLAB Apache
Airavata 2.0 Apache Airavata 1.0 Apache Airavata 1.0 Apache
Airavata 1.0 Apache Airavata 1.0 Apache Airavata 1.0
- Slide 25
- Slide 26