Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.
-
Upload
curtis-clark -
Category
Documents
-
view
224 -
download
0
Transcript of Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.
![Page 1: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/1.jpg)
Birmingham/Warwick AstroGrid Science Workshop
Meeting Goals
Silvia Dalla
Oct 10-11, 2006
![Page 2: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/2.jpg)
Workshop goals
• Aim to work with you, the early users of AstroGrid, to demonstrate its capabilities and the potential of the Virtual Observatory (VO)
• Introduce you to the elements of the system and how to do science with it
• Look at a science problem in your research area and use VO tools to tackle it
![Page 3: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/3.jpg)
Workshop goals (contd)
• Collect your feedback
What you like/dislike
What needs to be improved
Which data you need
Which applications you want to see running in AG
![Page 4: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/4.jpg)
Workshop Plan
• Today: background talks on AstroGrid and how the system works (am). Initial workgroup set up (pm)
• Tomorrow: workgroups work on science cases. Report and feedback Session (pm)
• Don’t miss the dinner (Today – 7:00 pm The Chamon)
![Page 5: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/5.jpg)
What you can do with AstroGrid
AstroGrid Science Team
Silvia Dalla, Eduardo Gonzalez-Solares, Jonathan Tedds, Anita Richards and Nicholas Walton
![Page 6: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/6.jpg)
Outline
1. Background: Virtual Observatories (VOs) and AstroGrid
2. Example of non-VO solution to a data retrieval and processing task: making a movie of solar images
3. The AstroGrid solution
4. Introduction to AstroGrid components
![Page 7: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/7.jpg)
What is the VO?
www.encyclopedia.com: VO = collection of integrated astronomical data archives and software tools that utilize computer networks to create an environment in which research can be conducted
Thanks to Ohishi Masatoshi (NAOJ) for the pointer…
![Page 8: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/8.jpg)
VOs worldwide
• Similar efforts now in 15 countries:– UK, USA, Canada, France,
Germany, Italy, Holland, Japan, Australia, India, China, Russia, Hungary, South Korea, ESO, Spain
• Active collaboration among projects– Standards, common demos– International VO roadmap being
developed– Regular telecons over 10
timezones
Formal collaboration:International Virtual Observatory Alliance (IVOA)
![Page 9: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/9.jpg)
AstroGrid: the UK’s VO
• Consortium of several UK Universities, funded by PPARC to build a VO for the UK.
• Part of the Euro-VO www.euro-vo.org
• Astrogrid release 2006.3: www.astrogrid.org/launch
• Data discovery, access to data and applications.
![Page 10: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/10.jpg)
AstroGrid Goals
• Enable science by:
– Improving the quality, easy, speed of on-line astronomy
– Making integration and comparison of data from diverse sources transparent
– Removing data access barriers to multiwavelength analysis
– Enabling access and manipulation of large datasets
![Page 11: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/11.jpg)
The need for a VO
• Data volume doubles every year
• Increase of size and multiplex capabilities of new instruments– WFCAM: 100Gb/night (100Tb/yr)– VISTA: 300Gb/night (300Tb/yr)– e-MERLIN/ALMA: 1Tb/day– SDO: 1 Tb/day
• We will have Petabytes of data by 2010
![Page 12: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/12.jpg)
The need for a VO
• Hubble UDF
• Million second exposure
• 6000x6000 pix
• 11.5 sq. arcmin
• 10,000 galaxies
![Page 13: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/13.jpg)
The need for a VO
• SWIRE ELAIS N1
• 9 sq. degrees (~3000 UDF)
• (moon ~0.2 sq. deg.)
• ~ 600,000 objects
![Page 14: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/14.jpg)
The need for a VO
• Solar Dynamics Observer
• Launch 2008
• >1 TB/day
• Over 5 years: 1.8 PB
![Page 15: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/15.jpg)
Making a solar movie – non VO
Web interface to database of SOHO/EIT observations
![Page 16: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/16.jpg)
Making a solar movie – non VO
Download imagesto local machine
![Page 17: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/17.jpg)
Making a solar movie – non VO
IDL SolarSoft requiredto calibrate images andmake a movie. Routinesavailable, however eachuser rewrites code calling them to produce a movie.
![Page 18: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/18.jpg)
Making a solar movie – non VO: weaknesses
• If user wishes to make a movie for a different time period, the above steps need to be repeated – by hand
• Similarly for astronomy data if one is interested in many objects
• The entire archive is not ‘visible’ to the user – only the downloaded subset can be processed
• Scripting eg with Python, Perl etc is possible though code is different for different archives – not easy for the ‘general’ user / for many datasets. This limits feasibility of multi-wavelength, multi-instrument work
![Page 19: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/19.jpg)
Making a solar movie – with AG
Solar Movie Maker science workflow. Other science workflows available:Redshift Maker, Colour Cutter, Cone Search, SWIRE images
![Page 20: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/20.jpg)
Making a solar movie – with AG: under the hood
• Access to database of observations – via AG DSA (DataSet Access) software
• Requests are sent using ADQL (Astronomy Data Query Language), similar to SQL
Database of observations(at archive)
DSAAG Workbench/workflow engine
Input: ADQL query
Output: table of observationssatisfying the query, in VOTable format
![Page 21: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/21.jpg)
Making a solar movie – with AG: under the hood: CEA
CEAAG workbench/workflow engine
Input: CEA applicationinput parameters
Output: whatever the outputof the application is, delivered to user’s Myspace
• Capability to call a set of processing routines that do operations on data, or a model (apps in any programming language can be wrapped) – CEA Application
Application. Eg software that
processes data, model etc
(on apps server)
![Page 22: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/22.jpg)
Making a solar movie – with AG: under the hood: Workflow
• Workflow capability so that queries to datasets and calls to applications can be managed
• Jobs are run remotely and asynchronously
• Queries and workflows can be re-used and shared
• AstroGrid is currently the only VO project with a workflow capability
![Page 23: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/23.jpg)
AstroGrid: Workbench
• Workbench gives access to AG services
• Data discovery: AstroScope and HelioScope
• Task Launcher – send a query to a database or launch a application
![Page 24: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/24.jpg)
AstroGrid: Workbench- contd
• Run a ready made science workflow
• Build your own workflow
• View your files in MySpace
![Page 25: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/25.jpg)
AstroGrid: MySpace
• Virtual disk space where you can store results, temporary files, and new things like query files and workflow files, so you can adjust and re-run jobs on a later day.
• Visible from any computer.
![Page 26: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/26.jpg)
Finding Information : the Registry
• How do you find the data you require?
• How do you decide which resource (data, application, information, disk, …) to use?
• The registries are the yellow pages for astronomical resources
• All VO registries harvesting each other: thus querying any one returns full list of globally held resources.
![Page 27: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/27.jpg)
VOTable: an interchange format
• XML standard for the interchange of data represented as a set of tables
• A table is an unordered set of rows, as specified in the table metadata
• Each row is a sequence of table cells, each of them containing a primitive data type or an array of such primitives
• It bridges two ways to express structured data: XML and FITS
• Use of Universal Column Descriptors (UCDs) to express the content of each parameter.
![Page 28: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/28.jpg)
<?xml version="1.0"?><VOTABLE version="1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://www.ivoa.net/xml/VOTable/VOTable/v1.1"> <COOSYS ID="J2000" equinox="J2000." epoch="J2000." system="eq_FK5"/> <RESOURCE name="myFavouriteGalaxies"> <TABLE name="results"> <DESCRIPTION>Velocities and Distance estimations</DESCRIPTION> <PARAM name="Telescope" datatype="float" ucd="phys.size;instr.tel" unit="m" value="3.6"/> <FIELD name="RA" ID="col1" ucd="pos.eq.ra;meta.main" ref="J2000" datatype="float" width="6" precision="2" unit="deg"/> <FIELD name="Dec" ID="col2" "pos.eq.dec;meta.main" ref="J2000" datatype="float" width="6" precision="2" unit="deg"/> <FIELD name="Name" ID="col3" ucd="meta.id;meta.main" datatype="char" arraysize="8*"/> <FIELD name="RVel" ID="col4" ucd="src.veloc.hc" datatype="int" width="5" unit="km/s"/> <FIELD name="e_RVel" ID="col5" ucd="stat.error;src.veloc.hc" datatype="int" width="3" unit="km/s"/> <FIELD name="R" ID="col6" ucd="phys.distance" datatype="float" width="4" precision="1" unit="Mpc"> <DESCRIPTION>Distance of Galaxy, assuming H=75km/s/Mpc</DESCRIPTION> </FIELD> <DATA> <TABLEDATA> <TR> <TD>010.68</TD><TD>+41.27</TD><TD>N 224</TD><TD>-297</TD><TD>5</TD><TD>0.7</TD> </TR> <TR> <TD>287.43</TD><TD>-63.85</TD><TD>N 6744</TD><TD>839</TD><TD>6</TD><TD>10.4</TD> </TR> <TR> <TD>023.48</TD><TD>+30.66</TD><TD>N 598</TD><TD>-182</TD><TD>3</TD><TD>0.7</TD> </TR> </TABLEDATA> </DATA> </TABLE> </RESOURCE></VOTABLE>
Table Metadata
Table data(XML, FITS)
VOTableExample
![Page 29: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/29.jpg)
![Page 30: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/30.jpg)
Visualising results: Topcat
![Page 31: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/31.jpg)
Visualising results: Topcat
![Page 32: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/32.jpg)
Astrogrid Help Pages
Linked from workshop pages
![Page 33: Birmingham/Warwick AstroGrid Science Workshop Meeting Goals Silvia Dalla Oct 10-11, 2006.](https://reader035.fdocuments.us/reader035/viewer/2022062301/56649e2b5503460f94b19c08/html5/thumbnails/33.jpg)
Summary
• Workshop is an opportunity to see AG in action
• Current capability can be of use to a diverse range of science problems
• New cases – perhaps emerging from this workshop – may demand new science services: input is welcome from you!