Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual...
-
Upload
randall-nicholson -
Category
Documents
-
view
215 -
download
1
Transcript of Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual...
![Page 1: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/1.jpg)
Some Grid Science
California Institute of Technology
Roy WilliamsPaul Messina
Grids and Virtual Observatory
Grids andand LIGO
![Page 2: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/2.jpg)
Virtual Sky: Image Federation
Xray (ROSAT) theme
Change scale
Change theme
http://virtualsky.org/fromCaltech CACRCaltech AstronomyMicrosoft Research
Optical (DPOSS)
Coma cluster
Virtual Sky has140,000,000 tiles
140 Gbyte
![Page 3: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/3.jpg)
VO and Grid ComputingSupercomputer center support for
Data Pipelines (not just MPI)
12 hr 250 Gb
HPSS
HP Superdome
VS1 Science productHPSS system
VS2 Web productWin2000 database
Raw data
CPU Utilization
VirtualSky.org Ingestion
Copy raw data and filter (background subtraction)
Resampling
Write VS1
Build VS2
Write VS2
Today ~ 3 TB
1 TB
140 GBComputation also takes about 12 hrs.
![Page 4: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/4.jpg)
VO and database JoinR
AD
ec
mag
Jm
ag
Hm
ag
K
glo
ng
lat
flu
x I
flu
x Q
flu
x U
Infraredcatalog
(indexed by RA)
Radiocatalog
(indexed by glon)
Crossmatchengine
SQL select
SQL select
Need same sorting for crossmatchWho does it?Or can we impose a standard?
order by glon?
order by RA?
sort here?
Sorting 10^9 objects is expensive
![Page 5: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/5.jpg)
VO Interoperability• Directory white/yellow pages
– VO Schema repository
• How to publish, how to define content• action> Document schema for
– archive content– web service capability– table/image/spectrum
• How to publish algorithms• Plug and play data services
– SOAP, UDDI, WSDL, Jini?
• Semantic web– “Question and answer in the language of the client” – Topic maps
![Page 6: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/6.jpg)
VO and Interoperability
Client application (e.g., OASIS)
asks for available catalogs
then for attributes
makes a query
then displays the result
What is the standard interactionwith a catalog service?
Client Service
Want to be dynamic, be able to
add catalogs and the applications still work
![Page 7: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/7.jpg)
VO and International
•Of course there are many astronomy data collections in the UK, Europe, Japan, etc.
•We intend to collaborate/coordinate with international efforts
![Page 8: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/8.jpg)
LaserInterferometricGravitational waveObservatory
Listening to Collisions ofBlack Holes andNeutron Stars
LIGO
![Page 9: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/9.jpg)
LIGO Pictures
![Page 10: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/10.jpg)
Grid LIGO Architecture
Clientseg Web, Script, Agent
Clientseg Web, Script, Agent
Text requestText request Request Manager
Request Manager
GriPhyN LDAS
Gatekeeper(GRAM)
Gatekeeper(GRAM)
Science AlgorithmsSoftware Collaboratory
Parallel ComputingGridFTPGridFTP
Local Disk
Data
HPSS
GridFTPGridFTP
Replica CatalogReplica ManagementTransformation CatalogVirtual Data Catalog
Virtual Data Request
Data Movement
Globus RPC
other LDASCondor jobs
Objective: Add security & wide-area data servicesLDAS is Ligo Data Analysis System
![Page 11: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/11.jpg)
Grid LIGO Logic• VD Request Request Manager
– XML, Key-value, script?– Action> Build document schema for
this• Do we have it already in data caches
– if yes, get from caches, use simple transformations
– else build LDAS script•Move relevant data to LDAS local
disk•Queue job to LDAS, get jobID•Pass updates, then result pointer
back to user•Perhaps cache result
![Page 12: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/12.jpg)
International GW Grid
50 kbyte/sec
GEO
![Page 13: Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f315503460f94c4ca60/html5/thumbnails/13.jpg)
International GW Grid• Coincidence in Ligo-Virgo data
– Not astrophysics data yet….– Try seismic, electromagnetic data for now
Caltech
VirgoElectromagneticSeismic
Coincidence events!
Currently using rsync
LLO
LHO
Merging 3 streams to 1
Correlate signals(LDAS)
~12 kbyte/sec each way