May 9, 2007Astro-RG, OGF201 Japanese Virtual Observatory and NaReGi Masatoshi Ohishi / NAOJ,...
18
May 9, 2007 Astro-RG, OGF20 1 Japanese Virtual Observat ory and NaReGi Masatoshi Ohishi / NAOJ, Sokendai & NII 大大大大 / 大大大大大 , 大大大大大大大大大 & 大大大大大大大大 [email protected]
Transcript of May 9, 2007Astro-RG, OGF201 Japanese Virtual Observatory and NaReGi Masatoshi Ohishi / NAOJ,...
May 9, 2007 Astro-RG, OGF20 1
Japanese Virtual Observatory and NaReGi
Masatoshi Ohishi / NAOJ, Sokendai & NII
大石雅寿 / 国立天文台 , 総合研究大学院大学 & 国立情報学研究所
May 9, 2007 Astro-RG, OGF20 2
Supported by
• JSPS, Core to Core Program
• MEXT, Kakenhi
• NII, CSI project
• NAOJ
May 9, 2007 Astro-RG, OGF20 3Accessible from anywhere at any time
Even for educational useAccelerate astronomical research, and sufficient time for research itself !! Virtual Observatory
Too hard to collect and analyze whole data.Need Cat’s help
Nobeyam Radio Obs.
SUBARU telescope
ALMA
Data Rate
~ 1 TB/yr
~ 20 TB/yr
~ 1 PB/yr
May 9, 2007 Astro-RG, OGF20 4
SkyNode1.0
SDSS QSO
… 2MASS
Portal
P. Reg.
S. Reg.
LDAP
SkyNode 0.7.4
SIAP/SSAP
解析解析
解析AnalysisServers
MDS
P. Reg.
User StorageWeb Service
HTTP GetFTP
LDAPOther VOs
User
Request
Invoke
Invoke
InvokeSer
vice
Dis
cove
ry
Auth.
Meta
data
Harvesti
ng
JVO
Subaru
Grid System
Registry
Data Service
Overview of the JVO Portal Service
May 9, 2007 Astro-RG, OGF20 5
MAG_B
Log(
F_I
R)
May 9, 2007 Astro-RG, OGF20 6
Spectral Data from Subaru
• queries
• display
• Baseline estimate
• Line detection
• VOSpecHigh Dispersion Spectrometer
May 9, 2007 Astro-RG, OGF20 7
Interconnected VO Data services in the World
Canada VO
More than 600 data resources are available
May 9, 2007 Astro-RG, OGF20 8
Improvement of Work Flow system
Items in FY2006• Improvement of WFDL
– Shorten the volume of DL– Array definition– Awk-like description (loop d
escription in a single line) – and so on
• Built-in functions– Available to register and us
e built-in functions
• Status and Log system
WF Editor
Execution monitor
May 9, 2007 Astro-RG, OGF20 9
SkyNode1.0
SDSS QSO
… 2MASS
Portal
P. Reg.
S. Reg.
LDAP
SkyNode 0.7.4
SIAP/SSAP
解析解析
解析AnalysisServers
MDS
P. Reg.
User StorageWeb Service
HTTP GetFTP
LDAPOther VOs
User
Request
Invoke
Invoke
InvokeSer
vice
Dis
cove
ry
Auth.
Meta
data
Harvesti
ng
JVO
Subaru
Grid System
Registry
Data Service
Overview of the JVO Portal Service
May 9, 2007 Astro-RG, OGF20 10
Subaru Grid Pipeline Arch.
Data Storage Service
Data Analysis Service
MDS Server Data Search Service
3. query a location of data
2. submit a job
1. search an available
server
4. get data
report status
query status
7. retrieve the result
5. result UR
L
6. Submit data transfer job
Client
May 9, 2007 Astro-RG, OGF20 11
Grid Service on VO• Subaru data reduction pipeline system
– Web service based GRID– Generates calibration data (Super-Flat) for all the Suprime
Cam data in a week– On-the-fly data reduction with the pre-prepared Super-Flat– Users can access to the data reduced with the most recent
algorithm– Also support the older version of pipeline
• Backend computing system of the JVO– catalog generation– photo-z calculation– Image arithmetic – …
May 9, 2007 Astro-RG, OGF20 12
– Reduction 10s (Bias+Flat+Distortion+ Astrometory)– Transfer 30s
Data Reduction:~24 hours ~1 hour
May 9, 2007 Astro-RG, OGF20 13
Study on Environment around QSOs
Present Past
Hig
h
C
on
centratio
n
Lo
w
May 9, 2007 Astro-RG, OGF20 14
Cyber Science Infrastructure toward Petascale Computing (planned 2006-2011)
Cyber-Science Infrastructure （ＣＳＩ）
（ IT Infra. for Academic Research and Education)
Operation/Maintenance
（ Middleware ）Networking Contents
NII Collaborative Operation Center
Delivyer
Delivery
Networking Infrastructure (Super-SINET/SINET3 ）
Univ./National Supercomputing
VODomain Specific
VOs
Feedback
Feedback
R&D Collaboration
Operaontional Collaborati
MiddlewareCA
NAREGI Site
Research
Dev.（ ）βver.V1.0V2.0
International Collaboration- EGEE- UNIGRIDS-Teragrid -GGF etc.
Feedback
Delivery
Project-Oriented
VO
Delivery
Domain SpecificVOs
CustomizationOperation/Maintenance
ナノ分野実証・評価
分子研
ナノ分野実証・評価
分子研
NanoProof of al.Concept
Eval.IMS
NanoProof, Eval.
IMSJoint Project
（ Bio ）Osaka-U
Joint Project
AIST
R&D Collaboration
IndustrialProjects
Project-orientedVO
Note: names of VO are tentative）
Peta-scaleSystem
VOCore Site
R&D Collaboration
Operation/Maintenance
（ UPKI,CA ）
IMS,AIST,KEK,NAO,etc.
May 9, 2007 Astro-RG, OGF20 15
Introduction of the NaReGi middleware into JVO
• NaReGi middleware version 1.0.2
• Trial introduction into the Subaru Grid pipeline– A part of middleware deployment program by the NII
• To evaluate how easy it is to use the middleware
• To find which components may be used for Virtual Observatories– SSO, Apps interface, and so on
May 9, 2007 Astro-RG, OGF20 16
NAREGI Grid environment
・ Registration of UserApplications・Generation of WF
・ Issue CAs・ Keep proxy certif.
・ Computation Node・ Job monitor at each site・ Job monitor at each node
・Monitor Job Execution・ Display Information
・ Find Computation Resources・ Job Scheduling
Data Flow
May 9, 2007 Astro-RG, OGF20 17
Requests to Improve
• Items to be improved toward version 1.0– Make it easier to install
– Make it much faster to install
– Need to support OSs other than RHEL
– and so on
• These requests were reflected to the modified road map of NaReGi.
May 9, 2007 Astro-RG, OGF20 18
Good Findings
• Good Graphical WF builder– Plan to import to the
JVO system
– Maybe referred to by the IVOA
![start [Open Grid Forum] - RNS Requirements from KEKRequirments...RNS Requirements from KEK based on operational experiences using LCG/SRB/NAREGI Go Iwai, KEK/CRC The 24th Open Grid](https://static.fdocuments.us/doc/165x107/605599d2807cdc19cf3b4d95/start-open-grid-forum-rns-requirements-from-kek-requirments-rns-requirements.jpg)
