SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
OAS Contribution
• The ACDS
• The new XCATDB based on Saada
Laurent MICHELChristian MOTCHFrançois Xavier PINEAU
Observatoire Astronomique de Strasbourg (Fr)Saada: http://astro.u-strasbg.fr/websaadaXCATDB: http://amwdb.u-starsbg.fr/jacds
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
The ACDS
• Delivered Products
– FITS catalogue extractions• X-ray source counterparts• Archival sources in the XMM field of view
– Graphical products (PDF)• Finding charts • Rosat Images• Catalogue plots
– HTML product• Browsing pages for ACDS data with many links to CDS resources
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
The ACDS
Vizier at CFA(Harvard USA)
Vizier at Cambridge(UK)
Vizier at CADC(Victoria Canada)
ACDS:• 2 Sparcs 400 Mhz• 1Gb RAM• 8 Agents in //
Vizier at CDS(Strasbourg France)
Simbad(Strasbourg France)
NED(Caltech USA)
Aladin(Strasbourg France)
Pipeline(Leicester)
• Correlation computing• A lot of remote server accesses• A few computation
• File product generation• Some remote server accesses (Aladin)• A lot of computation
• Correlation computing• A lot of remote server accesses• A few computation
• File product generation• Some remote server accesses (Aladin)• A lot of computation
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
The ACDS
20%
72%
8%
Cross CorrelationGraphical Products Other Tasks
• Actual Throughput (8 process in //)– 3300 sec /dataset
– (8*3600*24)/3300 = 209 datasets/day
• Most part of the time is pure CPU (graphical prd)– IDL, Distiller….
– Easy to improve
• new hardware
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
The ACDS
• Operation issues
– Support the pipeline throughput (but not more)• > 2200 datasets processed since October 13th
– 157 pending this morning due to a 2 days Aladin shutdown• No need to improve the hardware
– CDS database upgrades• Vizier is being moved on PSQL without serious consequences for us• Major Simbad to be release late 2006
– New error definition (no effect for standard users)
– External failures• Remote server shutdown• Corrupted products returned (cache miss-synchronization after crash)
– Internal failures• Empty Rosat images/wrong PPS index• Hardware failure
– Spare available/never used• Miss-configuration• Holidays
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
The ACDS
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
The ACDS
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
The XCATDB
• The XCATDB is not a data repository but a tool to make science
• N to M persistent relationship– Difficult to implement in an efficient way for complex queries– Possibility of doing more than with dynamic cross-matches
• Data-mining feature
X Source 1 Arch Source 1
Arch Source 2
Arch Source 3
Arch Source 4
Arch Source 5
X Source 2
X Source 3
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
Saada in a few Words
• Saada: An Astronomical Database generator– Making automatic the process
of building a database
– Hosting heterogeneous datasets
– Highlighting scientific content
– Publishing personal data into the VO
• Databases are installed on local machines– Any Linux/Windows/Mac box
– Tomcat 5.xx
– PostrgreSQL 8.xx
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
The XCATDB
• Based on Saada– Queries expressed in SaadaQL– Data loader
• How to build quickly millions of correlation links• How to restore archival source uniqueness
– GUI• Layout inspired from the 1XMM XCatDB• Architecture based on native Saada GUI
• Prototype based on RC2 data open for the community– http://amwdb.u-strasbg.fr/jacds
Source X
Source X
Source X
Arch Source 1
Arch Source 2
Arch Source 1
Arch Source 1
Arch Source 2
Source X
Source X
Source X
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
2nd Release of the XMM Catalogue
150,000 X-ray sourcesIn 2800 observations
1,400,000 archival sources
1,000,000 links
Implemented with Saada relationshipsSpecific Module
Native Saada Data-loader
Native Saada Data-loader
Demo
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
A simplified example: Looking for High Mass X-ray Binaries
Hardness ratio range for X-ray binaries
Hardness ratio limit for stars
HR3 = (F2-4.5keV – F1-2keV) / (F2-4.5keV + F1-2keV)HR4 = (F4.5-12keV– F2-4.5keV) / (F4.5-12keV + F2-4.5keV)
HR3 = (F2-4.5keV – F1-2keV) / (F2-4.5keV + F1-2keV)HR4 = (F4.5-12keV– F2-4.5keV) / (F4.5-12keV + F2-4.5keV)
Demo
< 1
.5ke
V
kT =
5 –
50k
eV
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
HR Condition
With bright counterparts
Without WR counterparts
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
Using UCDs in queries
• Selecting X-Rays sources by correlation patterns on 200 classes of X-Ray sources requires to handle a lot of meta-data
select oidsaada from Arch_9037AEntry where ( _Flux > 1e-13 )select oidsaada from Arch_7181AEntry where ( _Flux2 > 1e-16 ) select oidsaada from Arch_9032AEntry where ( _FX > 1e-16 ) select oidsaada from Arch_9015AEntry where ( _fX > 1e-16 ) select oidsaada from Arch_9031AEntry where ( _Fx > 1e-13 )………….
[phot.flux;em.X-ray] > 1e-16 [W/m2]
• Solution: Expressing queries using UCDs and Units
Demo
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
(att1>x1 and att1<x2) or (att2>x1 and att2<x2) or (att1>x1 and att2<x2) or (att2>x1 and att1<x2)
UCD Issue
[UCD]>x1 and [UCD]<x2
Usually makes no sense
Solution: Using unambiguous operators: [UCD]In(x1,x2)
(att1>x1 or att2>x1) and (att1<x2 or att2<x2)
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
UCD Issue (continued)
• Query language issue: using a simple syntax– One operator for one UCD
• Query execution– Editable execution plan
• UCD attribution: a sensible issue– Using multi-words UCDs as often as possible– Limit the columns accessible per UCDs– Take care with units
• Existence• Name: known by the converter
• Result presentation: another sensible issue– The client must understand how resources have been selected and accessed
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
XCATDB: Status and Prospects
• Data loader– Correlation setup works fine– Other products can be loaded with the native data-loader
• GUI– HTML form improvements– Display of constrained values
• Interoperability– External Xmatch in SaadaQL (F.X. Pineau)– Connection with ESAC resources (XSA, VOSpec)– Connections with CDS clients (VO portals)– VO protocols
SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds
Thank You for your Attention
Top Related