Discussed in KyotoDiscussed in Kyoto

Schema changes for the next version (Gerard Schema changes for the next version (Gerard Lemson) Lemson) will be included in VOTable1.2 will be included in VOTable1.2

ucd1 and ucd1+ ucd1 and ucd1+ just a replacement just a replacement datatypes limited to FITS set -- would datatypes limited to FITS set -- would

extensions be needed extensions be needed (e.g. unsigned (e.g. unsigned numbers, date+time, string, ...) numbers, date+time, string, ...) no no unanimityunanimity

<COOSYS> vs <stc:AstroCoordSystem> <COOSYS> vs <stc:AstroCoordSystem> no real conclusion, will be discussed no real conclusion, will be discussed

VOTable: back to basicsVOTable: back to basics

VOTable VOTable is not a data model !is not a data model ! A way to serialize data A way to serialize data collectionscollections ... ... ...while keeping full metadata contents...while keeping full metadata contents

VOTable schema remains simple and VOTable schema remains simple and flexible...flexible... to cope with existing data sets to cope with existing data sets (FITS)(FITS) to be adaptable to ever changing data to be adaptable to ever changing data

models.models.

VOTable is used in VOVOTable is used in VO

to express various collectionsto express various collections collections of astronomical sources collections of astronomical sources

(catalogues)(catalogues) collections of imagescollections of images large time serieslarge time series instrumentation, models, ...instrumentation, models, ...

and needs stability for applications to run!and needs stability for applications to run! how can large data collections remain valid if how can large data collections remain valid if

data model is changing ?data model is changing ?

Question of Astrometry Question of Astrometry DefinitionDefinition

Current VOTable way (from V1.0)Current VOTable way (from V1.0)<<COOSYSCOOSYS ID=" ID="J2000J2000" system="" system="eq_FK5eq_FK5" "

equinox="equinox="J2000J2000" " epoch="epoch="J1991.25J1991.25"/>"/> Does not apply to e.g. solar studies, moving Does not apply to e.g. solar studies, moving

frames -- VOTable has to be more genericframes -- VOTable has to be more generic VO components have to be shared --- if STC VO components have to be shared --- if STC

is the data model of space+time is the data model of space+time coordinates, VOTable as well as other VO coordinates, VOTable as well as other VO components has to refer to it.components has to refer to it.

Scenarios for Astrometry in Scenarios for Astrometry in VOTablesVOTables

2 basic ways:2 basic ways: include the STC schema and follow include the STC schema and follow

its directives in a changing worldits directives in a changing world make a reference to STC components make a reference to STC components

via the via the utypeutype reference reference

1. Addition of STC schema1. Addition of STC schema

Would be closer to a Would be closer to a COOSYSCOOSYS replacement, replacement, BUT:BUT:

AstroCoord AstroCoord requires all requires all time+velocity+space definitions: would be time+velocity+space definitions: would be so much better to have these disantangled so much better to have these disantangled (e.g. time series)(e.g. time series)

AstroCoordAstroCoord defines labels & units as e.g. defines labels & units as e.g.      <crd:AstroCoords coord_system_id="B1950-OPTICAL-ET">            <crd:Position2D unit="deg">                  <crd:Name>RA,Dec</crd:Name>                  <crd:Value2Ref>Column3</crd:Value2Ref>                  <crd:Error2Ref>Column4</crd:Error2Ref>                  <crd:Size2Ref>Column5</crd:Size2Ref>            </crd:Position2D>

Addition of STC schemaAddition of STC schema(continued)(continued)

AstroCoord AstroCoord requires coordinate requires coordinate components stored as an array -- components stored as an array -- inapplicable if e.g. single component inapplicable if e.g. single component is present (galactic latitude)is present (galactic latitude)

Other VO modelsOther VO models: should VOTable : should VOTable include all VO schemas (e.g. solar include all VO schemas (e.g. solar position, characterisation ...)?position, characterisation ...)?

2. STC referencing 2. STC referencing

utypeutype referencing: referencing: was originally created for DAL needswas originally created for DAL needs a consensus on an XPath expression:a consensus on an XPath expression:

model:element/element...model:element/element...[attribute].../[attribute].../

e.g.e.g.

votvot::VOTable/RESOURCE/TABLE[1]/FIELD[@name="RAVOTable/RESOURCE/TABLE[1]/FIELD[@name="RA"]"]

Examples of possible STC Examples of possible STC referencingreferencing

Referencing via a Referencing via a GROUPGROUP<<GROUPGROUP

utype="crd:AstroCoords[@coord_system_id='J200utype="crd:AstroCoords[@coord_system_id='J2000-OPTICAL-ET']/Position2D/Value2Ref">0-OPTICAL-ET']/Position2D/Value2Ref">

<FIELDref ref="RAJ2000"/><FIELDref ref="RAJ2000"/>

<FIELDref ref="DecJ2000"/><FIELDref ref="DecJ2000"/>

</</GROUPGROUP>>

<FIELD ID="RAJ2000" unit="deg" ... /><FIELD ID="RAJ2000" unit="deg" ... />

<FIELD ID="DecJ2000" unit="deg" ... /><FIELD ID="DecJ2000" unit="deg" ... />

STC ReferencingSTC Referencing

Referencing individual Referencing individual componentscomponents<<FIELD FIELD name="RAJ2000" datatype="double"name="RAJ2000" datatype="double"utype="crd:AstroCoords[@coord_system_id='J2000-OPTICAL-ET']/ utype="crd:AstroCoords[@coord_system_id='J2000-OPTICAL-ET']/ Position2D/Value2Position2D/Value2(1)(1)">"></</FIELDFIELD>>

<<FIELD FIELD name="DEJ2000" datatype="double"name="DEJ2000" datatype="double"utype="crd:AstroCoords[@coord_system_id='J2000-OPTICAL-ET']/ utype="crd:AstroCoords[@coord_system_id='J2000-OPTICAL-ET']/ Position2D/Value2Position2D/Value2(2)(2)">"> </</FIELDFIELD>>

Referencing problemReferencing problem

How to refer to a specific component How to refer to a specific component of an array in XPath ? of an array in XPath ?

Again, disentangling the 3 position / Again, disentangling the 3 position / time / velocity components would time / velocity components would help...help...

Would be better to adopt identical Would be better to adopt identical conventions in all VO componentsconventions in all VO components

ProposalProposal

adopt the adopt the utypeutype way with way with utype="crd:AstroCoords[@coord_system_id='J2000-utype="crd:AstroCoords[@coord_system_id='J2000-OPTICAL-OPTICAL-

ETET']/Position2D/Value2']/Position2D/Value2(1)(1)">"> individual <individual <FIELDFIELD> or <> or <GROUPGROUP> ?> ? is there a chance the components angular is there a chance the components angular

position / time / velocity are disentangled ?position / time / velocity are disentangled ? as soon as STC is adoptedas soon as STC is adopted crd:crd: or or stc:stc: ? ?

Share conventions with other VO Share conventions with other VO componentscomponents

VOTable AppendicesVOTable Appendiceswhat to do ?what to do ?

A.1 VOTable LINK substitutions LINK question A.2 VOTable Query Extension Registry ? A.3 Arrays of variable-length strings String ? A.4 FIELDs as data pointers LINK question A.5 Encoding individual table cells A.6 Very large arrays A.7 Additional TABLE attributes (OK in

VOTable1.1)

A.8 Other serialization(s)

A1. A1. LINKLINK Questions Questions

LINKLINK is required for several applications is required for several applications SIAP: one of the columns gives a link to actual SIAP: one of the columns gives a link to actual

image server, coded in a image server, coded in a <TD><!<TD><![CDATA[.......]]></TD>[CDATA[.......]]></TD>

Aladin: required to provide a link to the full Aladin: required to provide a link to the full data, coded in a data, coded in a LINKLINK with substitutions with substitutions

A. Wicenec proposes a A. Wicenec proposes a <FIELD <FIELD type="location"> type="location"> associated toassociated to <LINK <LINK href="cid:...">href="cid:...">

It is important !It is important !

<TABLE name="results"><DESCRIPTION>Velocities and Distance estimations</DESCRIPTION><FIELD name="RA" ID="col1" ucd="pos.eq.ra;meta.main" ref="J2000" datatype="float" width="6" precision="2" unit="deg"/><FIELD name="Dec" ID="col2" ucd="pos.eq.dec;meta.main" ref="J2000" datatype="float" width="6" precision="2" unit="deg"/><FIELD name="Name" ID="col3" ucd="meta.id" datatype="char" arraysize="8*"/><FIELD name="RVel" ID="col4" ucd="src.veloc;pos.heliocentric" datatype="int"width="5" unit="km/s"/><FIELD name="e_RVel" ID="col5" ucd="src.veloc;pos.heliocentric;stat.error" datatype="int" width="3" unit="km/s"/><FIELD name="R" ID="col6" ucd="pos.distance" datatype="float" width="4" precision="1" unit="Mpc"><DESCRIPTION>Distance of Galaxy, assuming H=75km/s/Mpc </DESCRIPTION></FIELD></TABLE>

Example

LINKLINK substitution substitution(Aladin way)(Aladin way)

<LINK href="http://ivoa.net/lookup?Galaxy=${Name}&amp;RA=${RA}&amp;DE=${DE}"/>

a substitution filter is applied in the context of a particular row. For the first row of the table, the substitution would result in the URL

http://ivoa.net/lookup?Galaxy=N++224&RA=010.68&DE=%2b41.27

LINKLINK definition definition(INES way)(INES way)

<FIELD name="Spectrum" datatype="char" arraysize="*">

<DESCRIPTION> Link to the spectrum file</DESCRIPTION>....<TD><STREAM href="http://ivoa.net/getSpectrum?

Name="N++224"/></TD>.

A2 – Query ExtensionA2 – Query Extension

It is another view on the It is another view on the LINKLINK: present : present the different parameters available for a the different parameters available for a query.query.

Role of the registry ?Role of the registry ? these details stored in a "fine-grain" these details stored in a "fine-grain"

registryregistry in the context of a "coarse-grain" registry, in the context of a "coarse-grain" registry,

some facility of getting these details must some facility of getting these details must exist...exist...

A3 – Arrays of variable-length A3 – Arrays of variable-length stringsstrings

Is related to the Is related to the StringString datatype datatype which does not exist in VOTable (FITS which does not exist in VOTable (FITS syndrome)syndrome)

FITS knows only a fixed-size array of FITS knows only a fixed-size array of variable-length strings (via the variable-length strings (via the ""Variable length arrayVariable length array" convention)" convention)

A4 - A4 - FIELDs as data pointers

when a column contains a when a column contains a reference reference or aor a pointer pointer to the data: UR[LI], cid to the data: UR[LI], cid (Content-ID), ...(Content-ID), ...

http://almasw.hq.eso.org/almasw/bin/view/Archive/Vhttp://almasw.hq.eso.org/almasw/bin/view/Archive/VOTableDataTransferOTableDataTransfer

AddAdd type="location" type="location" attributeattribute to to <FIELD> <FIELD>

AddAdd type="location" type="location" attributeattribute to <LINK> to <LINK>

A5 - A5 - Encoding individual table cells

To decide to insert To decide to insert binary databinary data at at the cell level – typically an imagethe cell level – typically an image – – without the need for a break in the without the need for a break in the

streamstream without having to encode the without having to encode the

whole tablewhole table

<TABLE name="SpectroLog"><FIELD name="Target" ucd="meta.id" datatype="char" arraysize="30*"/><FIELD name="Instr" ucd="instr.setup" datatype="char" arraysize="5*"/><FIELD name="Dur" ucd="time.expo" datatype="int" width="5" unit="s"/><FIELD name="Spectrum" ucd="phot.fluxDensity" datatype="float" arraysize="*"unit="mW/m2/nm" precision="E3"/><DATA><TABLEDATA><TR><TD>NGC6543</TD><TD>SWS06</TD><TD>2028</TD><TD encoding="base64">QJKPXECHvndAgMScQHul40CSLQ5ArocrQLxiTkC3XClAq0OWQKQIMUCblYFAh753QGij10BTEm9ARKwIQExqf0BqbphAieuFQJS0OUCJWBBAhcrBQJMzM0CmRaJAuRaHQLWZmkCyhytAunbJQLN87kC26XlA1KwIQOu+d0DsWh1A5an8QN0m6UDOVgRAxO2RQM9Lx0Din75A3o9cQMPfO0C/dLxAvUeuQKN87kCXQ5ZAjFodQH0vG0B/jVBAgaHLQI7Ag0CiyLRAqBBiQLaXjUDYcrBA8p++QPcKPUDg7ZFAwcKPQLafvkDDlYFA1T99QM2BBkCs3S9AjLxqQISDEkCO6XlAmlYEQKibpkC5wo9AvKPXQLGBBkCs9cNAuGp/QL0euEC4crBAuR64QL6PXEDOTdNA2987QN9T+EDoMSdA8mZmQOZumEDDZFpAmmZmQGlYEEBa4UhAivGqQLel40Dgan9A4WBCQLNcKUCIKPZAk1P4QNWRaEEPkWhBKaHLQTkOVkFEan9BUWBCQVyfvg==</TD></TR></TABLEDATA></DATA></TABLE>

A6 - A6 - Very large arrays

The BINARY serialization of variable-length arrays uses a 4-byte prefix for the number of items of the array. This convention imposes an absolute maximal number of 2^31

Possibility of an Possibility of an arrayprefix arrayprefix which could which could specify the size of the number of itemsspecify the size of the number of items

arrayprefix="unsignedChar" arrayprefix="unsignedChar" would imply would imply max. 255 elements in the arraymax. 255 elements in the array

arrayprefix="long" arrayprefix="long" would accept > 2billion would accept > 2billion itemsitems

A7 - Additional TABLE attributes

ucd and utype attributes ?

to drop from document (were adopted)

Data structure widely interpretable<TABLE name="Messier"><FIELD name="Number" ID="M" ucd="meta.id" datatype="int" ><DESCRIPTION>Messier Number</DESCRIPTION></FIELD><FIELD name="R.A.2000" ID="RA" ucd="pos.eq.ra" ref="J2000" unit="deg" datatype="float" width="5" precision="1" /><FIELD name="Dec.2000" ID="DE" ucd="pos.eq.dec" ref="J2000" unit="deg" datatype="float" width="5" precision="1" /><FIELD ID="T" name="Classification" datatype="char" arraysize="10*" ucd="src.class"><DESCRIPTION>Classification (galaxy, glubular cluster, etc)</DESCRIPTION></FIELD><DATA><CSV headlines="1"><![CDATA[M RA DE Classification3 205.5 +28.4 Globular Cluster31 010.7 +41.3 Galaxy]]></CSV></DATA></TABLE>

A8 - A8 - A new CSV serialization ?

Proposed RoadmapProposed Roadmap

Generalisation of the Generalisation of the utypeutype mode to refer to mode to refer to components of data modelscomponents of data models

Agreement of Agreement of utypeutype syntax with other WG syntax with other WG Precision of exact STC reference in Precision of exact STC reference in

agreement with DM and other WGsagreement with DM and other WGs deprecate COOSYS ?deprecate COOSYS ?

Rewrite the VOTable schema Rewrite the VOTable schema VOTable1.2 VOTable1.2 Appendices ? ?Appendices ? ?

And after?And after?

after discussion of the changes after discussion of the changes discussed here, discussion of a discussed here, discussion of a modified schema ...modified schema ... close the VOTable group ?close the VOTable group ? envisage other changes (e.g. Quantity-envisage other changes (e.g. Quantity-

based serialisation ?)based serialisation ?) ... ...