Post on 23-May-2022
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 1
Chajnantor (5000m)
Paranal (2600 m)
La Silla (2400 m)
Paul Eglitis and Dieter Suchar
Historical lessons, inter-disciplinary comparison, and their application to the future evolution of the ESO Archive Facility and Archive Services
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 1
ESO, Karl-Schwarzschild Str. 2, 85748 Garching near Munich, Germany
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 2
Introduction
Inter-disciplinary thoughts
The ESO Science Archive Facility
Physical experiences: storage and infrastructure
Future growth requirements
Lessons Learnt
Future evolution
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 3
Archive Requirements: Inter-disciplinary Variation and Similarity
Space Mission
Science
Stable (Mission)
On Opportunity
IPDA, SPASEEUROPLANET
Earth Observation
Service, Science
Stable (Long Term)
Periodic
GEOSS, INSPIRE, GIGAS ,HMA
Ground‐Based Astronomy
Science
Variable
Variable
FITS, VO, IVOA
Particle Physics
Science
Variable
In‐house++
E‐GRID
Customer
Configuration
Data Timeliness
Data Volume
Interoperability
Always more!
…
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 4
CHILE
USER INTERFACES
SECONDARYARCHIVE
LocalDatabases
OBSERVATORIES
PRIMARYARCHIVE
ReplicatedDatabase
Infrastructure
HQ
ReplicatedDatabase
Infrastructure
DATAHANDLING
Rep
licat
ion
STAGING
TravelingDisk
On-LineData Transfer
ESO SCIENCE ARCHIVE
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 5
DISK
0.250.52
Hardware Changes
Inexpensive Upgrades
CD DVD* *
****
GB
Year
Rac
ks
Physical Archive History
cited.be
Paranal
La Silla Chajnanator
Future
VLT Very Large Telescope
CRIRES CRyogenic high‐resolution InfraRed Echelle Spectrograph
FLAMES Fibre Large Array Multi Element Spectrograph
FORS1 FOcal Reducer/low dispersion Spectrograph 1
FORS2 FOcal Reducer/low dispersion Spectrograph 2
HAWK‐I High Acuity Wide field K‐band Imager
ISAAC Infrared Spectrometer And Array Camera
NACO Nasmyth Adaptive Optics System (NAOS)Near‐Infrared Imager and Spectrograph (CONICA)
SINFONI Spectrograph for INtegral Field Observations in the Near Infrared
UVES Ultraviolet and Visual Echelle Spectrograph
VIMOS VIsible MultiObject Spectrograph
VISIR VLT Imager and Spectrometer for mid Infrared
X‐SHOOTER a multi wavelength (300‐2500nm) medium resolution spectrograph
VLTI Very Large Telescope Interferometer
VLTI MIDI MID‐infrared Interferometric instrument
VLTI AMBER Astronomical Multi‐BEam combineR
VISTA Visible and Infrared Survey Telescope for Astronomy
MASCOT Mini All‐Sky Cloud Observation Tool
NTT ESO 3.6m MPG/ESO 2.2mWFI Wide Field Imager
FEROS The Fiber‐fed Extended Range Optical Spectrograph
EFOSC2 ESO Faint Object Spectrograph and Camera
HARPS High Accuracy Radial velocity Planet Searcher
SOFI Son of ISAAC
ALMA The Atacama Large Millimeter/submillimeter ArrayAPEX Atacama Pathfinder Experiment telescope
VST VLT Survey Telescope
E‐ELT European Extremely Large Telescope
“Multi-mission” Concept in Archival
Images Spectra Calibration and Quality Information
Data products managed in the ESO Data Centre and Archive Facilities
Leading observatories and state-of-the-art instrumentation
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 7
ESO Archive System 2009
Design and construction of the ESO Data Centre
Requirements:• Construction at different physical location• On an area of 7 offices in an existing building• Modular approach: Dismantle, move and rebuildthe Data Centre at ESO premises.
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 9
Archive Holdings Past and Projected
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 10
Paranal and the ESOVery Large Telescope
La Silla
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 11
VISTA-survey-telescope
Credit: Steven Beard/UKATC/ESO
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 12
Credit: ALMA
ALMA
~ 1 rack/yr
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 13
E-ELT 17 PB/yr ?
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 14
Lessons Learnt
Preconditioning: physical constraints of data flow a.o.
domain practices bias archive solutions.
Physical installation: in-house, out-source (all or part)?
Industry support: risks, costs, maintenance continuity.
Engineering changes: return on investment, cost of change,
operational costs.
Art-of-balance: especially when a facility serves multiple
purposes, e.g. Operations, Preservation, Exploitation.
Interoperability: many parallel efforts are ongoing.
Long-term view and inter-disciplinary initiatives like the PV-Series critical.
EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 15
ESO Archive Future Evolution“History of effective response to rapid changes has led to system heterogeneity”
Homogenisation of metadata and interfaces
Development of new HMIs
Standards compliance
Converge on next storage technology:
Analyse current usage of disk-based system in detail
Assessment of growth and access requirements
End-User, Operations, Maintenance and Development
requirements
The ESO Science Archive is a complex part of the complete operational Data Flow System at ESO.
Thank you
Acknowledgement: observatory images courtesy of the ESO Education and Public Outreach Department. See http://www.eso.org/gallery