THEMIS launch meeting Cape Canaveral Feb. 13-14, 2007
description
Transcript of THEMIS launch meeting Cape Canaveral Feb. 13-14, 2007
THEMIS launch meeting Cape Canaveral Feb. 13-14, 2007
C. Jacquey, V. Génot, E. Budnik, R. Hitier, M. BouchemitCDPP/CESR, Toulouse, France
AMDAAutomated Mutiple Dataset Analysis: application
to data simultaneously obtained onboard THEMIS, CLUSTER and other missions.
[email protected] http://cdpp.cesr.fr/
AMDA (Automated Multi-Dataset Analysis) is a service developed at CDPP. AMDA is a suite of integrated tools allowing to perform massive scientific processing of the content of multi-datasets in automated or semi-automated ways. This service is built for working with a local database or (via web-services) distant data sources as well. The functionalities offered in AMDA include (i) user edited visualization browse, (ii) semi-automated event search (visual identification), (iii) automated event search (based on user edited mathematical criteria applied to the content of the data), (iv) data extraction, (v) user edited parameter computation and (vi) access to data stored in distant sites.In this paper, we present AMDA, its functionalities and results obtained in studies based on the use of AMDA.
Abstract
The CDPP (Centre de Données de Physique des Plasmas) is the french data center for solar system plasma data. The CDPP has been created both CNRS and CNES in 1998 and is hosted at CESR (Centre d’Etudes Spatiales des Rayonnements), Toulouse, France. Primary objectives of CDPP is to (i) assure long term preservation of data obtained with french contribution, (ii) to provide technical and scientific services and (iii) to participate to the development of interoperability and virtual observatories.
THEMIS at CESR and CDPP
CL
CDPP base of standardised data
AMDA
CDPP
CESR
Distant data and services:CDAWeb, CAA, GAIA,
SSCweb, Darts, …
CL: server for interactive detailed visualisation and analysis, high resolution data, on flight calibration, high level products of data, user defined parameter or visualisation, …(see poster of E. Penou and J.A. Sauvaud, this session)
AMDA: server for automated, semi-automated or simply manual data analysis. AMDA works on a k-parameter database and provides facilities for visualisation, data extraction, semi-automated/automated event or signature search/ characterisation. AMDA produces event list and catalogs.
AMDA is developed for being interoperant with other data centers or services. For now, AMDA implemented the web-services offered by the CDAWeb. This allows AMDA to acceess to the CDAWeb database in a way transparent for the user.
THEMIS database: K-parameter
Mirror base of theTHEMIS space data
Cluster, DoubleStar,Ace, …, database
AMDA (Automated Mutiple Dataset Analysis ): the goals
Multiple Dataset: the study of the (multi-scale) dynamics of plasma objects requires to perform the integrated analysis of multi-point and multi-instrument data
Automated (or semi-automated) Analysis: help forSearch, characterisation, classification of events Extensive exploitation of the vast database (statistics)“historical” studies (example: through solar cycles)Building cataloguesBuilding virtual constellations
AMDA, fact sheet• Now, very first prototype. Operational version v1.0 sould be openend in march.
• Prototyping development.
• Built on an existing data system (DD_System, A. Fedorov)
• Built in IDL, C, FORTRAN, Javascript
• Works on standardised local database through universal requests
• Implementation of the CDAWeb web-services
• Forum for user feedback
• Already in use for scientific studies
• Output: data and derived parameter files, plots, event lists, catalogues.
Underway:
• models (Tsyganenko, Shock, MP, NS, …) implementation
• Interface development for running AMDA on other databases
Schematic description of the envisioned system
Userinterface
Systeminterface
Localdatabases
Services
CASSINIMAPSKP
CDAWebSPDF
Virtual objects
Virtual objects
AMDA
Distantservices
Distantdatabase
Distantdatabase
Distantservices
USER
Externalinterface
Web
-se
rvic
es
Local database. Themis and DoubleStar to be added
AMDA/workspace: user defined parameter computation
AMDA/plot: visualisation and semi-automated event/signature search (1)
AMDA/plot: visualisation and semi-automated event/signature search (2)
AMDA/download: data extraction
AMDA/Conditional search
Example: Looking for possible X-line with a pair of spacecraft
BZ1 * BZ2 < 0AND
VX1 * VX2 < 0
Output: event list + gap listIn XML format. Other formats are also envisioned
<?xml version="1.0" ?> - <TimeTable Name="essai1"> <Created>Sun Feb 11 12:21:25 2007</Created> - <Chain> <![CDATA[ B_C1(2)*B_C3(2)<0&MOM_C1(1)*MOM_C3(1)<0 ]]> </Chain> <Source>AMDA Search</Source> - <Interval> <Start>2001-08-01T23:50:00</Start> <Stop>2001-08-02T00:00:00</Stop> </Interval>- <Interval> <Start>2001-08-02T00:05:00</Start> <Stop>2001-08-02T00:30:00</Stop> </Interval>- <Interval> <Start>2001-08-02T02:00:00</Start> <Stop>2001-08-02T02:15:00</Stop> </Interval>- <Interval> <Start>2001-08-02T02:20:00</Start> <Stop>2001-08-02T02:40:00</Stop> </Interval>
AMDA/implementation of the CDAWeb web-services
ConclusionAMDA allows to attack standardised data bases either massively or case by caseComputation of user defined parameters from multi-datasetsDescriptive model computation (Tsyganenko, …)Multi-dataset visualisationData and computed parameter extractionCDAWeb web-servicesEvent or signature search, characterisation and classification. Event list creation and managment:
in an interactive way, by visual identificationIn an automated way, by editing mathetical criteria on the content of the data
Perspectives for exchanging event lists with other services (SPDF, CDAWeb, GAIA, CAA, DARTS, …)?
AMDA for ThemisIntegrated K-parameter database: Themis (space data), Cluster, DoubleStar, Geotail, … The level (SP or PP parameters) of Cluster-DoubleStar data depends on co-I/non co-ITransparent connection to CDAWeb… and others?Providing event list for CL… and others?
Some examples:AMDA + CL (or other softwares):1) Users produces time-tables on AMDA based on their own
defined criteria applied on the content of the data they selected2) Time-tables sent to CL3) Users browse user-defined plots on the periods of the time-
tables and then run detailed analysis on CL
SSCWeb (or DARTS) + AMDA:1) Users produces time-tables on SSCWeb based on criteria
applied on the orbitography and conjonctions.2) Time-tables sent to AMDA3) Users produces secondary time-tables on AMDA based on
criteria applied on the content of the data for the periods of SSCWeb time-tables
4) Then, these secondary time-tables are exported to detailed analysis softwares or are used inside AMDA for building catalogues, or …
Demo
Derived parameters computation (calculator)
Automated and semi-automated search
Sub-dataset conditional extraction
Distant access to complementary data
An example: search of the events when the cross-tail current is “encircled” by CLUSTER
Test:• X1< -10 Re • BX1 * BX2 * BX3 * BX4 > 0• min([BX1 BX2 BX3 BX4]) < 0• max([BX1 BX2 BX3 BX4]) > 0
Looking for the time periods when 2 s/c are above the neutral sheet and the 2 other ones are below
CLSoftware for detailed analysis
of multi-datasets in space physics
Developed by Emmanuel Penou ([email protected])Scientific specifications by Jean-André Sauvaud
THEMIS SWG meeting SSL Nov. 15-16, 2006
CL: fact sheet
Multi-missions (CLUSTER, VEX, MEX, DEMETER, ACE, STEREO, …) and multi-instrument (B,E, particles, Waves)
SOLARIS, LINUX, WINDOWS
IDL and C
L1 files
L2 files
Calibration files
Processing
Visualisation
Computation of derived products
Export data (CDF, ASCII)
(Conditionnal search)(Internal formatCDF, ASCII)
DemoPredefined plots
Building, editing, manipulating plots
External files. CDF files
High level products:-distribution function-spectrograms in PA, 1/V-partial moments
Remote running
Conclusion: what can be done at CESR and CDPP for the THEMIS community?
Mirror site for space THEMIS data
Distributing CL, but limited user help
Remote running of CL on dedicated machine(s)
AMDA for event search, catalogue building, statistics on “simple” THEMIS and complementary data
Coupling AMDA event search and CL (and any other softwares or system: CDAWeb, UCB, …) by exporting time-tables
Contacts
AMDA:[email protected]://cdpp.cesr.fr/ forum for feedback on the CDPP services (in french for now)
Example: event search
Old fashion: “paper” search
AE, AL ISEE-dataIMP-8
AE, AL
ISEEMAGIS
EE-el
ectro
n
IMP8
MAG
IMP8PLA
Current fashion: web
Event search takes time and energy
CDAWebSPDF
UCLA
CAA
CDPP
And others
AMDA, fact sheet
• Now, very first prototype (construction started in June 2006 after kind of phase-A)
• Built on an existing data system (DD_System, A. Fedorov)
• Built in IDL, C, FORTRAN, Javascript
• Works on standardised and “simple” data, NetCDF
• Includes models (Tsyganenko, Shock, MP, NS, …)
• Attempt for using web-services of CDAWeb and the up-coming VxOs
• Forum for user feedback
• Already in use for scientific studies