Inter-university Upper atmosphere Inter-university Upper atmosphere Global Observation NETwork Global Observation NETwork

(IUGONET) (IUGONET) H. Hayashi1, T. Hori5, Y. Koyama2, Y. Tanaka6, D.

Yoshida2, S. Ueno3, M. Kagitani7, T. Kouno5, S. Abe4, N. Kaneda3, Y. Miyoshi5, M. Okada6, M. Nose’2, and T. Nakamura6

1. RISH, Kyoto Univ., 2. WDC/Kyoto, Kyoto Univ., 3. Kwasan and Hida Observatories, Kyoto Univ.

4. SERC, Kyusyu Univ., 5. STE Lab., Nagoya Univ., 6. NIPR,

7. PPARC, Tohoku Univ.

1

Participating universities and research institutes

Planetary Plasma and Atmospheric Research Center, Tohoku University National Institute of Polar Research

Solar Terrestrial Environment Laboratory, Nagoya University

Space Environment Research Center, Kyushu University

Research Institute for Sustainable Humanosphere, Kyoto University

World Data Center for Geomagnetism, Kyoto University

Kwasan and Hida Observatories, Kyoto University

Purpose of this 6-year (2009-2014) project are: to investigate mechanism of long-term change in the upper atmosphere (UA). to develop the metadata data base (DB) of UA data accumulated over 50 develop the metadata data base (DB) of UA data accumulated over 50

years since IGY by Japanese institutes/universitiesyears since IGY by Japanese institutes/universities. to promote the effective use of the observational data and the comprehensive

studies of the UA.This project is supported by Special Educational Research Budget (Research

Promotion), MEXT, Japan

Inter-university Upper atmosphere Global Observation NETwork

(IUGONET)

2

Project members

3

Observations by IUGONET institutes/universities

■ MST radar▲ MF / meteor radar◆ MAGDAS magnetometer★ FM-CW radar■ OMTI imager● WDC magnetometer

SYOWA base SuperDARN radar x2 MF radar aurora imagers magnetometer chain ELF obs. (conjugate with Onagawa)

MU radar

@Shigaraki

Toromso IS radar (EISCAT) meteor radar MF radar

★

★

★

Iceland aurora imager x2 magnetometer x3 SuperDARN

Hokkaido HF radar

Equatorial Atmospheric Radar （ EAR ）

Svarbird ：　 IS radar (EISCAT),meteor radar, aurora imager

SuperDARN radar

Iitate, Onagawa

radio teloscope

magnetomter

Solar observatory(Kyoto Univ.)

4

Spatial coverage of IUGONET observations

Heliosphere

Plasmasphere

Atmosphere

Magnetosphere

Ionosphere

Mesosphere

Stratesphere

Troposphere

heig

ht

Antarctica

SD radarAurora

Magnetometer

MF/Meteor radar

Arctic

IS radarAurora

Magnetometer

MF/Meteor radar

Solar wind

Radiotelescope

Global magnetometer chain

Ionospheric radar

Imager magnet-ometer

Equator

MST radarMF radar

Meteor radar

Mid –lat.

IS radarLIDAR

Imager magnet-ometer

Antarctica Low-mid lat. Equator Low-mid lat. Arctic

WDC data 　　　　 magnetometer data, geomagnetic indices

Solar telescope

Solar wind

High-lat.

IS & SD radar

Imager

NIPR 　 Kyushu Univ.　　

Kyoto Univ.　

Tohoku Univ.　　

Nagoya Univ.　　

Solar telescope

5

Problem with STP data bases in Japan

?

?

Institute 2 University 3Research center 1

Aurora Data DB

Solar Data DBGeomag. Data

DB

Geomag. data

experts

・・・・・・

Not impossible, but hard to reach the other disciplinary data! 6

Metadata data base What’s “Metadata”?

What the metadata DB gives UA researchers?

Any kinds of information on data, typically appears in the header of data files

Smart data search to quickly find and obtain the UA data

Enable comprehensive studies based on various kinds of UA dataContribute to studies on long-term variations of UA

Easy access to data of other UA regions Promote multi-disciplinary researches

- Observation time- Instrument type- Location of data files- Rule of use for data

- Observation site- PI name- Data format- ... etc.

Facilitate research works with UA data

7

Data base access through metadata DB

Geomag. Data experts

Research center1 Institute 2 University 3

Geomag. Data DB

Solar Data DBAurora Data

DB

・・・・・・

Aurora Data experts

Solar Data experts

Easy to obtain various kinds of UA data from research institutes/universities!

Metadata DB Sharing MD among all MDB

MD

gen

erat

ion

XML

XML

8

9

Project Timeline

Task Y2009 Y2010 Y2011 Y2012 Y2013 Y2014 Detail

Virtual information center (VIC) of UA studies

System intallation

System update

Construct the integrated research environment (TV-conference system, ..)

Development of metadata DB system

System designPrototype

Release version

System update

Design and develop the metadata DB system

Design the Metadata format standards

Release of Ver.1

Release the format ver.1 and keep updating if necessary

Development of data analysis software tools

Specifications and basic Design

Develop and release analysis softwares for UA data

Maintenance&extension of existing DBs of Observation data

Incorporate non-DB’d data into the DBs

Metadata generation

Generate metadata in the designated format and add to metadata DB

Operation of metadata DB

Release the metadata DB for community

VIC extension to related fields

Wrap up the project and discuss further extension

Effort focused on old data from Y2012 on

Effort focused on old data from Y2012 on

10

Development of Metadata DB (1)

• Develop by modifying and customizing one of the existing repository softwares which:– supports DB system with advanced search engine – available freely and being widely used – wear in heavy-use

• Need to treat metadata of arbitrary structure– not only Dublin core but MD designed by IUGONET

• Have potential for metadata exchange with other DBs– Support typical exchange/harvesting protocols (OAI-

PMH, …)

11

We consider “DSpaceDSpace” most promisingWe consider “DSpaceDSpace” most promising

Development of Metadata DB (2)

We are now developing the prototype of system for registering, searching, providing, and collecting the metadata of our research field using repository software (DSpace).

It is possible to search by time and/or keywords.

Developing data-search system

<List of search results> <Details of item>

DSpace can deal with arbitrary metadata. We have confirmed that it is technically

possible to exchange metadata by OAI-PMH.12

Development of Metadata DB(3)

3.full text search by Apache-Lucene

4.capable of holding obs. data

1.customizable Web UI

2. support OAI-PMH for MD exchange

XML

metadatametadata IUGONET does not hold obs. data

+6.backend RDBMS

5.function of data storage

DSpace architecture

13

Design of Metadata format (1)

What needed for the metadata format?

• Elements, items, and terms enough to describe various data provided by participating institutes/universities

– Categorize various kinds of UA data– Describe both “datasets” and “data files”– Treat not only numerical data but also event lists

and non-digital data (recorded on film, as printed matter, …)

– Documents and application tools available for use

– Extendable by adding newly defined elements, items, and terms in future 14

Design of Metadata format(2)

General strategy

• First examine the existing metadata format standards

– Consider collaboration with other DBs by adopting one of international de facto standards

• Then select the one which matches most the Japanese UA data

– minor modifications might be made 15

Design of Metadata format(3)

Features:•Metadata description with XML files•Suitable for STP data (ongoing development/improvement)•Applicable to upper atmosphere (UA) data •Widely-used, such as VxO•Format, related softwares are open for community•Elements/terms easily apendable (XML)

Features:•Metadata description with XML files•Suitable for STP data (ongoing development/improvement)•Applicable to upper atmosphere (UA) data •Widely-used, such as VxO•Format, related softwares are open for community•Elements/terms easily apendable (XML)

Considering Data model/metadata format based on SPASE

SPASEhttp://www.spase-group.org/

Data model and associated metadata format developed by international consortium to comprehensively describe research resources regarding heliospheric and magnetospheric physics

16

Metadata examples in SPASE: SuperDARN Hokkaido HF radar

Relation between resources is described by “ResouceID”Relation between resources is described by “ResouceID”

For dataset (NumericalData)

For observation instrument (Instrument)

For observation site (Observatory)

17

Development of data analysis software

Tentative planTentative plan Developed on the basis of

IDL(Interactive Data Language)

Distribute the tools using IDL Virtual Machine for non-IDL users

Collaborate with the ERG Science Center in software development

Still investigate possibilities of softwares based on free/GPL tools

rough sketch of screen

stack plots of multi-data

based on GUI

2D image-type data

18

Short-term schedule

open!open (plan)

2009

19

IUGONET web site

URL: http://www.iugonet.org/

English pa

ge goes li

ve on Jan,

2010!Engli

sh page go

es live on

Jan,

2010!

20

Summary

“IUGONETIUGONET” has just started up as 6-year (2009-2014) project to seek to study the long-term variations of the upper atmosphere (UA) by developing the metadata DB of the observation data accumulated over 50 years since IGY by Japanese institutes/universities.

We develop the metadata DB to facilitate the efficient use of the Japanese UA data and thereby promote comprehensive, multi-disciplinary studies.

Currently the metadata DB system is being developed on the basis of DSpace in which metadata are stored in the SPASE-based format.

Data analysis softwares primarily based on IDL are developed in collaboration with ERG-SC and are supplied to the community.

21

22