DATA RESCUE IN AFRICA: SYNERGY WITH NATIONAL AND REGIONAL PROJECTS Climate&Environment Department...
-
Upload
maya-dwyer -
Category
Documents
-
view
214 -
download
2
Transcript of DATA RESCUE IN AFRICA: SYNERGY WITH NATIONAL AND REGIONAL PROJECTS Climate&Environment Department...
DATA RESCUE IN AFRICA: SYNERGY WITH NATIONAL AND REGIONAL
PROJECTS
Climate&Environment DepartmentACMAD
International Workshop on the Rescue and Digitization of Climate Records for countries in West Africa
19-23 November 2012, Accra, Ghana
- Historical Background
- ACMAD plans for African Data Rescue& Management
- Proposed support to other DARE projects in Africa
- Concluding Remarks
OUTLINE
HISTORICAL BACKGROUND
With funding from Belgium, WMO sent experts in 46 African countries to locate and image hydromet data
Microfiches of over 30 millions old observations from about 1000 sites were produced with copies to
countries
Microfiches were handed over to ACMAD around themid 90s
Discussions with IEDRO and recently with IRI at Columbia University for imaging and digitalization
Call for bid and selection of a firm to procure data rescue equipments and related training at ACMAD
( Aug-Oct 2012)
Training of ACMAD/country experts on scanning of
microfiches, creation of images library , data keying and
entry into the national and continental databases
Inventory , scan all microfiches ( about 100
thousands) and save as digital images
Key in the data and enter them into the ACMAD/Regional
Climate Centre’s African climate database
ACMAD DATA RESCUE PLANS
Provide on the job training on data rescue at ACMAD for countries
Countries data Rescue with seconded experts and visiting scientists programmes at ACMAD
Each country expert will bring back a copy of digitized
Data in a Climsoft managed database interfaced with application tools ( RCLIMDEX, CPT, Excell, GIS macros…) for climate monitoring and forecasting products
Organize workshops to produce climate indices, Atlases
and other added value products using rescued data
ACMAD DATA RESCUE PLANS ( support to country and sub-regional DARE projects)
DEVELOPMENT AND USE OF THE CONTINENTAL CLIMATE DATABASE AT ACMAD
Data base design and architecture
Ingestion of Daily Data
Ingestion of Synoptic data
Curent status of the Data base
DATA BASE DESIGN AND ARCHITECTURE DATA BASE DESIGN AND ARCHITECTURE Data model is the model for CLIMSOFT version 3, which was last
updated in 2010
Data model allows storage of different data formats: numeric, text, Boolean (Yes/No) and graphical images
Though originally designed in MS Access, the database design has been ported to MySQL and PostgreSQL. It can be ported to any other RDBMS.
For the ACMAD climatological database, the open source PostgreSQL was chosen for its efficiency, stability and robustness.
PostgreSQL data storage capacity is on the order of Terabytes (TB).
INGESTION OF DAILY SYNOP DATA
CLIMSOFT ingest GTS data via FTP.
The data are automatically imported into the CLIMSOFT database at a scheduled time.
This module for the ingestion of synoptic data became operational in June 2012.
Current Database StatusCurrent Database Status
Historical daily data from the NOAA/NCDC FTP site date back to 1972.
GTS SYNOP data imported from GTS data reception system at ACMAD.
Total database size is approximately 24 GB
Very Basic QC is carried out. We are planning to implement more rigorous Quality checks.
PROVISION OF FORMATED PROVISION OF FORMATED DATA FOR CLIMATE DATA FOR CLIMATE OPERATIONSOPERATIONS Table of 10-day summaries for selected stations for all Africa input to dekadal Bulletins
Data output in RClimDex, CPT and Instat inputs formats.
We have plans to include other outputs in formats required by any climate application tool
Long-term climatological means generated for all GTS stations in Africa.
Climatological means available for the period 1981-1990, 1991-2000, 2001-2010, 1981-2010.
Data products in XML format and can be viewed with any web browser
RCLIMDEX and INSTAT input data format from Chad RCLIMDEX and INSTAT input data format from Chad climsoft climsoft
Database at ACMADDatabase at ACMAD
Multiple key entry for countries data using on the job training, secondment and visits on data rescue to reduce possible errors
Continue development of CLIMSOFT for station data by building interfaces with the web, application tools, other environmental database management systems as well as GIS software
Expand CDMS training across Africa integrated into workshops for generating value added information for climate data