CHPS Era Archive/Data Backup

John Halquist

Development and Operations Officer

National Operational Hydrologic Remote Sensing Center (NOHRSC)

HIC Meeting – Feb 2010

Major categories for Archive use

General Archive requirements

Model development Unitgraph Routing

Calibration

Verification Analogs

Studies Case studies Applied Research Modeling/Product improvement Hindcasting/Reanalysis

Current situation NWS

IHFS(ops DB)

NWSRFS

ofs-de

raw SHEF-dec.

proc.SHEF-dec.

RAXarchive RDB

flat files(grids)

(field) data

acquisition

SHEF-dec.

FEWS fc DB

XSETS…

products

MPE

NAVX

ICPEVS

SHEFSHEF

SHEFSHEF

Current situation (FEWS archive)

FEWS archive:

• all data related to forecast bundled in one zip file

• forecast and associated inputs can be retrieved in stand alone FEWS

• data not directly accessible

Recommendation:intermediate solution only

(as short as possible)

FEWS fc DBmeta data

RDB

zip files archive-server

Where do we plan to go?

import

Exchange

Grid

Point

FEWSDB

Outbox (dir)

Archive Relational

DB

ArchiveFiles

External Sources

?

Could use

SOAP or PI

Possible NEW import JAVA

class

Enhanced or Additional FEWS

Export functionality

export

GGCalb

Verif

IWRSS

Key differences from existing Archive System

Data flows “through” the operational system to feed the Archiveseliminates Quality Control nightmareminimizes multiple “decode” pathways

Provide for a more “Integrated” Archive2 way data flow to support usage in operations and

analysis

Scalable, Flexible & Extensible architecturehardware & database schemas

Filling in the gaps …

Gaps in current requirements:

Meta-data What do we really need?What do we want?What about current archived data with little meta-

data?

Modifiers in CHPS Implemented significantly different than NWSRFSSome decisions need to be made on how we want to

handle modified time series

CHPS time series, example:

originaldatatimeseries

modifiers

time seriesused

Hardware modification:

chps2 (rdms)

Enclosure* (tuples)

Expansion…* Indicates New hardware

Schedule objectives:

Finalize conceptual design - Done

Determine Hardware requirements – Mar 2010

Data requirements finalization – May/Jun 2010

Build Prototype system – Sept 2010 Test system built at NOHRSC Operational field test at NWRFC

Provide initial DB Schema for review – Nov 2010

Field Deployment - ~2012

fini ...