Where does VIPER get its data from? Tom Pagano
[email protected] 503-414-3010
What data do we need?
VIPER’s input and storage of data
Old and new databasesAWDBWYFORCDBSSNOTEL
Real-time data schedule
Local file import
What data do we need?
MetadataStation lists, lat/lon/elev
Historical time series dataReal-time time series data
Snowpack, Precipitation, etc
Forecast equations, configurations
VIPER Workbook
StationMetadata
HistoricalTime-Series
1 station POR
Real-timeData
All stations2 years
Equations/Configurations
VIPER Workbook
StationMetadata
HistoricalTime-Series
1 station POR
Real-timeData
All stations2 years
Equations/Configurations
Metadata
Data
AWDB
VIPER Workbook
StationMetadata
HistoricalTime-Series
1 station POR
Real-timeData
All stations2 years
Equations/Configurations
USGS webpage
Metadata
Data
AWDB
Old time series databases
Old databases and new databases
CDBS•“Final” data• Period of record ~1 year out of date•Daily, monthly•Snow, Precip, Temp Resv, Strm
Old time series databases
Old databases and new databases
CDBS•“Final” data• Period of record ~1 year out of date•Daily, monthly•Snow, Precip, Temp Resv, Strm
SNOTEL•Provisional•Recent 2 years•Hourly (and daily)•All snotel elements•Archive kept (since ?)
Old time series databases
Old databases and new databases
CDBS•“Final” data• Period of record ~1 year out of date•Daily, monthly•Snow, Precip, Temp Resv, Strm
SNOTEL•Provisional•Recent 2 years•Hourly (and daily)•All snotel elements•Archive kept (since ?)
WYFOR•Ephemeral•Recent 2 years•Monthly•Snow, Prec, Strm Resv, “Misc”•Archive kept (since ‘93)
Old time series databases
Old databases and new databases
CDBS•“Final” data• Period of record ~1 year out of date•Daily, monthly•Snow, Precip, Temp Resv, Strm
SNOTEL•Provisional•Recent 2 years•Hourly (and daily)•All snotel elements•Archive kept (since ?)
WYFOR•Ephemeral•Recent 2 years•Monthly•Snow, Prec, Strm Resv, “Misc”•Archive kept (since ‘93)
AWDB•Provisional and final mixed•All years•All durations•All elements*
New time series database
*Planned
NOTE: AWDB load strategy being reworked so rest of slides may not apply in near future!
Example- where do historical monthly snow come from?
1960
1970
1980
1990
2000
Back estimated snowcourse
Site automated (aug 1980)
Brumley Snotel
Example- where do historical monthly snow come from?
1960
1970
1980
1990
2000
CDBS
1x load of cdbs… updated if Dana notified of changes
1x load daily data in off season into monthly slots (snotel.. Or daily cdbs?)
SNOTEL
Historical streamflow data
1960
1970
1980
1990
2000
CDBS
Wyfor “under” CDBS
Historical streamflow data
1960
1970
1980
1990
2000
1x load of old wyfor files since mid 1990’s
Then on top of that…
1x load of cdbs… updated if Dana notified of changes
The realtime data schedule and synchronization
AWDBdatabase
Wyfor SNOTEL
4x a day, complete load (1:20am, 5:20am, 11:20am, 10:20pm)
The realtime data schedule and synchronization
AWDBdatabase
Wyfor SNOTEL
Any DMP edits to snotel loadedevery 5 minutes to wyfor/awdb
DMPEdits
The realtime data schedule and synchronization
AWDBdatabase
Wyfor SNOTEL
Spreadsheet files loadedevery 10 minutes all day
to wyfor/awdb
CSV spreadsheet
files (non-snotel data)
CSVprocessor
The realtime data schedule and synchronization
AWDBdatabase
Wyfor SNOTEL
Direct edits of wyfor reach wyfor right awayBut reach AWDB 50 minutes after the hour
5-9am, 11am-4pm and 11pm. Worst method of entering data for viper.
Manual edits
If you have custom datasets, Viper can also access local CSV format files.
Same layout as individual station data sheets (e.g. data1, data2) Realtime data gets manually entered in “ExternalData” sheet.
OMG Viper is so slow!!!!
Viper is a network-hungry application.Within the USDA backbone, your connection should be very fast.Outside the backbone, speed varies from tolerable to disgruntling.
OMG Viper is so slow!!!!
Viper is a network-hungry application.Within the USDA backbone, your connection should be very fast.Outside the backbone, speed varies from tolerable to disgruntling.
The way viper connects to the database is the same way that users contact the new update report…. which can mean:
Top Related