Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of...
-
Upload
egbert-webb -
Category
Documents
-
view
217 -
download
0
Transcript of Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of...
![Page 1: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/1.jpg)
1
Extending FEWS with External functionalities
Zhengtao CuiNOAA National Weather ServicesOffice of Hydrologic Development
![Page 2: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/2.jpg)
2
Outline
• Day 11. FEWS Workflow (Page 3-7)2. The FEWS General Adapter Concept(Pages 8 –
19)3. OHD FEWS adapter framework (Pages 20 – 60)4. Examples (Pages 61 - 90)
• Day 2– Hands on Examples
![Page 3: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/3.jpg)
3
FEWS workflow, module and model
• A workflow specifies a sequence of other workflows or modules to execute– Usually executed to generate a forecast
• A module is a unit of execution– Can be a FEWS provided internal module, for example, the import module
• Flow of data/files is hidden from the user• http://publicwiki.deltares.nl/display/FEWSDOC/05+Configuring+the+available+DELFT-FEWS+modules
– A General Adapter is a reference to an external model• A FEWS General Adapter connects workflow with external models • Flow of data/files is visible to the user• Can have multiple execution units
• Model/External model– Numerical code (source, binary, scripts, etc.) that does some kinds of calculations based a particular
physical process,– Rainfall-runoff, routing, snow melt, etc.– Java model
• NWSRFS models written in Java
– Legacy model• NWSRFS models written in C++/C/Fortran
– http://publicwiki.deltares.nl/display/FEWSDOC/Models+linked+to+Delft-Fews
![Page 4: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/4.jpg)
4
Workflow example
![Page 5: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/5.jpg)
5
Module example
![Page 6: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/6.jpg)
6
Model example
• SAC-SMA• SNOW-17• ……
![Page 7: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/7.jpg)
7
Quiz
1. Describe the relationship of FEWS workflow and module.
Answer: A module is a part of a workflow.2. Which of the following could be listed inside a
workflow?1. A FEWS internal module2. Another workflow3. A general adapter module4. A binary executable of Snow-17
Answer: 1,2, and 3.
![Page 8: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/8.jpg)
8
The FEWS General Adapter Concept• The general adapter invokes an
external model allowing for FEWS workflow to execute that model• preAdapter, postAdapter,
Model• The user must provide a
model adapter to run an external model
• The general adapter communicates with external model using the Published Interface (PI)• PI is an XML based data
exchange schema• Pre/Post adapter converts PI
to/from native format
FEWS
Model Adapter/External Model
![Page 9: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/9.jpg)
9
The FEWS Published Interface (PI)• FEWS has no knowledge of any external models– Initiate by the FEWS General Adapter
• An external model has no knowledge about FEWS or any other external models– Read/Write PI XML data
• Including diagnostic information and model states
– physical model calculations• Communication between FEWS and external
models is done through PI– XML data files on disk
![Page 10: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/10.jpg)
10
FEWS Configuration Files1. Module Configuration
– Provides unique information about the module (name, input/output data, properties, parameter filename, external model, etc.)
2. Module Instance Descriptors– Defines each module ( e.g. module name, type). Type can be
FEWSTransformation or GeneralAdapter• FEWSTransformation are modules that come with FEWS• GeneralAdapter are modules developed by FEWS users
3. Workflows– Controls and defines the order in which to execute a series of workflows or
modules
4. Workflow Descriptors– Defines each workflow’s characteristics– Such as a descriptive name for a workflow
![Page 11: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/11.jpg)
11
…<workflowDescriptor … id=“workflow_file”</workflowDescriptor> …
WorkflowDescriptor.xml
…<activity> <moduleInstanceId>moduleConfigFile</moduleInstanceId></activity> …
…<moduleInstanceDescriptor id = “moduleConfigFile”> <moduleId>GeneralAdapter</moduleId></ModuleInstanceDescriptor> …
…< executeActivity> <command>cmd</command></executeActivity> …
Workflow_file.xml
ModuleInstanceDescriptors.xml
moduleConfigFile.xml
FEWSAdapter
![Page 12: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/12.jpg)
12
‘general’ section defines several some commontags that will be used by other sections.
‘startUpActivity’ defines activities that are runprior to a module run and export any data.
‘exportActivities’ defines all items to be exportedthrough the PI XML to the external module.
‘executeActivities’ defines external executablesor Java classes to be run.
‘importActivities’ defines all items to be importedFollowing successful completion of the module run.
Structure of General Adapter Module Configuration File
![Page 13: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/13.jpg)
13
Example: Basic Dummy AdapterDummyAdapter
• Has only one “main” method.• Prints “Hello World!” to the screen.• No input/output, parameters, properties, etc.• No external model is executed.
package dummy;
public class DummyAdapter{ public static void main(final String[] args) { System.out.println("Hello World"); }}
![Page 14: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/14.jpg)
14
General Adapter Module Configuration for the DummyAdapter
![Page 15: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/15.jpg)
15
Example: Basic Dummy AdapterDummyAdapter2
• Has only one “main” method.• Copy the output file in a property file to
another file named “rootDir/output/outputs.xml”.
• No external model is executed.
rootDirectory=Modules/dummy_module/dummy_basin2outputFile=output_benchmark.xml
args.file
![Page 16: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/16.jpg)
16
try { prop.load(bis); } catch(final IOException e) { e.printStackTrace(); }//get property identifying root directory final String rootDir = prop.getProperty("rootDirectory");
//get property identifying output file final String outputFile = prop.getProperty("outputFile");
//copy output file to output directory under root dir
try { FileUtils.copyFile(new File(rootDir, outputFile), new File(rootDir + "/output/outputs.xml")); } catch(final IOException e) { e.printStackTrace(); } }
package dummy;
import java.io.BufferedInputStream;import java.io.File;import java.io.FileInputStream;import java.io.FileNotFoundException;import java.io.IOException;import java.util.Properties;
import org.apache.commons.io.FileUtils;
public class DummyAdapter2{ public static void main(final String[] args) { final File propertyFile = new File(args[0]);
BufferedInputStream bis = null; try { bis = new BufferedInputStream(new FileInputStream(propertyFile)); } catch(final FileNotFoundException e) { e.printStackTrace(); } final Properties prop = new Properties();
DummyAdapter2.java
1
2
![Page 17: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/17.jpg)
17
General adapter configuration for DummyAdapter2
![Page 18: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/18.jpg)
18
• The <command> option can be any executable, for example,
![Page 19: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/19.jpg)
19
Quiz
1. How does FEWS exchange information with external modules?
Answer: Published Interface XML.2. Which configuration file specifies the model to run?Answer: The module configuration file.3. FEWS must have some knowledge about external
modules, true or false?Answer: False.4. What method a Java model adapter must implement?Answer: The main method.
![Page 20: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/20.jpg)
20
OHD FEWS Adapter Framework• An instance of the FEWS General Adapter concept.• Conforms the latest version of FEWS PI XML schema• Abstracts some common tasks for all OHD models.
– Encourage code reuse– Leveraging the effort of creating an adapter for each of the models.
• Can be used for both Java based model or legacy C/FORTRAN based model (a prebuilt binary executable).
• Configured through the general adapter configuration – Need a run info file to pass information to the OHD model, such as
input/output file locations, diagnostics files, etc.– The run info file is created by the <exportRunFileActivity> activity in the
module configuration file.• Has an executable jar in the <binDir> folder.
![Page 21: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/21.jpg)
21
OHD NWSRFS Java and non-Java (legacy) models
Java models• SACSMA• UNITHG• SNOW17• gridded SNOW17 for MARFC's
FFG application• TATUN• RSNWELEV• MUSKROUT• GLACIER• LAYCOEF• LAGK
Non-Java (Legacy) models• CHANLOSS(FORTRAN)• BASEFLOW(FORTRAN)• SARROUTE(FORTRAN)• CONSUSE(FORTRAN)• APICONT(FORTRAN)• gridded APICONT for MARFC's
FFG application(PCRASTER)• RESSNGL(FORTRAN)• RESJ(C++)• SSARRESV(FORTRAN)
![Page 22: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/22.jpg)
22
FEWS
params.xml
input.xml
mods.xml
states.xml
stateI.txt
output.xml
diag.xmlO
HD
Few
sAda
pter
ModelDriver
Extends
xxxModelDriver
Model output filesare PI XML files
Data are exchanged in memory.
Model specific filesLoaded into OHDFewsAdapter
Data Flow (Java model)PI
PI
PI
PI
PI
PI
![Page 23: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/23.jpg)
23
Model Parameter File Example
1. Follow the pi_moduleParameter.xsd schema2. XML tags are used to identify the parameter3. Use basic data types, string, double, int,
table
![Page 24: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/24.jpg)
24
Model State files
• states.xml
• statesI.txt/statesO.txt
![Page 25: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/25.jpg)
25
Important Java Classes (1)
• OHDFewsAdapter– Entry point of OHD models– Executed by the FEWS general adapter.– Invoke OHD model execution• Read in model configuration, timeseries, parameters,
and states• Calls the ModelDriver.executeDriver() method
– Can be executed independent of FEWS• Given all necessary information
![Page 26: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/26.jpg)
26
Important Java Classes (2)
• The ModelDriver Abstract Class– Abstracts all NWSRFS models– All models must implement the ‘execute()’
method• Executed by the ModelDriver.executeDriver() method• Output results and the diagnostics file in PI XML format • Execute the model using common data structures
(RegularTimeSeries, ModelState, ModelParameters)
![Page 27: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/27.jpg)
27
Important Java Classes (3)
• Common xml handling classes for– Parsing FEWS pi files • Input time series• Input states meta data • Input model parameters
– Writing FEWS pi files• Output time series
![Page 28: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/28.jpg)
28
How is it implemented?
class OHDFewsAdapter{ … void main() { … //instantiate modelDriver= Class.forName( “ohd.model.foo”); modelDriver.executeDriver(); … } …}
class ModelDriver{ … void executeDriver() { … //the physical model //starts execute(); … } …}
<executeActivity><className>ohd.model.util.fews.OHDFewsAdapter</className></executeActivity><exportRunFileActivity> <properties> <string key=“model” value=“ohd.model.foo”/> </properties></exportRunFileActivity>
XML
Java
Java
![Page 29: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/29.jpg)
29
OHDFewsAdapter
IDriver
Driver
ModelDriver
XXModelDriver
Implements
Extends
ExtendsInstantiate
Interface
Abstract class
User class
Class Hierarchy
OHD class
![Page 30: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/30.jpg)
30
The ModelDriver Abstract Class (1)
![Page 31: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/31.jpg)
31
The ModelDriver Abstract Class(2)
![Page 32: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/32.jpg)
32
The ModelDriver Abstract Class(3)
![Page 33: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/33.jpg)
33
OHD Java External Model Structure (1)
![Page 34: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/34.jpg)
34
OHD Java Model Structure (2)
![Page 35: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/35.jpg)
35
If rootDir is defined as the%TEMP_DIR%, the directoryis temporary; will beremoved after the adapter run. This behavior can be modified by the F12 menu.
![Page 36: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/36.jpg)
36
How to configure OHDFewsAdapter (1)
• Header Info
Can be missing. Set <ignoreDiagnostics> in <executeAcitvity>
Every external module is a generalAdapterRun
![Page 37: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/37.jpg)
37
How to configure OHDFewsAdapter (2)
• Startup activity
Most are standard directories thatmust be cleaned out prior to modelexecution.
Create working directories if they don’t exist
![Page 38: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/38.jpg)
38
How to configure OHDFewsAdapter (3)
• Export activity: Export States
In order to use relative dating,states must be exported. States can be empty.
![Page 39: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/39.jpg)
39
How to configure OHDFewsAdapter (4)
• Export activity: Input Time SeriesId of module that generated the needed data
Relative dating requires states be exported
![Page 40: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/40.jpg)
40
How to configure OHDFewsAdapter (5)
• Export activity: Parameters
If not specified, current module instance id is used.
![Page 41: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/41.jpg)
41
How to configure OHDFewsAdapter (6)
• Pass general information to the Adapter• Read by the OHDFewsAdapter
• Export activity: run info file
![Page 42: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/42.jpg)
42
Example of OHDFewsAdapter Runinfo File
This is the <properites> entry in the model configure file.
![Page 43: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/43.jpg)
43
Runinfo properties• “model” entry
– The name of the Java class that implements the ModelDriver interface• “printDebugInfo” entry
– Controls how detailed messages are printed to the diagnostic file.– Five levels (0 to 4).– Higher values mean more detailed information– Do not to be confused with the “Error Levels” in diag.xml
• 0 – fatal level, when an exception occurs• 1 – error level, similar to the fatal level• 2 - warning level, warning messages are printed• 3 – the regular level• 4 – debug level, the most detailed information about the execution of the
model.
• All other entries are model specific.
![Page 44: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/44.jpg)
44
How to configure OHDFewsAdapter (7)
• Execute activity
The runinfo file is passed as an argument to the adapter.
![Page 45: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/45.jpg)
45
How to configure OHDFewsAdapter (8)
• Import Activity: Import model output time series
The module instance ID of this module.
![Page 46: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/46.jpg)
46
Example: Basin Dummy ModelDummy Model
• Use the OHDFewsAdapter to drive the dummy model.– The dummy model is written in Java– Extends the OHD ModelDriver abstract class– Input a time series, add a value, and then subtract a
value from the time series– Configure the OHDFewsAdapter in FEWS
• Set input time series• The value to be added• The value to be subtracted
![Page 47: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/47.jpg)
47
package dummy.models;import java.util.List;import ohd.hseb.measurement.Measurement;import ohd.hseb.measurement.MeasuringUnit;import ohd.hseb.measurement.RegularTimeSeries;import ohd.hseb.ohdmodels.ModelDriver;import ohd.hseb.util.fews.FewsRegularTimeSeries;
public class DummyModelDriver extends ModelDriver{ public DummyModelDriver() { super._state = new ModelState(); super._parameters = new ModelParameters(); } @Override protected void execute() throws Exception { // get list of input time Series final List<RegularTimeSeries> inputTsList = getTsList();
_drivingTs = inputTsList.get(0); super.runModelDriverValidation();
final long startTime = _drivingTs.getStartTime(); final long endTime = _drivingTs.getEndTime(); final int interval = _drivingTs.getIntervalInHours(); final MeasuringUnit measuringUnit = _drivingTs.getMeasuringUnit();
// get measurement list; just a copy of input values final List<Measurement> measurementList = _drivingTs.getMeasurementList();// create an output time series using properties of input time series final FewsRegularTimeSeries tempTimeseries = new FewsRegularTimeSeries(startTime, endTime, nterval, measuringUnit);
// copy values in input time series to output time series final FewsRegularTimeSeries outputTimeSeries = new FewsRegularTimeSeries(tempTimeseries, _drivingTs);
// get run info properties final Properties runInfoProperties = getDriverProperties();
// get some properties in run_info.xml file final int valueToAdd = Integer.parseInt(runInfoProperties.getProperty("valueToAdd")); final int valueToSubtract = Integer.parseInt(runInfoProperties.getProperty("valueToSubtract"));
for(int index = 0; index < measurementList.size(); index++) { final Measurement measurement = easurementList.get(index); final double newValue = measurement.getValue() + valueToAdd - valueToSubtract; measurementList.set(index, new Measurement(newValue, measurement.getUnit())); }
outputTimeSeries.setMeasurementList(measurementList); super.addTStoResultMap(outputTimeSeries);
// each model driver must set the end state time in the states.xml file // this is important for validating that the state time is updated for the end of the run super._state.setDateTime(endTime);
}
}
DummModelDriver.Java
![Page 48: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/48.jpg)
48
…<exportRunFileActivity> <exportFile>%ROOT_DIR%/run_info.xml</exportFile> <properties> <string key="model" value="dummy.models.DummyModelDriver"/> <string key=“outputLocationId" value=“SELWE"/> <int key=“valueToAdd” value=8/> <int key=“valueToSubstract” value=8/> <int key="printDebugInfo" value="1"/> </properties></exportRunFileActivity>…<executeActivities> <executeActivity> <command> <className>ohd.hseb.ohdfewsadapter.OHDFewsAdapter</className> <binDir>$OHDBINDIR$</binDir> </command> <arguments> <argument>%ROOT_DIR%/run_info.xml</argument> </arguments> <timeOut>300000</timeOut> </executeActivity> </executeActivities>
Dummy Model Configuration
Set name of the Java classthat implements the model.
Explicitly set the location ID for model output time series
![Page 49: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/49.jpg)
49
OHD Legacy Models
• Uses the OHDFewsAdapter• Extends the ModelDriver• A framework has been developed to alleviate
the effort– Uses the LegacyModelAdapter to execute the
legacy model• A separate System Process (called from Java)
– LegacyModelAdapter has many helper methods such as writeStandAloneFiles().
![Page 50: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/50.jpg)
50
![Page 51: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/51.jpg)
51
Legacy Model Parameter PI XML File
1. All parameters are defined in one XML parameter element.
2. The OHD legacy adapter framework will export the parameters in native format and pass the filename to the model.
![Page 52: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/52.jpg)
OHDFewsAdapter
IDriver
Driver
ModelDriver
XXXLegacyModelDriver
Implements
Extends
Extends
Instantiate
Interface
Abstract class
User class
LegacyAdapter
LegacyModelAdapter
Extends
Instantiate
OHD class
Class Hierarchy
![Page 53: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/53.jpg)
53
How is it implemented?
class OHDFewsAdapter{ … void main() { … //instantiate modelDriver= Class.forName( “ohd.model.foo”); modelDriver.executeDriver(); … } …}
class ModelDriver{ … void executeDriver() { LegacyModelAdapter adapter = new LegacyModelAdapter(); adapter.executeModel( “executable”, args ); } …}
className>ohd.model.util.fews.OHDFewsAdapter</className></executeActivity><exportRunFileActivity> <properties> <string key=“model” value=“ohd.model.foo”/> <string key=“legacyLocation”, value=“executable”/> </properties></exportRunFileActivity>
XML
Java
Java
![Page 54: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/54.jpg)
54
These are model specific functions.
The framework helps wrap the existing models that are C or Fortran subroutines.
![Page 55: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/55.jpg)
55
![Page 56: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/56.jpg)
56
![Page 57: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/57.jpg)
57
• The example legacy model– Is a shell script– Uses the OHD legacy framework
• Configured as a OHDFewsAdapter• Executed by the OHD LegacyModelAdapter.executeModel()
– Takes an input time series and copies it to the output directory.
– Write diagnostic messages to the text file, diag.txt• The OHD LegacyAdapter will convert the text file to PI XML file,
diag.txt for FEWS to ingest.• ‘diag.txt’ has to be in a specific format.
– For example, 4| this is a log message
Example: Dummy Legacy Model
![Page 58: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/58.jpg)
58
public class DummyLegacyModelDriver extends ModelDriver{ @Override protected void execute() throws Exception { /** * Every legacy model needs to identify a driving time series from the time series read in the inputs.xml file. * This time series usually controls the run length and time step for executions In this example there is only * "1" input time series. Another approach is to search by time series type (e.g. parameterId) */ _drivingTs = getTsList().get(0);
final LegacyModelAdapter dummyAdapter = new LegacyModelAdapter(this, getWorkDir(), _logger);
dummyAdapter.writeStandAloneFiles();
/** * The line below is used to execute a stand alone program (separate executable) if there are any model specific * arguments to be sent, fill in a SortedProperties object and send to executeModel as last argument. Make sure the * executable returns a '0' exit code. The arguments will be inserted into a text file named arguments.txt and passed * as an argument to the executable. */ dummyAdapter.executeModel(getDriverProperties().getProperty(OHDConstants.LEGACY_LOCATION_DIR) + "/" + "dummy_executable", null);
getState().setDateTime(getComputationEndTime());
final List<RegularTimeSeries> resultsList = dummyAdapter.readFromFlatFileToRegularTimeSeriesList(); for(int i = 0; i < resultsList.size(); i++) { addTStoResultMap(resultsList.get(i)); }
}
}
Arguments will be written to a text file,arguments.txt. The file is passed to theExecutable as an argument.
![Page 59: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/59.jpg)
59
…<exportRunFileActivity> <exportFile>%ROOT_DIR%/run_info.xml</exportFile> <properties> <string key="model" value="dummy.models.DummyLegacyModelDriver"/> <string key=“legacyLocation" value=“Modules/bin"/> <string key=“outputLocationId" value=“SELWE"/> <int key="printDebugInfo" value="1"/> </properties></exportRunFileActivity>…<executeActivities> <executeActivity> <command> <className>ohd.hseb.ohdfewsadapter.OHDFewsAdapter</className> <binDir>$OHDBINDIR$</binDir> </command> <arguments> <argument>%ROOT_DIR%/run_info.xml</argument> </arguments> <timeOut>300000</timeOut> </executeActivity> </executeActivities>
Dummy Legacy Model Configuration
Set the location of the binary executable.
Explicitly set the location ID for model output time series
![Page 60: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/60.jpg)
60
Quiz1. OHDFewsAdapter is an executable Java class, true or false?Answer: True2. What ModelDriver method does the OHDFewsAdapter invoke?Answer: executeDriver()3. List at least three differences between an OHD model and an OHD legacy model.Answer: One possible answer 1) Java vs. C/Fortran binary; 2) Legacy model reads/writes text files; 3) Legacy model uses the LegacyModelAdapter 4. What argument is passed to the OHDFewsAdapter?Answer: run_info.xml5. Which property in the runinfo file specifies the class name for the model?Answer: The “model” property.6. The runinfo file shall be created by the user, true or false?Answer: False.7. Which ModelDriver method must be implemented by a modelAnswer: The “execute” method.
![Page 61: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/61.jpg)
61
NWSRFS Models: SACSMA• Rewrite in Java
OHDFewsAdapter
ModelDriver
SacSmaModelDriver
Extends
Used by
Abstract class
User class
SacSmaRainfallRunoffModelAdapterSacSmaRainfallRunoffModel
Instantiate
Used by
![Page 62: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/62.jpg)
62
SACSMA Java Code (1)
![Page 63: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/63.jpg)
63
SACSMA Java Code (2)
![Page 64: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/64.jpg)
64
Example SACSMA Configuration 1
![Page 65: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/65.jpg)
65
Example SACSMA Configuration 2
![Page 66: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/66.jpg)
66
Example SACSMA Configuration 3
![Page 67: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/67.jpg)
67
Example SACSMA Configuration 4
SACSMA can accept state modifications as separate input time series.
![Page 68: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/68.jpg)
68
Example SACSMA Configuration 5
![Page 69: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/69.jpg)
69
Example SACSMA Configuration 6
![Page 70: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/70.jpg)
70
SACSMA runinfo example
![Page 71: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/71.jpg)
71
SACSMA parameter PI XML
![Page 72: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/72.jpg)
72
SACSMA state files
• states.xml
• statesI.txt/statesO.txt
![Page 73: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/73.jpg)
73
NWSRFS Models: LAGK• A native binary executable
OHDFewsAdapter
ModelDriver
LagKModelDriver
Extends
Used by
Abstract class
User class
LegacyModelAdapter
Instantiate LegacyAdapter
Extends
Binaryexecutable
Executed by
![Page 74: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/74.jpg)
74
LagK Java Code (1)
![Page 75: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/75.jpg)
75
LagK Java Code (2)
![Page 76: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/76.jpg)
76
LagK Configuration(1)
![Page 77: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/77.jpg)
77
LagK Configuration(2)
![Page 78: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/78.jpg)
78
LagK Configuration(3)
![Page 79: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/79.jpg)
79
LagK Configuration(4)
![Page 80: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/80.jpg)
80
LagK runinfo
![Page 81: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/81.jpg)
81
LagK parameter PI XML files
![Page 82: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/82.jpg)
82
LagK state files• states.xml
• statesI.txt/statesO.txt
![Page 83: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/83.jpg)
83
LagK directorieslagk`-- ctwc1 |-- input | |-- inputs.xml | `-- params.xml |-- output | |-- diag.xml | |-- outputs.xml | `-- statesO.txt |-- run_info.xml |-- states | |-- states.xml | `-- statesI.txt `-- work |-- arguments.txt |-- diag.txt |-- nwsrfs_datatype_mapping_file.txt |-- nwsrfs_dataunit_file.txt |-- outputs.txt |-- params.txt |-- run_info.xml |-- states.xml |-- statesI.txt |-- statesO.txt `-- ts.txt
![Page 84: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/84.jpg)
84
HL-RDHM
• A software package for NWS distributed hydrologic models– Stand-alone tool for distributed hydrologic modeling research and
development.– A prototype for validating technique in NWS field offices prior to
operational software development• Written in C++/C/Fortran• Distributed models
– SACSMA, SAC-HT, SACHTET, Snow-17, and Hill slope/channel routing, DHMTF
– Auto-calibration– Models can be run in sequence
• Incorporated HL-RDHM functionalities into CHPS
![Page 85: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/85.jpg)
85
Approaches– Develop a FEWS distributed model general adapter– Develop a HL-RDHM model driver – Port HL-RDHM components as FEWS external modules• If a feature exists in FEWS, use FEWS otherwise use HL-
RDHM component• Use Java Native Interface (JNI) to link HL-RDHM
component with the model driver– Develop an external FEWS module that converts NetCDF
gridded time series to XMRG files– Reuse OHD code if possible
![Page 86: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/86.jpg)
86
Data flow in FEWS environment
![Page 87: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/87.jpg)
87
Software Design
![Page 88: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/88.jpg)
88
OHDDistFewsAdapter.javapublic class OHDDistFewsAdapter{ …… public static void main( final String[] args ) { if( all necessary data are not present or checks are not OK )
{ throw exception
}else{
//start logger defineModelRunInfoAndSetupModels() Call the ModelDriver's executeDriver() method // execute the model
//write logger
} } ……}
![Page 89: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/89.jpg)
89
DistModelDriverpublic class DistModelDriver extends ModelDriver{ …… public void executeDriver( ) throw Exception { …… execute() …… } ……}
public class RDHMDriver extends DistModelDriver{ …… public void execute( ) throw Exception { …… getRunInfo() modelFactory registerModels setupDomain call hlrms_algo() …… } ……}
![Page 90: Extending FEWS with External functionalities Zhengtao Cui NOAA National Weather Services Office of Hydrologic Development 1.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649ec55503460f94bd0921/html5/thumbnails/90.jpg)
90
Configuration