GFESuite Configuration Mark Mathewson. Sept. 24-27, 2002GFESuite Configuration2 Topics Documentation...
-
Upload
diana-loos -
Category
Documents
-
view
218 -
download
2
Transcript of GFESuite Configuration Mark Mathewson. Sept. 24-27, 2002GFESuite Configuration2 Topics Documentation...
GFESuite ConfigurationGFESuite Configuration
Mark Mathewson
Sept. 24-27, 2002 GFESuite Configuration 2
TopicsTopics
DocumentationDirectory StructureBase/Site ConfigurationBase/Site/User configurationProblem SolvingMisc. Items
Sept. 24-27, 2002 GFESuite Configuration 3
DocumentationDocumentation
Sept. 24-27, 2002 GFESuite Configuration 4
Where is it?Where is it?
GFE HELP menuFSL AWIPS doc web site
http://www-md.fsl.noaa.gov/eft/AWIPSdoc.html
Many other regional/national web sites contain information
listservers another source ndfd_team, ifpsrppinfo, ifps
Sept. 24-27, 2002 GFESuite Configuration 5
What is available?What is available?
User Training GuidesReference ManualsConfiguration GuidesProduct Generation GuidesREADME file
Sept. 24-27, 2002 GFESuite Configuration 6
How good is it?How good is it?Issues:
Little indexing available…. High priority for accurate info. Lower priority for indexing than s/w
enhancementsGenerally fairly accurate – especially the
configuration guidesSpend a few minutes with it – it will save
you time.
Sept. 24-27, 2002 GFESuite Configuration 7
Directory StructuresDirectory Structures
Sept. 24-27, 2002 GFESuite Configuration 8
Top-LevelTop-Level
Serverconfiguration
documentation
examples
Run scripts
Sept. 24-27, 2002 GFESuite Configuration 9
databases Structuredatabases Structure
KEEP OUT!
Sept. 24-27, 2002 GFESuite Configuration 10
databases Structuredatabases Structure
Sept. 24-27, 2002 GFESuite Configuration 11
Database grids StructureDatabase grids Structure
Sept. 24-27, 2002 GFESuite Configuration 12
GFE files on HPUXGFE files on HPUX
Sept. 24-27, 2002 GFESuite Configuration 13
Directory TipsDirectory Tips
Do not modify ANY files in etc/BASE Use etc/SITE instead
Do not modify ANY files under data/databases These are controlled by the ifpServer
README file contains list of directories that will be “purged” during installation
Sept. 24-27, 2002 GFESuite Configuration 14
BASE/SITE ConceptsBASE/SITE Concepts
ifpServer Configurability
Sept. 24-27, 2002 GFESuite Configuration 15
Base/Site conceptBase/Site concept
Applies to /etc/BASE, /etc/SITE Used to preserve YOUR changes from upgrade to
upgrade.
BASE file SITE file Purpose
serverConfig.py localConfig.py General server overrides
serverConfig.py localWxConfig.py Wx configuration override
Maps.py, MapFiles.py
localMaps.py, localMapFiles.py
Map background override
dfcConfig.py localDfcConfig.py Daily Forecast Critique override
MapFiles – new in IFPS12
Sept. 24-27, 2002 GFESuite Configuration 16
Base/Site conceptBase/Site concept
Use favorite text editor to change files.Normally have to restart ifpServer after
changing files.Use runIFPServer –n to check for errors.If server doesn’t come up, check for errors.
Look through log files.
Sept. 24-27, 2002 GFESuite Configuration 17
Base/Site conceptBase/Site concept
Configuration files are Python-executable code
Very particular to syntax.Local (e.g., SITE) overrides using Python
importing Can be tricky
Sept. 24-27, 2002 GFESuite Configuration 18
Server Configuration Server Configuration OverridesOverrides
serverConfig.py
Maps.py
localConfig.py
Maps.py
SITEBASE
Python Path:SITE, then BASE
CopiedThenmodified
importPartialoverrides
CompleteOverride (don’t do)
Sept. 24-27, 2002 GFESuite Configuration 19
Python “dual” ImportingPython “dual” ImportingserverConfig
TC6NG = (0*HOUR,6*HOUR,6*HOUR)
DFC_UPDATE_INTERVAL = 15
Dbs = DBS + localConfig.parms
Eta = (ETA, TC6NG)
import localConfig
localConfig
serverConfig.TC6NG = (1*HOUR,…)TC6NG = …
parms = [([MaxT], TC6NG)]
parms = [([QPF], TC6NG)]
DFC_UPDATE_INTERVAL = 10
import serverConfigfrom serverConfig import *
Sept. 24-27, 2002 GFESuite Configuration 20
Python “dual” ImportingPython “dual” ImportingserverConfig
DFC_UPDATE_INTERVAL = 15
import localConfig
localConfig
DFC_UPDATE_INTERVAL = 10
import serverConfigfrom serverConfig import *
Result?DFC_UPDATE_INTERVAL = ???
start
Sept. 24-27, 2002 GFESuite Configuration 21
Python “dual” ImportingPython “dual” Importing
import serverConfig Allows you to access variables/methods from the
module you import, via: serverConfig.varName from serverConfig import *
Makes a copy of the variables, you can access the copies of these variables directly, via: varName, but these are COPIES. If you change them, the originals aren’t changed.
Exceptions: dictionaries, arrays Don’t require serverConfig. in front of variable.
Sept. 24-27, 2002 GFESuite Configuration 22
Python “dual” ImportingPython “dual” ImportingserverConfig
TC6NG = (0,6,6)MaxT = (“MaxT”, SCALAR, "F", …)
Dbs = DBS + localConfig.parms
MaxT_PG = (MaxT, TC6NG)
import localConfig
localConfig
serverConfig.TC6NG = (1,6,6)TC6NG = (2,6,6)
parms = [([MaxT], TC6NG)]
parms = [([QPF], TC6NG)]
import serverConfigfrom serverConfig import *
What is TC6NG at this point?What is the TC for MaxT at this point? Two parm
lines
Sept. 24-27, 2002 GFESuite Configuration 23
Base/Site conceptBase/Site concept
GOTCHA – Python files .py .pyc .pyo
If you delete the .py file, such as localConfig.py, you may still have *pyc or *pyo files. The system will use those and will behave as if you never deleted the .py file.
Sept. 24-27, 2002 GFESuite Configuration 24
localConfig capabilitieslocalConfig capabilities
Adding new weather elementsAdding new modelsChanging characteristics (# of versions)Changing time constraints for elementsChanging domain/resolution
Sept. 24-27, 2002 GFESuite Configuration 25
localMaps/localMapFileslocalMaps/localMapFiles
Overrides Maps.py and MapFiles.py Add new maps Remove existing maps Modify characteristics of maps
FiltersAutomatic edit areas
MapFiles/localMapFiles (new IFPS12) Defines the names of the shapefiles
Sept. 24-27, 2002 GFESuite Configuration 26
Changing map backgrounds – Changing map backgrounds – filteringfiltering
Can filter by shapefile attributes. Make a new map just for ifpIMAGE generation.
CWAzones.filter(lambda x : x['CWA'] == CWA)
def cwaZoneFilt(x): return x['CWA'] == CWA CWAzones.filter(cwaZoneFilt)
ShapefileAttribute
Identical Filter
Sept. 24-27, 2002 GFESuite Configuration 27
Changing map backgrounds - Changing map backgrounds - filteringfiltering
def exampleFilt(x): myCounties = ['Summit', 'Sandusky', 'Huron', 'Medina'] return x['COUNTYNAME'] in myCounties and \ x['STATE'] == "OH" and x['CWA'] = 'CLE'
excludedCities = [‘Clinton’, ‘Pearl’, ‘Brandon’]cities.filter(lambda x,y=excludedCities: x[‘POP_1990’] > 10000 \ and x[‘NAME’] not in y
Note:
Sept. 24-27, 2002 GFESuite Configuration 28
Updating Map ShapefilesUpdating Map Shapefiles
Download new shapefiles from NWSHgzip each component:
*.dbf, *.shp, *.shxPlace into /awips/GFESuite/data/mapsIf the name changed, add/edit
localMapFiles.py [IFPS12] to point to the new file
Caution: shapefiles replaced during upgrade
Sept. 24-27, 2002 GFESuite Configuration 29
localWxConfiglocalWxConfig
Similar to localConfig, except for the Weather Definition
Uses: Adding additional attributes Adding additional weather types Adding additional coverage/prob, intensities
May make incompatible grids with other NWS offices.
Sept. 24-27, 2002 GFESuite Configuration 30
Tips (and Rules)Tips (and Rules)
DON’T edit files in etc/BASE They are wiped clean each upgrade
Check your changes immediately: runIFPServer –n (even with server running)
Some changes may not be compatible with IFPS Time constraints, domains/resolution
Sept. 24-27, 2002 GFESuite Configuration 31
BASE/SITE/USER ConceptsBASE/SITE/USER Concepts
GFE / ifpIMAGE Configurability
Sept. 24-27, 2002 GFESuite Configuration 32
Base/Site/User conceptBase/Site/User concept
Applies to: gfe and ifpIMAGE configuration files Sample sets, edit areas Color tables Smart Tools/Procedures User-defined time ranges Weather Element Groups
Sept. 24-27, 2002 GFESuite Configuration 33
Tips (and Rules)Tips (and Rules) Just like etc/BASE, do not edit files in
data/databases/BASE since they are wiped clean during the next upgrade.
Do not edit files directly in the databases directory. Some files require associated metadata files that can’t
be created with a text editor Changing files can confuse the ifpServer and GFEs if
they are running at the time. Use the GFE to edit files, OR use the ifpServerText
program.
Sept. 24-27, 2002 GFESuite Configuration 34
““Special” UsersSpecial” Users
SITE – site administrator login. Files in this category are seen by all users.
BASE – brings up GFE using BASE configuration only --- as supplied by FSL. Good way to verify a problem is a local configuration problem.
“BASE” and “SITE” will not appear in the GFE startup dialog.
Sept. 24-27, 2002 GFESuite Configuration 35
Python Errors…GFE is hung!Python Errors…GFE is hung!
FSL
Not any more…. Starting GFE witha bad configuration file will cause adialog to popup showing you theerror location.
Sept. 24-27, 2002 GFESuite Configuration 36
Why Can’t I find my config Why Can’t I find my config filesfiles
Are your config files lost?Perhaps you created them under a different
user?Use ifpServerStats to help you find them.
ifpIMAGE uses the user SITE for execution – you need to create files in correct place.
Sept. 24-27, 2002 GFESuite Configuration 37
I changed my config file, but I changed my config file, but the change isn’t working…the change isn’t working…
gfeConfig BASE
3. Later, log onto awipsusr,the BOUsiteConfig “seen” doesn’thave the latest changes.
BOUsiteConfig awipsusr (or other user)
1. User awipsusr createsBOUsiteConfig, to test out thenew configuration.
BOUsiteConfig SITE
2. Later, the same user logsonto user SITE and copiesthe awipsusr BOUsiteConfig using ifpServerText. Makes another change to it.
import
Sept. 24-27, 2002 GFESuite Configuration 38
The “READ-ONLY” flagThe “READ-ONLY” flag
Setting the Read-Only flag on the various configuration files will stop the user from overriding them.
No GFE GUI to set the read-only flag.Must do it directly on the file system.
chmod 444 filename
Fully implemented with IFPS12
Sept. 24-27, 2002 GFESuite Configuration 39
Some sites have…Some sites have…
Made copies of files in BASE and placed them in SITE Copied lots of files from SITE to “user” This practice is NOT recommended. You
might get different behavior based on login name.
BASE SITE user
gfeConfig gfeConfig
50 tools 50 tools
copied
copied
Sept. 24-27, 2002 GFESuite Configuration 40
Concepts to understandConcepts to understand
Python files (gfeConfig) Can use Python importer for partial overrides
Non-Python files (sample sets) Must use complete overrides, no importer capability
Sept. 24-27, 2002 GFESuite Configuration 41
Order of PrecedenceOrder of Precedence
ERROR
no SITE File
Exists?
yes
no
Return SITE File
BASE File
Exists?
no
yes
Return BASE File
USER File
Exists?
Request File
yes
Return USER File
Sept. 24-27, 2002 GFESuite Configuration 42
Retrieval of Color Table Retrieval of Color Table ExampleExample
QPFColorTable
BASE
QPFColorTable
SITE
QPFColorTable
USER
X X
Sept. 24-27, 2002 GFESuite Configuration 43
Some files are PythonSome files are Python
You can use the Python “import” mechanism and you should.
Advantage: You can override small pieces of files. You don’t need to redefine everything
Examples: Smart Tools, gfe/ifpIMAGE configuration files
Sept. 24-27, 2002 GFESuite Configuration 44
Python importingPython importing
A B C
import B
codecodecode
import A
codecodecode
codecodecode
Sept. 24-27, 2002 GFESuite Configuration 45
Python importingPython importing
C C
import C
codecodecode
codecodecode
XCan’t import from “yourself”, even if at different BASE/SITE/USER levels
Sept. 24-27, 2002 GFESuite Configuration 46
Python-file RetrievalPython-file Retrieval
Sept. 24-27, 2002 GFESuite Configuration 47
GFE Config file exampleGFE Config file example
…T_defaultColorTable=‘midRange’ImageOnActive=yes…
USER (joeConfig)
import gfeConfigT_defaultColorTable=‘lowRange’…
SITE (siteGfeConfig)
Site could also have redefined the color table
import siteGfeConfigT_defaultColorTable=‘myOwn’ImageOnActive=noContour_font=3T_spatialImageType = [‘Image’, ‘Contour’]…
User could also have imported directly from the base.
BASE (gfeConfig)
Sept. 24-27, 2002 GFESuite Configuration 48
TipsTips
Software doesn’t seem to be working. Start GFE as user “BASE”, config “gfeConfig”
Run ifpServerStats and look at your environment.
Move files around if needed using moveGFEData.
Sept. 24-27, 2002 GFESuite Configuration 49
Misc. TopicsMisc. Topics
Sept. 24-27, 2002 GFESuite Configuration 50
Installs/UpgradesInstalls/Upgrades
SITE and USER directories not touched by installs.
BASE files overwritten. Recommend you do partial overrides, not
complete files copies in order to ensure operability of software and to catch latest changes in the upgrade.
Watch out for directories that are overwritten! README file.
Sept. 24-27, 2002 GFESuite Configuration 51
Product Generation Product Generation
Site/base/user concept applies to all programs using the server.
By Default, many product generation programs become user “SITE”.
Prevents user overrides from affecting site’s products
Can start programs with specific user switch (-u username).
Sept. 24-27, 2002 GFESuite Configuration 52
Watch your localConfig Watch your localConfig configuration changes….configuration changes….
Some changes may “drop” your database. Changing domains, resolutions, data type
Some changes only “drop” a single weather element. Changing the Weather definition
Some changes may shift around the grids in time. Changing time constraints
Sept. 24-27, 2002 GFESuite Configuration 53
Problems…and how to Problems…and how to diagnose themdiagnose them
Each program uses logfiles. Located in /awips/GFESuite/data/logfiles
Search logfiles for “PROBLEM”, “BUG”
Many problems can be diagnosed simply by examining the logfiles.
But some entries in logfiles are cryptic to forecasters.