Improved Water Supply Forecasts for the Kootenay Basin · US Army Corps of Engineers Northwestern...

US Army CorpsUS Army Corpsof Engineersof EngineersNorthwestern DivisionNorthwestern Division

Improved Water SupplyImproved Water SupplyForecasts for the KootenayForecasts for the Kootenay

BasinBasin

Randal T. WortmanRandal T. WortmanHydraulic EngineerHydraulic Engineer

August 4, 2005August 4, 2005

Columbia Basin Water Management DivisionColumbia Basin Water Management DivisionNorthwestern Division, U.S. Army Corps of EngineersNorthwestern Division, U.S. Army Corps of Engineers

PO Box 2870, Portland, OR 97208PO Box 2870, Portland, OR 97208--28702870Ph: 503 808Ph: 503 808--3957 Email: [email protected] Email: [email protected]

US Army CorpsUS Army Corpsof Engineersof EngineersNorthwestern DivisionNorthwestern Division Improved Water SupplyImproved Water Supply

Forecasts for the KootenayForecasts for the KootenayBasinBasin

Randal T. WortmanRandal T. Wortman

Columbia River BasinColumbia River Basin

Canada

United States

Kootenay Basinabove Libby Dam1530 feet of head1530 feet of head

at 13 downstreamat 13 downstreamhydropowerhydropowerprojectsprojects

IDMT

WA

BC

AL

Fort Steel Subbasin

Libby Local Subbasin

70 0 70 140 Miles

N

EW

S

Kootenay Basin MapKootenay Basin Map

AB

Libby Dam

Area above LibbyDam: 8985 sq. miles(23,300 sq. km)

Lake Koocanusastorage: 5,000,000 AF


Kootenay Inflow aboveKootenay Inflow aboveLibby DamLibby Dam

Kootenay River above LibbyMonthly Inflow Volume

0500

10001500200025003000350040004500

Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep

Inflo

w in

KA

F_ MaxMeanMin

Forecast Season: Apr-Aug


Standard MultipleStandard Multiple--Variable Regression inVariable Regression in

Water Supply ForecastingWater Supply Forecasting

�� The dependent variable is a seasonal inflow volume, e.g.The dependent variable is a seasonal inflow volume, e.g.AprilApril--August runoff in thousandAugust runoff in thousand--acreacre--feet (KAF)feet (KAF)

�� Predictor variables are pseudoPredictor variables are pseudo--variables created fromvariables created fromweighted combinations of similar stations (e.g. sum orweighted combinations of similar stations (e.g. sum oraverage of three snow stations)average of three snow stations)


Libby Local BasinFt Steel BasinVariable

Σ Oct, NovLibby Local basin runoff

Σ Oct, NovFt Steele basin runoff

Fall Runoff (FRO)

Σ Apr, May, .8 Jun, .5 Jul, .2 Aug

Σ FTIM, PTHI, KASB,WHFM


Σ BRIB, KASB, PTHI,WASB, CRSB

Spring (Apr--Aug)Precipitation (SP)

Σ Oct, Nov, Dec, Jan, Feb, Mar

Σ ELKBELKB, FENB, FTIM,Σ Oct, Nov, Dec, Jan, Feb, Mar

Σ ELKBELKB, BABB, GRPB,BRIB, KASB

Winter (Oct--Mar)Precipitation (WP)

Σ SUMB, NFRB, RMTM,KIMBKIMB, WSLM,0.5*MORB

Σ MILB, MORB,KGHB, SUMB, MBCB,GRPB, NFRB

1 April Snow WaterEquivalent (SWE)

Libby Local BasinFt Steel BasinVariable

Σ Oct, NovLibby Local basin runoff

Σ Oct, NovFt Steele basin runoff

Fall Runoff (FRO)


Σ FTIM, PTHI, KASB,WHFM


Σ BRIB, KASB, PTHI,WASB, CRSB

Spring (Apr--Aug)Precipitation (SP)


Σ ELKBELKB, FENB, FTIM,LRSM, BONI, POLM


Σ ELKBELKB, BABB, GRPB,BRIB, KASB

Winter (Oct--Mar)Precipitation (WP)

Σ SUMB, NFRB, RMTM,KIMBKIMB, WSLM,0.5*MORB

Σ MILB, MORB,KGHB, SUMB, MBCB,GRPB, NFRB

1 April Snow WaterEquivalent (SWE)

Original Libby Forecast“Split-Basin” Regression

Equations


Fort Steele Regression ModelFort Steele Regression Model

1.3091.309 FROFRO + 0.067+ 0.067 SWESWE + 0.068+ 0.068 WPWP + 0.167+ 0.167 SPSP –– 5.1145.114

RR22=.914=.914 Sept 1 Forecast Std Error=213 KAFSept 1 Forecast Std Error=213 KAF

Libby Local Regression ModelLibby Local Regression Model

0.9210.921 FROFRO + 0.046+ 0.046 SWESWE + 0.086+ 0.086 WPWP + 0.152+ 0.152 SPSP –– 4.1834.183

RR22=.874=.874 Sept 1 Forecast Std Error=262 KAFSept 1 Forecast Std Error=262 KAF

Original Libby Forecast“Split-Basin” Regression

Equations


3000

4000

5000

6000

7000

8000

9000

10000

3000 4000 5000 6000 7000 8000 9000 10000Observed Apr-Aug Runoff

Fore

cast

Apr

-Aug

Run

off

1-April ForecastForecast=Observed

Underforecast

Overforecasterrorerror

1 April Forecast Standard Error = 707 KAF

Libby Forecast PerformanceLibby Forecast Performance–– 1 April Split1 April Split--Basin RegressionBasin Regression

Apr-Aug Runoff in KAF

0

200

400

600

800

1000

1200

1400

1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May 1-JunForecast Date

Stan

dard

Err

or in

KA

F_

Original Regression (Standard Error)

Libby Water Supply Forecast usingLibby Water Supply Forecast using“Split“Split--Basin” Standard RegressionBasin” Standard Regression


Concerns with TraditionalConcerns with TraditionalRegression ModelsRegression Models

�� Subjective station selectionSubjective station selection�� Subjective station weighting/aggregatingSubjective station weighting/aggregating�� Use of “normal subsequent” variable as a surrogateUse of “normal subsequent” variable as a surrogate

for a “future value” variable (Avoid)for a “future value” variable (Avoid)�� Predictor variables are frequently highlyPredictor variables are frequently highly

intercorrelated. Intercorrelated variables produceintercorrelated. Intercorrelated variables produceinteractions and problems with the regressioninteractions and problems with the regressioncoefficients and goodnesscoefficients and goodness--ofof--fit model statistics.fit model statistics.


�� Single regression model for the entire basinSingle regression model for the entire basin�� New equation for each monthNew equation for each month�� Choose models to maintain monthChoose models to maintain month--toto--month consistency of variablesmonth consistency of variables

�� Investigate climate variablesInvestigate climate variables�� SOI and PDOISOI and PDOI

�� Eliminate intercorrelation between predictor variablesEliminate intercorrelation between predictor variables�� Optimize variable weighting and selection proceduresOptimize variable weighting and selection procedures�� Model evaluation/selection utilizes crossModel evaluation/selection utilizes cross--validationvalidation

statisticsstatistics

=>Regression variables selected and fitted utilizing NRCS=>Regression variables selected and fitted utilizing NRCSPrincipal Components regression procedurePrincipal Components regression procedure

Objectives for NewObjectives for NewForecast EquationsForecast Equations

Historic Monthly SOI1933-2002

1953-54

1943-44

2000-01

1938-39

1978-79

-5.0

-4.0-3.0

-2.0

-1.00.0

1.0

2.03.0

4.0

Jan-

1930

Jan-

1935

Jan-

1940

Jan-

1945

Jan-

1950

Jan-

1955

Jan-

1960

Jan-

1965

Jan-

1970

Jan-

1975

Jan-

1980

Jan-

1985

Jan-

1990

Jan-

1995

Jan-

2000

Year

Inde

x Va

lue

Correlation: SOI vs Subsequent Apr-Aug KAF

June Jun+Jul JJA JJAS

-0.10

0.00

0.10

0.20

0.30

0.40

0.50

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Ending Month of Index Sums

Cor

rela

tion

Coe

f (r)

1-Month SOI 2-Month SOI 3-Month SOI 4-Month SOI 6-Month SOI 8-month SOI

US Army CorpsUS Army Corpsof Engineersof EngineersNorthwestern DivisionNorthwestern Division Principal ComponentsPrincipal Components

X11 X21 X31

X12 X22 X32

X13 X23 X33

X14 X24 X34

X15 X25 X35

X16 X26 X36

… … …

X1N X2N X3N

PC11 PC21 PC31

PC12 PC22 PC32

PC13 PC23 PC33

PC14 PC24 PC34

PC15 PC25 PC35

PC16 PC26 PC36

… … …

PC1N PC2N PC3N

Observed DataObserved Data Principal ComponentsPrincipal Components

US Army CorpsUS Army Corpsof Engineersof EngineersNorthwestern DivisionNorthwestern Division Principal ComponentsPrincipal Components

�� Creates surrogate variables (principal components) thatCreates surrogate variables (principal components) thatare a weighted combination of the original variables.are a weighted combination of the original variables.

�� The principal components have the property of beingThe principal components have the property of beingfully independent of one another (zero intercorrelation)fully independent of one another (zero intercorrelation)

�� Most of the “variability” in the predictor variables isMost of the “variability” in the predictor variables isloaded into the first one or two components.loaded into the first one or two components.

�� EigenvaluesEigenvalues reflect the proportion of the variability inreflect the proportion of the variability inoriginal variables loaded into each componentoriginal variables loaded into each component

�� Note: PCs combine the informationNote: PCs combine the information within the predictorwithin the predictorvariablesvariables, but “have no knowledge” of the dependent, but “have no knowledge” of the dependentvariable, the variable to be forecasted.variable, the variable to be forecasted.


Principal ComponentsPrincipal ComponentsRegressionRegression

3*2*1* 3210 XXXY ββββ +++=

X11 X21 X31

X12 X22 X32

X13 X23 X33

X14 X24 X34

X15 X25 X35

X16 X26 X36

… … …

X1N X2N X3N

Observed DataObserved Data

PC11 PC21 PC31

PC12 PC22 PC32

PC13 PC23 PC33

PC14 PC24 PC34

PC15 PC25 PC35

PC16 PC26 PC36

… … …

PC1N PC2N PC3N

Principal ComponentsPrincipal Components3*2*1* 3210 PCPCPCY ββββ +++=

Traditional Regression ModelTraditional Regression Model

Principal ComponentPrincipal ComponentRegression ModelRegression Model


Principal ComponentsPrincipal ComponentsRegressionRegression

�� Properties of the Principal Component Regression ModelProperties of the Principal Component Regression Modelwith all “P” Components (“p”= # of original variables)with all “P” Components (“p”= # of original variables)�� Component RComponent R--squared valuessquared values�� Component RComponent R--squared loadingsquared loading�� Eigenvalue loadingEigenvalue loading

�� 7 original variables7 original variables --> 7 principal components:> 7 principal components:PC Analysis PC_1 PC_2 PC_3 PC_4 PC_5 PC_6 PC_7R-Square 0.84144 0.00383 0.00040 0.00003 0.00495 0.00336 0.00068R-Square % 98.5% 0.4% 0.0% 0.0% 0.6% 0.4% 0.1%Cumul R-Sqr 0.8414 0.8453 0.8457 0.8457 0.8506 0.8540 0.8547

Eigenvalues 4.41 0.80 0.74 0.55 0.36 0.09 0.04

Example using SOI, 2 Precip & 4 snow variablesExample using SOI, 2 Precip & 4 snow variables


Principal Components RegressionPrincipal Components Regression

�� Variable Selection (Which components do I keep?)Variable Selection (Which components do I keep?)�� Component retention criteria:Component retention criteria:

�� Eigenvalue: provides the proportion of variability of XEigenvalue: provides the proportion of variability of Xvariables contained in each PCvariables contained in each PC

�� significant Rsignificant R--squared: indicates the variability in the Ysquared: indicates the variability in the Yvariable explained by this PC in a linear model, i.e. thevariable explained by this PC in a linear model, i.e. theusefulness of this PC in predicting the Y variableusefulness of this PC in predicting the Y variable

�� significantsignificant Beta:Beta: reject this component when the regressionreject this component when the regressioncoefficient is indistinguishable from zero.coefficient is indistinguishable from zero.

�� sign ofsign of BetaBeta: be wary of this component if the sign is negative: be wary of this component if the sign is negative(applies to water supply forecasting)(applies to water supply forecasting)


Comparing and Evaluating ModelsComparing and Evaluating Models� Be cautious of statistical models that have too many

variables in comparison to the number of observations(years of data).

� Fitting too many variables leads to a model that is“overfit”, i.e. is not parsimonious. Overfit modelsusually produce poor forecasts!

� Both the Adjusted R-square and Standard Errorstatistics are useful in comparing models, as theyinclude a “degrees-of-freedom adjustment” to accountfor the number of coefficients used to fit the regressionmodel to the data.

�� Be cautious of statistical models that have too manyBe cautious of statistical models that have too manyvariables in comparison to the number of observationsvariables in comparison to the number of observations(years of data).(years of data).

�� Fitting too many variables leads to a model that isFitting too many variables leads to a model that is““overfitoverfit”, i.e. is not parsimonious.”, i.e. is not parsimonious. OverfitOverfit modelsmodelsusually produce poor forecasts!usually produce poor forecasts!

�� Both theBoth the Adjusted RAdjusted R--squaresquare andand Standard ErrorStandard Errorstatistics are useful in comparing models, as theystatistics are useful in comparing models, as theyinclude a “degreesinclude a “degrees--ofof--freedom adjustment” to accountfreedom adjustment” to accountfor the number of coefficients used to fit the regressionfor the number of coefficients used to fit the regressionmodel to the data.model to the data.

-4 -2 0 2 4-3 -1 1 3 5Principal Component No.1 for 1-April Variables

2000

3000

4000

5000

6000

7000

8000

9000

10000

Run

off(

KA

F)

Standard Deviation = 1408 KAF

Libby Apr-Aug Runoffand Deviation from Average

-4 -2 0 2 4-3 -1 1 3 5Principal Component No.1 for 1-April Variables

2000

3000

4000

5000

6000

7000

8000

9000

10000

Run

o ff(

KA

F)

Standard Error = 537 KAF

Libby Apr-Aug RunoffD viation from r

Standard ErrorStandard Error

withn

eStdDev

n

ii

,1

1

2)(

−=∑

=_

)()( YYe ii −=

withpn

eErrorStd

n

ii

,1

2)(

−=∑

=^

)()( YYe ii −=

}{

US Army CorpsUS Army Corpsof Engineersof EngineersNorthwestern DivisionNorthwestern Division Model Comparison:Model Comparison:

Validation StatisticsValidation Statistics

�� Calibration statisticsCalibration statistics reflect the errors of the modelreflect the errors of the modeloptimized to fit to a given set of data.optimized to fit to a given set of data.

�� Adjusted RAdjusted R--SquareSquare andand Standard ErrorStandard Error are both statisticsare both statisticsof the calibration model.of the calibration model.

�� Validation statisticsValidation statistics reflect the errors of the calibratedreflect the errors of the calibratedmodelmodel being applied to data not used in the calibration.being applied to data not used in the calibration.

�� Calibration statisticsCalibration statistics tend to be overly optimistic.tend to be overly optimistic.Forecast models are best suited to be evaluated andForecast models are best suited to be evaluated andcompared based oncompared based on validation statisticsvalidation statistics..


Model validationModel validation

Calibration Data Validation Data

Fit your model to this data Compute your errorstatistics using this data

Calibration DataX

est

X X

est est

�� SplitSplit--Sample validationSample validation

�� LeaveLeave--oneone--out validationout validation


CrossCross--Validation Standard ErrorValidation Standard Error

�� CrossCross--validation standard error (“Jackknife” Std Err)validation standard error (“Jackknife” Std Err)�� CVSE supports model parsimony by includingCVSE supports model parsimony by including dd--ff adj.adj.�� CVSE better indicator of how the model performs withCVSE better indicator of how the model performs with

data not used in the calibration, that it “hasn’t seen yet”data not used in the calibration, that it “hasn’t seen yet”�� CVSE = PRESS statistic adjusted for degreesCVSE = PRESS statistic adjusted for degrees--ofof--freedomfreedom�� CVSE can be directly calculated from either ProjectionCVSE can be directly calculated from either Projection

matrix or “Hat” matrix (calculated by NRCS PCREG)matrix or “Hat” matrix (calculated by NRCS PCREG)

pne

CVSE i

−= ∑ 2

)(Where e(i) is the forecast errorfor the leave-one-out forecastof observation i

Apr-Aug Runoff in KAF

0

200

400

600

800

1000

1200

1400

1-Nov 1-Dec 1-Jan 1-Feb 1-Mar 1-Apr 1-May 1-JunForecast Date

Stan

dard

Err

or in

KA

F_ Principal Components Regression (CVSE)Standard Regression (Standard Error)

Libby Water Supply Forecast usingLibby Water Supply Forecast usingPrincipal Components RegressionPrincipal Components Regression

US Army CorpsUS Army Corpsof Engineersof EngineersNorthwestern DivisionNorthwestern Division EndorsementsEndorsements

The following agencies use Principal Components regression in thThe following agencies use Principal Components regression in theireirWater Supply Forecasting procedures:Water Supply Forecasting procedures:

�� National Water and Climate Center, Natural ResourcesNational Water and Climate Center, Natural ResourcesConservation ServiceConservation Service

�� Northwestern Division, U.S. Army Corps of EngineersNorthwestern Division, U.S. Army Corps of Engineers�� Northwest River Forecasting Center, National OceanicNorthwest River Forecasting Center, National Oceanic

and Atmospheric Administrationand Atmospheric Administration�� Columbia River Treaty Operating Committee; CanadianColumbia River Treaty Operating Committee; Canadian

and United States Entitiesand United States Entities�� Bonneville Power Administration, Dept of EnergyBonneville Power Administration, Dept of Energy�� BC HydroBC Hydro


Questions?Questions?

Improved Water Supply Forecasts for the Kootenay Basin · US Army Corps of Engineers Northwestern...

Documents

Transcript of Improved Water Supply Forecasts for the Kootenay Basin · US Army Corps of Engineers Northwestern...