Waterflooding optimization in uncertain geological...

Comput GeosciDOI 10.1007/s10596-013-9371-1

ORIGINAL PAPER

Waterflooding optimization in uncertain geological scenarios

Andrea Capolei · Eka Suwartadi · Bjarne Foss ·John Bagterp Jørgensen

Received: 21 March 2013 / Accepted: 21 August 2013© Springer Science+Business Media Dordrecht 2013

Abstract In conventional waterflooding of an oil field,feedback based optimal control technologies may enablehigher oil recovery than with a conventional reactivestrategy in which producers are closed based on waterbreakthrough. To compensate for the inherent geologicaluncertainties in an oil field, robust optimization has beensuggested to improve and robustify optimal control strate-gies. In robust optimization of an oil reservoir, the waterinjection and production borehole pressures (bhp) are com-puted such that the predicted net present value (NPV) of anensemble of permeability field realizations is maximized.In this paper, we both consider an open-loop optimizationscenario, with no feedback, and a closed-loop optimizationscenario. The closed-loop scenario is implemented in amoving horizon manner and feedback is obtained using anensemble Kalman filter for estimation of the permeabilityfield from the production data. For open-loop implementa-tions, previous test case studies presented in the literature,

A. Capolei · J. B. Jørgensen (�)Department of Applied Mathematics and Computer Scienceand Center for Energy Resources Engineering,Technical University of Denmark (DTU),2800 Kgs. Lyngby, Denmarke-mail: [email protected]

A. Capoleie-mail: [email protected]

E. Suwartadi · B. FossDepartment of Engineering Cybernetics,Norwegian University of Science and Technology (NTNU),7491 Trondheim, Norway

E. Suwartadie-mail: [email protected]

B. Fosse-mail: [email protected]

show that a traditional robust optimization strategy (RO)gives a higher expected NPV with lower NPV standarddeviation than a conventional reactive strategy. We presentand study a test case where the opposite happen: The reac-tive strategy gives a higher expected NPV with a lower NPVstandard deviation than the RO strategy. To improve the ROstrategy, we propose a modified robust optimization strat-egy (modified RO) that can shut in uneconomical producerwells. This strategy inherits the features of both the reactiveand the RO strategy. Simulations reveal that the modifiedRO strategy results in operations with larger returns andless risk than the reactive strategy, the RO strategy, and thecertainty equivalent strategy. The returns are measured bythe expected NPV and the risk is measured by the stan-dard deviation of the NPV. In closed-loop optimization, weinvestigate and compare the performance of the RO strategy,the reactive strategy, and the certainty equivalent strategy.The certainty equivalent strategy is based on a single real-ization of the permeability field. It uses the mean of theensemble as its permeability field. Simulations reveal thatthe RO strategy and the certainty equivalent strategy give ahigher NPV compared to the reactive strategy. Surprisingly,the RO strategy and the certainty equivalent strategy givesimilar NPVs. Consequently, the certainty equivalent strat-egy is preferable in the closed-loop situation as it requiressignificantly less computational resources than the robustoptimization strategy. The similarity of the certainty equiv-alent and the robust optimization based strategies for theclosed-loop situation challenges the intuition of most reser-voir engineers. Feedback reduces the uncertainty and this isthe reason for the similar performance of the two strategies.

Keywords Robust optimization · Ensemble Kalmanfilter · Oil reservoir · Production optimization · Automatichistory matching

mailto:[email protected]




Comput Geosci

Mathematics Subject Classifications (2010) 90C30 ·49M37 · 49J20

1 Introduction

In the oil industry, closed-loop reservoir management(CLRM) has been suggested to maximize oil recovery or afinancial measure such as the net present value of a givenoil reservoir [1–12]. Fig. 1 illustrates the components inclosed-loop reservoir management. The controller consistsof model based data assimilation, also known as a param-eter and state estimator, and a model based optimizer formaximizing the oil recovery or some predicted financialmeasure such as the net present value. The inputs to thecontroller is production measurements, forecasts of the oilprice, the interest rate, and the operating unit costs. Basedon these inputs the controller computes water injection tra-jectories as well as borehole pressure trajectories. Only thefirst part of these trajectories are implemented in the realoil reservoir. As new measurements become available, theprocess is repeated. The parameters and the states of themodel are re-estimated using the data assimilation compo-nent. These filtered states and parameters are used in themodel based optimization for computation of optimal tra-jectories for the manipulated variables, and the first part ofthe trajectories are implemented. This form of control isalso known as Nonlinear Model Predictive Control (NMPC)[13–19]. A key difference of NMPC applied to reservoirmanagement and traditional process control applications isthe size of the model describing the system. In reservoirmanagement, spatial discretization of the partial differen-tial equation (PDE) system describing the flow results ina system of differential equations that is much larger than

Real Oil Reservoir

Reservoir models

Optimizer

Data assimilation

Measurements

Noise NoiseOutputInput

Control Input

Fig. 1 Closed-loop reservoir management

the systems typically encountered in process control appli-cations. The large-scale nature of the closed-loop reservoirmanagement problem requires special numerical techniquesfor the data assimilation [20] as well as the optimization[21, 22].

In this paper, we study and discuss two closed-loopapproaches for real-time production optimization. Both ofthe two closed-loop approaches consist of three key ele-ments: 1) A gradient based optimization algorithm forcomputation of the control input, 2) an ensemble Kalmanfilter (EnKF) for model updating through data assimila-tion (history-matching), and 3) use of the moving horizonprinciple for data assimilation and implementation of thecomputed control input.The first closed-loop approach isa certainty equivalent strategy. In this strategy, the EnKFis used to estimate permeabilities of each member of theensemble. The average of these permeabilities is used in theoptimization. The second closed-loop approach is a strategybased on robust optimization. In this strategy, all membersof the ensemble are used to compute the mean net presentvalue for the optimization.

We know from finance that strategies which attempt toincrease returns are often accompanied by an increaseduncertainty [23]. The robust strategy typically reducesthe uncertainty of the expected outcome compared to anon-robust strategy and one would consequently expect adecrease in return [24, 25]. Figure 2 sketches this phe-nomenon. For the test cases used in the oil industry totest robust optimization strategies, this phenomenon has notbeen reported [26]. Due to the large-scale nature of an oilreservoir model, we cannot compute the entire distribu-tion of the net present value for the closed-loop system.Accordingly, the ambition in this paper is in a computationaltractable way, using a few realizations, to demonstrate theclosed-loop performance of the certainty equivalent and therobust optimization strategy.

0 5 10 15 200

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

NPV

pdf

RobustCertainty Equivalent

Fig. 2 Conceptual sketch of the distribution of the net present valuefor two strategies in finance. A robust strategy has lower variance andtypically also lower mean value than non-robust strategies such as acertainty equivalent strategy

Comput Geosci

Data assimilation by the EnKF is a popular method forhistory matching as well as closed-loop reservoir manage-ment [1, 3, 4, 6, 27]. In [28], different data assimilation andoptimization methods are tested on the synthetic ”Bruggefield” to maximize its NPV. The three best results are allobtained by methods using an EnKF for data assimilation.The EnKF method is a Monte Carlo implementation of theKalman filter [29]. The literature available on the EnKFin petroleum engineering is rather large and mature. Dataassimilation using the ensemble Kalman filter has beenreviewed by [30–32] and [20, 33, 34] provide overviewsof filtering techniques. A review of various issues of theEnKF, including sampling error because of small ensem-bles, covariance localization (limiting the influence of theobservations to the state variables that are located spatiallyclose to them), filter divergence, and model error, is givenin [31] and [30]. [35] describes the necessity of introduc-ing a confirming step to ensure consistency of the updatedstatic and dynamic variables with the flow equations, while[30] discusses the reduction of the ensemble size with aresampling scheme. The problem of ensemble collapse isdiscussed in [36]. [37] considers a way to handle modelconstraints within the EnKF. [38] investigates an updatestep that preserves multi-point statistics and not only twopoint-statistics.

In the model based optimization part of CLRM, a tra-ditional choice is to use methods based on one realization,usually the ensemble mean from the EnKF. To reduce therisk arising from uncertainty in the geological description,[26] proposes to optimize the expectation of net presentvalue over a set of reservoir models using a gradient basedmethod. This procedure is referred to as robust optimization(RO). In open-loop simulations, [26] compares the resultsof the RO procedure to two alternative approaches: a nomi-nal optimization (NO) and a reactive control approach. Theyfind that RO yields a much smaller variance than the alter-natives. Moreover the RO strategy significantly improvesthe expected NPV over the alternative methods (on aver-age 9.5 % higher than using reactive-control and 5.9 %higher than the average of NO strategies). [27, 39, 40] doclosed-loop reservoir management using an EnKF for dataassimilation and robust optimization with a gradient-freeensemble based optimization scheme for the model basedoptimization. [39] reports that an ensemble based optimiza-tion results in a NPV improvement of 22 % compared to areactive strategy. However, they do not compare the closed-loop robust strategy to a closed-loop certainty equivalentstrategy.

To our knowledge, there is no closed-loop applicationof the gradient-based robust optimization strategy as imple-mented in [26] available in the literature. Furthermore,the CLRM literature misses an open-loop as well as aclosed-loop comparison of the performance of an ensem-

ble based optimization scheme [39] or a gradient-basedrobust optimization scheme [26] with a certainty equivalentoptimization strategy based on the ensemble mean. In thiswork we partially fill this gap and do CLRM comparinga RO strategy [26] to three alternative approaches: a reac-tive strategy, a nominal strategy, and a certainty equivalentstrategy. By using feedback, the ensemble of permeabilityfields converge to a point such that the RO strategy becomesequivalent to the certainty equivalent strategy based on theensemble mean. The RO is more expensive computationallythan the certainty equivalent strategy. In this paper, we usea case study to compare the RO strategy in closed-loop toother strategies.

The paper is organized as follows. Section 2 defines thereservoir model. Section 3 states the constrained optimalcontrol problem and describes the robust optimization strat-egy. The ensemble Kalman filter for data assimilation isdescribed in Section 4. Section 5 describes the numericalcase study and conclusions are presented in Section 6.

2 Reservoir model

In this work, we assume that the reservoirs are in the sec-ondary recovery phase where the pressures are above thebubble point pressure of the oil phase. Therefore, two-phaseimmiscible flow, i.e. flow without mass transfer betweenthe two phases, is a reasonable assumption. We focus onwater-flooding cases for two-phase (oil and water) reser-voirs. Further, we assume incompressible fluids and rocks,no gravity effects or capillary pressure, no-flow boundaries,constant porosity, and finally isothermal conditions. Thestate equations in an oil reservoir �, with boundary ∂�

and outward facing normal vector n, can be represented bypressure and saturation equations. The pressure equation isdescribed as

v = −λtK∇p, ∇ · v = q in �

v · n = 0 on ∂� (1)

v is the Darcy velocity (total velocity), K is the permeability,p is the pressure, q is the volumetric well rate, and λt is thetotal mobility, which in this setting is the sum of the waterand oil mobility functions,

λt = λw(s) + λo(s) = krw(s)/μw + kro(s)/μo (2)

The saturation equation is given by

φ∂

∂tSw + ∇ · (fw(Sw)v

) = qw

ρw

(3)

φ is the porosity, s is the saturation, fw(s) is the water frac-tional flow which is defined as λw

λt, and qw is the volumetric

water rate at the well. We use the MRST [41] reservoirsimulator to solve the pressure and saturation equations,

Comput Geosci

(1) and (3), sequentially. Wells are implemented using thePeaceman well model [42]

qi = −λtWIi

(pi − pBHP

i

)(4)

qi is the flow rate into grid block i, pBHPi is the wellbore

pressure, and WIi is the Peaceman well-index.

3 Production optimization

Production optimization aims at maximizing a performanceindex, net present value or oil recovery, for the life time ofthe oil reservoir. Spatial and temporal discretization of themodel equations and the performance index yield a finitedimensional nonlinear constrained optimization problem ona time horizon from 0 to N that can be formulated as

max{xk}Nk=0,{uk}N−1

k=0

J =N−1∑

k=0

Jk(xk, xk+1, uk) (5a)

subject to

x0 = x0 (5b)

gk(xk, xk+1, uk; θ) = 0 k ∈ N := {0, 1, . . . , N − 1}(5c)

c({uk}N−1

k=0

)≤ 0 (5d)

x0 is the initial states, θ is a parameter vector in an uncer-tain space � (in our case the permeability field), xk is thestate vector, uk is a piecewise constant control vector, gk

is the discretization of the dynamical model, (1) and (3),

and c({uk}N−1

k=0

)are linear bounds on the control vector. In

our formulations we do not allow nonlinear state or outputconstraints, see e.g. [43].

3.1 Objective function

The optimizer maximizes the net present value by manipu-lating the well bhps. Hence, the manipulated variable at time

period k ∈ N is uk ={{pbhp

i,k }i∈I , {pbhp

i,k }i∈P

}with I

being the set of injectors and P being the set of producers.For i ∈ I , p

bhpi,k is the bhp (bar) in time period k ∈ N at

injector i. For i ∈ P , pbhp

i,k is the bhp (bar) at producer i intime period k ∈ N .

The stage cost, Jk , in the objective function for a netpresent value (NPV) maximization can be expressed as

Jk = − �tk

(1 + d)tk+1

τ

[∑

i∈P

ro qk+1o,i (uk, xk+1)

−∑

i∈P

rwp qk+1w,i (uk, xk+1)+

∑

l∈I

rwi qk+1l (uk, xk+1)

]

(6)

ro, rwp, and rwi represent the oil price, the water separationcost, and the water injection cost, respectively. The waterflow rate (bbl/day) in producer i at time period k is qk

w,i =fwqk

i and the oil flow rate is qko,i = (1 − fw)qk

i . qki is the

flow rate at producer i as given by (4). The well rates atthe injector wells are denoted by ql (only water is injected).Note that from the well model (4), it follows that the flowrates q are negative for the producer wells and positive forthe injector wells. d is the discount factor, �tk is the timeinterval, and N is the number of control steps. Note that inthe special case when the discount factor is zero (d = 0) andthe water injection and separation costs are zero as well, theNPV is equivalent to the quantity of produced oil.

3.2 Control and constraints

We control the bhp of the wells and assume that these con-trol inputs are piecewise constant functions. The bhps areconstrained by well and reservoir conditions. To maintainthe two phase situation we require the pressure to be abovethe bubble point pressure (290 bar). To avoid fracturing therock, the pressure must be below the fracture pressure ofthe rock (350 bar). To maintain flow from the injectors tothe producers, the injection pressure is maintained above310 bar and the producer pressures are kept below 310bar. With these bounds we did not experience that the flowwas reversed. Without these pressure bounds, however, stateconstraints, like bounds on flow rates (4), must be appliedto avoid flow reversion.

3.3 Single-shooting optimization

We use a single shooting algorithm [11, 44] for solu-tion of (5a). Alternatives are multiple-shooting [45, 46]and collocation methods [47]. Despite the fact that themultiple shooting and the collocation methods offer bet-ter convergence properties than the single-shooting method[45–47], their application in production optimization isrestricted by the large state dimension of such prob-lems. The use of multiple-shooting is prevented by theneed for computation of state sensitivities. The colloca-tion method do not allow for adaptive time stepping andwould need to solve huge-scale optimization problems. In

Comput Geosci

the single shooting optimization algorithm, we define thefunction

ψ = ψ({uk}N−1

k=0 ; x0, θ)

={J =

N−1∑

k=0

Jk(xk, xk+1, uk) :

x0 = x0, gk(xk, xk+1, uk; θ) = 0, k ∈ N

}(7)

such that (5a) can be expressed as the optimization problem

max{uk}N−1

k=0

ψ = ψ({uk}N−1

k=0 ; x0, θ)

s.t. c({uk}N−1

k=0

)≤ 0 (8)

Gradient based optimization algorithms for solving (8)

require evaluation of ψ = ψ({uk}N−1

k=0 ; x0, θ)

, ∇ukψ for

k ∈ N , c({uk}N−1

k=0 , and ∇uk c({uk}N−1

k=0

)for k ∈ N . The

constraint function is in the cases considered linear boundssuch that evaluation of these constraint functions and theirgradients is trivial. Given an iterate, {uk}N−1

k=0 , ψ is com-puted by solving (7) marching forwards. ∇ukψ for k ∈ Nis computed by the adjoint method [9, 11, 43, 48–51]. Inthis method, the gradients {∇ukψ}N−1

k=0 are computed usingAlgorithm 1 with

∇xk gk = ∇xk gk(xk, xk+1, uk; θ) k ∈ N (9a)

∇xk+1gk = ∇xk+1gk(xk, xk+1, uk; θ) k ∈ N (9b)

∇ukgk = ∇ukgk(xk, xk+1, uk; θ) k ∈ N (9c)

∇xk Jk = ∇xk Jk(xk, xk+1, uk) k ∈ N \ {0} (9d)

∇xk+1Jk = ∇xk+1Jk(xk, xk+1, uk) k ∈ N (9e)

∇ukJk = ∇ukJk(xk, xk+1, uk) k ∈ N (9f)

that are computed and stored during the forward solutionof (7).

Algorithm 1 Adjoint method for computing {∇ukψ}N−1k=0 .

Solve for λN in ∇xNgN−1λN = ∇xN

JN−1fork = N − 1,N − 2, . . . , 1 do

Compute ∇ukψ = ∇uk

Jk − ∇ukgkλk+1

Solve for λk in ∇xkgk−1λk = ∇xk

Jk−1 +∇xkJk −∇xk

gkλk+1end forCompute ∇u0ψ = ∇u0J0 − ∇u0g0λ1

To solve (8), we use two commercial optimizationsoftware packages: Knitro [52] and Matlab’s fminconfunction [53]. Knitro as well as fmincon, allows us touse an interior point or an active-set methods. We useup to 10 different initial guesses when running the opti-mizations and we find similar qualitative results with bothsoftware packages. Further, similar results are found withinterior point and active-set methods. When using Knitroas well as fmincon, we select an interior point methodsince we experience the lowest computation times with thismethod. A local optimal solution is reported if the KKTconditions are satisfied to within a relative and absolute tol-erance of 10−6. The current best but non-optimal iterate isalso returned in cases when the optimization algorithm usesmore than 100 iterations. Similarly, the current best, butnon-optimal, iterate is also returned in the case of a relativecost function or step size change less than 10−8. Further-more, in our simulations we noted that normalizing the costfunction improved the convergence.

3.4 Certainty-equivalent and robust optimization

In certainty equivalent optimization, (8) is solved using theexpected value for the parameters, η = E[θ ], such that theobjective used in (8) is

ψCE = ψ({uk}N−1

k=0 ; x0, η)

(10)

[26] introduces robust optimization to reduce the effectof geological uncertainties in the field development phase.Robust optimization uses a set of realizations that reflectthe range of possible geological structures honoring thestatistics of the geological uncertainties. In reservoir mod-els, geological uncertainty is generally profound because ofthe noisy and sparse nature of seismic data, core samples,and borehole logs. The consequence of a large number ofuncertain model parameters (θ ) is the broad range of pos-sible models that may satisfy the seismic and core-sampledata. Nevertheless, in many cases, a single reservoir modelis adopted in which the uncertain parameters, θ , are con-verted to deterministic parameters η by taking their expectedvalues (i.e. η = E[θ ]). However, because the NPV isused as our measure of performance, we are more inter-ested in the expected NPV over the uncertainty space, �

(spanned by the uncertain parameters θ ), than the meanof the parameters. The expected NPV over the uncertaintyspace, �, is in general not the same as the NPV com-puted using the expected values of the uncertain parameters,η = E[θ ], as NPV is a nonlinear function of the parameters,θ . Consequently

Eθ

[ψ({uk}N−1

k=0 ; x0, θ)]

�= ψ({uk}N−1

k=0 ; x0, Eθ [θ ])

, θ ∈ � (11)

Comput Geosci

Consider a discretization, �d = {θ1, . . . , θnd }, of theuncertainty space, �, such that expected NPV may beapproximated by

Eθ

[ψ({uk}N−1

k=0 ; x0, θ ∈ �)]

≈ Eθ

[ψ({uk}N−1

k=0 ; x0, θ ∈ �d

)](12)

This approximation of the expected NPV is a better approx-imation than the NPV computed using the expected param-eters, η. In the special case of equiprobable realizations, theright-hand side of (12) is the arithmetic average

Eθ

[ψ({uk}N−1

k=0 ; x0, θ ∈ �d

)]

= 1

nd

nd∑

i=1

ψ({uk}N−1

k=0 ; x0, θi)

(13)

The robust optimization method uses the ensemble averageas its objective function in (8)

ψ rob = 1

nd

nd∑

i=1

ψ({uk}N−1

k=0 ; x0, θi)

= 1

nd

nd∑

i=1

ψ i (14)

The corresponding gradients may be computed by

∇ukψ rob = 1

nd

nd∑

i=1

∇ukψ({uk}N−1

k=0 ; x0, θi)

k ∈ N

(15)

Compared to a certainty-equivalent computation, the com-putation of the robust cost function (14) and its gradient (15)results in an increased computational effort by a factor nd .As the addends in these computations are decoupled, theycan be computed in parallel.

3.5 Permeability field

In our study, the uncertainty lies in the permeability field.We generate 100 permeability field realizations of a 2Dreservoir in a fluvial depositional environment with a knownvertical main-flow direction, see Fig. 3. To generate the per-meability fields we started by creating a set of 100 binary(black and white) training images by using the sequentialMonte Carlo algorithm ’SNESIM’ [54]. Then a Kernel PCA[55] procedure is used to preserves the channel structuresand smooths the original binary images. The realizations soobtained are quite heterogeneous with permeabilities in therange 6 − 2734 mD.

4 Ensemble Kalman filter

We use the Ensemble Kalman filter (EnKF) for estimatingthe permeability field based on production data measure-ments. The EnKF is a Monte Carlo implementation of the

Kalman filter [13, 56–58] using an ensemble of nd realiza-tions to represent the necessary first and second moments(means and covariances). In this section we describe theEnKF.

Consider the discrete time system

xk+1 = F(xk,uk,θ ) (16a)

yk = G(xk,uk ) + vk vk ∼ N(0, R) (16b)

The dynamic equation (16a) is a representation of the modeldynamics (5c) in explicit form. It should be noted that in thisrepresentation we do not consider stochastic model errors(process noise) [20]. The uncertain parameters θ are the log-arithm of the permeability field, θ = log(K). The statesare the pressure and water saturation in each grid block,x = [P ; Sw]. The initial states, x0, can also be consid-ered uncertain. However, in this work we fix them to theiraverage value. uk is the control input which represents theborehole pressures.

The measurements, y, are the fractional flow for each pro-ducer well and the water injection rate for each injector well.In the measurement equation (16b), G(xk, uk) includes thePeaceman well model (4) as well as the equations relat-ing fractional flow to pressures and water saturations. vk ismeasurement noise that we assume is normally distributed.

4.1 Basic Ensemble Kalman filter

(16a) includes the states, x, and the parameters, θ . There-fore, we form the augmented state space model

xk+1 = F(xk, uk, θk) (17a)

θk+1 = θk (17b)

and apply the EnKF to the dynamic equation (17a) and themeasurement equation (16b). In the EnKF all means andcovariances are represented by samples of the stochasticvariables. Therefore, the initial mean and covariance of theaugmented states, [xk, θk], are represented by

{xi0|0, θ

i0|0}nd

i=1 ={x0, θ

i0|0}nd

i=1(18)

It should be noted that the initial states, x0, in our case areassumed to be known exactly. Only the parameters, θ , areuncertain. Index i refers to each of the nd members of theensemble, i.e. each realization.

In the following we describe the algorithm for discretetime instant k = 1, 2, . . .. In general, at discrete time instantk, both the states and the parameters from the previousinstant, k − 1, are uncertain. This is denoted{xik−1|k−1, θ

ik−1|k−1

}nd

i=1k = 1, 2, . . . (19)

Comput Geosci

Fig. 3 A selection of realizations from the ensemble of 100 permeability fields. The realizations are quite heterogeneous, values are in the range6 − 2734 mD

In the EnKF, the one-step prediction step is conducted bypassing each ensemble member through the dynamics (17a)such that for i = 1, 2, . . . , nd

xik|k−1 = F

(xik−1|k−1, uk−1, θ

ik−1|k−1

), (20a)

θ ik|k−1 = θ i

k−1|k−1, (20b)

where the previous input, uk−1, is known. Then the output,zik|k−1, and the measurement, yi

k|k−1, at discrete time k maybe computed as

zik|k−1 = G

(xik|k−1, uk−1

)i = 1, 2, . . . , nd (21a)

yik|k−1 = zi

k|k−1 + vik i = 1, 2, . . . , nd (21b)

To obtain the correct covariances of the state estimates in theEnKF, it is important that each ensemble member, yi

k|k−1,

contain measurement noise, vik|k−1 [59]. It should also be

noted that uk−1 is used in the evaluation of G in (21a). Theexplanation for the use of uk−1 is that we use a zero-order-hold representation of u(t), i.e. u(t) = uk−1 for tk−1 ≤ t <

tk , and that we assume the measurement is conducted at timet−k = limt<tk t . Then, at time tk , the EnKF and optimal con-trol computations are conducted infinitely fast such the nextdecisions, u(t) = uk for tk ≤ t < tk+1, can be implementedat time tk .

The innovation, eik , for each ensemble member is com-

puted using the actual measurement, yk , and the predictedmeasurement

εik = zi

k|k−1 − yk i = 1, 2, . . . , nd (22a)

eik = yk − yi

k|k−1 = −εik − vi

k i = 1, 2, . . . , nd (22b)

In these equations, yk is the actual measurement and there-fore a deterministic variable. In the EnKF, the realizedtrajectory of the system and an ensemble of different

Comput Geosci

Table 1 Parameters for thetwo phase model, thediscounted state cost function(6), and the measurement noise

Symbol Description Value Unit

φ Porosity 0.2 −cr Rock compressibility 0 Pa−1

ρo Oil density (300 bar) 700 kg/m3

ρw Water density (300 bar) 1000 kg/m3

μo Dynamic oil viscosity 3 · 10−3 Pa · s

μw Dynamic water viscosity 0.3 · 10−3 Pa · s

Sor Residual oil saturation 0.1 −Sow Connate water saturation 0.1 −no Corey exponent for oil 2 −nw Corey exponent for water 2 −Pinit Initial reservoir pressure 300 bar

Sinit Initial water saturation 0.1 −ro Oil price 120 USD/bbl

rwp Water production cost 20 USD/bbl

rwi Water injection cost 10 USD/bbl

d Discount factor 0 −R Cov. matrix for measurements noises Diag( 5 · 10−3, 5 · 10−3, 5 · 10−3, 5 · 10−3, 30 )

state trajectories are considered. In the derivation of thestandard Kalman filter [13, 57, 58], it is the other wayaround. A (infinite) number of system realizations are con-sidered, while the filter is represented by one deterministictrajectory (the mean).

The optimal linear estimator conditioned on the innova-tions are [57]

xik|k = xi

k|k−1 + Kx,keik i = 1, 2, . . . , nd (23a)

θ ik|k = θ i

k|k−1 + Kθ,keik i = 1, 2, . . . , nd (23b)

with the Kalman filter gains computed as

Kx,k = 〈xk|k−1, ek〉〈ek, ek〉−1 (24a)

Kθ ,k = 〈θk|k−1, ek〉〈ek, ek〉−1 (24b)

using the covariances

〈xk|k−1, ek〉 = 〈xk|k−1, εk〉 (25a)

〈θk|k−1, ek〉 = 〈θk|k−1, εk〉 (25b)

〈ek, ek〉 = 〈εk, εk〉 + 〈vk, vk〉 ≈ 〈εk, εk〉 + R (25c)

The Kalman gains may be based on direct computation ofthe empirical estimates (〈xk|k−1, ek〉, 〈θk|k−1, ek〉, 〈ek, ek〉)or the relations in (25a). We choose to base the computations

on (25a), the approximate first moments (means) computedas

zk|k−1 = E{zk|k−1} ≈ 1

nd

nd∑

i=1

zik|k−1

εk = E{εk} ≈ 1

nd

nd∑

i=1

εik = zk|k−1 − yk

xk|k−1 = E{xk|k−1} ≈ 1

nd

nd∑

i=1

xik|k−1

θk|k−1 = E{θk|k−1} ≈ 1

nd

nd∑

i=1

θ ik|k−1

log10(K) [Darcy]

−2

−1.5

−1

−0.5

0

Fig. 4 Permeability mean, θ0|0, of the ensemble given in Fig. 3

Comput Geosci

Table 2 Key indicators for theopen-loop optimized cases.Improvements are relative tothe nominal case

NO Reactive Certainty equivalent RO RO modified

106 USD 106 USD, % 106 USD, % 106 USD, % 106 USD, %

Eθ [ψ] 33.84 56.47, +66.9 42.72, +26.2 44.11, +30.3 58.18, +71.9

Std. dev. 26.35 6.05 18.27 13.19 6.25

and the approximate second moments (covariances) com-puted by

〈xk|k−1, εk〉 ≈ 1

nd − 1

nd∑

i=1

(xik|k−1 − xk|k−1

) (εi

k − εk

)′

〈θk|k−1, εk〉 ≈ 1

nd − 1

nd∑

i=1

(θ i

k|k−1 − θk|k−1

)(εi

k − εk

)′

〈εk, εk〉 ≈ 1

nd − 1

nd∑

i=1

(εi

k − εk

) (εi

k − εk

)′

The result of (23a) in this procedure is an ensemble{xik|k, θ

ik|k}nd

i=1k = 1, 2, . . . (28)

representing the states and parameters at time k given mea-surements up until time k. Using this ensemble, a robustoptimization may be performed or various statistics such asthe mean may be computed.

(23a) may result in non-physical updates. Therefore, wemodify the EnKF such that the ensemble (28) satisfiesphysical constraints, e.g. that the permeabilities are in cer-tain ranges. To mitigate such effects, we clip the solutionaccording to the constraints

θ ik|k :=

⎧⎪⎨

⎪⎩

θmin θ ik|k < θmin

θ ik|k θmin ≤ θ i

k|k ≤ θmax

θmax θ ik|k > θmax

(29)

and compute the filtered states, xik|k , by solving the dynamic

model equations

xij+1|k = F

(xij |k, uj , θ

ik|k)

, xi0|k = x0,

j = 0, 1, . . . , k − 1 (30)

for each ensemble member, i ∈ {1, . . . , nd }, using theclipped parameter estimates computed by (29). In this way,

state updates consistent with the model is guaranteed [35].In particular, this eliminates the possibility of nonphysicalstates (nonphysical pressures and saturations). The compu-tational load can potentially be reduced by only doing theinitial-value simulation when the estimated saturation andpressure changes passes a certain threshold [39]. The mod-ifications (29) and (30) provides the ensemble (28) that isused for the optimal control computations and for the initi-ation of the EnKF at the next time step. Finally, the choiceof the ensemble size nd in the EnKF is a topic of researchitself [60]. It affects the performance of the filter. In reser-voir engineering an ensemble’s size of 100 is a commonchoice based on experience [4], [61]. However, this numberis problem dependent and in some cases good results canalso be obtained using ensembles with fewer members [61].

4.2 Performance metrics

To measure the convergence of the Kalman filter estimates,we consider the mean standard deviation

σk =√√√√ 1

np

(1

nd − 1

nd∑

i=1

∥∥∥θ ik|k − θk|k

∥∥∥2

2

)

(31)

of the parameters in the parameter vector, θk|k . σk mea-sures the ensemble spread. We also consider the root-mean-square-error of the parameter estimates compared to the trueparameters, θ0:

RMSEk =∥∥∥θk|k − θ0

∥∥∥2√

np

(32)

θk can be computed for real as well as synthetic cases, whileRMSEk can only be computed for synthetic cases in whichthe true parameters, θ0, are available.

Fig. 5 log10 K [D] of the firsttwo realizations of the ensemblein Fig.3 and their ensemblemean θ0|0

p2

p4

i1

p1

p3

(1) First realization

p2

p4

i1

p1

p3

(2) Second realization

p2

p4

i1

p1

p3

−2

−1.5

−1

−0.5

0

(3) Ensemble mean

Comput Geosci

In the ideal case, the spread (31) should converge toa number related to the measurement noise and the root-mean-square-error (32) should converge to 0. In practice,(32) will not converge to zero due to e.g. factors likemodel-plant mismatch. Cases with divergence of the root-mean-square-error may indicate that the ensemble is toosmall to represent the true uncertainty.

5 Case study

We consider a conventional horizontal oil field that can bemodeled as a two phase flow in a porous medium [62–64].The reservoir size is 450 m × 450 m × 10 m. By spatial dis-cretization this reservoir is divided into 45 × 45 × 1 gridblocks. The permeability field is uncertain. We assume thatthe ensemble in Fig. 3 represents the range of possible geo-logical uncertainties. The configuration of injection wellsand producers is illustrated in Fig. 5(1). As indicated inFig. 5(1), the four producers are located in the corners ofthe field, while the single injector is located in the center ofthe field.

The reservoir’s petrophysical parameters are listed inTable 1. The initial reservoir pressure is 300 bar every-where in the reservoir. The initial water saturation is 0.1everywhere in the reservoir. This implies that initially,the reservoir has a uniform oil saturation of 0.9. Themanipulated variables are the bhp of the five wells (fourproducers, one injector) over the life of the reservoir.In this study, we consider a zero discount factor d inthe cost function (6). This means that we maximize NPVat the final time, without caring about the shorter horizon[11].

The case study is divided into an open-loop optimiza-tion part and a closed-loop optimization part. In open-loopoptimization, we compute the control strategy without usingmeasurement feedback to update the parameters, i.e. theensemble in Fig. 3 is fixed in time. In closed-loop opti-mization, we use production measurements and the EnKFto estimate the permeability field parameters. To simulatethe reservoir and create production data, the first realizationof the permeability field, θ1

0|0, in Fig. 3 is used. This per-meability field represents the true permeability field of thereservoir.

In reality, we never know the true model when perform-ing data assimilation with EnKF. We can only implicitlyassume that we can generate a reasonable approxima-tion of the true reservoir. Since we focus on the opti-mizer formulation and separate the effects of the qual-ity in data assimilation from the quality of CE, RO andreactive strategies as much as possible, we assume thatthe true reservoir is contained in the ensemble of initialguesses.

200 400 600 800 1000 1200 14000

10

20

30

40

50

60

time [days]

ψ1 (

mill

ion

US

D)

Optimal ψ1

ReactiveROModified RO

(1) First realization

200 400 600 800 1000 1200 14000

10

20

30

40

50

60

time [days]

ψ2 (

mill

ion

US

D)

Optimal ψ2


(2) Second realization

200 400 600 800 1000 1200 14000

10

20

30

40

50

60

time [days]

E[ψ

] (m

illio

n U

SD

)


(3) Combined NPV

Fig. 6 Profit evolutions for open-loop optimization of the two ensem-ble case. The optimal trajectories computed using the true permeabilityfields give the highest possible profit. The profit of the RO strategyis below the profit of the reactive strategy for the first permeabilityrealization and slightly above the second permeability realization. Onaverage the RO strategy gives less profit than the reactive strategy. Themodified RO strategy produces for all cases a higher profit than thereactive strategy

5.1 Open-loop optimization

We consider a prediction horizon of tN = 4 · 365 = 1460days divided in N = 60 control periods (i.e. a control

Comput Geosci

0 200 400 600 800 1000 1200 1400290

295

300

305

310bh

p (b

ar)

Producer 1

0 200 400 600 800 1000 1200 1400290

295

300

305

310

bhp

(bar

)

Producer 2

0 200 400 600 800 1000 1200 1400290

295

300

305

310

bhp

(bar

)

Producer 3

0 200 400 600 800 1000 1200 1400290

295

300

305

310

bhp

(bar

)

Producer 4

0 200 400 600 800 1000 1200 1400310

320

330

340

350

bhp

(bar

)

time (days)

Injector1

Optimal ψ1

Optimal ψ2

ROmodified RO

Fig. 7 Control trajectories for open-loop optimization of the twoensemble case. The control trajectories for the considered optimizationstrategies are very different

period Ts ≈ 24 days). We control the reservoir using fivedifferent strategies that we call: the reactive strategy, thenominal strategy (NO), the certainty equivalent strategy, therobust optimization strategy (RO), and the modified robustoptimization strategy (modified RO).

The reactive strategy develops the field at the maximumproduction rate (setting the producers at the lowest allowedvalue of 290 bar and the injector at the maximum allowedvalue of 350 bar) and subsequently shut-in each productionwell when it is no longer economical. From the values inTable 1, we observe that a producer well becomes uneco-nomical when the fractional flow fw is above the value120/(120 + 20) = 0.857. The nominal strategy is basedon a single realization. For each realization in the ensemblewe compute the optimal control trajectory. Then we applyeach of these 100 optimal control trajectories to each ofthe ensemble members obtaining 100 NPV values for eachcontrol trajectory. The certainty equivalent strategy is basedon solving problem (5a) using the certainty equivalent costfunction ψCE (10). It uses the mean of the ensemble as itspermeability field. Figure 4 illustrates the mean of the per-meability field ensemble given in Fig. 3. The RO strategy isbased on solving problem (5a) using the robust cost func-tion ψ rob (5a) and the robust gradient ∇ukψ rob (15). Themodified robust optimization is the RO strategy with anadded reactive strategy, i.e. we solve problem (5a) using(14) and (15) but we shut in a producer well when it is noneconomical. This means that when we solve the flow equa-tions (5c), the number of active producer wells can change.This in turn means that once a well is shut-in, its later con-tribution to the NPV and its gradient will be zero. We couldsay that for each realization we manipulate producer wellsbhps as long as they are profitable. Further, this strategystops the production of a reservoir when all wells are non-economical. To our knowledge, there exist no extension ofrobust optimization that includes reactive control. However,the idea of adding reactive control has been used to improvethe NPV of a single reservoir model. In [65] they considerproduction optimization in the absence of uncertainty byincluding a watercut constraint on the well completions.This results in increased NPV and a faster convergence ofthe optimizer.

Simulations reveal that for the present case, the RO strat-egy yields an higher expected NPV Eθ [ψ] and a lowerstandard deviation of the NPV (see Table 2) compared tothe certainty equivalent strategy. However, both the RO andthe certainty equivalent strategies are worse than the reac-tive strategy because of a much lower expected NPV Eθ [ψ]with a much higher NPV standard deviation. The modifiedRO strategy has a NPV standard deviation comparable tothe reactive strategy, but a higher expected NPV Eθ [ψ]. It isimportant to stress that the results concerning the merits ofthe different strategies are particular to this case study and

Comput Geosci

200 400 600 800 1000 1200 14000

10

20

30

40

50

60

time [days]

Eθ[ψ

] (m

illio

n U

SD

)

Certainty equivalentReactiveROModified RO

Fig. 8 Profit evolution for open-loop optimization in the hundredensemble case

not universal. [26] presents a case in which the RO strat-egy provides higher expected NPV and lower NPV standarddeviation than the reactive strategy. In making a compari-son with [26], there are a number of things we should stress.First of all, in [26] they are controlling directly the liquidrates of 12 wells with no direct control on the bhp values. Inour test case, we control the bhp values of only 5 wells withno direct control on the liquid rates. In [26], all the realiza-tions are giving positive NPV for all the control strategies.Further, they find that the reactive strategy is the worst touse. In our test case, the NO strategy is the worst to use,and it gives a substantial negative NPV contribution. Hence,it seems like the test case in [26] facilitates optimal con-trol strategies. In our case, however, the heterogeneities inthe ensemble realizations make it hard for optimal controlstrategies to improve on a reactive strategy. To summarize,the problems treated in [26] and in this paper have quitedifferent characteristics. Hence, different preferences withrespect to open-loop strategies is not necessarily surprising.

The results in our paper indicates the value of feedback.The reactive strategy as well as the modified robust strategyboth use a simple form of feedback. The nominal strategy,the certainty equivalent strategy, and the robust optimiza-tion strategy are pure open-loop strategies that do not usefeedback.

To illustrate the results in a tutorial way, we split thediscussion of the open-loop optimization into a two ensem-ble case and a hundred ensemble case. In the two ensemblecase, we present the results of open-loop optimization usingan ensemble of two realizations. Figure 5 illustrates thetwo realizations of the uncertain permeability field for thiscase. In the case with hundred ensemble members, we usethe entire ensemble in Fig. 3 to represent the uncertainpermeability field.

5.1.1 Case - ensemble with two members

In this subsection, we describe the performance of the ROstrategy for the case with an ensemble consisting of the twopermeability field realizations illustrated in Fig. 5. We com-pare the results of the RO control strategy with the resultsof the reactive, the modified RO and the optimal controlstrategies. By the optimal control strategies for the two real-izations in Fig. 5, we mean the optimal control strategies,{uk}N−1

k=0 , that are computed by solving the optimizationproblem (8) using the true permeability fields. These are

ψ = ψ({uk}N−1

k=0 ; x0, θ1)

= ψ1

ψ = ψ({uk}N−1

k=0 ; x0, θ2)

= ψ2

The NPVs computed using these optimal control strategiesact as an upper bound for the NPVs computed using theother control strategies. To choose the two realizations touse, we first compute optimal control trajectories for therealizations in the ensemble of Fig. 3. Then we choose two

0 200 400 600 800 1000 1200 1400

0

20

40

60

time (days)

ψi (

mili

on U

SD

)

(1) RO strategy

0 200 400 600 800 1000 1200 1400

0

20

40

60

time (days)

ψi (

mili

on U

SD

)

(2) Modified RO strategy

Fig. 9 Profit evolution of the open-loop RO strategy and the open-loop modified RO-strategy for each realization of the permeability field. Somescenarios in the RO strategy gives negative profits while the modified RO strategy avoids that by shutting in uneconomical producer wells

Comput Geosci

realizations with large differences in the optimal productionstrategies.

Figures 6(1), 6(2) and 6(3) show the terms ψ1, ψ2 andEθ [ψ] (13) for the reactive strategy, the RO strategy, andthe modified RO strategy, respectively. As expected, theNPVs computed using the optimal control strategies givethe highest values for ψ1 and ψ2. Compared to the reactivestrategy, the RO strategy gives a lower NPV, ψ1, for realiza-tion 1, and a higher NPV,ψ2, for realization 2. As illustratedin Fig. 6(3), this results in a lower NPV mean, Eθ [ψ],for the RO strategy compared to the reactive strategy. The

modified RO control strategy gives the highest NPVs for allthe realizations.

Furthermore, it is interesting to observe the differencein production times for the different strategies. For the ROstrategy, the production continue for the entire time horizon(1460 days) considered. In the reactive strategy, the produc-tion lasts 949 days in the first realization (ψ2) and 1119days in the second realization (ψ2). So there is an importantdifference in the field developing time of the two realiza-tions. In the modified RO strategy, the production lasts 1289days in the first realization and 1240 days in the second

p2

p4

i1

t = 170 days

p1

p3

0.2

0.4

0.6

0.8

t = 365 days t = 706 days t = 949 days t = 1460 days

(1) Certainty equivalent

p2

p4

i1

t = 170 days

p1

p3

0.2

0.4

0.6

0.8


(2) RO

p2

p4

i1

t = 170 days

p1

p3

0.2

0.4

0.6

0.8


(3) modified RO

p2

p4

i1

t = 170 days

p1

p3

0.2

0.4

0.6

0.8


(4) Reactive

Fig. 10 Saturation profiles of the first realization for the open-loop optimization strategies in the hundred ensemble case

Comput Geosci

p2

p4

i1

t = 170 days

p1

p3

0.2

0.4

0.6

0.8



p2

p4

i1

t = 170 days

p1

p3

0.2

0.4

0.6

0.8


(2) RO

p2

p4

i1

t = 170 days

p1

p3

0.2

0.4

0.6

0.8


(3) modified RO

p2

p4

i1

t = 170 days

p1

p3

0.2

0.4

0.6

0.8


(4) Reactive

Fig. 11 Saturation profiles of the second realization for the open-loop optimization strategies in the hundred ensemble case

realization. We note that with the modified RO strategy, theproduction time is longer than the production time of thereactive strategy.

Figure 7 shows the control trajectories of the RO, themodified RO and the optimal strategies. We note thatbecause of the heterogeneity between the realizations, theresulting optimal control trajectory of one realization can bevery different and conflicting with the optimal control tra-jectory for the other realization. To find a common optimalcontrol that takes all these differences into account can bedifficult if not impossible. Especially if we don’t allow forchanges in the configuration of active wells, e.g. producernumber 4 is producing at its minimum (310 bar) in the solu-tion for ensemble 1 and at its maximum (290 bar) in the

solution for ensemble 2. The RO and modified RO solutionsfor the producer number 4 stay in between the two optimaltrajectories.

In conclusion, the two-ensemble case demonstrates thatthe optimizer produces the maximal profit for the optimalcases. Therefore, the optimizer works well and the lowerprofit of the RO strategy is not the result of lack of conver-gence in the optimizer, but rather the result of heterogeneouspermeability fields giving conflicting control trajectories.

5.1.2 Case - ensemble with hundred members

In this subsection, we describe the results for the casein which we do open-loop optimization using the entire

Comput Geosci

0 200 400 600 800 1000 1200 1400294

296

298

300

302

304

306

308

310bh

p (b

ar)

Producer 1

0 200 400 600 800 1000 1200 1400294

296

298

300

302

304

306

308

310

bhp

(bar

)

Producer 2

0 200 400 600 800 1000 1200 1400294

296

298

300

302

304

306

308

310

bhp

(bar

)

Producer 3

0 200 400 600 800 1000 1200 1400294

296

298

300

302

304

306

308

310

bhp

(bar

)

Producer 4

0 200 400 600 800 1000 1200 1400310

315

320

325

330

335

340

345

350

bhp

(bar

)

time (days)

Injector1

ROmodified ROCertainty equivalent

Fig. 12 Control trajectories for open-loop optimization in the hundredensemble case

−60 −40 −20 0 20 40 600

0.2

0.4

0.6

0.8

1

Cumulative distribution function Fψ(x) = Prob(ψ ≤ x)

million dollars

RO modifiedROReactiveCertainty equivalentNO

Fig. 13 Cumulative distribution function for the open-loop opti-mization strategies. The cumulative distribution functions are for theensemble with hundred members

ensemble of 100 realizations in Fig. 3. Figure 4 illu-strates the mean permeability field for the ensemble of per-meability fields in Fig. 3. Figure 8 shows the profit evolutionin the case of an ensemble consisting of 100 permeabil-ity fields for the certainty equivalent strategy, the reactivestrategy, the RO strategy, and the the modified RO strategy.

Table 2 reports the corresponding key performance indi-cators (expected NPV Eθ [ψ] and standard deviation of theNPV). As in the two ensemble case, the reactive strategyyields both a larger expected NPV and a smaller standarddeviation for the NPV compared to the certainty equiva-lent and the the RO strategies. The reasons for the inferiorperformance of the RO strategy should be searched in theconflicting controls required for the different realizations.Figure 9(1) shows that the RO strategy cannot avoid thatsome ψ i gives a negative contribution to the expected NPV

0 1000 2000 3000 4000 50000

10

20

30

40

50

60

time [days]

ψ({

u k}; x

0,θ0 )

(Mill

ion

dolla

rs)

OptimalCertainty equivalentROReactiveCE (open loop)RO (open loop)mod. RO (open loop)

Fig. 14 Profit evolution for the closed-loop optimization strategies.The profit evolution of the true model controlled by different strategiesbased on the ensemble in Fig. 3. Both the RO and the certainty equiv-alent strategies give a higher NPV than the the reactive strategy. Theoptimal control strategy represents the best possible solution. By usingthe RO strategy and the certainty equivalent strategy we get profitsclose to the maximum possible profit

Comput Geosci

Table 3 Key indicators for the closed-loop optimized cases. Improvements are relative to the reactive case

Meas. noise Reactive Certainty equivalent RO Optimal

106 USD 106 USD, % 106 USD, % 106 USD, %

5 · R 51.24 59.13, +15.4 58.34, +13.9 60.41, +17.3

R 51.24 59.79, +16.7 59.09, +15.3 60.41, +17.3

5−1 · R 51.24 59.84, +16.8 59.52, +16.2 60.41, +17.3

5−2 · R 51.24 59.95, +17.0 59.56, +16.2 60.41, +17.3

Eθ [ψ]. In contrast, as illustrated in Fig. 9(2), the modi-fied RO strategy does not produce realizations with negativeprofit. Furthermore, each realization of the modified ROstrategy seems to increase the profit compared to the ROstrategy. The reactive strategy performs better than both theRO and ceratinty equivalent strategies because it can shut ina well when it is no longer profitable to operate the well.The modified RO strategy inherits the ability of the reac-tive strategy to shut in unprofitable wells. This is in essencea simple feedback mechanism. Figures 10 and 11 show thesaturation profiles of the first two realizations for the open-loop strategies. We note that the reactive strategy and themodified strategy inject a higher water quantity and displacethe oil more uniformly compared to the RO and the certaintyequivalent strategies.

Figures 12 shows the control trajectories of the RO, themodified RO and the certainty equivalent strategies. Com-pared to the the trajectories in Fig. 7, for the two ensemblecase, it seems that the RO and certainty equivalent strategiesinclude some averaging (smoothing) in the resulting con-trol trajectories that limits their effectiveness. The result isa control trajectory that produces less oil than the modifiedRO strategy that can shut in uneconomical producer wells.

As indicated by Fig. 7, the RO control trajectories may bethe average of conflicting control trajectories and thereforeinefficient for the uncertain system.

Figure 13 shows the cumulative distribution function forthe different control strategies, i.e. the probability to get aNPV � x. These curves are similar to the ones reportedin [26] with the difference that the NO and the certaintyequivalent strategies have a positive probability of givingnegative NPVs. Figure 13 confirms that the modified ROstrategy is superior to the other open-loop strategies.

5.2 Closed-loop optimization

The closed-loop optimization strategies are implementedusing the moving horizon principle. In this method, eachtime new measurements from the real or simulated reservoirare available, the EnKF uses these measurements to updatethe estimates of the permeability field, and an open-loopoptimization problem is solved using the updated perme-ability field. Only the first part of the resulting optimalcontrol trajectory is implemented. As new measurementsbecomes available, the procedure is repeated. The samplingtime for the system is Ts = 146 days, i.e. the data assim-ilation and optimization is performed every 146 days. Theopen-loop optimization uses a prediction and control hori-zon of 4 · 365 = 1460 days that is divided into N = 60periods (the same as for the open-loop optimization strate-gies). With this parametrization, the control steps for thefirst six periods are implemented to the system, and then

0 1000 2000 3000 4000 50000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

time [days]

Inje

cted

wat

er


(1) Cumulative water injection

0 1000 2000 3000 4000 50000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

time [days]

Prod

uced

Oil


(2) Cumulative oil production

Fig. 15 Closed-loop. Water injected and produced oil trajectories for different optimization strategies. The values are normalized with respect tothe pore volume

Comput Geosci

we receive new measurements to do new data assimilationand optimization computations for a shifted time window. Inthis paper, we consider 35 of these steps such that the totalproduction horizon is 146 · 35 = 5110 days.

We compare three closed-loop optimization strategies:A reactive strategy, a certainty equivalent strategy, and aRO strategy. We did not implement a modified RO strategybecause that would be complicated by the need to manage

Fig. 16 Saturation profiles ofthe true field for the closed-loopoptimization strategies

p2

p4

i1

t = 170 days

p1

p3

0.2

0.4

0.6

0.8

t = 365 days t = 949 days t = 1460 days



p2

p4

i1

t = 170 days

p1

p3

0.2

0.4

0.6

0.8



(4) RO

p2

p4

i1

t = 170 days

p1

p3

0.2

0.4

0.6

0.8



(6) Optimal

Comput Geosci

0 1000 2000 3000 4000 5000295

300

305

310

315

320

bhp

(bar

)

Producer 1

0 1000 2000 3000 4000 5000295

300

305

310

315

320

bhp

(bar

)

Producer 2

0 1000 2000 3000 4000 5000295

300

305

310

315

320

bhp

(bar

)

Producer 3

0 1000 2000 3000 4000 5000295

300

305

310

315

320

bhp

(bar

)

Producer 4

0 1000 2000 3000 4000 5000310

315

320

325

330

335

340

bhp

(bar

)

time (days)

Injector1

OptimalCertainty equivalentRO

Fig. 17 Control trajectories for the closed-loop optimizationstrategies

situations with a variable number of active wells and mea-surements for different ensemble realizations. Further, itwould require a strategy to replace ensemble realizationswhen all the producing wells are shut-in.

Figure 14 shows the NPV ψ({uk}N−1

k=0 ; x0, θ10|0)

for the

reactive strategy, the closed-loop RO strategy, the closed-loop certainty equivalent strategy, the optimal control strat-egy, and the open-loop strategies introduced in the previoussection. The optimal control strategy is obtained solvingthe optimization problem (7) using the true permeabil-ity field (the first realization of the permeability field inFig. 3). The NPV computed by the optimal control strat-egy represents the best possible operation of the reservoir.Table 3 reports key indicators (expected NPV and improve-ments compared to the reactive strategy) for the closed-loop strategies at different levels of measurement noise.Figure 14 and Table 3 shows that for all investigated cases,both the closed-loop certainty equivalent strategy and theclosed-loop RO strategy yields significantly higher NPVthan the reactive strategy. As is also evident from Fig. 14and Table 3, the closed-loop certainty equivalent strategy

0 1000 2000 3000 4000 5000

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

time [days]

(1) RMSE

0 1000 2000 3000 4000 5000

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

time [days]

Meas. Cov. = 5 RMeas. Cov. = R

Meas. Cov. = 5−1 R

Meas. Cov. = 5−2 R

(2) Ensemble spread

Fig. 18 Convergence measures for the EnKF with various levels ofmeasurement noise for the closed-loop certainty equivalent strategy.(1) shows that the EnKF does not converge to the true parameters.However, the estimate captures enough features to be useful. (2) illus-trates that the parameter uncertainty decreases as more production datais assimilated in the estimates

Comput Geosci

Fig. 19 Estimates of the meanpermeability field as function oftime for the closed-loopcertainty equivalent strategy.The initial estimate is a fourchannel structure. The estimates,θk|k−1, converge towards thetrue two-channel structure asmore measurements areassimilated

(1) t=0

(36) t = 35 Δ T (37) true

Comput Geosci

yields higher NPV than the closed-loop RO strategy. Fur-thermore, the NPV of the closed-loop certainty equivalentstrategy is very close to the NPV of the optimal strategy.Consequently, the closed-loop certainty equivalent strategyis preferable over the closed-loop RO strategy as it yieldshigher NPV and requires significantly less computationaleffort. This observation is also confirmed when using thesecond realization in Fig. 3 as the permeability field.

From Fig. 14 we note also that the modified RO is thebest open-loop strategy and that the closed-loop strategiesCE and RO provide better results compared to the modi-fied RO starting from the assimilation steps 13 (t ≈ 1898days) and 16 (t ≈ 2336 days), respectively. Figure 15 showsthe cumulative water injection and the cumulative oil pro-duction for different strategies. The slope of the curves isthe reservoir injection/production rate. In general, we notethat the closed-loop strategies are injecting at a lower ratecompared to the open-loop strategies. This happens sincewe use a zero discount factor, i.e. we focus on long termbehaviour. Moreover, there are no direct bounds on the liq-uid rates. We note that the open-loop strategies, so as theoptimal strategy, have an upward concavity. This meansthat the water injection rate is increasing with time: thesestrategies increase the injection at the final time to exploitthe high oil-to-water price ratio. The closed-loop strategies,instead, have a downward concavity. At the beginning (first300 days) the closed-loop strategies are injecting at a sim-ilar pace as their open-loop counterparts (same slope inthe initial part of the curves). However, as the data assim-ilation proceed, and a better estimate of the true field isgiven, the closed-loop strategies try to inject/produce asmuch as the optimal strategy (black curves in figure). Thisexplains the change in concavity of the closed-loop strate-gies i.e. why the closed-loop strategies reduce the waterinjection rate with time. Figure 16 illustrates the satura-tion profiles of the true field for the closed-loop strategies.We note that they have similar field sweep at the finaltime. Figure 17 shows the corresponding control trajec-tories of the different closed-loop control strategies. It isevident that the control trajectories of the optimal controlstrategy are very different from the control trajectories forthe closed-loop certainty equivalent and the closed-loop ROstrategies.

Figure 18 illustrates the RMSE (32) and the ensemblespread (31) of the EnKF when applied together with thecertainty equivalent strategy. The RMSE indicates whetherthe permeability parameter estimate of the EnKF convergestoward the true permeability parameters. The ensemblespread indicates the uncertainty in the estimated perme-ability parameters. The RMSE and the ensemble spreadsequences are computed for different levels of measure-ment noise, i.e. different values of R in (16b). Figure 18(2)indicates that decreasing levels of measurements noise, R,

decrease the ensemble spread (31). This decrease does notalways results in a lower RMSE (32) value. However, as isevident from Fig. 18(2), in most of the cases, lower mea-surement noise levels reduce the RMSE. In this case studythere is no ensemble collapse. In fact, Fig. 20 shows thatthe ensemble realizations have different distances at the lastassimilation time, this is an index of the variability in theensemble of realizations.

10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

|| (θ

i k|k −

θ0 )

/ np ||

2

Ensemble member #

(1) t=0

10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

Ensemble member #

|| (θ

i k|k −

θ0 )

/ np ||

2

Certainty equivalentRO

(2) t=5110 days, Meas. noise = 5 R

10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

Ensemble member #

|| (θ

i k|k −

θ0 )

/ np ||

2

Certainty equivalentRO

(3) t=5110 days, Meas. noise = R/5

Fig. 20 Closed-loop. Distance of the ensemble realizations of the per-meability field respect to the true permeability field for (1) the initialensemble, (2) the final ensemble with a measurement noise of 5 R and(3) the final ensemble with a measurement noise of R/5. We note thatalso in the case with the lowest measurement noise, the ensemble isnot collapsing

Comput Geosci

In the EnKF, at each data assimilation step, we updatethe estimated permeability field for each ensemble mem-ber. Figure 19 illustrates the time evolution of the mean,θk|k−1, of these estimated permeability field ensembles forthe closed-loop certainty equivalent optimization strategy.Figure 19 indicates that the estimated mean permeabilityfield captures the main features of the true permeabilityfield. We start out with a mean permeability field havingfour channel structures and converge towards the correcttwo channel structure. Figure 20 illustrates the associateduncertainty with this permeability estimate.

6 Conclusions

In this paper, we demonstrate the open-loop and the closed-loop performance of the certainty equivalent strategy andthe RO strategy. For the open-loop case we present a mod-ified RO strategy that performs significantly better than theother open-loop strategies. In the closed-loop situation forthe case studied, we arrive at the surprising conclusion thatthe certainty equivalent strategy is slightly better than theRO strategy.

For the case presented, the open-loop RO strategy yields3 % higher expected NPV and 28 % lower NPV standarddeviation than the open-loop certainty equivalent strategy.Yet, the reactive strategy performed even better than theopen-loop RO strategy. Simulations indicate that the infe-rior performance of the open-loop RO strategy compared tothe reactive strategy is due to the inability of the RO strategyto efficiently encompass ensembles with very different andconflicting optimal control trajectories. We propose a mod-ified RO strategy that allow shut in of uneconomical wells.The modified RO strategy performs significantly better thanthe other open-loop strategies and the reactive strategy. Themodified RO optimization strategy yields an expected NPVthat is 36 % higher than the expected NPV of the open-loop certainty equivalent strategy and 3 % higher than theexpected NPV for the reactive strategy. The NPV standarddeviation of the modified RO strategy is similar to the NPVstandard deviation of the reactive strategy. These observa-tions are non-trivial, as previous literature suggests that theopen-loop RO strategy performs better than the reactivestrategy [26]. The improved economic performance of theopen-loop modified RO strategy justifies the computationaleffort used in determining the trajectories for this strategy.

The simulations for the closed-loop strategies, revealthat the RO strategy and the certainty equivalent strategyyields significantly higher NPV than the reactive strat-egy. Surprisingly, the closed-loop certainty equivalent strat-egy yields a higher NPV than the closed-loop RO strat-egy for the case studied. The uncertainty reduction ofthe permeability field estimate due to data assimilation

explains the good performance of the closed-loop certaintyequivalent optimization strategy. Consequently, in closed-loop, the increased computational effort of the RO strategycompared to the certainty equivalent strategy is not justifiedfor the particular case studied in this paper.

Future work will include a test of the strategies discussedin this paper on a more complex scenario (many wells,3D grid, state/output constraints, spurious correlations), andwe plan to work on the “Brugge field” [28]. In open-loopsimulations, we expect that the modified RO strategy willimprove the RO strategy as seen here. This result is in someway anticipated in [65], where, despite they do not con-sider uncertainty in the reservoir parameters, they get anincreased NPV for the ”Brugge field” by adding a reac-tive control to an optimal control strategy. In closed-loopsimulations, we expect to obtain similar results for the ROand the CE strategies provided the data assimilation con-verges properly, as in the case showed in this paper. Finally,the optimization strategies presented in this paper and toour knowledge all literature on closed-loop reservoir man-agement deals with optimization of the expected NPV. Theapproaches only implicitly considers risk and variance ofthe NPV. Future approaches should more directly includerisk and variance of the NPV in the optimization.

Acknowledgments This research project is financially supported bythe Danish Research Council for Technology and Production Sciences.FTP Grant no. 274-06-0284 and the Center for Integrated Operationsin the Petroleum Industry at NTNU.

References

1. Brouwer, D., Nævdal, G., Jansen, J.: Improved reservoir man-agement through optimal control and continuous model updating.In: SPE Annual Technical Conference and Exhibition, Houston(2004)

2. Brouwer, D.R., Jansen, J.D.: Dynamic optimization of waterflood-ing with smart wells using optimal control theory. SPE J. 9,391–402 (2004)

3. Sarma, P., Durlofsky, L., Aziz, K.: Efficient closed-loop pro-duction optimization under uncertainty. In: SPE Europec/EAGEAnnual Conference, Madrid (2005)

4. Nævdal, G., Brouwer, D.R., Jansen, J.-D.: Waterflooding usingclosed-loop control. Comput. Geosci. 10, 37–60 (2006)

5. Jansen, J.-D., Bosgra, O.H., Van den Hof, P.M.J.: Model-basedcontrol of multiphase flow in subsurface oil reservoirs. J. ProcessControl 18, 846–855 (2008)

6. Jansen, J.D., et al.: Closed-loop reservoir management. In: 2009SPE Reservoir Simulation Symposium, SPE 119098 The Wood-lands, Texas (2009)

7. Lorentzen, R.J., Shafieirad, A., Nævdal, G.: Closed loop reser-voir management using the ensemble Kalman filter and sequentialquadratic programming. In: 2009 SPE Reservoir Simulation Sym-posium, SPE 119101. The Woodlands, Texas (2009)

8. Foss, B., Jensen, J.P.: Performance analysis for closed-loop reser-voir management. SPE J. 16, 183–190 (2011)

9. Capolei, A., Stenby, E.H., Jørgensen, J.B.: High order adjointderivatives using esdirk methods for oil reservoir production

Comput Geosci

optimization. In: ECMOR XIII 13th European Conference on theMathematics of Oil Recovery (2012)

10. Van den Hof, P.M.J., Jansen, J.D., Heemink, A.: Recent devel-opments in model-based optimization and control of subsurfaceflow in oil reservoirs. In: Proceedings of the 2012 IFAC Work-shop on Automatic Control in Offshore Oil and Gas Production,pp. 189–200, Trondheim (2012)

11. Capolei, A., Volcker, C., Frydendall, J., Jørgensen, J.B.: Oil reser-voir production optimization using single shooting and ESDIRKmethods. In: Proceedings of the 2012 IFAC Workshop on Auto-matic Control in Offshore Oil and Gas Production, pp. 286–291,Trondheim (2012)

12. Foss, B.: Process control in conventional oil and gas fields -challenges and opportunities. Control. Eng. Pract. 20, 1058–1064(2012)

13. Rawlings, J.B., Mayne, D.Q.: Model predictive control: Theoryand design. Nob Hill Publishing, Madison (2009)

14. Grune, L., Pannek, J.: Nonlinear model predictive control theoryand algorithms. Springer, London (2011)

15. Grotschel, M., Krumke, S.O., Rambau, J. (eds.): Online optimiza-tion of large scale systems. Springer, Heidelberg (2001)

16. Allgower, F., Zheng, A. (eds.): Nonlinear Model Predictive Con-trol, vol. 26 Progress in Systems and Control Theory. Birkhauser,Basel (2000)

17. Findeisen, R., Allgower, F., Biegler, L.T. (eds.): Assessment andfuture directions of nonlinear model predictive control. LectureNotes in Control and Information Sciences, vol. 358, Springer,Heidelberg (2007)

18. Magni, L., Raimondo, D.M., Allgower, F. (eds.) : Nonlinearmodel predictive control. Towards New Challenging Applications.Lecture Notes in Control and Information Sciences, vol. 384,Springer, Heidelberg (2009)

19. Lazar, M., Allgower, F. (eds.): 4th IFAC Nonlinear Model Predic-tive Control Conference (NMPC’12). IFAC Noordwijkerhout, NL(2012)

20. Evensen, G.: Data Assimilation: The Ensemble Kalman Filter, 2ndedn. Springer (2009)

21. Biegler, L.T., Ghattas, O., Heinkenschloss, M., van BloemenWaanders, B. (eds.): Large-Scale PDE-Constrained Optimization.Springer (2003)

22. Biegler, L.T., Ghattas, O., Heinkenschloss, M., Keyes, D., vanBloemen Waanders, B. (eds.): Real-Time PDE-Constrained Opti-mization SIAM (2007)

23. Markowitz, H.: Portfolio selection. J. Finance 7, 77–91 (1952)24. Terwiesch, P., Ravemark, D., Schenker, B., Rippin, D.W.: Semi-

batch process optimization under uncertainty: Theory and experi-ments. Comput. Chem. Eng. 22, 201–213 (1998)

25. Srinivasana, B., Bonvina, D., Vissera, E., Palankib, S.: Dynamicoptimization of batch processes: II role of measurements in han-dling uncertainty. Comp. Chem. Eng. 27, 27–44 (2003)

26. Van Essen, G.M., Zandvliet, M.J., Van den Hof, P.M.J., Bosgra,O.H., Jansen, J.D.: Robust waterflooding optimazation of multiplegeological scenarios. SPE J. 14, 202–210 (2009)

27. Chen, C., Wang, Y., Li, G., Reynolds, A.C.: Closed-loop reservoirmanagement on the Brugge test case. Comput. Geosci. 14, 691–703 (2010)

28. Peters, L., et al.: Results of the Brugge benchmark study for flood-ing optimization and history matching. SPE Reserv. Eval. Eng. 13,391–405 (2010)

29. Evensen, G.: The ensemble Kalman filter: theoretical formu-lation and practical implementation. Ocean Dyn. 53, 342–367(2003)

30. Wen, X.-H., Chen, W.H.: Some practical issues on real-time reser-voir model updating using ensemble Kalman filter. SPE J. 12,156–166 (2007)

31. Ehrendorfer, M.: A review of issues in ensemble-based Kalmanfiltering. Meteorol. Z. 16, 795–818 (2007)

32. Aanonsen, S.I., Nævdal G., Oliver, D.S., Reynolds, A.C., Valls,B.: The ensemble Kalman filter in reservoir engineering-a review.SPE J. 14, 393–412 (2009)

33. Simon, D.: Optimal State Estimation Kalman, H∞, and NonlinearApproaches. Wiley, Hoboken (2006)

34. Rawlings, J.B., Bakshi, B.R.: Particle filtering and moving horizonestimation. Comput. Chem. Eng. 30, 1529–1541 (2006)

35. Wen, X.-H., Chen, W.H.: Real-time reservoir model updatingusing ensemble Kalman filter with confirming option. SPE J. 11,431–442 (2006)

36. Sarma, P., Chen, W. Preventing ensemble collapse and preserv-ing geostatistical variability across the ensemble with the subspaceenkf. In: ECMOR XIII-13th European Conference on the Mathe-matics of Oil Recovery. Biarritx, France (2012)

37. Sarma, P., Chen, W.H. Robust and efficient handling of modelcontraints with the kernal-based ensemble Kalman filter. In:Reservoir Simulation Symposium. The Woodlands, Texas (2011)

38. Sarma, P., Chen, W. Generalization of the ensemble Kalman filterusing kernels for nongaussian random fields. In: SPE ReservoirSimulation Symposium. The Woodlands, Texas (2009)

39. Chen, Y., Oliver, D.S., Zhang, D.: Efficient ensemble-basedclosed-loop production optimization. SPE J. 14, 634–645 (2009)

40. Chen, Y., Oliver, D.S.: Ensemble-based closed-loop optimiza-tion applied to Brugge field. SPE Reserv. Eval. Eng. 13, 56–71(2010)

41. Lie, K.A., et al.: Open source matlab implementation of consistentdiscretisations on complex grids. Comput. Geosci. 16, 297–322(2012)

42. Peaceman, D.W.: Interpretation of well-block pressures innumerical reservoir simulation with nonsquare grid blocks andanisotropic permeability. SPE J. 23(3), 531-543 (1983)

43. Suwartadi, E., Krogstad, S., Foss, B.: Nonlinear output constraintshandling for production optimization of oil reservoirs. Comput.Geosci. 16, 499–517 (2012)

44. Schlegel, M., Stockmann, K., Binder, T., Marquardt, W.: Dynamicoptimization using adaptive control vector parameterization. Com-put. Chem. Eng. 29, 1731–1751 (2005)

45. Capolei, A., Jørgensen, J.B. Solution of constrained optimal con-trol problems using multiple shooting and esdirk methods. In:American Control Conference (ACC), 295–300 (2012)

46. Bock, H.G., Plitt, K.J. A multiple shooting algorithm fordirect solution of optimal control problems. In: Proceedings9th IFAC World Congress Budapest, pp. 243–247. Pergamon Press(1984)

47. Biegler, L.T.: Solution of dynamic optimization problems bysuccessive quadratic programming and orthogonal collocation.Comput. Chem. Eng. 8, 243–248 (1984)

48. Jansen, J.: Adjoint-based optimization of multi-phase flowthrough porous media - A review. Comput. Fluids 46, 40–51(2011)

49. Sarma, P., Aziz, K., Durlofsky, L.J. Implementation of adjointsolution for optimal control of smart wells. In: SPE Reservoir Sim-ulation Symposium, 31 January-2 Feburary 2005, The Woodlands,Texas (2005)

50. Jørgensen, J.B. Adjoint sensitivity results for predictive con-trol, state- and parameter-estimation with nonlinear models. In:Proceedings of the European Control Conference 2007, pp. 3649–3656. Kos, Greece (2007)

51. Volcker, C., Jørgensen, J.B., Stenby, E.H. Oil reservoir productionoptimization using optimal control. In: 50th IEEE Conference onDecision and Control and European Control Conference, 7937–7943 Orlando, Florida (2011)

Comput Geosci

52. Byrd, R.H., Nocedal, J., Waltz, R.A.: Knitro: An integratedpackage for nonlinear optimization. In: Large Scale NonlinearOptimization, pp. 35–59 (2006)

53. MATLAB. version 7.13.0.564 (R2011b) (The MathWorks Inc.,Natick, Massachusetts, 2011)

54. Liu, Y.: Using the snesim program for multiple-point statisticalsimulation. Comput. Geosci. 32, 1544–1563 (2006)

55. Scholkopf, B., Smola, A., Muller, K.-R.: Nonlinear componentanalysis as a kernel eigenvalue problem. Neural Comput. 10,1299–1319 (1998)

56. Kalman, R.E.: A new approach to linear filtering and predictionsproblems. J. Basic Eng. 82, 35–45 (1960)

57. Kailath, T., Sayed, A.H., Hassibi, B.: Linear Estimation. PrenticeHall (2000)

58. Jazwinski, A.H.: Stochastic Processes and Filtering Theory. Aca-demic Press (1970)

59. Burgers, G., van Leeuwen, P.J., Evensen, G.: Analysis scheme inthe ensemble Kalman filter. Mon. Weather Rev. 126, 1719–1724(1998)

60. Wen, X.H., Chen, W.: Real-time reservoir model updating usingensemble Kalman filter. In: SPE Reservoir Simulation Sympo-sium. The Woodlands, Texas (2005)

61. Gu, Y., Oliver, D.S.: History matching of the punq-s3 reservoirmodel using the ensemble Kalman filter. SPE J. 10, 217–224(2005)

62. Chen, Z. Reservoir Simulation Mathematical Techniques in OilRecovery. SIAM Philadelphia (2007)

63. Aziz, K., Durlofsky, L., Tchelepi, H.: Notes on petroleumreservoir simulation. Department of Petroleum EngineeringSchool of Earth Sciences, Stanford University (2005)

64. Volcker, C., Jørgensen, J.B., Thomsen, P.G., Stenby, E.H. Simula-tion of subsurface two-phase flow in an oil reservoir. In: Proceed-ings of the European Control Conference 2009, pp. 1221–1226.Budapest, Hungary (2009)

65. Dehdari, V., Oliver, D.S.: Sequential quadratic programming forsolving constrained production optimization – case study fromBrugge field. SPE J. 17, 874–884 (2012)

Waterflooding optimization in uncertain geological...

Documents

Transcript of Waterflooding optimization in uncertain geological...