Available online at www.sciencedirect.com

ScienceDirect

Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145www.elsevier.com/locate/cma

Faster computation of the Karhunen–Loeve expansion using itsdomain independence property

Srikara Pranesh, Debraj Ghosh∗

Department of Civil Engineering, Indian Institute of Science, Bangalore 560012, India

Received 14 June 2014; received in revised form 27 September 2014; accepted 31 October 2014Available online 13 November 2014

Abstract

The goal of this work is to reduce the cost of computing the coefficients in the Karhunen–Loeve (KL) expansion. The KLexpansion serves as a useful and efficient tool for discretizing second-order stochastic processes with known covariance function. Itsapplications in engineering mechanics include discretizing random field models for elastic moduli, fluid properties, and structuralresponse. The main computational cost of finding the coefficients of this expansion arises from numerically solving an integraleigenvalue problem with the covariance function as the integration kernel. Mathematically this is a homogeneous Fredholmequation of second type. One widely used method for solving this integral eigenvalue problem is to use finite element (FE) bases fordiscretizing the eigenfunctions, followed by a Galerkin projection. This method is computationally expensive. In the current work itis first shown that the shape of the physical domain in a random field does not affect the realizations of the field estimated using KLexpansion, although the individual KL terms are affected. Based on this domain independence property, a numerical integrationbased scheme accompanied by a modification of the domain, is proposed. In addition to presenting mathematical arguments toestablish the domain independence, numerical studies are also conducted to demonstrate and test the proposed method. Numericallyit is demonstrated that compared to the Galerkin method the computational speed gain in the proposed method is of three to fourorders of magnitude for a two dimensional example, and of one to two orders of magnitude for a three dimensional example, whileretaining the same level of accuracy. It is also shown that for separable covariance kernels a further cost reduction of three to fourorders of magnitude can be achieved. Both normal and lognormal fields are considered in the numerical studies.c⃝ 2014 Elsevier B.V. All rights reserved.

Keywords: Stochastic finite element; Random field; KL expansion; Integral equation; Numerical integration

1. Introduction

While newer mathematical and computational tools are regularly being developed for studying engineering systemsof increasing complexity and size, reliability of the predictions from these tools can be improved by systematicallyconsidering the underlying uncertainties. This consideration has led to the development of the area of uncertaintyquantification (UQ). In a probability theory based UQ problem the uncertain parameters are modeled as randomquantities such as variables, vectors, and processes or fields. For structural mechanics problems such parameters

∗ Corresponding author. Tel.: +91 080 2293 2818.E-mail address: dghosh@civil.iisc.ernet.in (D. Ghosh).

http://dx.doi.org/10.1016/j.cma.2014.10.0530045-7825/ c⃝ 2014 Elsevier B.V. All rights reserved.

126 S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145

are loads such as earthquake or winds [1,2], material properties such as Young’s modulus and density [3–5], supportconditions and geometric properties [6]. These uncertainties are then propagated through methods such as Monte Carlosimulation [7], perturbation [8], and spectral stochastic finite element method [9], to name a few. For computationalpurpose often the random process (or field) models need to be discretized. Accordingly, a tensor product space isfirst defined on the cartesian product of the underlying probability space (Ω , F , P) and the physical domain D. Forspatial processes, D is a closed interval in the space Rd , where d is 1, 2, or 3. Then a process κ(x, θ) is approximatedusing a set of bases in this product space, where x ∈ D and θ ∈ Ω . This discretization or approximation is finitedimensional, and a lower dimensionality helps in optimizing the cost of uncertainty propagation. In the literature anumber of discretization techniques such as midpoint [10,11], local (spatial) averaging [12,13], series expansions suchas polynomial chaos (PC) [9], Karhunen–Loeve (KL) expansion [14], optimal linear estimation [15], and weightedintegral [16] have been proposed. An attempt to unify some of these techniques has been made in [17]. When themean and the covariance of the process are known, the KL expansion can be used to discretize the process, and thisrepresentation is optimal in the sense of dimensionality of the discretized process [15].

When the mean κ(x, θ) and the continuous covariance kernel C(x1, x2) are known, a square-integrable processκ(x, θ) can be discretized in its KL expansion as

κ(x, θ) = ω(x, θ) ≡ κ(x, θ) +

∞i=1

λiφi (x)ξi (θ). (1)

Here ξi (θ) are zero-mean unit-variance uncorrelated random variables, λi and φi (x) are eigenvalues and eigenfunc-tions, respectively, of the covariance kernel satisfying the integral eigenvalue problem (IEVP)

DC(x1, x2)φi (x1)dx1 = λiφi (x2). (2)

This integral equation is a homogeneous Fredholm equation of second type. Symmetry and positive semi-definitenessof a covariance kernel ensure that the eigenvalues are non-negative and the eigenfunctions are real-valued. Eq. (1) isconvergent in mean-square sense. For computational purpose a finite-dimensional truncated version of the KL expan-sion is adopted as

κ(x, θ) = ω(x, θ) ≡ κ(x, θ) +

Li=1

λiφi (x)ξi (θ). (3)

The truncation level L depends upon the correlation length of the process; a higher correlation length leads to a fasterdecay in the eigenvalues, thus a lower L . In many practical cases an exponentially decaying covariance kernel isused, which often needs to be modified [18] to remove the non-differentiability at the origin and thus improving thesmoothness of the kernel. Although the KL expansion is used mostly for Gaussian processes, a few non-Gaussiandistributions such as lognormal or beta distribution also have been modeled successfully [19–21]. Exact solutions ofEq. (2) are available only for a few specific types of covariance kernels over simple domains [14]. For most practicalcases a numerical method is required to solve it. The current work is regarding this numerical method. Two successfulmethods for solving this IEVP are the finite element based Galerkin method (FE) [9] and numerical quadrature such asNystrom method [22]. In the structural mechanics community FE is the most popular choice since the FE mesh devel-oped for solving the mechanics problem can be re-used to solve the IEVP [9,23]. The FE approximation followed bya Galerkin projection leads to a generalized algebraic eigenvalue problem. A number of works have been conductedin this regard for accelerating the computation. In [24] an adaptive FE scheme was followed for stationary processes.In [25] the aforementioned algebraic eigenvalue problem was solved using a generalized fast multipole method. Thusthe cubic complexity of the computational cost was reduced to log-linear. In [26] the eigenvalue problem was solvedusing a hierarchical sparse matrix approximation. Other modifications on the KL expansion have also been proposed.In [27] a Fourier–Karhunen–Loeve representation is followed where a truncated Fourier approximation of a widesense stationary process is followed by a principal component analysis (PCA).

Although the convenience of re-using the mechanics mesh in solving the IEVP is an advantage, one importantconcern naturally arises here. Should the optimal mesh for the mechanics problem be accurate and optimal for dis-cretizing the random field as well? This issue has already been raised in the literature [10,17,23], where it was found

S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145 127

that such optimality does not hold. Similar mesh resolution issue appears in other discretization techniques as well[15,28]. A few alternate ways have been explored and reported in the literature. In [29] it was found numerically thatif the eigenvectors are known to have a periodic behavior, then a Rayleigh–Ritz method with trigonometric basis setis better than a collocation scheme. In [30] a numerical parametric study was conducted on the factors affecting theconvergence of the KL expansion, and the requirement of an efficient numerical scheme to solve Eq. (2) was high-lighted. In this context another related question can be asked as follows. While specifying the characteristics of therandom process only the mean, variance, and probability distribution are mentioned, the exact shape of the physicaldomain D is not required. However, solutions of the eigenvalue problem stated in Eq. (2) would be dependent uponD. Therefore, should the KL approximated process depend upon D? Motivated by these questions, here we proceedas follows. First, using Mercer’s theorem it is mathematically argued that the KL expansion for a process is physicaldomain independent. That is, although a modification in the domain D changes the eigenvalues and eigenfunctions ofthe covariance kernel, it does not change the realizations of the process estimated using the KL expansion. It is alsoshown that the truncated KL expansion should include the most dominant terms to ensure this invariance. Based onthis domain independence property, a new method is proposed here. Accordingly, a complicated domain is replacedby a bounding box such as a rectangle in two-dimensions and a box in three-dimensions, over which the IEVP issolved using a numerical quadrature. Through two numerical studies – one two- and one three-dimensional – it isdemonstrated that the proposed method reduces the computational cost significantly compared to the FE approach.Both Gaussian and lognormal processes are modeled. It is further demonstrated that for separable kernels the costcan be further reduced by using the separability. A set of preliminary numerical studies on the proposed approachapplied to a two-dimensional problem was reported in [31]. The approach of embedding a complicated domain intoa simpler domain is also used in [32], based on the finite cell method in mechanics [33]. In the current work, thedomain-independence of the KL-approximated random field is proved (in Proposition 1), and thus it is establishedthat any numerical method can be used on a simpler domain, without the need for a local mesh refinement as in [32].

The paper is organized as follows. In Section 2 a mathematical proof for the domain independence of the KLexpansion for a Gaussian process is presented. The role of truncation is also discussed in this context. Based onthe results in this section the proposed method is presented in Section 3. The domain independence is numericallydemonstrated and the proposed method is applied and tested on two numerical problems in Section 4. A comparativestudy of the proposed method along with the conventional Galerkin method is performed in this section. Finally theconcluding remarks are made.

2. Domain independence of the KL expansion

Following the motivations mentioned in the previous section, now it will be demonstrated that if the domains D andD′ are such that D ∩D′

= φ, then the mean and second moments of the process remain unaltered. That is, at a commonspatial location of two domains x ∈ D∩D′, the mean and variance remain the same. Furthermore, the covariance of theprocess between two spatial locations x, y ∈ D ∩D′ also remains unaltered. This domain independence property, whenproved, would imply that if the process is completely characterized by first two moments – such as Gaussian processes– the marginal and joint probability density functions (pdfs) remain the same on the common spatial locations of D andD′. The fundamental tools used to prove these domain independence properties are (i) the fact that the KL expansionis a second order representation of a random process, and (ii) Mercer’s theorem. A few definitions will be stated first,followed by the statement of Mercer’s theorem and KL theorem for the completeness of the discussion.

Definition 1. Let D ⊂ Rd be a bounded domain and let k : D × D → R.

If

D|k(x, y)|2dxdy < ∞ (4)

then k is referred to as a Hilbert–Schmidt kernel.

Definition 2. Let k(x, y) be a Hilbert–Schmidt kernel and u(y) ∈ L2(D). An integral operator K : L2(D) → L2(D)

defined as

[K u](x) =

D

k(x, y)u(y)dy (5)

is called as a Hilbert–Schmidt operator.

128 S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145

Mercer’s Theorem. Let k : D × D → R be a continuous Hilbert–Schmidt kernel. Let the correspondingHilbert–Schmidt operator K : L2(D) → L2(D) be positive definite, and self-adjoint with the i th eigenvalue andeigenfunction denoted as λi and φi , respectively. Then

k(x, y) =

∞i=1

λiφi (x)φi (y) ∀x, y ∈ D. (6)

The convergence of this series is absolute and uniform.

Karhunen–Loeve Theorem. Consider a centered, mean square continuous stochastic process κ : D × Ω → R,which is also square integrable, that is, κ ∈ L2(D × Ω). Then there exist a basis φi (x) of L2(D) and randomvariables ξi (θ) such that

ni=1

λiξi (θ)φi (x)

L2(Ω)−−−→ κ (7)

where the convergence is in L2(Ω) sense, that is,

limn→∞

∥κ(x, θ) −

ni=1

λiξi (θ)φi (x)∥L2(Ω) → 0 ∀ x ∈ D. (8)

Here ξi (θ) =1

√λi

D κ(x, θ)φi (x)dx, holding the properties

1. E[ξi (θ)] = 02. E[ξi (θ)ξ j (θ)] = δi j3. V ar [ξi (θ)] = 1

where E[·] denotes the expectation operator with respect to the probability space (Ω , F , P) and V ar(·) denotes thevariance.

Upon using Mercer’s theorem in the Karhunen–Loeve theorem, it is found that λi -s and φi -s are eigenvalues andcorresponding eigenfunctions of the covariance kernel of the random process, that is, are solutions of Eq. (2). Thisleads to the KL expansion stated in Eq. (1). Based on these definitions and theorems the domain independence propertyis stated and proved next.

Proposition 1. Let κ(x, θ) ∈ L2(D × Ω) be a mean-square continuous random field defined on D ⊂ Rd , specifiedup to second order moments with κ(x) as the mean and C(x, y) as the covariance function. Consider anotherdomain D′

⊂ Rd such that D ∩ D′ is not a null set, and define another mean-square continuous random fieldκ ′(x, θ) ∈ L2(D′

× Ω) with the same mean κ(x) and covariance function C(x, y). Let (λi , φi ) be the i th eigenpair ofIEVP equation (2) on D and (λ′

i , φ′

i ) be the i th eigenpair on D′. Let ω(x, θ) and ω′(x, θ) denote the complete (thatis, without truncation) KL expansions of κ(x, θ) and κ ′(x, θ), respectively. Then for any x ∈ D ∩ D′

1. E[ω(x, θ)] = E[ω′(x, θ)]

2. E[ω2(x, θ)] = E[ω′2(x, θ)] and for any x, y ∈ D ∩ D′

3. E[ω(x, θ)ω(y, θ)] = E[ω′(x, θ)ω′(y, θ)].

Proof. Since the covariance C(x, y) between any two points x, y ∈ D ∩ D′ is the same for both the domains, byMercer’s theorem we can conclude that

C(x, y) =

∞i=1

λiφi (x)φi (y) =

∞i=1

λ′

iφ′

i (x)φ′

i (y) ∀ x, y ∈ D ∩ D′. (9)

The KL expansions ω(x, θ) and ω′(x, θ) of random fields κ(x, θ) and κ ′(x, θ), respectively, are given by

ω(x, θ) = κ(x) +

∞i=1

λiφi (x)ξi (θ)

ω′(x, θ) = κ(x) +

∞i=1

λ′

iφ′

i (x)ξi (θ).

S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145 129

Each of the conclusions will be shown separately.

1.

E[ω(x, θ)] = E

κ(x) +

∞i=1

λiφi (x)ξi (θ)

= κ(x) +

∞i=1

λiφi (x)E[ξi (θ)]

= κ(x) following property 1 of the Karhunen–Loeve theorem.

Similarly E[ω′(x, θ)] = κ(x). Hence E[ω(x, θ)] = E[ω′(x, θ)].2.

E[ω2(x, θ)] = E

∞

i=1

∞j=1

λi

λ jφi (x)φ j (x)ξi (θ)ξ j (θ)

=

∞i=1

∞j=1

λi

λ jφi (x)φ j (x)E[ξi (θ)ξ j (θ)].

Using the property 2 of ξi -s from the Karhunen–Loeve theorem

E[ω2(x, θ)] =

∞i=1

λiφ2i (x).

Similarly E[ω′2(x, θ)] =

∞

i=1 λ′

iφ′2i (x) and by substituting x = y in Eq. (9), E[ω2(x, θ)] = E[ω

′2(x, θ)]

3.

E[ω(x, θ)ω(y, θ)] = E

∞

i=1

∞j=1

λi

λ jφi (x)φ j (y)ξi (θ)ξ j (θ)

=

∞i=1

∞j=1

λi

λ jφi (x)φ j (y)E[ξi (θ)ξ j (θ)].

Using the property 2 of ξi -s from the Karhunen–Loeve theorem

E[ω(x, θ)ω(y, θ)] =

∞i=1

λiφi (x)φi (y),

similarly

E[ω′(x, θ)ω′(y, θ)] =

∞i=1

λ′

iφ′

i (x)φ′

i (y).

Using Eq. (9)

E[ω(x, θ)ω(y, θ)] = E[ω′(x, θ)ω′(y, θ)].

Remark 1. Proposition 1 implies that the first and second order moments of a random process generated by the KLexpansion are invariant to a change in the physical domain. However, no invariance is claimed for the individual KLcoefficients.

Remark 2. If the random field is Gaussian, the mean and second order moments completely specify the randomfield. Proposition 1 implies that a Gaussian random field is invariant to the change of the physical domain providedfunctional form of the covariance kernel is retained the same. For an example in mechanics, consider a beam wherethe modulus of elasticity is modeled as a Gaussian random field. Proposition 1 implies that making a hole in the beamwill not alter the probability distribution of the modulus of elasticity in the remaining part of the beam.

130 S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145

Note that the domain independence property is proved here for the complete KL expansion. However, as mentionedearlier, for computational purpose the expansion is truncated after a few terms, as expressed in Eq. (3). Now the effectof this truncation on the domain independence property is explored and an error bound is derived. Note that fromProposition 1, the KL expansions ω(x, θ) and ω′(x, θ) lead to the same process. Therefore, now onwards only thenotation κ(x, θ) is used and the notation κ ′(x, θ) is discarded.

Proposition 2. Let ω, ω′ denote the truncated versions of ω and ω′, respectively. Let

ω(x, θ) − ω(x, θ) = ϵ(x, θ)

ω′(x, θ) − ω′(x, θ) = ϵ′(x, θ).

Then

∥ω(x, θ) − ω′(x, θ)∥L2(Ω) ≤ (∥ϵ′(x, θ)∥L2(Ω) + ∥ϵ(x, θ)∥L2(Ω)) (10)

where ∥ · ∥L2(Ω) denotes the L2(Ω) norm.

Proof. For brevity the explicit dependence on the arguments x and θ will not be mentioned. Now

∥ω − ω∥L2(Ω) = ∥ϵ∥L2(Ω) (11)

∥ω′− ω′

∥L2(Ω) = ∥ϵ′∥L2(Ω). (12)

An error bound for

∥ω − ω′∥L2(Ω) (13)

is now derived. Add and subtract ω to (ω − ω′) and then use the triangle inequality in Eq. (13), to get

∥ω − ω′∥L2(Ω) = ∥ω − ω + ω − ω′

∥L2(Ω)

≤ ∥ω − ω∥L2(Ω) + ∥ω − ω′∥L2(Ω)

= ∥ϵ∥L2(Ω) + ∥ω − ω′∥L2(Ω). (14)

Now consider ∥ω − ω′∥L2(Ω). Add and subtract κ to ω − ω′, take the norm, and use the triangle inequality to get

∥ω − ω′∥L2(Ω) = ∥ω − κ + κ − ω′

∥L2(Ω)

≤ ∥κ − ω∥L2(Ω) + ∥κ − ω′∥L2(Ω). (15)

Following the convergence of the KL theorem as mentioned in Eq. (8), ∥κ − ω∥L2(Ω) = 0 and ∥κ − ω′∥L2(Ω) = 0

∀x, y ∈ D ∩ D′. Therefore using the non-negative property of norms in Eq. (15), it can be concluded that∥ω − ω′

∥L2(Ω) = 0. Next consider the last term on the right hand side of inequality (14)

∥ω − ω′∥L2(Ω) = ∥ω − ω′

+ ω′− ω′

∥L2(Ω)

≤ ∥ω − ω′∥L2(Ω) + ∥ω′

− ω′∥L2(Ω) (using triangle inequality)

= ∥ω′− ω′

∥L2(Ω) (using ∥ω − ω′∥L2(Ω) = 0)

This inequality, combined with Eq. (12), leads to

∥ω − ω′∥L2(Ω) ≤ ∥ϵ′

∥L2(Ω). (16)

Substituting inequality (16) into inequality (14) we obtain

∥ω − ω′∥L2(Ω) ≤ (∥ϵ′

∥L2(Ω) + ∥ϵ∥L2(Ω)).

Note that for a desired level of accuracy the number of required terms L in the truncated KL expansions in twodomains may vary. Proposition 2 implies that if the KL expansions in both the domains are truncated after sufficientnumber of terms, then the random processes generated by them differ only by a very small amount. Inclusion of moreterms will reduce this difference further. Based on these observations the proposed method is presented next.

S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145 131

Fig. 1. Example 1: 2D problem, a plate with a hole. An FE mesh is also shown.

Fig. 2. Example 2: 3D problem, a structural steel angle with bolting holes. An FE mesh is also shown.

3. Proposed method using domain independence

The domain independence property of the KL expansion of a random process will be used in the following way.The first step is to simplify the spatial domain. That is, the original physical domain D is replaced by a new domain D′

over which the numerical solution of the IEVP will be easier and faster. Here D′ is chosen as the minimal boundingbox around D, such that D ⊂ D′. For demonstration of the proposed idea consider two examples. First, consider atwo-dimensional domain shown in Fig. 1. The mechanics problem here is a plane stress problem on a rectangulardomain with a circular hole at the center—call this domain as D. For solving the KL eigenvalue problem this domainwill be replaced simply by a rectangle whose sides coincide with the sides of the outer boundary of the original domainD—call this new domain as D′. Then, consider a three-dimensional domain shown in Fig. 2, this is a structural steelconnector angle with bolting holes. For this problem, the modified domain D′ is chosen as a (solid) cuboid as shownin Fig. 3, whose sides are of minimal dimensions to fit around the object. In other words, D′ is the smallest cuboidwhich completely contains the angle.

The next step is to use the Nystrom method for solving the integral equation (2) over the new domain D′. Accord-ingly, a numerical quadrature rule is chosen to evaluate the integral on domain D′. Due to the rectangular (in twodimensions) or cuboid (in three dimensions) nature of the domain D′, finding a quadrature rule over D′ will be farsimpler than to find a quadrature rule over the complex shaped domain D. Furthermore, in this approach the accuracyand cost of solving the IEVP will no longer be affected by the FE mesh chosen for the mechanics problem. Using anN point quadrature rule the IEVP becomes

Nj=1

w( j)C(x( j)1 , x2)φi (x

( j)1 ) = λiφi (x2) (17)

132 S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145

Fig. 3. Bounding box corresponding to the steel angle.

Fig. 4. Eigenvalue decay in the 2D problem.

Fig. 5. Comparison of pdfs for plate with and without hole using exponential covariance.

S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145 133

Fig. 6. Comparison of pdfs for plate with and without hole using triangular covariance.

Fig. 7. Comparison of pdfs for the lognormal process in the 2D problem.

where x( j)1

Nj=1 denotes the set of quadrature points and w( j)

Nj=1 denotes the set of corresponding weights. Dis-

cretizing the above equation completely over N quadrature points by replacing x2 = x(k)2 , k = 1, 2, . . . , N , we obtain

N simultaneous equations

Nj=1

w( j)C(x( j)1 , x(k)

2 )φi (x( j)1 ) = λiφi (x

(k)2 ), k = 1, 2, . . . , N . (18)

The above system of equations can be written in the form of a matrix eigenvalue problem as

Kφi = λiφi (19)

where K ∈ RN×N with the (k, j)th element as w( j)C(x( j)1 , x(k)

2 ), and φi ∈ RN with the kth element as φi (x(k)2 ). This

symmetric, positive definite algebraic eigenvalue problem can be solved using a standard solver. Note that only a fewlargest eigenvalues and associated eigenvectors are sufficient for the KL approximation. Therefore methods such asthe power iterations or Krylov-subspace based solvers might be efficient choices.

134 S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145

Fig. 8. Decay of eigenvalues in the 3D problem.

Fig. 9. Comparison of pdfs for steel angle and the cuboid in the 3D problem.

The proposed method thus can be summarized as follows.

• Step 1: Enclose the given domain in the minimal bounding box.• Step 2: Choose an appropriate quadrature rule for numerical integration over this box and calculate the correspond-

ing weights.• Step 3: Construct the matrix K by evaluating w( j)C(x( j)

1 , x(k)2 ) for k = 1, 2, . . . , N , j = 1, 2, . . . , N .

• Step 4: Solve the algebraic eigenvalue problem defined in Eq. (19).• Step 5: Post processing: When required, interpolate the eigenfunctions as required following the quadrature rule

used. For a mechanics problem the eigenfunctions need to be evaluated at the Gauss points of the FE mesh.

Because of the regular shape of the domain D′ a uniform numerical mesh can be used, which makes the imple-mentation simple. Quadrature rules such as Simpsons rule, trapezoidal rule, and mid-point rule can be used. In thefollowing numerical study the mid-point rule has been used.

4. Numerical studies

Two stochastic mechanics problems are considered for numerical studies. The first one is a plane stress problem,where a thin plate with a circular hole at center – as shown in Fig. 1 – is subjected to an in-plane uniaxial tensile

S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145 135

Fig. 10. Match of pdfs using correlation length as 1/12 of the width and six KL expansion terms in the 2D problem.

Fig. 11. Match of pdfs using correlation length as 1/12 of the width and fifty KL expansion terms in the 2D problem.

force. The second one is a three-dimensional one: a connector angle with bolting holes – as shown in Fig. 2 – issubjected to a uniaxial tensile force. Young’s modulus is assumed to be a random field in these two examples, witha known covariance function. For the two-dimensional (2D) problem two probability distributions are consideredfor the field: Gaussian and lognormal, whereas for the three-dimensional (3D) problem only Gaussian distribution isconsidered. For the mechanics problem, in both cases the presence of holes leads to stress concentration around them,thus demands a refined FE mesh in these areas. For instance, see the typical meshes in Figs. 1 and 2. Whereas therandom field should not require any such localized refinement. Using these two numerical examples three differentnumerical studies are conducted here. First, the domain independence of the KL approximation of the randomfield is numerically demonstrated and the effect of the correlation length on the truncation of the KL expansion isstudied. Next, a computational cost comparison between the proposed method and the existing FE-discretization basedGalerkin method is carried out. Both CPU time and memory requirement are compared for this purpose. Accuracyof the proposed method is also verified. Finally, for a separable kernel it is demonstrated that further cost saving ispossible by a corresponding separation of the eigenvalue problem.

136 S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145

Fig. 12. Convergence of stress with mesh refinement in the 2D problem.

Fig. 13. Convergence of variance of the simulated process with mesh refinement in the 2D problem using the Galerkin method.

Table 1Percentage relative error of eigenvalues using the proposed method: the 2D example.

Mode No Nystrom method Analytical value Percentage error

1 6.59 6.63 0.62 0.51 0.49 3.93 0.26 0.26 04 0.14 0.13 7.1

4.1. Numerical demonstration of domain independence

For demonstrating the domain independence of the KL approximations, the widely used Galerkin approach in anFE-discretization [9,30] is used to solve the IEVP. Accordingly, a set of FE bases – usually the one used for themechanics problem – is used to approximate the eigenfunctions. To find the unknown coefficients, this approximationis substituted in the IEVP—that is, in Eq. (2), and a Galerkin projection of the residual is taken on the same FE basisset. These steps lead to a generalized algebraic eigenvalue problem, which is then solved by a standard solver.

S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145 137

Fig. 14. Percentage relative error with mesh refinement for mode 1 in 2D problem using the proposed method.

Fig. 15. Convergence of variance of the simulated process with mesh refinement in the 2D problem using the proposed method.

First the 2D problem is considered. The thin plate is chosen to have an aspect ratio of two with the diameter ofthe hole being 0.5 times the width of the plate. For a Gaussian random field model of Young’s modulus the mean isassumed to be 200 MPa and standard deviation as 10 MPa. Two covariance kernels are considered: one exponentialwith the form

C(x, y) = e−|x1−x2|

α e−|y1−y2|

α (20)

and one triangular with the form

C(x, y) =

1 −

|x1 − x2|

α

1 −

|y1 − y2|

α

(21)

in two dimensions [30], where (x1, y1) and (x2, y2) denote the coordinates of two points. In both the cases thecorrelation length α is chosen to be five times the width of the plate. Constant strain triangle (CST) plane stresselements were used in the FE modeling. From a mesh convergence study for the mechanics problem – details ofwhich are given in the next subsection – it is found that about 1000 elements can capture the stress field accurately.Therefore this converged mesh is used for the Galerkin method of solution of the IEVP. As mentioned earlier, the

138 S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145

Fig. 16. Accuracy comparison between the proposed and Galerkin methods, the 2D example.

Fig. 17. Convergence of stress with mesh refinement in the 3D problem.

minimal bounding box here is a thin rectangular plate of same dimensions, but without the hole. For brevity here thesetwo domains are referred to as with hole and without hole. The FE mesh generated for the modified domain (withouthole) is independent from the mesh used in the original domain (with hole). Upon solving the eigenvalue problemsfor these two domains, decay of the eigenvalues of the covariance function with unit variance is plotted in Fig. 4.From this figure it is noted that both the domains have similar trend of decay, however, the numerical values of theeigenvalues vary. Based on these decay patterns six terms were retained in constructing the KL approximation. Foran arbitrarily chosen spatial location the pdfs of Young’s modulus for these two domains are then estimated usingthe truncated KL expansion. These pdfs are plotted in Fig. 5 for the exponential covariance, and in Fig. 6 for thetriangular covariance, respectively. In both of these figures an excellent match between the pdfs in two domains isobserved, which was verified using a 5% Kolmogorov–Smirnov test. Often the log-normal distribution is suggestedas a more realistic model for elastic properties [4]. To study the domain independence property, a lognormal processis next considered. The statistical moments for this lognormal field model of Young’s modulus are taken from [4] andan exponential covariance function is considered. Upon finding the KL approximations in two domains, the pdfs ofYoung’s modulus at the same spatial location are plotted in Fig. 7. Again an excellent agreement is found and this isverified using a 5% Kolmogorov–Smirnov test.

S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145 139

Fig. 18. Convergence of variance with mesh refinement in the 3D problem using the Galerkin method.

Fig. 19. Convergence of variance with mesh refinement in the 3D problem using the proposed method.

The 3D example is considered next. As mentioned earlier, the minimal bounding box here is a solid cuboid of sameouter dimensions. Constant strain tetrahedron elements were used for the FE modeling. The modulus of elasticity ofthe angle was modeled as a Gaussian random field with a mean of 200 MPa and standard deviation of 10 MPa, withthe covariance function being of exponential type. Decay of the eigenvalues for two domains is plotted in Fig. 8, andthe pdfs of the KL approximated process evaluated at an arbitrarily chosen spatial location are plotted in Fig. 9. Theobservations are similar to the 2D case. That is, although the individual eigenvalues may vary with a modification inthe domain, pdfs of the KL-approximated field remain domain independent. The match in the pdfs is again verifiedusing a 5% Kolmogorov–Smirnov test.

Recall that in Section 2 the importance of including sufficient number of terms was highlighted for this domainindependence. This sufficiency requirement is now numerically studied for the 2D problem. In general, with areduction in the correlation length, the eigenvalue decay becomes slower and thus the KL approximation requiresmore terms to be included. Here the correlation length is changed to 1

12× width of the plate. The pdfs from twodifferent domains are compared with different levels of truncation. In Fig. 10 six terms were included, that is, L = 6in Eq. (3). It is observed that the pdfs have different variance. Next the level of truncation L is increased to 50 andthe pdfs are plotted in Fig. 11. Now there is an excellent agreement between the generated pdfs, which highlights theimportance of including more number of terms.

140 S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145

Fig. 20. Comparison of memory used for the 2D problem.

Therefore by numerical experiments it is found that although the individual terms λi , φi (x) in the KL expansionmay vary with a change in the domain, the overall KL approximation

Li=1

√λiφi (x)ξi (θ) — that is, the summation

over a sufficient number of KL modes L — does not depend upon the domain.

4.2. Cost and accuracy analysis of the proposed method

Next the accuracy and computational cost of the proposed method is studied. In these studies the Galerkin methodis applied to the original domains and the proposed method is applied to the modified domains. First the 2D exampleis considered. Since the accuracy is of concern, first the convergence of meshes in both the methods is studied.Convergence of the FE mesh is studied for both the mechanics problem and the random field discretization problem.Accordingly, a sequence of meshes with successive refinement is considered. The axial stress at an arbitrarily chosenpoint in the stress concentration region is computed for these meshes and the trend is plotted in Fig. 12. The stresshas converged in a mesh with around 1000 elements. Note that this converged mesh was used in the previous section.The variance of the process estimated using the Galerkin method is plotted against the mesh size in Fig. 13. It isnoted that the variance converges at around 200 elements, thus much earlier than the mechanics problem. To study theconvergence of the proposed method the number of quadrature points is increased successively. Let λN ys denote theeigenvalue found using the Nystrom method and λAnal denote the eigenvalue computed analytically. For the highesteigenvalue the relative error, defined as |λN ys−λAnal |

λAnal∗ 100%, is plotted against the refinement in the quadrature mesh

in Fig. 14. Here the word element refers to the rectangular region within four quadrature points, this terminology isintroduced only to compare with the FE meshes. For a mesh with 100 elements the relative errors in highest foureigenvalues are tabulated in Table 1. The convergence of the variance of the process with respect to the refinementin the quadrature mesh is plotted in Fig. 15. It is found that around 1400 elements yield a converged estimate. Toverify the accuracy of the proposed method the pdf of Young’s modulus estimated using the proposed method (on themodified domain) and the Galerkin method (on the original domain) is plotted in Fig. 16, and an excellent match isfound. This match shows that the proposed method is accurate. The same set of experiments is carried out for the 3Dexample. Using FE mesh the convergences of the axial stress and the variance of the process are plotted in Figs. 17and 18, respectively. Again the variance converges with a much coarser mesh compared to the stress. This observationfor both the examples suggests that the optimal meshes for the mechanics problem and the random field discretizationneed not be the same. Convergence of the variance using the proposed method is plotted in Fig. 19.

The computational cost is studied next. For the 2D example the memory requirement and the CPU time on a singleprocessor with 2.8 GHz speed are plotted in Figs. 20 and 21, respectively. A few important data points in these plotsare tabulated in Table 2. From Fig. 20 it is noted that for any fixed mesh size the Galerkin method requires morememory compared to the proposed method. However, recall that the optimal mesh sizes for these two methods are notsame. Therefore, the cost comparison should be made for the optimal meshes. To this end, consider three points in this

S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145 141

Fig. 21. Comparison of CPU times for the 2D problem.

Table 2Time and memory comparison for various optimal meshes for the 2D problem.

Number of elements Time (s) Memory (Mb)

Optimal Mechanics mesh 1000 1.2 × 104 29Optimal mesh for Galerkin method 200 250 4.8Optimal mesh for proposed method 1400 0.8 16.2

figure marked as Q1, Q2, and Q3. They correspond to the optimal mesh for the proposed method, the optimal mesh forthe mechanics problem in FE—referred to here as the optimal mechanics mesh, and the optimal mesh for the randomfield discretization using the FE mesh—referred to here as the optimal stochastic mesh, respectively. From these threepoints it is observed that the memory requirement of the proposed method is lower than the optimal mechanics meshbut higher than the optimal stochastic mesh in the Galerkin method. Next consider the CPU time comparison that isplotted in the semi-log scale in Fig. 21. Here it is noted that the proposed method is much faster compared to theGalerkin method, and the difference is of three to four orders of magnitude. The exact figures are given in Table 2.Both the optimal meshes in the Galerkin method are far more expensive than the optimal mesh in the proposed method.Note that the optimal mesh in the proposed method is finer than the stochastic optimal mesh in the Galerkin method.Thus the higher memory requirement in the proposed method can be attributed to higher mesh density. However, thecost saving can be explained using a computational complexity analysis for 2D problem. Solving Eq. (2) using theGalerkin method with FE basis results in a generalized eigenvalue problem

Ax = λBx . (22)

Here the cost of formulation is O(e2n2e +e2n2

e F +en2e G) where e is the number of elements, n is the number of nodes,

ne is the number of nodes per element, F is the cost of computation for each entry in the elemental A matrix, andG is the cost of computation for each entry in the elemental B matrix. Evaluation of each entry involves numericalevaluation of the integral whose cost depends on the chosen integration rule. On the other hand, in the proposedmethod let the rectangular domain be divided into N × N rectangular elements. If E is the cost of computation ofeach entry, then the total computational cost would be O(N 2 E). It must be observed that the proposed method leadsto an ordinary algebraic eigenvalue problem Eq. (19), hence requires only computation of a single matrix. Further Eis much lower than F or G, as it is only the cost of evaluation of the covariance kernel.

These investigations in terms of a computational complexity analysis reveal that the computation of individualelements in the matrices is far more expensive in the Galerkin method compared to the proposed method. Thus the

142 S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145

Fig. 22. Comparison of memory used for the 3D problem.

Fig. 23. Comparison of time taken with mesh refinement in the proposed method and the Galerkin method in the 3D problem.

stage of assembling of matrices contributes dominantly to the total cost in the Galerkin method. This relative costsaving makes the proposed method computationally inexpensive.

Similar observations are made for the 3D example as well. For this example the memory requirement and CPUtimes are plotted in Figs. 22 and 23, respectively. In Fig. 22 it is noted that the memory requirement is very close inboth the methods. However, from Fig. 23 it is noted that the proposed method is about one to two orders of magnitudefaster than the Galerkin method. The point Q2 is not generated here as it required far more computational time, andits contribution to the conclusions here may not be significant.

4.3. Further cost reduction for separable covariance functions

Many of the commonly used covariance functions are separable [30,18]. For these functions the proposed methodcan be further enriched to exploit this separability, and thus to reduce the computation further. For these kernels theeigenvalues and eigenfunctions can be obtained as products of solutions of one-dimensional problems [25]. However,in this case the domain must be of a rectangular shape in two-dimensions and cuboid shape in three-dimensions. Thus,irrespective of the numerical method used to solve the integral eigenvalue problem, modification of the domain fromD to D′ is necessary and the domain independence property plays a key role. In the current work, although the chosen

S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145 143

Fig. 24. Effect of using separability of the covariance kernel: CPU time comparison for the 2D example.

Fig. 25. Effect of using separability of the covariance kernel: CPU time comparison for the 3D example.

kernels are separable, the separability was not used in the numerical results presented in the previous subsection. Thischoice was made to test the proposed method for any general kernel. This separability is next used for both 2D and 3Dexamples, and the computational cost is presented in Figs. 24–27. All results are generated using the Nystrom method.In both of these plots it is noted that the separability reduces the cost by a few orders of magnitude. For both the 1400elements mesh in 2D and the 5000 elements mesh 3D the cost saving is of four orders of magnitude. Furthermore, inboth the problems the cost growth rate with respect to the mesh size is far lower while using the separability.

5. Conclusion

By using the domain independence property of the KL expansion along with the Nystrom method, thecomputational cost of solving the integral eigenvalue problem is reduced by up to four orders of magnitude comparedto the finite element discretization. For separable kernels the cost is reduced even further. Since the mechanicsproblem and the IEVP are discretized using different discretizations, optimality of the meshes in terms of cost andaccuracy for these two problems can be achieved independently. The proposed method, in its current form, is simpleto implement. Thus it could serve as a very efficient alternative to currently used methods, especially to the FE based

144 S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145

Fig. 26. Effect of using separability of the covariance kernel: Memory requirement comparison for the 2D example.

Fig. 27. Effect of using separability of the covariance kernel: Memory requirement comparison for the 3D example.

Galerkin method. It is found to be successful for a non-Gaussian processes as well. Further potential developmentsof this method should include exploring various quadrature rules, finding a more suitable algebraic eigensolver, andparallelizing.

Acknowledgments

The authors would like to acknowledge financial support from the Board of Research in Nuclear Sciences (BRNS)Grant No. 2011/36/41-BRNS/1977 and the ISRO-IISc Space Technology Cell Grant No. ISTC/CCE/DG/273.

References

[1] K.R. Gurley, M.A. Tognarelli, A. Kareem, Analysis and simulation tools for wind engineering, Probab. Eng. Mech. 12 (1) (1997) 9–31.[2] G.I. Schueller, H.J. Pradlwarter, C.A. Schenk, Non-stationary response of large linear FE models under stochastic loading, Comput. Struct.

81 (8) (2003) 937–947.[3] G. Stefanou, M. Papadrakakis, Stochastic finite element analysis of shells with combined random material and geometric properties, Comput.

Methods Appl. Mech. Engrg. 193 (1) (2004) 139–160.

S. Pranesh, D. Ghosh / Comput. Methods Appl. Mech. Engrg. 285 (2015) 125–145 145

[4] JCSS-probabilistic model code. part III resistance models — steel. Joint Committee on Structural Safety, 2001.[5] M. Ostoja-Starzewski, Random field models of heterogeneous materials, Int. J. Solids Struct. 35 (19) (1998) 2429–2455.[6] A. Nouy, A. Clement, F. Schoefs, N. Moes, An extended stochastic finite element method for solving stochastic partial differential equations

on random domains, Comput. Methods Appl. Mech. Engrg. 197 (51) (2008) 4663–4682.[7] C.A. Schenk, G.I. Schueller, Uncertainty Assessment of Large Finite Element Systems, Springer, Berlin/Heidelberg, 2005.[8] M. Kleiber, T.D. Hien, The Stochastic Finite Element Method: Basic Perturbation Technique and Computer Implementation, Wiley, 1992.[9] R. Ghanem, P.D. Spanos, Stochastic Finite Elements: A Spectral Approach, revised ed., Dover Publications, 2003.

[10] A. Der Kiureghian, J.-B. Ke, The stochastic finite element method in structural reliability, Probab. Eng. Mech. 3 (2) (1988) 83–91.[11] S. Mahadevan, A. Haldar, Practical random field discretization in stochastic finite element analysis, Struct. Saf. 9 (4) (1991) 283–304.[12] E. Vanmarcke, M. Grigoriu, Stochastic finite element analysis of simple beams, J. Eng. Mech. 109 (1983) 1203–1214.[13] G.A. Fenton, E.H. Vanmarcke, Simulation of random fields via local average subdivision, J. Eng. Mech. 116 (8) (1990) 1733–1749.[14] H.L. Van Trees, Detection, Estimation and Modulation Theory, Part I, Wiley, 2001.[15] C.-C. Li, A. Der Kiureghian, Optimal discretization of random fields, J. Eng. Mech. 119 (6) (1993) 1136–1154.[16] G. Deodatis, M. Shinozuka, The weighted integral method. II: response variability and reliability, J. Eng. Mech. 117 (8) (1991) 1865–1877.[17] B.A. Zeldin, P.D. Spanos, On random field discretization in stochastic finite elements, Trans. ASME J. Appl. Mech. 65 (1998) 320–327.[18] P.D. Spanos, M. Beer, J. Red-Horse, Karhunen–Loeve expansion of stochastic processes with a modified exponential covariance kernel, J.

Eng. Mech. 133 (7) (2007) 773–779.[19] R. Ghanem, The nonlinear Gaussian spectrum of log-normal stochastic processes and variables, J. Appl. Mech. 66 (4) (1999) 964–973.[20] K.K. Phoon, S.P. Huang, S.T. Quek, Simulation of second-order processes using Karhunen–Loeve expansion, Comput. Struct. 80 (12) (2002)

1049–1060.[21] K.K. Phoon, H.W. Huang, S.T. Quek, Simulation of strongly non-gaussian processes using Karhunen–Loeve expansion, Probab. Eng. Mech.

20 (2) (2005) 188–198.[22] K.E. Atkinson, The Numerical Solution of Integral Equations of the Second Kind, Cambridge University Press, 1997.[23] G. Stefanou, The stochastic finite element method: past, present and future, Comput. Methods Appl. Mech. Engrg. 198 (2009) 1031–1051.[24] D.L. Allaix, V.I. Carbone, Numerical discretization of stationary random processes, Probab. Eng. Mech. 25 (2010) 332–347.[25] C. Schwab, R.A. Todor, Karhunen–Loeve approximation of random fields by generalized fast multipole methods, J. Comput. Phys. 217 (1)

(2006) 100–122.[26] B. Khoromskij, A. Litvinenko, H. Matthies, Application of hierarchical matrices for computing the Karhunen–Loeve expansion, Computing

84 (2009) 49–67.[27] C.F. Li, Y.T. Feng, D.R.J. Owen, D.F. Li, I.M. Davis, A Fourier–Karhunen-0Loeve discretization scheme for stationary random material

properties in SFEM, Internat. J. Numer. Methods Engrg. 73 (2008) 1942–1965.[28] D.C. Charmpis, M. Papadrakakis, Improving the computational efficiency in finite element analysis of shells with uncertain properties,

Comput. Methods Appl. Mech. Engrg. 194 (2005) 1447–1478.[29] R. Gutierrez, J.C. Ruiz, M.J. Valderrama, On the numerical expansion of a second order stochastic process, Appl. Stoch. Models Data Anal.

8 (2) (1992) 67–77.[30] S.P. Huang, S.T. Quek, K.K. Phoon, Convergence study of the truncated Karhunen–Loeve expansion for simulation of stochastic processes,

Internat. J. Numer. Methods Engrg. 52 (9) (2001) 1029–1043.[31] S. Choudhary, Numerical methods for solving the eigenvalue problem involved in the Karhunen-Loeve decomposition (Ph.D. thesis), Indian

Institute of Science, 2012.[32] W. Betz, I. Papaioannou, D. Straub, Numerical methods for the discretization of random fields by means of the Karhunen–Loeve expansion,

Comput. Methods Appl. Mech. Engrg. 271 (2014) 109–129.[33] J. Parvizian, A. Duster, E. Rank, Finite cell method, Comput. Mech. 41 (1) (2007) 121–133.