Solution Qualification and Uncertainty Estimation forLocal Quantities

G.B. DengEquipe de Modelisation Numerique

Laboratoire de Mecanique des FluidesEcole Centrale de Nantes

1 Rue de la Noe, 44321 Nantes, Francee-mail: Ganbo.Deng@ec-nantes.fr

1 Introduction

Method based on Richardson extrapolation using the concept of Grid Convergence Index(GCI) proposed by Roache [4] is the most commonly used and the most convincing ap-proach for numerical uncertainty estimation. With a carefully designed grid set, the GCIapproach can give reliable uncertainty estimation for global quantity such as the resis-tance. However, it becomes more problematic when it is applied to local quantity. Localquantity on a systematically refine grid set usually does not exhibit a monotonic conver-gence behaviour. In the case where monotonic convergence behaviour is observed, theapparent order of convergence is often quite different from the expected theoretical order.In these cases, the result of the uncertainty estimation can not be consider as reliable. Itdepends on the ad-hoc choice in the uncertainty estimation procedure adopted and thedata set selected by a user. In the present paper, we propose a procedure that combinesthe result of error estimation for a global quantity and data difference between two gridsto obtain the uncertainty estimation for a local quantity. This procedure is tested onthe two test cases proposed for the present workshop. For each test case, different typesof grid with different convergence behaviour are employed. It has been found that thisprocedure provides not only a plausible uncertainty estimation for local quantity, but alsouseful informations that can be used to qualify the numerical solution.

2 Uncertainty Estimation Procedure

To evaluate the uncertainty for a local quantity, we first determine the L1 norm errorfor this quantity using the Grid Convergence Index approach. This can be proceeded asfollow. First, we choose a target region. The target region can be a simple rectangulardomain inside the computational domain. Then we build a test grid in the target region.A uniform Cartesian grid is sufficient for this purpose. Next, we interpolate the solutionfrom all grids to the test grid. In the present study, a 4th order accurate interpolationmethod using a least squares approach is employed. This interpolated solution will be

noted as φk. After that, we evaluate the L1 norm of the solution difference between thegrid k and the grid 1 (the finest grid) as:

Errk =∑

i

|φki − φ1

i | V oli

Here, V oli is the volume of the ith element of the test grid. Richardson extrapolation isapplied to the data set Errk to determine the estimated true value Err0 which is expectedto be negative. Select a grid k as reference grid. A grid with a refinement ratio of 2 will bechosen in the present study. Construct a data field by scaling the data difference betweenthe reference grid and the finest grid as

| Err0 |Errk

(φk

i − φ1i

)The L1 norm of this data field is equal to the estimated L1 norm error on the finestgrid. Hence, it can be considered as an approximation to the error on the finest grid.Applying a safety factor Fs according to the common practice in a GCI approach, thenumerical uncertainty in the ith element of the test grid for the finest grid solution canbe approximated by

Fs | Err0 |Errk

(φk

i − φ1i

)The value for the safety factor Fs can be 1.25 or 3 depending on the reliability of theextrapolation for Err0.

3 The Numerics

Computations are performed with the ISIS-CFD flow solver developed by EMN (EquipeModelisation Numerique, i.e. CFD Department of the Fluid Mechanics Laboratory). Tur-bulent flow is simulated by solving the incompressible Reynolds-averaged Navier-Stokesequations (RANSE). The solver is based on the finite volume method to build the spatialdiscretization of the transport equations. The velocity field is obtained from the momen-tum conservation equations and the pressure field is extracted from the mass conservationconstraint, or continuity equation, transformed into a pressure-equation. In the case ofturbulent flows, additional transport equations for modeled variables are discretized andsolved using the same principles. The gradients are computed with a least square approachbased on linear polynomial or an approach based on Gauss’s theorem, both ensuring a for-mal first order accuracy and giving a second order accurate result on a nearly symmetricstencil. Inviscid flux is computed with a piecewise linear reconstruction associated withan upwinding stabilizing procedure which ensures an second order formal accuracy whenflux limiter is not applied. Viscous flux are computed with a central difference schemewhich guarantee a first order formal accuracy. We have to rely on mesh quality to obtaina second order discretization for the viscous term.

2

4 Manufactured Solution Test Case

4.1 Manufactured solution and boundary conditions

In this exercise, except for one case where the turbulence eddy-viscosity is prescribedusing the manufactured solution, we compute the numerical solution to the Navier-Stokesequation with the Spalart-Allmaras model. Different choices for the turbulence eddy-viscosity are possible in [3]. We have chosen the solution ν for the Spalart-Allmaras modeldesigned as MS2 in [3] that has a y2 asymptotic behaviour near the wall. The kinematicviscosity is 10−6 as suggested for the Workshop. To reduce the numerical uncertainty,the viscous damping function fv1 in the Spalart-Allmaras model is set to 1 both in themanufactured solution and in the flow solver. This modification is applied only to theMMS test case. For the backward facing step test case, no modification is made to theSA model.

The computational domain is defined by 0.5 ≤ x ≤ 1 and 0 ≤ y ≤ 0.5. The exactmass flux is imposed on all boundaries. The pressure at the boundary is updated as:

P numbnd = P num

internal + P exactbnd − P exact

internal (1)

At the outlet boundary x=1, the velocity field is updated in the same way. Those arespecial boundary conditions for this verification exercise. Dirichlet boundary conditionsare applied to the velocity field at the wall y=0, at the inlet x=0.5 and at the upperboundary y=0.5.

4.2 Mesh description

To evaluate the uncertainty estimation approach, three different types of grid have beenemployed. The first one is a simple Cartesian grid, the second one is an unstructuredtriangular grid, and the last one is a locally refined quadrilateral unstructured grid.

Cartesian grid

The manufactured solution is quite smooth. There is no large gradient near the wall.The solution can be represented correctly with a uniform grid. However, due to thehigh Reynolds number, numerical experiments show that it is necessary to use a non-uniform grid in the wall normal direction in order to ensure that the solution is in theasymptotic convergence range. For this reason, we employ a uniform grid distribution inthe x direction and a wall-stretched grid distribution in the y direction. The grid aspectratio hx/hy for the first grid cell next to the wall is about 12.5. The dimensions of the sixgrids used in the present study are 33 × 65, 49 × 97, 65 × 129, 97 × 192, 129 × 257 and193× 385.

Unstructured triangular grid

As it is necessary to employ a wall-stretched grid, it is not trivial to generate an un-structured triangular mesh using grid generation software. Therefore, the triangular gridemployed in the present study is obtained by triangulating the previous Cartesian grid.

3

Unstructured quadrilateral grid

For high Reynolds number viscous flows, it is not easy to generate a high quality tetrahe-dral/triangular grid for the viscous layer. This is the reason why hexahedral/quadrilateralgrid is preferred for such applications. Generation of a block-structured hexahedral gridfor a complex geometry involving many appendages, for instance, is a not a trivial task.It may take weeks of human time, which explains why hexahedral unstructured grid isa promising alternative. Recent grid generation software such as HEXPRESS, developedby NUMECA, allows to generate such a kind of grid. This software is now routinelyemployed in our CFD team for applications involving very complex geometry like fullyappended ship or yacht. HEXPRESS uses a successive refinement technique to capturethe geometry, which leads to a grid containing an abrupt change in mesh size and non-conformal elements near the refinement interfaces. It is mandatory to ensure that thenumerical solution converges towards the exact solution when such a kind of unstruc-tured grid is employed. Consequently, a set of 6 grids has been generated with a similarnumber of grid nodes and refinement ratio as the previous Cartesian grid set. However,a perfect grid similarity can not be ensured. Figure 1 displays a zoom of the coarsestgrid where the above-mentioned special features can be easily identified. Two refinementregions have been intentionally introduced in order to study the convergence behaviourof the code under this situation. The number of grid cells for the 6 grids are 2189, 4601,8480, 16521, 32919 and 73629.

X

Y

0.6 0.65

0

0.02

0.04

0.06

0.08

0.1

Figure 1: Unstructured quadrilateral grid

4

4.3 Convergence behaviour

4 test cases will be considered. The first test case (referred as test case A hereafter)solve the Navier-Stokes equation on the Cartesian grids with prescribed turbulent eddy-viscosity. All equations are solved for other test cases. The test case B, C and D use theCartesian grid, the locally refined quadrilateral grid and the triangular grid respectively.In this paper, we consider only the module of the velocity φ =

√u2 + v2. Figure 2 displays

the L1 norm error for each test case together with the observed order of convergence. Sec-ond order convergence is observed for the first three test cases with some data scatteringproblem for the test case B and C. It is unexpected that the order of convergence for thetest case D is lower than 1. This deterioration is caused by the fact that the gradienton the triangular grid is computed only with first order accuracy everywhere. As thesource terms in the transport equation for turbulent quantity depend on the gradient,the accuracy of the numerical scheme is considerably deteriorated. It must be mentionedthat this deterioration is only due to the source term in the turbulent transport equationrather than the discretization of the convection and the diffusion terms in the transportequation. In fact, a previous study using the same manufactured solution shows that weobtain a numerical solution with second order accuracy to the Navier-Stokes equationwith prescribed turbulent eddy-viscosity using the same triangular grids [2].

Number of cells

L1no

rmer

roro

f|V

|

104 105

10-6

10-5

Case ACase BCase CCase D

2.0

2.0

1.9

1.9

1.8

2.6

1.9

2.1

1.9

1.7

2.6

1.7

2.0

2.4

1.8

0.8 0.50.5

0.50.6

Figure 2: L1 norm error of the velocity module for different cases.

5

4.4 Uncertainty estimation

The target domain is the same as the computational domain. The test grid is a 100×100uniform grid. The L1 norm errors of the velocity module for different test cases are8.407E-7, 1.284E-6, 1.697E-6 and 7.721E-6 respectively when it is evaluated directly onthe finest grid. The same quantities evaluated on the test grid using the interpolatedfinest grid numerical solution and the finest grid exact solution are 8.248E-7, 1.175E-6,1.611E-6 and 6.505E-6, giving a relative error of 1.9%, 8.5%, 5.1% and 15.8% respectively.Except for the triangular grid, error due to the interpolation to the test grid is acceptable.

Case A

In this test case, we solve the Navier-Stokes equation on the Cartesian grid set with aprescribed turbulence eddy-viscosity. This test case is selected to test the uncertaintyestimation procedure in a case for which the numerical solutions are in the asymptoticconvergence range. Table 1 gives the solution difference on 6 different grids. We need toextrapolate the data in order to obtain the L1 norm error on the finest grid. The reliabilityof the extrapolated result depends on the extrapolation approach and the data set usedfor the extrapolation. Their appropriate choice is also one of the issue we investigate inthe present study. We employ Richardson extrapolation using one term expansion withunknown exponent. The least squares approach is employed when more that 3 data valuesare used for the extrapolation. Table 2 gives the extrapolated results for different datasets, together with the expected result computed with the interpolated exact solution.

Grid ID Case A Case B Case C Case DGrid 1 0 0 0 0Grid 2 9.088027E-7 1.167446E-6 1.753194E-6 1.906818E-6Grid 3 2.121582E-6 2.737624E-6 5.711813E-6 3.451730E-6Grid 4 5.531578E-6 7.418501E-6 1.186993E-5 6.419474E-6Grid 5 1.031215E-5 1.351308E-5 2.207135E-5 1.028203E-5Grid 6 2.439133E-5 4.386837E-5 6.288166E-5 2.226599E-5

Table 1: Solution difference for different test cases

Grid set Case A Case B Case C Case D6-5-4 -0.0289/2.1 0.3947/3.5 0.6434/3.5 0.2121/2.25-4-3 -0.0598/2.0 -0.1677/1.8 -0.1459/1.9 0.0650/1.84-3-2 -0.0697/2.0 -0.0704/2.2 -0.4613/1.4 -0.3416/0.93-2-1 -0.0809/1.9 -0.1021/1.9 -0.0933/2.6 -1.1306/0.4All -0.0606/2.0 0.0306/2.8 0.0392/2.7 -0.0451/1.7

5 to 1 -0.0706/2.0 -0.1057/1.9 -0.2158/1.8 -0.2643/1.14-3-2-1 -0.0758/1.9 -0.0872/2.0 -0.2041/1.8 -0.5953/0.7

Expected -0.0825 -0.1175 -0.1611 -0.6505

Table 2: Estimated Err0 × 105 and apparent order of convergence

The apparent order of convergence obtained with different grid triplets suggests thatthe numerical solutions are in the asymptotic convergence range. The fact that the

6

apparent order decreases from 2.1 to 1.9 is due to the fact that the pressure equation hasan undefined order of convergence ranging from 1 to 2 as demonstrated in the previousstudy [2]. Since the expected true value is 8.25E-7, the best estimation 8.10E-7 is obtainedwith the finest grid triplet. This observation suggests that in an uncertainty estimationprocedure, when there is little data scattering, the finest grid triplet is the best choice.To estimate the uncertainty on the grid 1, we choose the grid 3 as reference grid. WithErr0=-8.10E-7, Err3=2.12E-6, and a safety factor Fs=1.25, the numerical uncertaintyon the finest grid is estimated as:

Fs | Err0 |Err3

(φ3

i − φ1i

)= 0.48

(φ3

i − φ1i

)Case B

The same grid set is employed as for the previous test case. But the turbulent eddy-viscosity is determined numerically. Table 1 and 2 give the solution difference and theresult of the extrapolation. The apparent orders of convergence observed with grid tripletsare acceptable except for the coarsest one (triplet 6-5-4). It is preferred to exclude the grid6 from the extrapolation procedure. As there is not too much data scattering, we selectthe finest grid triplet (3-2-1), giving the result Err0=-1.02E-6. With Err3=2.74E-6, anda safety factor Fs=1.25, the numerical uncertainty on the finest grid is estimated as:

Fs | Err0 |Err3

(φ3

i − φ1i

)= 0.47

(φ3

i − φ1i

)Case C

Results for this test case is obtained with the locally refined quadrilateral grid sets. Fromthe convergence behaviour based on the exact L1 norm error shown in the previous section,data scattering is expected. Apparent orders of convergence obtained with grid tripletsindicate that the coarsest grid is too coarse to be included in the extrapolation procedure.The apparent orders of convergence on the two finest grid triplet (4-3-2) and (3-2-1) arequite different from the theoretical order p=2, which is an indication of data scattering.For this case, least squares approach with more data points is preferred. The least squaresapproach with the grid set (5-4-3-2-1) is an appropriate choice. This gives an estimationErr0=-2.158E-6 which is considerably higher than the expected value -1.611E-6. WithErr3=5.71E-6, and a safety factor Fs=1.25, the numerical uncertainty on the finest gridis estimated as:

Fs | Err0 |Err3

(φ3

i − φ1i

)= 0.47

(φ3

i − φ1i

)Case D

The results for this test case is obtained with the triangular grid sets. Verification usingthe exact solution reveals that the order of convergence for this solution is smaller than 1.Apparent orders of convergence observed on grid triplet decrease monotonically from 2.2with the coarsest triplet to 0.4 with the finest triplet (Table 2). This observation clearlyindicates that the convergence behaviour of the numerical solution is problematic. Wecan also observe that the estimated solution for Err0 depends strongly on the choice ofthe data set. It is evident that reliable estimation is impossible for this test case. To give

7

an uncertainty estimation anyway, we choose the most conservative estimation, namelyErr0=-1.1306E-5 with the finest grid triplet. This estimation is much higher than theexpected value 6.505E-6. The apparent order of convergence is so low that we don’tneed to increase the safety factor. With Err3=3.45E-6, and a safety factor Fs=1.25, thenumerical uncertainty on the finest grid is estimated as:

Fs | Err0 |Err3

(φ3

i − φ1i

)= 4.1

(φ3

i − φ1i

)We can also use the procedure proposed by Eca and Hoekstra [1] who suggest to use threetimes the data range as uncertainty in the case where the apparent order of convergenceis smaller that 0.5.

4.5 Verification of the uncertainty estimation

Figure 3 shows the profile of error and the estimated uncertainty at x=0.75 for differenttest cases. Fairly good results are obtained for the test cases A and B. The error barsalmost coincide with the error line. This observation suggests that when the estimationof the L1 norm error is reliable, the uncertainty estimation for local quantity is alsoreliable, both in magnitude and in sign. Due to data scattering, accurate estimation forthe L1 norm error is difficult for the test case C. Resulting uncertainty estimation forlocal quantity is less satisfactory in the near wall region where the error bars is about2 times higher than the error. But the uncertainty estimation can be still consider assuccessful, since the error line is bounded by the error bars in region where the error ishigh. For case D, the uncertainty estimation is too pessimistic globally. It is interestingto note that although the L1 norm errors are over-estimated for the test cases C and D,the local maximum error near y=0.15 is correctly estimated for both cases.

5 Backward facing step test case

5.1 Mesh for different test cases

4 different type grids are employed for the backward facing step test case. The first one(test case A) is a Cartesian grid (Figure 4). With this kind of topology, grid nodes areuselessly clustered in some region, making the convergence of the computation extremelydifficult. The most attractive feature of this type of grid is that there is no grid qualityproblem. We expect that it can give a reliable reference solution. A block-structured gridis employed for the test case B. Except a L type block employed around the upper corner,Cartesian blocks are used elsewhere. The grid set B suffers from a grid quality problemaround the upper corner of the step. Grid resolution is not sufficient in this region. Inaddition, there is a mis-alignment default which is typical in a block-structured grid. Thetest case C employs an unstructured quadrilateral grid generated by the commercial gridgeneration software HEXPRESS. High grid density observed in figure 4 is the result oflocal refinement around the corner. It doesn’t represent the global grid density of the gridset C. The last test case (test case D) uses a single block structured grid. It is the gridset A proposed for the last workshop [1].

8

Y

Erro

r

0.05 0.1 0.15 0.2-4E-05

-2E-05

0

2E-05

4E-05 Case B

Y

Erro

r

0.05 0.1 0.15 0.2-4E-05

-2E-05

0

2E-05

4E-05 Case A

Y

Erro

r

0.05 0.1 0.15 0.2-4E-05

-2E-05

0

2E-05

4E-05 Case C

Y

Erro

r

0.05 0.1 0.15 0.2-0.0002

-0.0001

0

1E-04

0.0002 Case D

Figure 3: Verification of the estimated uncertainty at x=0.75.

5.2 Uncertainty estimation

The target region is a rectangular domain defined by (0.05≤x≤8, 0.05≤y≤1.5). It iscovered by a 200×100 uniform test grid. Uncertainty estimation is performed with theprocedure described above.

Test case A, Cartesian grid

Cartesian grid is employed for this test case. 6 grids have been employed for the computa-tion. The divergence observed on the finest grid triplet may due to insufficient convergenceon the finest grid. Data scattering is observed. But error estimation is quite reliable whenleast squares approach is employed. The solution converges with second order accuracyas expected. We choose the result obtained with the least square approach using the gridset (6 to 1). Grid 6 is chosen as reference grid. With Err0=-0.0034, Err6=0.00932, anda safety factor Fs=1.25, the numerical uncertainty on the finest grid is estimated as:

Fs | Err0 |Err6

(φ6

i − φ1i

)= 0.46

(φ6

i − φ1i

)

9

X

Y

0 0.1 0.2

1

1.1 Case A

X

Y

0 0.1 0.2

1

1.1 Case B

X

Y

0 0.1 0.2

1

1.1 Case D

X

Y

0 0.1 0.2

1

1.1 Case C

Figure 4: Mesh for different test cases.

Grid ID Case A Case B Case C Case DGrid 1 0 /35200 0 /25200 0 /95500 0 /57600Grid 2 8.69E-4/28512 4.79E-3/16128 5.41E-3/55661 0.0540/40000Grid 3 1.78E-3/22528 7.54E-3/12348 1.72E-2/26328 0.0871/32400Grid 4 3.35E-3/17248 1.21E-2/ 9072 2.93E-2/15808 0.125/25600Grid 5 5.62E-3/12672 1.58E-2/ 6300 6.05E-2/ 7793 0.171/19600Grid 6 9.32E-3/ 8800 2.74E-2/ 4032 -/- 0.226/14400

Table 3: Solution difference and number of grid cells for different test casesGrid set Case A Case B Case C Case D Case D2

6-5-4 -0.0033/ 1.9 0.0093/ 4.6 -/- -1.279/0.2 -0.3125-4-3 -0.0050/ 1.6 0.0000/-2.3 -0.0009/2.0 -0.489/0.5 -0.2844-3-2 -0.0010/ 3.3 -0.0019/ 2.6 -0.0128/1.3 -1.225/0.2 -0.3513-2-1 -/-0.6 -/-0.2 -0.0120/1.4 -0.723/0.4 -0.3506 to 1 -0.0034/ 1.9 -0.0107/ 1.3 -/- -0.782/0.4 -0.3245 to 1 -0.0034/ 1.9 -0.1096/ 0.2 -0.0082/1.7 -0.749/0.4 -0.3344 to 1 -0.0035/ 1.9 -0.0180/ 1.0 -0.0124/1.4 -0.859/0.3 -0.351

Table 4: Richardson extrapolation results for different test cases

10

Test case B, block-structured grid

6 block-structured grids are used for the computation. Due to severe data scattering,least squares approach is the only choice. As the result obtained with the data set 5 to1 is unrealistic due to the too low apparent order of convergence, we choose the mostconservative estimation among the remaining results, namely Err0=-0.0180. With theresult for the reference grid Err5=0.0153, and a safety factor Fs=1.25, the numericaluncertainty on the finest grid is estimated as:

Fs | Err0 |Err5

(φ5

i − φ1i

)= 1.47

(φ5

i − φ1i

)Test case C, unstructured grid

The 5 grids employed in this test case are generated with the commercial grid generationsoftware HEXPRESS. There is no data scattering problem. The coarsest grid is toocoarse. The apparent order of convergence is about 1.4. We choose the result obtainedwith the finest grid triplet, namely Err0=-0.0120. With the result for the reference gridErr3=0.0172, and a safety factor Fs=1.25, the numerical uncertainty on the finest grid isestimated as:

Fs | Err0 |Err3

(φ3

i − φ1i

)= 0.87

(φ3

i − φ1i

)Test case D, single block structured grid

The grid used for this test case is the grid set A employed in the last workshop. There islittle variation in the apparent order of convergence. But the value is too low (less than0.4). The type of grid and the nature of the numerical scheme make us believe that sucha convergence order is unrealistic. To obtain a better estimation for the uncertainty, weemploy a polynomial method with a first and a second order term in the Taylor expansion.Results are shown in the last column in Table 4. Almost identical extrapolated resultsare obtained with the two finest grid triplets, which justifies the the use of the polynomialmethod. We choose the result obtained with the finest grid triplet, namely Err0=-0.350.With the result for the reference grid Err6=0.226, and a safety factor Fs=1.25, thenumerical uncertainty on the finest grid is estimated as:

Fs | Err0 |Err6

(φ6

i − φ1i

)= 1.94

(φ6

i − φ1i

)Comparison between different test cases

The estimated L1 norm error and the apparent order of convergence for different testcases are shown in figure 5. The quality of the numerical solutions obtained with dif-ferent grids are quite different, both in the magnitude of the error and in the order ofconvergence. Result obtained with the single block structured grid is problematic. Theerror level is two order higher compared with the best result obtained with the Cartesiangrid. In addition, the apparent order of convergence is too low (p=0.4). It is unlikelypossible that we can obtain a reliable uncertainty estimation for this case. Such kind ofnumerical solution should be disqualified in a certification procedure. On the order hand,results obtained with the block-structured grid and the unstructured quadrilateral grid

11

are validated solutions, in spite of the fact that the error is about 5 times higher comparedwith the result obtained with the Cartesian on the fine grid using about the same numberof grid points. Compared with the block-structured grid, the unstructured grid providesa numerical solution with a higher level of error, but a better convergence order. In bothcases, the order of convergence is lower than second order. For the test case with block-structured grid, we believe that the lost of accuracy is due to the grid quality defaultdue to mis-alignment error in the up corner of the step. In addition, grid density is notsufficient both in the region around the up corner and in the region after the step wherefree shear layer develops. For the case with unstructured grid, grid is intentionally refinedin the above mentioned two regions. We believe that the lost of accuracy is due to thefirst order accurate viscous flux reconstruction scheme employed at the grid refinementinterface rather than a grid resolution issue.

Number of cells

L1no

rmer

ror

50000 100000

10-2

10-1

100

Case A, Cartesian gridCase B, block structured gridCase C, unstructured quadrilateral gridCase D, single block structured grid

p=0.4

p=1.4p=1.0

p=1.9

Figure 5: Estimated L1 norm error and order of convergence

5.3 Uncertainty self consistency check

In this section, we compare the uncertainty estimation for different test cases to see ifthe error bars overlap. The solution differences with respect to the test case C togetherwith the uncertainty level are compared at three different location x=1h, 4h and 7h infigures 6, 7 and 8 respectively. At x=1h, the error bars for the test cases C and A do not

12

overlap (figure 6). Careful inspection reveals that the numerical solutions obtained forthe test case A is not the correct solution to the physical problem due to the activationof non-negative-value limiter for the turbulent eddy-viscosity near the lower wall. Thisalso explains the non-overlapping error bars near the lower wall between the test case Cand A at the other two location x=4h (figure 7) and x=7h (figure 8). At x=7h, the errorbars for the test cases C and A do not overlap even way from the wall. Error bars for thetest cases C and B overlap very well for all locations. In addition, they do not excessivelyoverlap, indicating that the level of the estimated uncertainty is appropriate. The level ofthe uncertainty for the test case D is about 20 times higher compared with the test case Band C. This order of magnitude is appropriate. Near the wall at the location x=4h (figure7), the uncertainty level is under estimated. Such failure in uncertainty estimation is notsurprising when we look at the convergence behaviour of the solution shown in figure 5.

Y

Solu

tion

diff

eren

ce

0.2 0.4 0.6 0.8 1 1.2 1.4-0.03

-0.02

-0.01

0

Case CCase A

Y

Solu

tion

diff

eren

ce

0.2 0.4 0.6 0.8 1 1.2 1.4

-0.006

-0.004

-0.002

0

0.002

0.004

0.006

Case CCase B

Y

Solu

tion

diff

eren

ce

0.2 0.4 0.6 0.8 1 1.2 1.4

-0.15

-0.1

-0.05

0

0.05

Case CCase D

Figure 6: Uncertainty self consistency check at x=1.

6 Conclusion

In addition to uncertainty estimation, we believe that the convergence behaviour of errornorms in a grid refinement study is equally important in a solution verification exercise.

13

Y

Solu

tion

diff

eren

ce

0.2 0.4 0.6 0.8 1 1.2 1.4

-0.004

-0.002

0

0.002

Case CCase A

Y

Solu

tion

diff

eren

ce

0.2 0.4 0.6 0.8 1 1.2 1.4

-0.006

-0.004

-0.002

0

0.002

0.004

0.006

Case CCase B

Y

Solu

tion

diff

eren

ce

0.2 0.4 0.6 0.8 1 1.2 1.4

-0.1

-0.05

0

0.05

Case CCase D

Figure 7: Uncertainty self consistency check at x=4.

This procedure is commonly employed in code verification execise using exact solution.A solution qualification procedure is proposed in this paper that allows to estimate er-ror norms with confidence in a solution verrification exercise where the exact solution isunknown. The convergence behaviour of the estimated error norms provides revealinginformations to the numerical solution. It can also provide an uncertainty estimation forlocal quantity. This procedure is verified with the manufactured solution and is appliedto the backward facing step test case. It is capable to qualify numerical solution withreliable confidence. For numerical solution with good convergence behaviour, we can ex-pect to obtain a reliable uncertainty estimation for local quantity. When the convergencebehaviour is poor, the uncertainty estimation for local quantity may not be correct insome region, but the level of the uncertainty is still realistic.

In order to obtain a good uncertainty estimation, the procedure needs to dynamicallyadjust itself according to the data. More than 3 grids are necessary in the verificationprocedure in order to obtain a reliable solution qualification and uncertainty estimation.When there is little data scattering and little variation in apparent order of convergence,Richardson extrapolation with finest grid triplet is the best choice. In the case wherethe apparent order of convergence changes too much or when it is too different comparedwith the expected theoretical order, the polynomial method may gives better estimation.

14

Y

Solu

tion

diff

eren

ce

0.2 0.4 0.6 0.8 1 1.2 1.4-0.004

-0.003

-0.002

-0.001

0

0.001

Case CCase A

Y

Solu

tion

diff

eren

ce

0.2 0.4 0.6 0.8 1 1.2 1.4

-0.004

-0.002

0

0.002

Case CCase B

Y

Solu

tion

diff

eren

ce

0.2 0.4 0.6 0.8 1 1.2 1.4

-0.08

-0.06

-0.04

-0.02

0

0.02

Case CCase D

Figure 8: Uncertainty self consistency check at x=7.

If data scattering problem is present, then the least squares approach is the only choice.

References

[1] L. Eca and M. Hoekstra. An Uncertainty Estimation Exercise with the Finite-Difference and Finite-Volume Versions of PARNASSOS. In L. Eca and M. Hoekstra,editors, Workshop on CFD Uncertainty Analysis. Lisbon, 2004.

[2] M. Visonneau G.B. Deng, P. Queutey. A Code Verification Exercise for the Unstruc-tured Finite-Volume CFD Solver ISIS-CFD. In European Conference on Computa-tional Fluid Dynamics. Egmond aan Zee, The Netherlands, 2006.

[3] A. Hay L. Eca, M. Hoekstra and D. Pelletier. A Manufactured Solution for a Two-Dimensional Steady Wall-Bounded Incompressible Turbulent Flow. IST Report D72-34, 2005.

[4] Patrick J. Roache. Verification and Validation in Computational Science and Engi-neering. Hermosa Publishers, 1998.

15