Sparse Observability using LP Presolve and LTDL Factorization in IMPL (IMPL-SparseObservability)

i n d u s t r IAL g o r i t h m s LLC. (IAL)

www.industrialgorithms.com August 2014

Introduction Presented in this short document is a description of our technology we call “Sparse Observability”. Observability is the estimatability metric (Bagajewicz, 2010) to structurally determine that an unmeasured variable or regressed parameter is either uniquely solvable (observable) or otherwise unsolvable (unobservable) in data reconciliation and regression (DRR) applications. Ultimately, our purpose to use efficient sparse matrix techniques is to solve large industrial DRR flowsheets quickly and accurately. Most other implementations of observability calculation use dense linear algebra such as reduced row echelon form (RREF), Gauss-Jordan decomposition (Crowe et. al. 1983; Madron 1992), QR factorization which can now be considered as semi-sparse (Swartz, 1989; Sanchez and Romagnoli, 1996), Schur complements, Cholesky factorization (Kelly, 1998a) and singular value decomposition (SVD) (Kelly, 1999). A sparse LU decomposition with complete-pivoting from Albuquerque and Biegler (1996) for dynamic data reconciliation observability computation was used but it is uncertain if complete-pivoting causes extreme “fill-ins” of the lower and upper triangular matrices essentially making them near-dense. There is another sparse observability method using an LP sub-solver found in Kelly and Zyngier (2008) but this requires solving as many LP sub-problems as there are unmeasured variables which can be considered as somewhat inefficient. IMPL’s sparse observability technique uses the variable classification and nomenclature found in Kelly (1998b) given that if we partition or separate the unmeasured variables into independent (B12) and dependent (B34) sub-matrices then all dependent unmeasured variables by definition are unobservable. If any independent unmeasured variable is a (linear) function of any dependent variable then this independent variable is of course also unobservable because it is dependent on another non-observable variable. This can be easily seen using the following (linearized) DRR equation (Kelly, 1998b):

A * x + [B12 B34] * [y12T y34T]T + C * z = 0 or

A * x + B12 * y12 + B34 * y34 + C * z = 0 where x represents the measured variables in column vector form, y are the unmeasured variables and z are the fixed variables or constants with corresponding sparse Jacobian matrices A, B12, B34 and C respectively. By multiplying both sides by B12T and rearranging we arrive at essentially the Normal equation found in typical ordinary least squares (OLS) regression which is unique to the classification method of Kelly (1998b):

B12T * B12 * y12 = -B12T * (A * x + B34 * y34 + C * z) or

y12 = -(B12T * B12)-1 * B12T * (B34 * y34 + A * x + C * z)

Therefore, if (B12T * B12)-1 * B12T * B34 has any non-zero elements (relative to machine precision) in any row then the corresponding independent unmeasured y12 variable is a linear function of a non-observable unmeasured y34 variable. Hence, the technology described below is an effective and reliable way to identify unobservable unmeasured variables in large-scale and industrial-sized flowsheet DRR problems. It should be mentioned that once the observability calculations have been performed, then the same matrices can be used to identify redundant and non-redundant measured variables described in Kelly (1998b). Redundant variables are observable if there measurement reading is missing, deleted or removed from the data content of the DRR problem. Non-redundant variables are unobservable if their measurement is no longer available given the remaining model and data of the DRR problem. Obviously, it is important to have as much hardware (measurement readings) and software (model relationships) redundancy in the system as possible given the capital, operational and maintenance costs where this sensor network design optimization problem is addressed in Kelly and Zyngier (2008). Technology IMPL’s sparse observability has two steps or parts: a presolve or preprocessing step to identify essentially trivial observable (strongly independent) unmeasured variables and a LTDL factorization for sparse symmetric indefinite matrices on the remaining equations and unmeasured variables to determine the dependent unmeasured variables. Our DRR presolve is taken directly from primal LP presolve techniques (Andersen and Andersen, 1995) and operates on the B sparse matrix (i.e., [B12 B34]) and to our knowledge, we are the first to adopt primal LP presolving techniques to the application of identifying observable unmeasured variables in DRR problems. First, it involves removing singleton rows with a single column indicating that the column or unmeasured variable is strongly observable because it is not a function of any other unmeasured variable. And second, it removes doubleton rows of the form b * yi – b * yj = 0 and one of the yi or yj unmeasured variables are removed because they are mirror images of one another; thus reducing the number of rows and columns of B. These two operations are repeated until there is no more improvement in reducing the rows and columns of B or there are no rows or columns left in B. These two straightforward primal LP presolve operations are very effective in practice and usually shrink the B matrix considerably. As an example, Figure 1 shows the unmeasured variable’s sparse matrix B taken from Crowe et. al. (1983)’s example #3.

Figure 1. Crowe et. al. (1983)’s Example #3.

All columns or unmeasured variables are observable and we determined these to be observable using our LP presolving techniques. For instance, the first pass or iteration of our presolve finds eight (8) singleton rows that can be removed i.e., “Node1 N2”, “Node1 Ar”, “Node2 N2”, “Node2 Ar”, “Node3 N2”, “Node3 H2”, “Node3 NH3”, “Splitter Constraints H2”. This means, that of the total thirteen (13) unmeasured variables, 8 are determined to be strongly observable easily. When these 8 rows and columns are removed a second pass further removes rows “Node2 NH3”, “Node2 H2”, “Node3 Ar” and “Node4 H2”. The third and final pass removes row “Node1 H2” and now completely identifies all columns as observable. When not all unmeasured variables are declared to be observable using our presolve method, we then need to proceed to identifying from the reduced or presolved set, which are independent and dependent columns using either of the methods mentioned previously. Instead, our approach in IMPL is to use the LTDL (Cholesky-like) factorization for symmetric indefinite sparse matrices from Intel’s Math Kernel Library (MKL) PARDISO (Schenk and Gartner, 2006). LTDL is similar to Cholesky factorization except that the former performs symmetric and other types of pivoting required to precisely determine if a symmetric matrix is singular (Hansen, 1987) where the later does not. More specifically, indefinite matrices have both positive and negative eigenvalues and some can be zero (semi-definite) and these types of indefinite matrices are found in non-convex and nonlinear interior-point optimization problems such as those solved by IPOPT. IMPL also supports locating independent and dependent unmeasured variables using a sparse version of the modified Cholesky factorization from Kelly (1998a) and, even though it does not employ any pivoting techniques, it is surprisingly accurate even for B matrices with large condition numbers i.e., maximum singular value divided by its minimum singular value. And, IMPL further supports a sparse LU decomposition technique with partial-pivoting to check for near-zero diagonal elements of the upper triangular matrix (U). If Ui,i ~ 0 then this (usually) indicates that column [B]i is dependent but is less accurate than the LTDL and our modified Cholesky.

In summary, when the LTDL factorization in PARDISO is performed using parallel-processing applied to the presolved BTB matrix, this provides a very fast, accurate and robust technique to identify observable and unobservable unmeasured variables in large-scale systems as well as its complement of redundant and non-redundant measured variables. For nonlinear problems, the A, B and C sparse matrices are linearized from the converged x and y solution values and provide only “point or local estimates” of the observability and redundancy indicators. References Crowe, C.M., Garcia Campos, Y.A., Hrymak, A., “Reconciliation of process flowrates by matrix projection, part 1: linear case”, American Institute of Chemical Engineering Journal, 29, 881-888, (1983). Hansen, P.C., “Detection of near-singularity in Cholesky and LDL factorizations”, Journal of Computational and Applied Mathematics, 19, 293-299, (1987). Swartz, C.L.E., “Data reconciliation for generalized flowsheet applications”, 197th National Meeting, American Chemical Society (ACS), Dallas, Texas, (1989). Madron, F., “Process plant performance: measurement and data processing for optimization and retrofits”, Ellis Horwood, New York, NY, (1992). Andersen, E.D., Andersen, K.D., “Presolving in linear programming”, Mathematical Programming, 71, 221-245, (1995). Albuquerque, J.S., Biegler, L.T., “Data reconciliation and gross-error detection for dynamic systems”, American Institute of Chemical Engineering Journal, 42, 10, (1996). Sanchez, M., Romagnoli, J., “Use of orthogonal transformations in data classification-reconciliation”, Computers & Chemical Engineering, 20, 483-493, (1996). Kelly, J.D., "On finding the matrix projection in the data reconciliation solution", Computers & Chemical Engineering, 1553, (1998a). Kelly, J.D., "A regularization approach to the reconciliation of constrained data sets", Computers & Chemical Engineering, 1771, (1998b). Kelly, J.D., "Reconciliation of process data using other projection matrices", Computers & Chemical Engineering, 785, (1999). Schenk, O., Gartner, K., “On fast factorization pivoting methods for sparse symmetric indefinite systems”, Electronic Transactions on Numerical Analysis, 23, 158-179, (2006). Kelly, J.D., Zyngier, D., "A new and improved MILP formulation to optimize observability, redundancy and precision for sensor network problems", American Institute of Chemical Engineering Journal, 54, 1282, (2008). Bagajewicz, M., “Smart process plants: software and hardware for accurate data and profitable operations”, McGraw-Hill (ISBN:978-0-07-160471), (2010).